Deepfake pornography, or simply fake pornography, is a type of synthetic pornography that is created via altering already-existing pornographic material by applying deepfake technology to the faces of the actors. The use of deepfake porn has sparked controversy because it involves the making and sharing of realistic videos featuring non-consenting individuals, typically female celebrities, and is sometimes used for revenge porn. Efforts are being made to combat these ethical concerns through legislation and technology-based solutions.
The term "deepfake" was coined in 2017 on a Reddit forum where users shared altered pornographic videos created using machine learning algorithms. It is a combination of the word "deep learning", which refers to the program used to create the videos, and "fake" meaning the videos are not real. [1]
Deepfake porn was originally created on a small individual scale using a combination of machine learning algorithms, computer vision techniques, and AI software. The process began by gathering a large amount of source material (including both images and videos) of a person's face, and then using a deep learning model to train a Generative Adversarial Network (GAN) to create a fake video that convincingly swaps the face of the source material onto the body of a porn performer. However, the production process has significantly evolved since 2018, with the advent of several public apps that have largely automated the process. [2]
In June 2019, a downloadable Windows and Linux application called DeepNude was released which used GAN to remove clothing from images of women. The app had both a paid and unpaid version, the paid version costing $50. [3] On June 27, the creators removed the application and refunded consumers, although various copies of the app, both free and for charge, continue to exist. [4] On GitHub, the open-source version of this program called "open-deepnude" was deleted. [5] The open-source version had the advantage of allowing to be trained on a larger dataset of nude images to increase the resulting nude image's accuracy level. [6]
In July 2019 a deepfake bot service was launched on messaging app Telegram that uses AI technology to create nude images of women. The service is free and has a user-friendly interface, enabling users to submit photos and receive manipulated nude images within minutes. The service is connected to seven Telegram channels, including the main channel that hosts the bot, technical support, and image sharing channels. While the total number of users is unknown, the main channel has over 45,000 members. As of July 2020, it is estimated that approximately 24,000 manipulated images have been shared across the image sharing channels. [7]
Deepfake technology has been used to create non-consensual and pornographic images and videos of famous women. One of the earliest examples occurred in 2017 when a deepfake pornographic video of Gal Gadot was created by a Reddit user and quickly spread online. Since then, there have been numerous instances of similar deepfake content targeting other female celebrities, such as Emma Watson, Natalie Portman, and Scarlett Johansson. [8] Johansson spoke publicly on the issue in December 2018, condemning the practice but also refusing legal action because she views the harassment as inevitable. [9]
In 2018, Rana Ayyub, an Indian investigative journalist, was the target of an online hate campaign stemming from her condemnation of the Indian government, specifically her speaking out against the rape of an eight-year-old Kashmiri girl. Ayyub was bombarded with rape and death threats, and had doctored pornographic video of her circulated online. [10] In a Huffington Post article, Ayyub discussed the long-lasting psychological and social effects this experience has had on her. She explained that she continued to struggle with her mental health and how the images and videos continued to resurface whenever she took a high-profile case. [11]
In 2023, Twitch streamer Atrioc stirred controversy when he accidentally revealed deepfake pornographic material featuring female Twitch streamers while on live. The influencer has since admitted to paying for AI generated porn, and apologized to the women and his fans. [12] [13]
In January 2024, AI-generated sexually explicit images of American singer Taylor Swift were posted on X (formerly Twitter), and spread to other platforms such as Facebook, Reddit and Instagram. [14] [15] [16] One tweet with the images was viewed over 45 million times before being removed. [17] [15] A report from 404 Media found that the images appeared to have originated from a Telegram group, whose members used tools such as Microsoft Designer to generate the images, using misspellings and keyword hacks to work around Designer's content filters. [18] [19] After the material was posted, Swift's fans posted concert footage and images to bury the deepfake images, and reported the accounts posting the deepfakes. [20] Searches for Swift's name were temporarily disabled on X, returning an error message instead. [21] Graphika, a disinformation research firm, traced the creation of the images back to a 4chan community. [22] [23]
A source close to Swift told the Daily Mail that she would be considering legal action, saying, "Whether or not legal action will be taken is being decided, but there is one thing that is clear: These fake AI-generated images are abusive, offensive, exploitative, and done without Taylor's consent and/or knowledge." [20] [24]
The controversy drew condemnation from White House Press Secretary Karine Jean-Pierre, [25] Microsoft CEO Satya Nadella, [26] the Rape, Abuse & Incest National Network, [27] and SAG-AFTRA. [28] Several US politicians called for federal legislation against deepfake pornography. [29] Later in the month, US senators Dick Durbin, Lindsey Graham, Amy Klobuchar and Josh Hawley introduced a bipartisan bill that would allow victims to sue individuals who produced or possessed "digital forgeries" with intent to distribute, or those who received the material knowing it was made non-consensually. [30]
Deepfake technology has made the creation of child sexual abuse material (CSAM), also often referenced to as child pornography, faster, safer and easier than it has ever been. Deepfakes can be used to produce new CSAM from already existing material or creating CSAM from children who have not been subjected to sexual abuse. Deepfake CSAM can, however, have real and direct implications on children including defamation, grooming, extortion, and bullying. [31]
Most of deepfake porn is made with faces of people who did not consent to their image being used in such a sexual way. In 2023, Sensity, an identify verification company, has found that "96% of deepfakes are sexually explicit and feature women who didn’t consent to the creation of the content." [32] Oftentimes, deepfake porn is used to humiliate and harass primarily women in ways similar to revenge porn.
Deepfake detection has become an increasingly important area of research in recent years as the spread of fake videos and images has become more prevalent. One promising approach to detecting deepfakes is through the use of Convolutional Neural Networks (CNNs), which have shown high accuracy in distinguishing between real and fake images. One CNN-based algorithm that has been developed specifically for deepfake detection is DeepRhythm, which has demonstrated an impressive accuracy score of 0.98 (i.e. successful at detecting deepfake images 98% of the time). This algorithm utilizes a pre-trained CNN to extract features from facial regions of interest and then applies a novel attention mechanism to identify discrepancies between the original and manipulated images. While the development of more sophisticated deepfake technology presents ongoing challenges to detection efforts, the high accuracy of algorithms like DeepRhythm offers a promising tool for identifying and mitigating the spread of harmful deepfakes. [33]
Aside from detection models, there are also video authenticating tools available to the public. In 2019, Deepware launched the first publicly available detection tool which allowed users to easily scan and detect deepfake videos. Similarly, in 2020 Microsoft released a free and user-friendly video authenticator. Users upload a suspected video or input a link, and receive a confidence score to assess the level of manipulation in a deepfake.
As of 2023, there is a lack of legislation that specifically addresses deepfake pornography. Instead, the harm caused by its creation and distribution is being addressed by the courts through existing criminal and civil laws.
Victims of deepfake pornography often have claims for revenge porn, tort claims, and harassment. [34] The legal consequences for revenge porn vary from state to state and country to country. [35] [36] For instance, in Canada, the penalty for publishing non-consensual intimate images is up to 5 years in prison, [37] whereas in Malta it is a fine of up to €5,000. [38]
The "Deepfake Accountability Act" was introduced to the United States Congress in 2019 but has died in 2020. [39] It aimed to make the production and distribution of digitally altered visual media that was not disclosed to be such, a criminal offense. The title specifies that making any sexual, non-consensual altered media with the intent of humiliating or otherwise harming the participants, may be fined, imprisoned for up to 5 years or both. [36] A newer version of bill was introduced in 2021 which would have required any "advanced technological false personation records" to contain a watermark and an audiovisual disclosure to identify and explain any altered audio and visual elements. The bill also includes that failure to disclose this information with intent to harass or humilitate a person with an "advanced technological false personation record" containing sexual content "shall be fined under this title, imprisoned for not more than 5 years, or both." However this bill has since died in 2023. [40]
While the legal landscape remains undeveloped, victims of deepfake pornography have several tools available to contain and remove content, including securing removal through a court order, intellectual property tools like the DMCA takedown, reporting for terms and conditions violations of the hosting platform, and removal by reporting the content to search engines. [41]
Several major online platforms have taken steps to ban deepfake pornography. As of 2018, gfycat, reddit, Twitter, Discord, and Pornhub have all prohibited the uploading and sharing of deepfake pornographic content on their platforms. [42] [43] In September of that same year, Google also added "involuntary synthetic pornographic imagery" to its ban list, allowing individuals to request the removal of such content from search results. [44]
Pornographic films (pornos), erotic films, adult films, sex films, 18+ films, or also known as blue movie or blue film, are films that present sexually explicit subject matter in order to arouse, fascinate, or satisfy the viewer. Pornographic films present sexual fantasies and usually include erotically stimulating material such as nudity (softcore) and sexual intercourse (hardcore). A distinction is sometimes made between "erotic" and "pornographic" films on the basis that the latter category contains more explicit sexuality, and focuses more on arousal than storytelling; the distinction is highly subjective.
Human image synthesis is technology that can be applied to make believable and even photorealistic renditions of human-likenesses, moving or still. It has effectively existed since the early 2000s. Many films using computer generated imagery have featured synthetic images of human-like characters digitally composited onto the real or other simulated film material. Towards the end of the 2010s deep learning artificial intelligence has been applied to synthesize images and video that look like humans, without need for human assistance, once the training phase has been completed, whereas the old school 7D-route required massive amounts of human work .
Pornography has been defined as sexual subject material "such as a picture, video, or text" that is intended for sexual arousal. Made for the consumption by adults, pornography depictions have evolved from cave paintings, some forty millennia ago, to virtual reality presentations. A general distinction of adult content is made classifying it as pornography or erotica.
There has been demand for imagery of nude celebrities for many decades. It is a lucrative business exploited by websites and magazines.
Amateur pornography is a category of pornography that features models, actors or non-professionals performing without pay, or actors for whom this material is not their only paid modeling work. Reality pornography is professionally made pornography that seeks to emulate the style of amateur pornography. Amateur pornography has been called one of the most profitable and long-lasting genres of pornography.
Legal frameworks around fictional pornography depicting minors vary depending on country and nature of the material involved. Laws against production, distribution and consumption of child pornography generally separate images into three categories: real, pseudo, and virtual. Pseudo-photographic child pornography is produced by digitally manipulating non-sexual images of real minors to make pornographic material. Virtual child pornography depicts purely fictional characters. "Fictional pornography depicting minors", as covered in this article, includes these latter two categories, whose legalities vary by jurisdiction, and often differ with each other and with the legality of real child pornography.
In the United States, child pornography is illegal under federal law and in all states and is punishable by up to life imprisonment and fines of up to $250,000. U.S. laws regarding child pornography are virtually always enforced and amongst the harshest in the world. The Supreme Court of the United States has found child pornography to be outside the protections of the First Amendment to the United States Constitution. Federal sentencing guidelines on child pornography differentiate between production, distribution, and purchasing/receiving, and also include variations in severity based on the age of the child involved in the materials, with significant increases in penalties when the offense involves a prepubescent child or a child under the age of 18. U.S. law distinguishes between pornographic images of an actual minor, realistic images that are not of an actual minor, and non-realistic images such as drawings. The latter two categories are legally protected unless found to be obscene, whereas the first does not require a finding of obscenity.
PhotoDNA is a proprietary image-identification and content filtering technology widely used by online service providers.
Revenge porn is the distribution of sexually explicit images or videos of individuals without their consent. The material may have been made by a partner in an intimate relationship with the knowledge and consent of the subject at the time, or it may have been made without their knowledge. The subject may have experienced sexual violence during the recording of the material, in some cases facilitated by narcotics such as date rape drugs which also cause a reduced sense of pain and involvement in the sexual act, dissociative effects and amnesia. The possession of the material may be used by the perpetrators to blackmail the subjects into performing other sexual acts, to coerce them into continuing a relationship or to punish them for ending one, to silence them, to damage their reputation, and/or for financial gain. In the wake of civil lawsuits and the increasing numbers of reported incidents, legislation has been passed in a number of countries and jurisdictions to outlaw the practice, though approaches have varied and been changed over the years. The practice has also been described as a form of psychological abuse and domestic violence, as well as a form of sexual abuse.
Celeb Jihad is a website known for sharing leaked private videos and photos as well as faked ones of celebrities as a form of jihad satire. The Daily Beast describes it as a "satirical celebrity gossip website."
Deepfakes were originally defined as synthetic media that have been digitally manipulated to replace one person's likeness convincingly with that of another. The term was coined in 2017 by a Reddit user, and has later been expanded to cover any videos, pictures, or audio made with artificial intelligence to appear real, for example realistic-looking images of people who do not exist. While the act of creating fake content is not new, deepfakes leverage tools and techniques from machine learning and artificial intelligence, including facial recognition algorithms and artificial neural networks such as variational autoencoders (VAEs) and generative adversarial networks (GANs). In turn the field of image forensics develops techniques to detect manipulated images. Deepfakes have garnered widespread attention for their potential use in creating child sexual abuse material, celebrity pornographic videos, revenge porn, fake news, hoaxes, bullying, and financial fraud. The spreading of disinformation and hate speech through deepfakes has a potential to undermine core functions and norms of democratic systems by interfering with people's ability to participate in decisions that affect them, determine collective agendas and express political will through informed decision-making. Both the information technology industry and government have responded with recommendations to detect and limit their use.
OnlyFans is an internet content subscription service based in London, United Kingdom. The service is used primarily by sex workers who produce pornography, but it also hosts the work of other content creators, such as physical fitness experts and musicians.
Artificial intelligence art is visual artwork created through the use of an artificial intelligence (AI) program.
Fake nude photography is the creation of nude photographs designed to appear as genuine nudes of an individual. The motivations for the creation of these modified photographs include sexual gratification, the stigmatization or embarrassment of the subject, and commercial gain, such as through the sale of the photographs via pornographic websites. Fakes can be created using image editing software or through machine learning. Fake pornographic images created using the latter method are called deepfakes.
Synthetic media is a catch-all term for the artificial production, manipulation, and modification of data and media by automated means, especially through the use of artificial intelligence algorithms, such as for the purpose of misleading people or changing an original meaning. Synthetic media as a field has grown rapidly since the creation of generative adversarial networks, primarily through the rise of deepfakes as well as music synthesis, text generation, human image synthesis, speech synthesis, and more. Though experts use the term "synthetic media," individual methods such as deepfakes and text synthesis are sometimes not referred to as such by the media but instead by their respective terminology Significant attention arose towards the field of synthetic media starting in 2017 when Motherboard reported on the emergence of AI altered pornographic videos to insert the faces of famous actresses. Potential hazards of synthetic media include the spread of misinformation, further loss of trust in institutions such as media and government, the mass automation of creative and journalistic jobs and a retreat into AI-generated fantasy worlds. Synthetic media is an applied form of artificial imagination.
An audio deepfake is a product of artificial intelligence used to create convincing speech sentences that sound like specific people saying things they did not say. This technology was initially developed for various applications to improve human life. For example, it can be used to produce audiobooks, and also to help people who have lost their voices to get them back. Commercially, it has opened the door to several opportunities. This technology can also create more personalized digital assistants and natural-sounding text-to-speech as well as speech translation services.
Synthesia is a synthetic media generation company that develops software used to create AI generated video content. It is based in London, England.
The AI boom, or AI spring, is an ongoing period of rapid progress in the field of artificial intelligence (AI) that started in the late 2010s. Known examples include protein folding prediction led by Google DeepMind and generative AI led by OpenAI.
In late January 2024, sexually explicit AI-generated deepfake images of American musician Taylor Swift were proliferated on social media platforms 4chan and X. The images led Microsoft to enhance Microsoft Designer's text-to-image model to prevent future abuse. Several artificial images of Swift of a sexual or violent nature were quickly spread, with one post reported to have been seen over 47 million times before its eventual removal. These images prompted responses from anti-sexual assault advocacy groups, US politicians, Swifties, Microsoft CEO Satya Nadella, among others, and it has been suggested that Swift's influence could result in new legislation regarding the creation of deepfake pornography.
Graphika is an American social network analysis company known for tracking online disinformation. It was established in 2013.