Audio deepfake

Last updated

Audio deepfake technology, also referred to as voice cloning or deepfake audio, is an application of artificial intelligence designed to generate speech that convincingly mimics specific individuals, often synthesizing phrases or sentences they have never spoken. [1] [2] [3] [4] Initially developed with the intent to enhance various aspects of human life, it has practical applications such as generating audiobooks and assisting individuals who have lost their voices due to medical conditions. [5] [6] Additionally, it has commercial uses, including the creation of personalized digital assistants, natural-sounding text-to-speech systems, and advanced speech translation services. [7]

Contents

Incidents of fraud

Audio deepfakes, referred to as audio manipulations beginning in the early 2020s, are becoming widely accessible using simple mobile devices or personal computers. [8] These tools have also been used to spread misinformation using audio. [3] This has led to cybersecurity concerns among the global public about the side effects of using audio deepfakes, including its possible role in disseminating misinformation and disinformation in audio-based social media platforms. [9] People can use them as a logical access voice spoofing technique, [10] where they can be used to manipulate public opinion for propaganda, defamation, or terrorism. Vast amounts of voice recordings are daily transmitted over the Internet, and spoofing detection is challenging. [11] Audio deepfake attackers have targeted individuals and organizations, including politicians and governments. [12]

In 2019, scammers using AI impersonated the voice of the CEO of a German energy company and directed the CEO of its UK subsidiary to transfer € 220,000. [13] In early 2020, the same technique impersonated a company director as part of an elaborate scheme that convinced a branch manager to transfer $35 million. [14]

According to a 2023 global McAfee survey, one person in ten reported having been targeted by an AI voice cloning scam; 77% of these targets reported losing money to the scam. [15] [16] Audio deepfakes could also pose a danger to voice ID systems currently used by financial institutions. [17] [18] In March 2023, the United States Federal Trade Commission issued a warning to consumers about the use of AI to fake the voice of a family member in distress asking for money. [19]

In October 2023, during the start of the British Labour Party's conference in Liverpool, an audio deepfake of Labour leader Keir Starmer was released that falsely portrayed him verbally abusing his staffers and criticizing Liverpool. [20] That same month, an audio deepfake of Slovak politician Michal Šimečka falsely claimed to capture him discussing ways to rig the upcoming election. [21]

During the campaign for the 2024 New Hampshire Democratic presidential primary, over 20,000 voters received robocalls from an AI-impersonated President Joe Biden urging them not to vote. [22] [23] The New Hampshire attorney general said this violated state election laws, and alleged involvement by Life Corporation and Lingo Telecom. [24] In February 2024, the United States Federal Communications Commission banned the use of AI to fake voices in robocalls. [25] [26] That same month, political consultant Steve Kramer admitted that he had commissioned the calls for $500. He said that he wanted to call attention to the need for rules governing the use of AI in political campaigns. [27] In May, the FCC said that Kramer had violated federal law by spoofing the number of a local political figure, and proposed a fine of $6 million. Four New Hampshire counties indicted Kramer on felony counts of voter suppression, and impersonating a candidate, a misdemeanor. [28]

Categories

Audio deepfakes can be divided into three different categories:

Replay-based

Replay-based deepfakes are malicious works that aim to reproduce a recording of the interlocutor's voice. [29]

There are two types: far-field detection and cut-and-paste detection. In far-field detection, a microphone recording of the victim is played as a test segment on a hands-free phone. [30] On the other hand, cut-and-paste involves faking the requested sentence from a text-dependent system. [11] Text-dependent speaker verification can be used to defend against replay-based attacks. [29] [31] A current technique that detects end-to-end replay attacks is the use of deep convolutional neural networks. [32]

Synthetic-based

The synthetic-based approach diagram TTS diagram.png
The synthetic-based approach diagram

The category based on speech synthesis refers to the artificial production of human speech, using software or hardware system programs. Speech synthesis includes text-to-speech, which aims to transform the text into acceptable and natural speech in real-time, [33] making the speech sound in line with the text input, using the rules of linguistic description of the text.

A classical system of this type consists of three modules: a text analysis model, an acoustic model, and a vocoder. The generation usually has to follow two essential steps. It is necessary to collect clean and well-structured raw audio with the transcripted text of the original speech audio sentence. Second, the text-to-speech model must be trained using these data to build a synthetic audio generation model.

Specifically, the transcribed text with the target speaker's voice is the input of the generation model. The text analysis module processes the input text and converts it into linguistic features. Then, the acoustic module extracts the parameters of the target speaker from the audio data based on the linguistic features generated by the text analysis module. [8] Finally, the vocoder learns to create vocal waveforms based on the parameters of the acoustic features. The final audio file is generated, including the synthetic simulation audio in a waveform format, creating speech audio in the voice of many speakers, even those not in training.

The first breakthrough in this regard was introduced by WaveNet, [34] a neural network for generating raw audio waveforms capable of emulating the characteristics of many different speakers. This network has been overtaken over the years by other systems [35] [36] [37] [38] [39] [40] which synthesize highly realistic artificial voices within everyone’s reach. [41]

Text-to-speech is highly dependent on the quality of the voice corpus used to realize the system, and creating an entire voice corpus is expensive.[ citation needed ] Another disadvantage is that speech synthesis systems do not recognize periods or special characters. Also, ambiguity problems are persistent, as two words written in the same way can have different meanings.[ citation needed ]

Imitation-based

The Imitation-based approach diagram Imitation-based approach.png
The Imitation-based approach diagram

Audio deepfake based on imitation is a way of transforming an original speech from one speaker - the original - so that it sounds spoken like another speaker - the target one. [42] An imitation-based algorithm takes a spoken signal as input and alters it by changing its style, intonation, or prosody, trying to mimic the target voice without changing the linguistic information. [43] This technique is also known as voice conversion.

This method is often confused with the previous synthetic-based method, as there is no clear separation between the two approaches regarding the generation process. Indeed, both methods modify acoustic-spectral and style characteristics of the speech audio signal, but the Imitation-based usually keeps the input and output text unaltered. This is obtained by changing how this sentence is spoken to match the target speaker's characteristics. [44]

Voices can be imitated in several ways, such as using humans with similar voices that can mimic the original speaker. In recent years, the most popular approach involves the use of particular neural networks called generative adversarial networks (GAN) due to their flexibility as well as high-quality results. [29] [42]

Then, the original audio signal is transformed to say a speech in the target audio using an imitation generation method that generates a new speech, shown in the fake one.

Detection methods

The audio deepfake detection task determines whether the given speech audio is real or fake.

Recently, this has become a hot topic in the forensic research community, trying to keep up with the rapid evolution of counterfeiting techniques.

In general, deepfake detection methods can be divided into two categories based on the aspect they leverage to perform the detection task. The first focuses on low-level aspects, looking for artifacts introduced by the generators at the sample level. The second, instead, focus on higher-level features representing more complex aspects as the semantic content of the speech audio recording.

A generic audio deepfake detection framework Audio deepfake detection.png
A generic audio deepfake detection framework

Many machine learning models have been developed using different strategies to detect fake audio. Most of the time, these algorithms follow a three-steps procedure:

  1. Each speech audio recording must be preprocessed and transformed into appropriate audio features;
  2. The computed features are fed into the detection model, which performs the necessary operations, such as the training process, essential to discriminate between real and fake speech audio;
  3. The output is fed into the final module to produce a prediction probability of the Fake class or the Real one. Following the ASVspoof [45] challenge nomenclature, the Fake audio is indicated with the term "Spoof," the Real instead is called "Bonafide."

Over the years, many researchers have shown that machine learning approaches are more accurate than deep learning methods, regardless of the features used. [8] However, the scalability of machine learning methods is not confirmed due to excessive training and manual feature extraction, especially with many audio files. Instead, when deep learning algorithms are used, specific transformations are required on the audio files to ensure that the algorithms can handle them.

There are several open-source implementations of different detection methods, [46] [47] [48] and usually many research groups release them on a public hosting service like GitHub.

Open challenges and future research direction

The audio deepfake is a very recent field of research. For this reason, there are many possibilities for development and improvement, as well as possible threats that adopting this technology can bring to our daily lives. The most important ones are listed below.

Deepfake generation

Regarding the generation, the most significant aspect is the credibility of the victim, i.e., the perceptual quality of the audio deepfake.

Several metrics determine the level of accuracy of audio deepfake generation, and the most widely used is the mean opinion score (MOS), which is the arithmetic average of user ratings. Usually, the test to be rated involves perceptual evaluation of sentences made by different speech generation algorithms. This index showed that audio generated by algorithms trained on a single speaker has a higher MOS. [44] [34] [49] [50] [39]

The sampling rate also plays an essential role in detecting and generating audio deepfakes. Currently, available datasets have a sampling rate of around 16 kHz, significantly reducing speech quality. An increase in the sampling rate could lead to higher quality generation. [37]

In March 2020, a Massachusetts Institute of Technology researcher demonstrated data-efficient audio deepfake generation through 15.ai, a web application capable of generating high-quality speech using only 15 seconds of training data, [51] [52] compared to previous systems that required tens of hours. [53] The system implemented a unified multi-speaker model that enabled simultaneous training of multiple voices through speaker embeddings, allowing the model to learn shared patterns across different voices even when individual voices lacked examples of certain emotional contexts. [54] The platform integrated sentiment analysis through DeepMoji for emotional expression and supported precise pronunciation control via ARPABET phonetic transcriptions. [55] The 15-second data efficiency benchmark was later corroborated by OpenAI in 2024. [56]

Deepfake detection

Focusing on the detection part, one principal weakness affecting recent models is the adopted language.

Most studies focus on detecting audio deepfake in the English language, not paying much attention to the most spoken languages like Chinese and Spanish, [57] as well as Hindi and Arabic.

It is also essential to consider more factors related to different accents that represent the way of pronunciation strictly associated with a particular individual, location, or nation. In other fields of audio, such as speaker recognition, the accent has been found to influence the performance significantly, [58] so it is expected that this feature could affect the models' performance even in this detection task.

In addition, the excessive preprocessing of the audio data has led to a very high and often unsustainable computational cost. For this reason, many researchers have suggested following a self-supervised learning approach, [59] dealing with unlabeled data to work effectively in detection tasks and improving the model's scalability, and, at the same time, decreasing the computational cost.

Training and testing models with real audio data is still an underdeveloped area. Indeed, using audio with real-world background noises can increase the robustness of the fake audio detection models.

In addition, most of the effort is focused on detecting synthetic-based audio deepfakes, and few studies are analyzing imitation-based due to their intrinsic difficulty in the generation process. [11]

Defense against deepfakes

Over the years, there has been an increase in techniques aimed at defending against malicious actions that audio deepfake could bring, such as identity theft and manipulation of speeches by the nation's governors.

To prevent deepfakes, some suggest using blockchain and other distributed ledger technologies (DLT) to identify the provenance of data and track information. [8] [60] [61] [62]

Extracting and comparing affective cues corresponding to perceived emotions from digital content has also been proposed to combat deepfakes. [63] [64] [65]

Another critical aspect concerns the mitigation of this problem. It has been suggested that it would be better to keep some proprietary detection tools only for those who need them, such as fact-checkers for journalists. [29] That way, those who create the generation models, perhaps for nefarious purposes, would not know precisely what features facilitate the detection of a deepfake, [29] discouraging possible attackers.

To improve the detection instead, researchers are trying to generalize the process, [66] looking for preprocessing techniques that improve performance and testing different loss functions used for training. [10] [67]

Research programs

Numerous research groups worldwide are working to recognize media manipulations; i.e., audio deepfakes but also image and video deepfake. These projects are usually supported by public or private funding and are in close contact with universities and research institutions.

For this purpose, the Defense Advanced Research Projects Agency (DARPA) runs the Semantic Forensics (SemaFor). [68] [69] Leveraging some of the research from the Media Forensics (MediFor) [70] [71] program, also from DARPA, these semantic detection algorithms will have to determine whether a media object has been generated or manipulated, to automate the analysis of media provenance and uncover the intent behind the falsification of various content. [72] [68]

Another research program is the Preserving Media Trustworthiness in the Artificial Intelligence Era (PREMIER) [73] program, funded by the Italian Ministry of Education, University and Research (MIUR) and run by five Italian universities. PREMIER will pursue novel hybrid approaches to obtain forensic detectors that are more interpretable and secure. [74]

DEEP-VOICE [75] is a publicly available dataset intended for research purposes to develop systems to detect when speech has been generated with neural networks through a process called Retrieval-based Voice Conversion (RVC). Preliminary research showed numerous statistically-significant differences between features found in human speech and that which had been generated by Artificial Intelligence algorithms.

Public challenges

In the last few years, numerous challenges have been organized to push this field of audio deepfake research even further.

The most famous world challenge is the ASVspoof, [45] the Automatic Speaker Verification Spoofing and Countermeasures Challenge. This challenge is a bi-annual community-led initiative that aims to promote the consideration of spoofing and the development of countermeasures. [76]

Another recent challenge is the ADD [77] —Audio Deepfake Detection—which considers fake situations in a more real-life scenario. [78]

Also the Voice Conversion Challenge [79] is a bi-annual challenge, created with the need to compare different voice conversion systems and approaches using the same voice data.

See also

Related Research Articles

Speech recognition is an interdisciplinary subfield of computer science and computational linguistics that develops methodologies and technologies that enable the recognition and translation of spoken language into text by computers. It is also known as automatic speech recognition (ASR), computer speech recognition or speech-to-text (STT). It incorporates knowledge and research in the computer science, linguistics and computer engineering fields. The reverse process is speech synthesis.

Speech synthesis is the artificial production of human speech. A computer system used for this purpose is called a speech synthesizer, and can be implemented in software or hardware products. A text-to-speech (TTS) system converts normal language text into speech; other systems render symbolic linguistic representations like phonetic transcriptions into speech. The reverse process is speech recognition.

<span class="mw-page-title-main">Human image synthesis</span> Computer generation of human images

Human image synthesis is technology that can be applied to make believable and even photorealistic renditions of human-likenesses, moving or still. It has effectively existed since the early 2000s. Many films using computer generated imagery have featured synthetic images of human-like characters digitally composited onto the real or other simulated film material. Towards the end of the 2010s deep learning artificial intelligence has been applied to synthesize images and video that look like humans, without need for human assistance, once the training phase has been completed, whereas the old school 7D-route required massive amounts of human work .

Music and artificial intelligence is the development of music software programs which use AI to generate music. As with applications in other fields, AI in music also simulates mental tasks. A prominent feature is the capability of an AI algorithm to learn based on past data, such as in computer accompaniment technology, wherein the AI is capable of listening to a human performer and performing accompaniment. Artificial intelligence also drives interactive composition technology, wherein a computer composes music in response to a live performance. There are other AI applications in music that cover not only music composition, production, and performance but also how music is marketed and consumed. Several music player programs have also been developed to use voice recognition and natural language processing technology for music voice control. Current research includes the application of AI in music composition, performance, theory and digital sound processing.

<span class="mw-page-title-main">Deep learning</span> Branch of machine learning

Deep learning is a subset of machine learning that focuses on utilizing neural networks to perform tasks such as classification, regression, and representation learning. The field takes inspiration from biological neuroscience and is centered around stacking artificial neurons into layers and "training" them to process data. The adjective "deep" refers to the use of multiple layers in the network. Methods used can be either supervised, semi-supervised or unsupervised.

Adversarial machine learning is the study of the attacks on machine learning algorithms, and of the defenses against such attacks. A survey from May 2020 exposes the fact that practitioners report a dire need for better protecting machine learning systems in industrial applications.

Multimodal learning is a type of deep learning that integrates and processes multiple types of data, referred to as modalities, such as text, audio, images, or video. This integration allows for a more holistic understanding of complex data, improving model performance in tasks like visual question answering, cross-modal retrieval, text-to-image generation, aesthetic ranking, and image captioning.

<span class="mw-page-title-main">Generative adversarial network</span> Deep learning method

A generative adversarial network (GAN) is a class of machine learning frameworks and a prominent framework for approaching generative artificial intelligence. The concept was initially developed by Ian Goodfellow and his colleagues in June 2014. In a GAN, two neural networks contest with each other in the form of a zero-sum game, where one agent's gain is another agent's loss.

WaveNet is a deep neural network for generating raw audio. It was created by researchers at London-based AI firm DeepMind. The technique, outlined in a paper in September 2016, is able to generate relatively realistic-sounding human-like voices by directly modelling waveforms using a neural network method trained with recordings of real speech. Tests with US English and Mandarin reportedly showed that the system outperforms Google's best existing text-to-speech (TTS) systems, although as of 2016 its text-to-speech synthesis still was less convincing than actual human speech. WaveNet's ability to generate raw waveforms means that it can model any kind of audio, including music.

<span class="mw-page-title-main">Generative audio</span> Creation of audio files from databases of audio clips

Generative audio refers to the creation of audio files from databases of audio clips. This technology differs from synthesized voices such as Apple's Siri or Amazon's Alexa, which use a collection of fragments that are stitched together on demand.

<span class="mw-page-title-main">Deepfake</span> Realistic artificially generated media

Deepfakes are images, videos, or audio which are edited or generated using artificial intelligence tools, and which may depict real or non-existent people. They are a type of synthetic media and modern form of a Media prank.

<span class="mw-page-title-main">Tsetlin machine</span> Artificial intelligence algorithm

A Tsetlin machine is an artificial intelligence algorithm based on propositional logic.

<span class="mw-page-title-main">Artificial intelligence art</span> Visual media created with AI

Artificial intelligence art is visual artwork created or enhanced through the use of artificial intelligence (AI) programs.

Synthetic media is a catch-all term for the artificial production, manipulation, and modification of data and media by automated means, especially through the use of artificial intelligence algorithms, such as for the purpose of misleading people or changing an original meaning. Synthetic media as a field has grown rapidly since the creation of generative adversarial networks, primarily through the rise of deepfakes as well as music synthesis, text generation, human image synthesis, speech synthesis, and more. Though experts use the term "synthetic media," individual methods such as deepfakes and text synthesis are sometimes not referred to as such by the media but instead by their respective terminology Significant attention arose towards the field of synthetic media starting in 2017 when Motherboard reported on the emergence of AI altered pornographic videos to insert the faces of famous actresses. Potential hazards of synthetic media include the spread of misinformation, further loss of trust in institutions such as media and government, the mass automation of creative and journalistic jobs and a retreat into AI-generated fantasy worlds. Synthetic media is an applied form of artificial imagination.

Deepfake pornography, or simply fake pornography, is a type of synthetic pornography that is created via altering already-existing photographs or video by applying deepfake technology to the images of the participants. The use of deepfake pornography has sparked controversy because it involves the making and sharing of realistic videos featuring non-consenting individuals, typically female celebrities, and is sometimes used for revenge porn. Efforts are being made to combat these ethical concerns through legislation and technology-based solutions.

Deep learning speech synthesis refers to the application of deep learning models to generate natural-sounding human speech from written text (text-to-speech) or spectrum (vocoder). Deep neural networks are trained using large amounts of recorded speech and, in the case of a text-to-speech system, the associated labels and/or input text.

<span class="mw-page-title-main">Text-to-video model</span> Machine learning model

A text-to-video model is a machine learning model that uses a natural language description as input to produce a video relevant to the input text. Advancements during the 2020s in the generation of high-quality, text-conditioned videos have largely been driven by the development of video diffusion models.

ElevenLabs is a software company that specializes in developing natural-sounding speech synthesis software using deep learning.

<span class="mw-page-title-main">Generative artificial intelligence</span> AI system capable of generating content in response to prompts

Generative artificial intelligence is a subset of artificial intelligence that uses generative models to produce text, images, videos, or other forms of data. These models learn the underlying patterns and structures of their training data and use them to produce new data based on the input, which often comes in the form of natural language prompts.

15.ai was a free non-commercial web application that used artificial intelligence to generate text-to-speech voices of fictional characters from popular media. Created by an artificial intelligence researcher known as 15 during their time at the Massachusetts Institute of Technology, the application allowed users to make characters from video games, television shows, and movies speak custom text with emotional inflections faster than real-time. The platform was notable for its ability to generate convincing voice output using minimal training data—the name "15.ai" referenced the creator's claim that a voice could be cloned with just 15 seconds of audio. It was an early example of an application of generative artificial intelligence during the initial stages of the AI boom.

References

  1. ↑ Smith, Hannah; Mansted, Katherine (April 1, 2020). Weaponised deep fakes: National security and democracy. Vol. 28. Australian Strategic Policy Institute. pp. 11–13. ISSN   2209-9689.{{cite book}}: CS1 maint: date and year (link)
  2. ↑ Lyu, Siwei (2020). "Deepfake Detection: Current Challenges and Next Steps". 2020 IEEE International Conference on Multimedia & Expo Workshops (ICMEW). pp. 1–6. arXiv: 2003.09234 . doi:10.1109/icmew46912.2020.9105991. ISBN   978-1-7281-1485-9. S2CID   214605906 . Retrieved 2022-06-29.
  3. 1 2 Diakopoulos, Nicholas; Johnson, Deborah (June 2020). "Anticipating and addressing the ethical implications of deepfakes in the context of elections". New Media & Society. 23 (7) (published 2020-06-05): 2072–2098. doi:10.1177/1461444820925811. ISSN   1461-4448. S2CID   226196422.
  4. ↑ Murphy, Margi (20 February 2024). "Deepfake Audio Boom Exploits One Billion-Dollar Startup's AI". Bloomberg.
  5. ↑ Chadha, Anupama; Kumar, Vaibhav; Kashyap, Sonu; Gupta, Mayank (2021), Singh, Pradeep Kumar; Wierzchoń, Sławomir T.; Tanwar, Sudeep; Ganzha, Maria (eds.), "Deepfake: An Overview", Proceedings of Second International Conference on Computing, Communications, and Cyber-Security, Lecture Notes in Networks and Systems, vol. 203, Singapore: Springer Singapore, pp. 557–566, doi:10.1007/978-981-16-0733-2_39, ISBN   978-981-16-0732-5, S2CID   236666289 , retrieved 2022-06-29
  6. ↑ "AI gave Val Kilmer his voice back. But critics worry the technology could be misused". Washington Post. ISSN   0190-8286 . Retrieved 2022-06-29.
  7. ↑ Etienne, Vanessa (August 19, 2021). "Val Kilmer Gets His Voice Back After Throat Cancer Battle Using AI Technology: Hear the Results". PEOPLE.com. Retrieved 2022-07-01.
  8. 1 2 3 4 Almutairi, Zaynab; Elgibreen, Hebah (2022-05-04). "A Review of Modern Audio Deepfake Detection Methods: Challenges and Future Directions". Algorithms. 15 (5): 155. doi: 10.3390/a15050155 . ISSN   1999-4893.
  9. ↑ Caramancion, Kevin Matthe (June 2022). "An Exploration of Mis/Disinformation in Audio Format Disseminated in Podcasts: Case Study of Spotify". 2022 IEEE International IOT, Electronics and Mechatronics Conference (IEMTRONICS). pp. 1–6. doi:10.1109/IEMTRONICS55184.2022.9795760. ISBN   978-1-6654-8684-2. S2CID   249903722.
  10. 1 2 Chen, Tianxiang; Kumar, Avrosh; Nagarsheth, Parav; Sivaraman, Ganesh; Khoury, Elie (2020-11-01). "Generalization of Audio Deepfake Detection". The Speaker and Language Recognition Workshop (Odyssey 2020). ISCA: 132–137. doi:10.21437/Odyssey.2020-19. S2CID   219492826.
  11. 1 2 3 Ballesteros, Dora M.; Rodriguez-Ortega, Yohanna; Renza, Diego; Arce, Gonzalo (2021-12-01). "Deep4SNet: deep learning for fake speech classification". Expert Systems with Applications. 184: 115465. doi:10.1016/j.eswa.2021.115465. ISSN   0957-4174. S2CID   237659479.
  12. ↑ Suwajanakorn, Supasorn; Seitz, Steven M.; Kemelmacher-Shlizerman, Ira (2017-07-20). "Synthesizing Obama: learning lip sync from audio". ACM Transactions on Graphics. 36 (4): 95:1–95:13. doi:10.1145/3072959.3073640. ISSN   0730-0301. S2CID   207586187.
  13. ↑ Stupp, Catherine. "Fraudsters Used AI to Mimic CEO's Voice in Unusual Cybercrime Case". WSJ. Retrieved 2024-05-26.
  14. ↑ Brewster, Thomas. "Fraudsters Cloned Company Director's Voice In $35 Million Bank Heist, Police Find". Forbes. Retrieved 2022-06-29.
  15. ↑ "Generative AI is making voice scams easier to believe". Axios. 13 June 2023. Retrieved 16 June 2023.
  16. ↑ Bunn, Amy (15 May 2023). "Artificial Imposters—Cybercriminals Turn to AI Voice Cloning for a New Breed of Scam". McAfee Blog. Retrieved 16 June 2023.
  17. ↑ Cox, Joseph (23 February 2023). "How I Broke Into a Bank Account With an AI-Generated Voice". Vice. Retrieved 16 June 2023.
  18. ↑ Evershed, Nick; Taylor, Josh (16 March 2023). "AI can fool voice recognition used to verify identity by Centrelink and Australian tax office". The Guardian. Retrieved 16 June 2023.
  19. ↑ "Scammers use AI to enhance their family emergency schemes". Consumer Advice. 2023-03-17. Retrieved 2024-05-26.
  20. ↑ "Deepfake audio of Sir Keir Starmer released on first day of Labour conference".
  21. ↑ Meaker, Morgan. "Slovakia's Election Deepfakes Show AI is a Danger to Democracy". Wired.
  22. ↑ "Political consultant behind fake Biden AI robocall faces charges in New Hampshire".
  23. ↑ "Political consultant accused of hiring magician to spam voters with Biden deepfake calls". Law & Crime. 2024-03-15. Retrieved 2024-05-23.
  24. ↑ David Wright; Brian Fung; Brian Fung (February 6, 2024). "Fake Biden robocall linked to Texas-based companies, New Hampshire attorney general announces". CNN.
  25. ↑ Brian Fung (February 8, 2024). "FCC votes to ban scam robocalls that use AI-generated voices". CNN.
  26. ↑ "FCC Makes AI-Generated Voices in Robocalls Illegal | Federal Communications Commission". www.fcc.gov. 2024-02-08. Retrieved 2024-05-26.
  27. ↑ Kramer, Marcia (2024-02-26). "Steve Kramer explains why he used AI to impersonate President Biden in New Hampshire - CBS New York". www.cbsnews.com. Retrieved 2024-05-23.
  28. ↑ "A political consultant faces charges and fines for Biden deepfake robocalls".
  29. 1 2 3 4 5 Khanjani, Zahra; Watson, Gabrielle; Janeja, Vandana P. (2021-11-28). "How Deep Are the Fakes? Focusing on Audio Deepfake: A Survey". arXiv: 2111.14203 [cs.SD].
  30. ↑ Pradhan, Swadhin; Sun, Wei; Baig, Ghufran; Qiu, Lili (2019-09-09). "Combating Replay Attacks Against Voice Assistants". Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies. 3 (3): 100:1–100:26. doi:10.1145/3351258. S2CID   202159551.
  31. ↑ Villalba, Jesus; Lleida, Eduardo (2011). "Preventing replay attacks on speaker verification systems". 2011 Carnahan Conference on Security Technology. pp. 1–8. doi:10.1109/CCST.2011.6095943. ISBN   978-1-4577-0903-6. S2CID   17048213 . Retrieved 2022-06-29.
  32. ↑ Tom, Francis; Jain, Mohit; Dey, Prasenjit (2018-09-02). "End-To-End Audio Replay Attack Detection Using Deep Convolutional Networks with Attention". Interspeech 2018. ISCA: 681–685. doi:10.21437/Interspeech.2018-2279. S2CID   52187155.
  33. ↑ Tan, Xu; Qin, Tao; Soong, Frank; Liu, Tie-Yan (2021-07-23). "A Survey on Neural Speech Synthesis". arXiv: 2106.15561 [eess.AS].
  34. 1 2 Oord, Aaron van den; Dieleman, Sander; Zen, Heiga; Simonyan, Karen; Vinyals, Oriol; Graves, Alex; Kalchbrenner, Nal; Senior, Andrew; Kavukcuoglu, Koray (2016-09-19). "WaveNet: A Generative Model for Raw Audio". arXiv: 1609.03499 [cs.SD].
  35. ↑ Kuchaiev, Oleksii; Li, Jason; Nguyen, Huyen; Hrinchuk, Oleksii; Leary, Ryan; Ginsburg, Boris; Kriman, Samuel; Beliaev, Stanislav; Lavrukhin, Vitaly; Cook, Jack; Castonguay, Patrice (2019-09-13). "NeMo: a toolkit for building AI applications using Neural Modules". arXiv: 1909.09577 [cs.LG].
  36. ↑ Wang, Yuxuan; Skerry-Ryan, R. J.; Stanton, Daisy; Wu, Yonghui; Weiss, Ron J.; Jaitly, Navdeep; Yang, Zongheng; Xiao, Ying; Chen, Zhifeng; Bengio, Samy; Le, Quoc (2017-04-06). "Tacotron: Towards End-to-End Speech Synthesis". arXiv: 1703.10135 [cs.CL].
  37. 1 2 Prenger, Ryan; Valle, Rafael; Catanzaro, Bryan (2018-10-30). "WaveGlow: A Flow-based Generative Network for Speech Synthesis". arXiv: 1811.00002 [cs.SD].
  38. ↑ Vasquez, Sean; Lewis, Mike (2019-06-04). "MelNet: A Generative Model for Audio in the Frequency Domain". arXiv: 1906.01083 [eess.AS].
  39. 1 2 Ping, Wei; Peng, Kainan; Gibiansky, Andrew; Arik, Sercan O.; Kannan, Ajay; Narang, Sharan; Raiman, Jonathan; Miller, John (2018-02-22). "Deep Voice 3: Scaling Text-to-Speech with Convolutional Sequence Learning". arXiv: 1710.07654 [cs.SD].
  40. ↑ Ren, Yi; Ruan, Yangjun; Tan, Xu; Qin, Tao; Zhao, Sheng; Zhao, Zhou; Liu, Tie-Yan (2019-11-20). "FastSpeech: Fast, Robust and Controllable Text to Speech". arXiv: 1905.09263 [cs.CL].
  41. ↑ Ning, Yishuang; He, Sheng; Wu, Zhiyong; Xing, Chunxiao; Zhang, Liang-Jie (January 2019). "A Review of Deep Learning Based Speech Synthesis". Applied Sciences. 9 (19): 4050. doi: 10.3390/app9194050 . ISSN   2076-3417.
  42. 1 2 Rodríguez-Ortega, Yohanna; Ballesteros, Dora María; Renza, Diego (2020). "A Machine Learning Model to Detect Fake Voice". In Florez, Hector; Misra, Sanjay (eds.). Applied Informatics. Communications in Computer and Information Science. Vol. 1277. Cham: Springer International Publishing. pp. 3–13. doi:10.1007/978-3-030-61702-8_1. ISBN   978-3-030-61702-8. S2CID   226283369.
  43. ↑ Zhang, Mingyang; Wang, Xin; Fang, Fuming; Li, Haizhou; Yamagishi, Junichi (2019-04-07). "Joint training framework for text-to-speech and voice conversion using multi-source Tacotron and WaveNet". arXiv: 1903.12389 [eess.AS].
  44. 1 2 Sercan, Ö Arık; Jitong, Chen; Kainan, Peng; Wei, Ping; Yanqi, Zhou (2018). "Neural Voice Cloning with a Few Samples". Advances in Neural Information Processing Systems (NeurIPS 2018). 31 (published 12 October 2018): 10040–10050. arXiv: 1802.06006 .
  45. 1 2 "| ASVspoof". www.asvspoof.org. Retrieved 2022-07-01.
  46. ↑ resemble-ai/Resemblyzer, Resemble AI, 2022-06-30, retrieved 2022-07-01
  47. ↑ mendaxfz (2022-06-28), Synthetic-Voice-Detection , retrieved 2022-07-01
  48. ↑ HUA, Guang (2022-06-29), End-to-End Synthetic Speech Detection , retrieved 2022-07-01
  49. ↑ Kong, Jungil; Kim, Jaehyeon; Bae, Jaekyoung (2020-10-23). "HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis". arXiv: 2010.05646 [cs.SD].
  50. ↑ Kumar, Kundan; Kumar, Rithesh; de Boissiere, Thibault; Gestin, Lucas; Teoh, Wei Zhen; Sotelo, Jose; de Brebisson, Alexandre; Bengio, Yoshua; Courville, Aaron (2019-12-08). "MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesis". arXiv: 1910.06711 [eess.AS].
  51. ↑ Ng, Andrew (April 1, 2020). "Voice Cloning for the Masses". DeepLearning.AI. Archived from the original on December 28, 2024. Retrieved December 22, 2024.
  52. ↑ Chandraseta, Rionaldi (January 21, 2021). "Generate Your Favourite Characters' Voice Lines using Machine Learning". Towards Data Science. Archived from the original on January 21, 2021. Retrieved December 18, 2024.
  53. ↑ "Audio samples from "Semi-Supervised Training for Improving Data Efficiency in End-to-End Speech Synthesis"". 2018-08-30. Archived from the original on 2020-11-11. Retrieved 2022-06-05.
  54. ↑ Temitope, Yusuf (December 10, 2024). "15.ai Creator reveals journey from MIT Project to internet phenomenon". The Guardian. Archived from the original on December 28, 2024. Retrieved December 25, 2024.
  55. ↑ Kurosawa, Yuki (January 19, 2021). "ゲームキャラ音声読み上げソフト「15.ai」公開中。『Undertale』や『Portal』のキャラに好きなセリフを言ってもらえる" [Game Character Voice Reading Software "15.ai" Now Available. Get Characters from Undertale and Portal to Say Your Desired Lines]. AUTOMATON (in Japanese). Archived from the original on January 19, 2021. Retrieved December 18, 2024.
  56. ↑ "Navigating the Challenges and Opportunities of Synthetic Voices". OpenAI. March 9, 2024. Archived from the original on November 25, 2024. Retrieved December 18, 2024.
  57. ↑ Babbel.com; GmbH, Lesson Nine. "The 10 Most Spoken Languages In The World". Babbel Magazine. Retrieved 2022-06-30.
  58. ↑ Najafian, Maryam; Russell, Martin (September 2020). "Automatic accent identification as an analytical tool for accent robust automatic speech recognition". Speech Communication. 122: 44–55. doi:10.1016/j.specom.2020.05.003. S2CID   225778214.
  59. ↑ Liu, Xiao; Zhang, Fanjin; Hou, Zhenyu; Mian, Li; Wang, Zhaoyu; Zhang, Jing; Tang, Jie (2021). "Self-supervised Learning: Generative or Contrastive". IEEE Transactions on Knowledge and Data Engineering. 35 (1): 857–876. arXiv: 2006.08218 . doi:10.1109/TKDE.2021.3090866. ISSN   1558-2191. S2CID   219687051.
  60. ↑ Rashid, Md Mamunur; Lee, Suk-Hwan; Kwon, Ki-Ryong (2021). "Blockchain Technology for Combating Deepfake and Protect Video/Image Integrity". Journal of Korea Multimedia Society. 24 (8): 1044–1058. doi:10.9717/kmms.2021.24.8.1044. ISSN   1229-7771.
  61. ↑ Fraga-Lamas, Paula; Fernández-Caramés, Tiago M. (2019-10-20). "Fake News, Disinformation, and Deepfakes: Leveraging Distributed Ledger Technologies and Blockchain to Combat Digital Deception and Counterfeit Reality". IT Professional. 22 (2): 53–59. arXiv: 1904.05386 . doi:10.1109/MITP.2020.2977589.
  62. ↑ Ki Chan, Christopher Chun; Kumar, Vimal; Delaney, Steven; Gochoo, Munkhjargal (September 2020). "Combating Deepfakes: Multi-LSTM and Blockchain as Proof of Authenticity for Digital Media". 2020 IEEE / ITU International Conference on Artificial Intelligence for Good (AI4G). pp. 55–62. doi:10.1109/AI4G50087.2020.9311067. ISBN   978-1-7281-7031-2. S2CID   231618774.
  63. ↑ Mittal, Trisha; Bhattacharya, Uttaran; Chandra, Rohan; Bera, Aniket; Manocha, Dinesh (2020-10-12), "Emotions Don't Lie: An Audio-Visual Deepfake Detection Method using Affective Cues", Proceedings of the 28th ACM International Conference on Multimedia, New York, NY, USA: Association for Computing Machinery, pp. 2823–2832, doi:10.1145/3394171.3413570, ISBN   978-1-4503-7988-5, S2CID   220935571 , retrieved 2022-06-29
  64. ↑ Conti, Emanuele; Salvi, Davide; Borrelli, Clara; Hosler, Brian; Bestagini, Paolo; Antonacci, Fabio; Sarti, Augusto; Stamm, Matthew C.; Tubaro, Stefano (2022-05-23). "Deepfake Speech Detection Through Emotion Recognition: A Semantic Approach". ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Singapore, Singapore: IEEE. pp. 8962–8966. doi:10.1109/ICASSP43922.2022.9747186. hdl:11311/1220518. ISBN   978-1-6654-0540-9. S2CID   249436701.
  65. ↑ Hosler, Brian; Salvi, Davide; Murray, Anthony; Antonacci, Fabio; Bestagini, Paolo; Tubaro, Stefano; Stamm, Matthew C. (June 2021). "Do Deepfakes Feel Emotions? A Semantic Approach to Detecting Deepfakes Via Emotional Inconsistencies". 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). Nashville, TN, USA: IEEE. pp. 1013–1022. doi:10.1109/CVPRW53098.2021.00112. hdl:11311/1183572. ISBN   978-1-6654-4899-4. S2CID   235679849.
  66. ↑ Müller, Nicolas M.; Czempin, Pavel; Dieckmann, Franziska; Froghyar, Adam; Böttinger, Konstantin (2022-04-21). "Does Audio Deepfake Detection Generalize?". arXiv: 2203.16263 [cs.SD].
  67. ↑ Zhang, You; Jiang, Fei; Duan, Zhiyao (2021). "One-Class Learning Towards Synthetic Voice Spoofing Detection". IEEE Signal Processing Letters. 28: 937–941. arXiv: 2010.13995 . Bibcode:2021ISPL...28..937Z. doi:10.1109/LSP.2021.3076358. ISSN   1558-2361. S2CID   235077416.
  68. 1 2 "SAM.gov". sam.gov. Retrieved 2022-06-29.
  69. ↑ "The SemaFor Program". www.darpa.mil. Retrieved 2022-07-01.
  70. ↑ "The DARPA MediFor Program". govtribe.com. Retrieved 2022-06-29.
  71. ↑ "The MediFor Program". www.darpa.mil. Retrieved 2022-07-01.
  72. ↑ "DARPA Announces Research Teams Selected to Semantic Forensics Program". www.darpa.mil. Retrieved 2022-07-01.
  73. ↑ "PREMIER". sites.google.com. Retrieved 2022-07-01.
  74. ↑ "PREMIER - Project". sites.google.com. Retrieved 2022-06-29.
  75. ↑ Bird, Jordan J.; Lotfi, Ahmad (2023). "Real-time Detection of AI-Generated Speech for DeepFake Voice Conversion". arXiv: 2308.12734 [cs.SD].
  76. ↑ Yamagishi, Junichi; Wang, Xin; Todisco, Massimiliano; Sahidullah, Md; Patino, Jose; Nautsch, Andreas; Liu, Xuechen; Lee, Kong Aik; Kinnunen, Tomi; Evans, Nicholas; Delgado, Héctor (2021-09-01). "ASVspoof 2021: accelerating progress in spoofed and deepfake speech detection". arXiv: 2109.00537 [eess.AS].
  77. ↑ "Audio Deepfake Detection: ICASSP 2022". IEEE Signal Processing Society. 2021-12-17. Retrieved 2022-07-01.
  78. ↑ Yi, Jiangyan; Fu, Ruibo; Tao, Jianhua; Nie, Shuai; Ma, Haoxin; Wang, Chenglong; Wang, Tao; Tian, Zhengkun; Bai, Ye; Fan, Cunhang; Liang, Shan (2022-02-26). "ADD 2022: the First Audio Deep Synthesis Detection Challenge". arXiv: 2202.08433 [cs.SD].
  79. ↑ "Joint Workshop for the Blizzard Challenge and Voice Conversion Challenge 2020 - SynSIG". www.synsig.org. Archived from the original on 2022-07-02. Retrieved 2022-07-01.
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.