Video manipulation

Last updated
A presentation showing examples of Deepfakes. Deepfake.jpg
A presentation showing examples of Deepfakes.

Video manipulation is a type of media manipulation that targets digital video using video processing and video editing techniques. The applications of these methods range from educational videos [1] to videos aimed at (mass) manipulation and propaganda, a straightforward extension of the long-standing possibilities of photo manipulation. This form of computer-generated misinformation has contributed to fake news, and there have been instances when this technology was used during political campaigns. [2] Other uses are less sinister; entertainment purposes and harmless pranks provide users with movie-quality artistic possibilities.

Contents

History

The concept of manipulating video can be traced back as far as the 1950s when the 2-inch Quadruplex tape used in videotape recorders would be manually cut and spliced. After being coated with ferrofluid, the two ends of tape that were to be joined were painted with a mixture of iron filings and carbon tetrachloride, a toxic and carcinogenic compound to make the tracks in the tape visible when viewed through a microscope so that they could be aligned in a splicer designed for this task [3]

As the video cassette recorder developed in the 1960s, 1970s, 1980s, and 1990s, the ability to record over an existing magnetic tape became possible. This led to the concept of overlaying specific parts of film to give the illusion of one consistently recorded video, which is the first identifiable instance of video manipulation.

In 1985, Quantel released The Harry, the first all-digital video editing and effects compositing system. It recorded and applied effects to a maximum of 80 seconds of 8-bit uncompressed digital video. A few years later, in 1991, Adobe released its first version of Premiere for the Mac, a program that has since become an industry standard for editing and is now commonly used for video manipulation. In 1999, Apple released Final Cut Pro, which competed with Adobe Premiere and was used in the production of major films such as The Rules of Attraction and No Country for Old Men . [4]

Face detection became a major research subject in the early 2000s that has continued to be studied in the present. In 2017, an amateur coder named “DeepFakes” was altering pornography videos by digitally substituting the faces of celebrities for those in the original videos. The word deepfake has become a generic noun for the use of algorithms and facial-mapping technology to manipulate videos.

On the consumer side, popular video manipulation programs FaceApp and Faceswap, developed from similar technology, have become increasingly sophisticated.

The proof-of-principle software Face2Face was developed at the University of Erlangen-Nuremberg, the Max-Planck Institute for Informatics, and Stanford University. [5] Such advanced video manipulation must be ranked alongside and beyond previous examples of deepfakes.

Types of Video Manipulation

Computer applications are becoming more advanced in terms of being able to generate fake audio and video content that looks real. [6] A video published by researchers depicts how video and audio manipulation works using facial recognition. [6] Though video manipulation could be thought of as piecing together different video clips, the types of video manipulation extend further than that. For example, an actor can sit in front of a camera moving his face. The computer then generates the same facial movement in real time on an existing video of Barack Obama. When the actor shakes his head, Obama also shakes his head, and the same happens when the actor speaks. [6] Not only does this create fake content, but it masks the content as even more authentic than other types of fake news, as video and audio were once the most reliable types of media for many people.

One of the most dangerous parts of video manipulation is the concept of politics; campaign videos are being manipulated to pose a threat to other nations. [2] Dartmouth University computer science professor Hany Farid commented on video manipulation and its dangers. Farid said that actors could generate videos of Trump claiming to launch nuclear weapons. These fabricated videos could be shared on social media before the mistake can be fixed, possibly resulting in war. [2] Despite the presence of manipulated video and audio, research teams are working to combat the issue. Prof. Christian Theobalt, a member of a team working on the technology at the Max-Planck-Institute for informatics in Germany, states that researchers have created forensic methods to detect fakes. [6]

The Washington Post 's fact-checking team has identified six forms of video manipulation, classified into three categories: [7]

  1. Missing context
    • Misrepresentation: Placing original video footage into an incorrect context to misinform the audience
    • Isolation: Publishing a short segment from a video that presents a different narrative than the full video
  2. Deceptive editing
    • Omission: Removing major segments from a video to present a different story
    • Splicing: Combining segments from different videos to form a narrative not supported by any of the individual videos
  3. Malicious transformation
    • Doctoring: Directly modifying video frames
    • Fabrication: Using technology to construct bogus videos, such as deepfakes

Video Manipulation and Fake News

With fake news becoming increasingly prominent in popular culture and with rapid advancements in audio and video manipulation technology, the public is increasingly encountering fake news that is supported by deceptive videos. [2] In terms of types of fake news, the potential to be classified is ever-expanding but includes five main types — satire or parody, selective reporting, sloppy journalism, clickbait, and conspiracies. [2] Though the five main types of fake news are prominent globally, one of the most destructive types of fake news lies within all five types and is video and audio manipulation. Video and audio manipulation are defined as a new variant of media manipulation that targets digital video using a combination of traditional video processing and video editing techniques with auxiliary methods from artificial intelligence like face recognition. The results range from artistic videos produced for aesthetic effects to videos aimed at (mass) manipulation and propaganda, a straightforward extension of the long-standing possibilities of photo manipulation.

Digital Fakes

A digital fake refers to a digital video, photo, or audio file that has been altered or manipulated by digital application software. Deepfake videos fall within the category of digital fake media, but a video may be digitally altered without being considered a deepfake. The alterations can be done for entertainment purposes, or more nefarious purposes such as spreading disinformation. The information can be used to conduct malicious attacks, political gains, financial crimes, or fraud.

Video Manipulation Regulations & Policy

Due to the social and political impacts caused by Deepfake, many national states implement regulations in order to combat these effects of video manipulation. Technical regulations range from real-name verification requirements, labeling information, censorships, and banning synthetic images, audio, and video. [8]

China

China issued the "Provision on the Administration of Deep Synthesis Internet Information Service" on January 28, 2022. China's State Internet Information Office enforced this regulation as a way to control manipulated content on the Internet and increase technological stability within the Chinese Communist Party (CCP). There are 25 articles in total and each article section ultimately explains the terms and conditions of the regulation itself.

"Article 5: Encourage relevant industry organizations to strengthen industry self-discipline, establish and improve industry standards, industry guidelines, and self-regulatory management systems, supervise and guide deep synthesis service providers to formulate and improve service specifications, strengthen information content security management, provide services in accordance with the law, and accept social supervision." [9]

One of the policy articles that were mentioned in Emmie Hine and Luciano Floridi's text was Article 5, which goes over that while the government will look over the information being posted publicly, industry corporations are also responsible for keeping track of content that is published on their social platforms. This particular policy pushes companies in China to be more aware of what is shown online because if not, the companies themselves will be fined. [8]

United States

The United States issued the "DEEP FAKES Accountability Act" in 2019. However, Hine and Luciano stated this policy only prevents the act around "unauthorized digital recreations of people". This established bill ultimately puts a criminal penalties for related impersonations and pornography violations. [10] Because of this regulation, many well-known platforms like Facebook and Twitter began to removed unlawful information that violated the policy and even require labeling regulations. [8]

Video Manipulation Market Failures

Some of the main reasons why large national states like the United States and China are implementing video manipulation regulations/policies are because of market failures. [8] In particular, market failures that revolve around the topics of public goods and negative externalities. Public goods refer to products and services that can be used by everyone. On the other hand, negative externalities are negative actions that affect someone who did not fully consent or was not involved with the situation.

Public Goods

Synthetic information, including content taken by Deepfake, is shown and available to everyone in the public eye (characteristics of general public goods). Because of the easy and worldwide access, misinformation is spread rapidly quickly and sometimes out of control. In turn, lawmakers and governmental organizations create policies that control what is being put on the internet. [11] One example that explicitly demonstrates content being "over-consumed" within society is "The Voice of April" event. The Voice of April was a day when numerous videos of Shanghai citizens were being censored and banned during Covid-19. However, because these videos were published on the Internet, it continued to circulate throughout China and some people even made different versions when the original content got deleted. [12] Covered in MIT Technology Review, Zeyi Yang informed that videos being censored by the government were re-uploaded into copies and stored through outside social platforms. [13] Due to the challenges faced by the open-access content published online, nations continue to implement more regulations in order to maintain verified information.

Negative Externalities

Video manipulation essentially creates fake scenarios that never happened. As a result of this act, many people can be impacted by synthetic information despite not being directly part of the scene. Jack Langa states in a Law Journal article that "a bad actor could take advantage of region's instability by using a deepfake to inflame a local population, which could lead to civilian casualties..." [11] One example that showcases negative externalities from misleading/manipulated information is the January 6, 2021 attack on the US Capitol. This event led to approximately 140 police officers being assaulted (noted by the United States Department of Justice). [14] However, Judy Woodruff declared in a PBS News Hours that the whole incident was caused by a big lie about election fraud in 2020. Even though the situation was based on manipulated content, it ultimately led to numerous officers being harmed in the process.

Related Research Articles

Disinformation is false information deliberately spread to deceive people. Disinformation is an orchestrated adversarial activity in which actors employ strategic deceptions and media manipulation tactics to advance political, military, or commercial goals. Disinformation is implemented through attacks that "weaponize multiple rhetorical strategies and forms of knowing—including not only falsehoods but also truths, half-truths, and value judgements—to exploit and amplify culture wars and other identity-driven controversies."

<span class="mw-page-title-main">Media manipulation</span> Techniques in which partisans create an image that favours their interests

Media manipulation refers to orchestrated campaigns in which actors exploit the distinctive features of broadcasting mass communications or digital media platforms to mislead, misinform, or create a narrative that advance their interests and agendas.

<span class="mw-page-title-main">Photograph manipulation</span> Transformation or alteration of a photograph

Photograph manipulation involves the transformation or alteration of a photograph. Some photograph manipulations are considered to be skillful artwork, while others are considered to be unethical practices, especially when used to deceive. Motives for manipulating photographs include political propaganda, altering the appearance of a subject, entertainment and humor.

Fact-checking is the process of verifying the factual accuracy of questioned reporting and statements. Fact-checking can be conducted before or after the text or content is published or otherwise disseminated. Internal fact-checking is such checking done in-house by the publisher to prevent inaccurate content from being published; when the text is analyzed by a third party, the process is called external fact-checking.

<span class="mw-page-title-main">Misinformation</span> Incorrect or misleading information

Misinformation is incorrect or misleading information. Misinformation can exist without specific malicious intent; disinformation is distinct in that it is deliberately deceptive and propagated. Misinformation can include inaccurate, incomplete, misleading, or false information as well as selective or half-truths.

<span class="mw-page-title-main">Digital rhetoric</span> Forms of communication via digital mediums

Digital rhetoric can be generally defined as communication that exists in the digital sphere. As such, digital rhetoric can be expressed in many different forms, including text, images, videos, and software. Due to the increasingly mediated nature of our contemporary society, there are no longer clear distinctions between digital and non-digital environments. This has expanded the scope of digital rhetoric to account for the increased fluidity with which humans interact with technology.

<span class="mw-page-title-main">Human image synthesis</span> Computer generation of human images

Human image synthesis is technology that can be applied to make believable and even photorealistic renditions of human-likenesses, moving or still. It has effectively existed since the early 2000s. Many films using computer generated imagery have featured synthetic images of human-like characters digitally composited onto the real or other simulated film material. Towards the end of the 2010s deep learning artificial intelligence has been applied to synthesize images and video that look like humans, without need for human assistance, once the training phase has been completed, whereas the old school 7D-route required massive amounts of human work .

Hany Farid is an American university professor who specializes in the analysis of digital images and the detection of digitally manipulated images such as deepfakes. Farid served as Dean and Head of School for the UC Berkeley School of Information. In addition to teaching, writing, and conducting research, Farid acts as a consultant for non-profits, government agencies, and news organizations. He is the author of the book Photo Forensics (2016).

Content storage management (CSM) is a technique for the evolution of traditional media archive technology used by media companies and content owners to store and protect valuable file-based media assets. CSM solutions focus on active management of content and media assets regardless of format, type and source, interfaces between proprietary content source/destination devices and any format and type of commodity IT centric storage technology. These digital media files most often contain video but in rarer cases may be still pictures or sound. A CSM system may be directed manually but is more often directed by upper-level systems, which may include media asset management (MAM), automation, or traffic.

Social hacking describes the act of attempting to manipulate outcomes of social behaviour through orchestrated actions. The general function of social hacking is to gain access to restricted information or to a physical space without proper permission. Most often, social hacking attacks are achieved by impersonating an individual or group who is directly or indirectly known to the victims or by representing an individual or group in a position of authority. This is done through pre-meditated research and planning to gain victims’ confidence. Social hackers take great measures to present overtones of familiarity and trustworthiness to elicit confidential or personal information. Social hacking is most commonly associated as a component of “social engineering”.

<span class="mw-page-title-main">Fake news</span> False or misleading information presented as real

Fake news or information disorder is false or misleading information presented as news. Fake news often has the aim of damaging the reputation of a person or entity, or making money through advertising revenue. Although false news has always been spread throughout history, the term fake news was first used in the 1890s when sensational reports in newspapers were common. Nevertheless, the term does not have a fixed definition and has been applied broadly to any type of false information presented as news. It has also been used by high-profile people to apply to any news unfavorable to them. Further, disinformation involves spreading false information with harmful intent and is sometimes generated and propagated by hostile foreign actors, particularly during elections. In some definitions, fake news includes satirical articles misinterpreted as genuine, and articles that employ sensationalist or clickbait headlines that are not supported in the text. Because of this diversity of types of false news, researchers are beginning to favour information disorder as a more neutral and informative term.

Internet manipulation is the co-optation of online digital technologies, including algorithms, social bots, and automated scripts, for commercial, social, military, or political purposes. Internet and social media manipulation are the prime vehicles for spreading disinformation due to the importance of digital platforms for media consumption and everyday communication. When employed for political purposes, internet manipulation may be used to steer public opinion, polarise citizens, circulate conspiracy theories, and silence political dissidents. Internet manipulation can also be done for profit, for instance, to harm corporate or political adversaries and improve brand reputation. Internet manipulation is sometimes also used to describe the selective enforcement of Internet censorship or selective violations of net neutrality.

<span class="mw-page-title-main">Deepfake</span> Realistic artificially generated media

Deepfakes are images, videos, or audio which are edited or generated using artificial intelligence tools, and which may depict real or non-existent people. They are a type of synthetic media.

Digital cloning is an emerging technology, that involves deep-learning algorithms, which allows one to manipulate currently existing audio, photos, and videos that are hyper-realistic. One of the impacts of such technology is that hyper-realistic videos and photos makes it difficult for the human eye to distinguish what is real and what is fake. Furthermore, with various companies making such technologies available to the public, they can bring various benefits as well as potential legal and ethical concerns.

Synthetic media is a catch-all term for the artificial production, manipulation, and modification of data and media by automated means, especially through the use of artificial intelligence algorithms, such as for the purpose of misleading people or changing an original meaning. Synthetic media as a field has grown rapidly since the creation of generative adversarial networks, primarily through the rise of deepfakes as well as music synthesis, text generation, human image synthesis, speech synthesis, and more. Though experts use the term "synthetic media," individual methods such as deepfakes and text synthesis are sometimes not referred to as such by the media but instead by their respective terminology Significant attention arose towards the field of synthetic media starting in 2017 when Motherboard reported on the emergence of AI altered pornographic videos to insert the faces of famous actresses. Potential hazards of synthetic media include the spread of misinformation, further loss of trust in institutions such as media and government, the mass automation of creative and journalistic jobs and a retreat into AI-generated fantasy worlds. Synthetic media is an applied form of artificial imagination.

Deepfake pornography, or simply fake pornography, is a type of synthetic pornography that is created via altering already-existing photographs or video by applying deepfake technology to the images of the participants. The use of deepfake porn has sparked controversy because it involves the making and sharing of realistic videos featuring non-consenting individuals, typically female celebrities, and is sometimes used for revenge porn. Efforts are being made to combat these ethical concerns through legislation and technology-based solutions.

Disinformation attacks are strategic deception campaigns involving media manipulation and internet manipulation, to disseminate misleading information, aiming to confuse, paralyze, and polarize an audience. Disinformation can be considered an attack when it occurs as an adversarial narrative campaign that weaponizes multiple rhetorical strategies and forms of knowing—including not only falsehoods but also truths, half-truths, and value-laden judgements—to exploit and amplify identity-driven controversies. Disinformation attacks use media manipulation to target broadcast media like state-sponsored TV channels and radios. Due to the increasing use of internet manipulation on social media, they can be considered a cyber threat. Digital tools such as bots, algorithms, and AI technology, along with human agents including influencers, spread and amplify disinformation to micro-target populations on online platforms like Instagram, Twitter, Google, Facebook, and YouTube.

An audio deepfake is a product of artificial intelligence used to create convincing speech sentences that sound like specific people saying things they did not say. This technology was initially developed for various applications to improve human life. For example, it can be used to produce audiobooks, and also to help people who have lost their voices to get them back. Commercially, it has opened the door to several opportunities. This technology can also create more personalized digital assistants and natural-sounding text-to-speech as well as speech translation services.

Identity replacement technology is any technology that is used to cover up all or parts of a person's identity, either in real life or virtually. This can include face masks, face authentication technology, and deepfakes on the Internet that spread fake editing of videos and images. Face replacement and identity masking are used by either criminals or law-abiding citizens. Identity replacement tech, when operated on by criminals, leads to heists or robbery activities. Law-abiding citizens utilize identity replacement technology to prevent government or various entities from tracking private information such as locations, social connections, and daily behaviors.

Nolan Higdon is a critical media literacy scholar and media personality. He is also an author and university lecturer of history and media studies. Higdon has been a lecturer at University of California, Santa Cruz and California State University, East Bay. Higdon is considered an expert in critical media literacy, digital culture, higher education, journalism, fake news, and news media history. Higdon is frequently featured as an expert voice in documentaries and news outlets such as ABC, CBS, CNBC, NewsNation, NBC, New York Times, PBS, and the San Francisco Chronicle.

References

  1. Pea, Roy (2006). "Chapter 55: Video-as-Data and Digital Video Manipulation Techniques for Transforming Learning Sciences Research, Education, and Other Cultural Practices". In Weiss, J.; et al. (eds.). The International Handbook of Virtual Learning Environments. Springer. pp. 100–110.
  2. 1 2 3 4 5 "Fake news: The dangerous new technology that will make us question our basic idea of reality — Quartz". qz.com. 5 December 2017. Retrieved 2018-11-14.
  3. "Edit Suite: Once Upon a Time: The History of Videotape Editing - Videomaker". Videomaker. 1997-07-01. Archived from the original on 2021-04-22. Retrieved 2018-11-14.
  4. "Fun Facts and Dates in Digital Editing 'Firsts' - The Beat: A Blog by PremiumBeat". The Beat: A Blog by PremiumBeat. 2011-12-10. Archived from the original on 2018-10-04. Retrieved 2018-11-14.
  5. "Face2Face: Real-time Face Capture and Reenactment of RGB Videos". gvv.mpi-inf.mpg.de. Archived from the original on 2019-12-31. Retrieved 2019-12-18.
  6. 1 2 3 4 Schellmann, Hilke. "Deepfake Videos Are Getting Real and That's a Problem". WSJ. Retrieved 2018-11-14.
  7. "Seeing Isn't Believing: The Fact Checker's guide to manipulated video". The Washington Post . June 25, 2019. Archived from the original on December 25, 2020. Retrieved January 19, 2021.
  8. 1 2 3 4 Hine, Emmie; Floridi, Luciano (2022-07-20). "New deepfake regulations in China are a tool for social stability, but at what cost?". Nature Machine Intelligence. 4 (7): 608–610. doi: 10.1038/s42256-022-00513-4 . ISSN   2522-5839.
  9. "国家互联网信息办公室关于《互联网信息服务深度合成管理规定(征求意见稿)》公开征求意见的通知-中共中央网络安全和信息化委员会办公室". www.cac.gov.cn. Archived from the original on 2023-08-29. Retrieved 2023-08-29.
  10. MST Students (2020-07-13). "H.R. 4286 (116th Congress) - Virtual Apprenticeship Tax Credit Act of 2019". The Contemporary Tax Journal. 9 (2). doi: 10.31979/2381-3679.2020.090214 . ISSN   2381-3679.
  11. 1 2 Kigwiru, Vellah Kedogo (2022). "Deepfake Technology and Elections in Kenya: Can Legislation Combat the Harm Posed by Deepfakes?". SSRN Electronic Journal. doi:10.2139/ssrn.4229272. ISSN   1556-5068.
  12. 四月之声The Voice of April Eng translation | China 2022 Shanghai Lockdown Record #shanghai #lockdown, 23 April 2022, archived from the original on 2023-08-29, retrieved 2023-08-29
  13. "WeChat wants people to use its video platform. So they did, for digital protests". MIT Technology Review. Archived from the original on 2022-04-30. Retrieved 2023-08-29.
  14. "24 Months Since the January 6 Attack on the Capitol". www.justice.gov. 2022-02-15. Archived from the original on 2023-01-04. Retrieved 2023-08-29.