Immersive journalism

Immersive Journalism is a form of journalism production that allows first person experience of the events or situations described in news reports and documentary film. Using 3D gaming and immersive technologies that create a sense of "being there" and offer the opportunity to personally engage with a story, immersive journalism puts an audience member directly into the event. By accessing a virtual version of the location where the story is occurring as a witness/participant, or by experiencing the perspective of a character depicted in the news story, the audience could be afforded unprecedented access to the sights and sounds, and even the feelings and emotions, which accompany the news. ^[1]

Historical precedents

Well-crafted journalism always aims to elicit a connection between the audience and the news story. Creating that connection via different kinds of ‘immersion’ has long been considered ideal. Describing her reporting during World War II, reporter Martha Gellhorn called it ‘The view from the ground’. Writer George Plimpton actually joined the Detroit Lions American football team in order to give his readers the most intimate sense of playing in this team. Television news correspondent Walter Cronkite made a series of documentaries recreating historical events where he would offer a brief introduction before an announcer would give the date and the event, proclaiming, ‘You Are There!’ More recently, attempts to combine audio, video and photographs on the Internet have created what some journalists call ‘immersive storytelling.’ As technology editor at MSNBC, Jonathan Dube, said that he believes this can bring the reader or viewer ‘closer to the truth’.

Concepts

Immersive journalism constructs allow the audience to enter a virtually recreated scenario representing the story. The pieces are constructed as CGI (computer graphic imagery) virtual environments which can be inserted into persistent online virtual worlds, such as Second Life or a web-based Unity game engine environment and viewed conventionally on a monitor or in fully immersive systems such as a CAVE or head-tracked and head-mounted display systems (HMD). Head and body tracking can be achieved by any number of commercial systems, including inexpensive consumer-end products, which makes this a viable and growing field. (For more on how virtual reality can create a sense of presence and connection to a virtual body, see ^[2] and ^[3] )

Video and audio captured from the physical world are used to reinforce the concept that participants are experiencing a nonfiction story. For example, video triggers at key points in the virtual landscape remind a participant that the computer-generated environment is grounded in a real news story.

Scripted events

Scripted events that create a first person interaction with the reportage can also help create a feeling of “being there.” By using programming to move participants from one virtual location to the next, thus moving the embodied presence along the timeline, the experience employs “embodied edits.” ^[4] These experiences blur the line between recreations and cinema verité because audio and video captured live from the physical world (cinema verité material) is embedded throughout the computer-generated environment and encountered in scripted/programmed variations. The computer-generated environment also can act as a replica of an actual space from the physical world, a recreation but with verité intent.

A digital avatar

Participants in immersive constructs are typically represented in the form of a digital avatar, which can move freely through the virtually recreated scenario that represents the news story. This animated 3D digital representation allows them to see the world from the first-person perspective of that avatar. In an immersive system such as a Cave, individuals may see their own real body, with their avatar established through shadows and reflections in the environment (though other online people could also see the avatar directly.) In a system such as an HMD the participant will see their avatar as if it were actually their own body and they will be offered a first-person point of view. Ideally, depending on the extent of body tracking, the movements of the virtual body will match those of the movements on the person’s real body. The news story can be constructed to control how the participant may enter the story: as oneself, a visitor gaining first-hand access to a virtual version of the location where the story is occurring, or through the perspective of a character depicted in the story.

Example

For example, Gone Gitmo, a virtual recreation of Guantanamo Bay prison, uses real audio and photographs from the prison to create an immersive journalism experience. ^[5] Similarly, a piece on the complicated carbon markets attempts to help an audience better understand cap and trade by literally following the money. ^[6]

Comparisons and contrasts

Immersive nonfiction and immersive journalism might be compared to news or documentary games because the pieces are typically set in what has been previously solely the terrain of gaming platforms and relies on computer generated graphics. However, there are a number of distinctions, particularly to news games, with the most important being that games work best as systems. Games are better at reproducing the conditions under which events unfold rather than outlining the details of the events themselves. That means linear narrative structures or presentation of multiple specific facts, which can be key to nonfiction stories, do not work as well with a gaming setup. Often, players advance through the game by passing “levels” that do not necessarily relate to the inherently unchangeable nature of a nonfiction narrative (no matter from whose perspective it has been constructed).

Example

One example is Cutthroat Capitalism (2009), a news game appended to Wired Magazine’s story on Somali pirates. ^[7] Since the audience is asked to become a Somali pirate, a deeper understanding of the economics of the highjack-and-ransom system is achieved through the play. However, it does not attempt to delineate any individual case in particular, and what happens to the player is based on his/her choices rather than reflecting the facts connected to one or more physical world events that have already transpired. Also, in contrast to immersive nonfiction, there is no virtual embodiment.

Documentary games

The difference between immersive nonfiction and documentary games, however, is more difficult to tease out, especially given that there has yet to be a body of research to help determine what constitutes news games versus documentary games. In Tracy Fullerton’s piece on documentary games, she offers a number of examples of games that recreate the events of Pearl Harbor, allow a player to reenact John Kerry’s Vietnam swift boat tour of duty, participate in a recreation of real Iraqi war scenario in which Saddam Hussein’s sons were killed, or “play” a 9/11 victim. ^[8] As Keith Halper, CEO of KUMA Reality Games, notes: “Games allow us to be who we are not, to do what we cannot, to be in places and times we cannot go… Through entertainment and action, we can educate in novel and powerful ways.” ^[9] However, each of these examples has shifting narratives—the player does not necessarily kill Kennedy or follow the exact documented path of any particular victim of 9/11. This includes the swiftboat exercise, which offers a player the ability to choose variations on the story.

Embodied experience

Perhaps key to clarifying the difference would be that immersive journalism uses an embodied experience in an unchangeable narrative that allows queries to the environment without changing any individual’s story trajectory. This reflects more closely a traditional journalistic or documentary practice. Consider the IPSRESS project which uses an HMD in which individuals were put in a virtual body of a detainee in a stress position. ^[10]

The individual in that experience heard an audio of a real transcript of an interrogation of Mohammed al-Qahtani read by actors as if it was coming from another room. This was a recreation of a real event and the experience proved to be journalistically accurate—after this immersive piece was complete, the British government released a video associated with the court trial over the death of civilian Baha Mousa. The released video helped establish the veracity of this particular immersive journalism construct, which had relied on Freedom of Information Act reports and International Red Cross descriptions to inform the design.

This is one example where, by being “embodied” in a location, an audience might understand and have an empathetic response to a nonfiction story that might otherwise be more difficult to convey. However, some may consider this an entry point for criticism, claiming that immersive journalism undermines objectivity because the audience can be put through an experience with a controlled viewpoint. Certainly in the digital age, visual and audio material can be rapidly manipulated to create a semblance of truth which has no relationship to facts. While immersive journalism is a digital platform and therefore vulnerable to similar manipulation, if appropriate news and nonfiction best practices are applied, the experience can be built with journalistic and editorial integrity. By retaining the same factual and ethical approaches as employed in traditional journalism and non-fiction storytelling, which includes explicit description of sources, immersive journalism can successfully become another platform for nonfiction storytelling.

Short characteristics

In general, immersive journalism is likely to include the following:

1. The audience should participate spatially in the story.
2. There is a place illusion which relates to where the news or non-fiction has occurred or is occurring.
3. The images, audio and environment must contextualize the story.
4. Audio and video from the physical world used in the piece must have been captured at a real event or place that is applicable to the immersive news story.
5. The story should reflect good journalistic/documentary practices for gathering and research material depicted in the immersive construct.

Journalism awards

In 2017, the Online News Association (ONA) introduced a category to the Online Journalism Awards to honor Excellence in Immersive Storytelling. The award was originally launched as part of the Journalism 360 initiative — a partnership between Knight Foundation, Google News Initiative and ONA — and features a $5,000 cash prize. ^{[11] [12]}

2021 winners and finalists

• Inside Xinjiang’s Prison State, The New Yorker (Winner)
• Covid-19 Particle and Airflow Simulations, The New York Times

2020 winners and finalists

• See How the World’s Most Polluted Air Compares With Your City’s, The New York Times (Winner)
• 1619: Searching for answers, USA TODAY
• 50th Anniversary: “Apollo 11: America’s Journey To The Moon," Florida Today, USA TODAY
• Who We Are, USC / JOVRNALISM

2019 winners and finalists

• Remembering Emmett Till: The Legacy of a Lynching, The New York Times (Winner)
• 12 Seconds of Gunfire, The Washington Post
• SwampScapes, University of Miami

2018 winners and finalists

• Yemen’s Skies of Terror, Al Jazeera Contrast (Winner)
• Augmented Reality: David Bowie in Three Dimensions, The New York Times

2017 winners and finalists

• After Solitary, Emblematic Group, FRONTLINE (Winner)
• Cash Cow, Al Jazeera English
• Fight for Falluja, The New York Times

References

1. Nonny de la Peña, Peggy Weil, Joan Llobera, Elias Giannopoulos, Ausiàs Pomés, Bernhard Spanlang, Doron Friedman, Maria V. Sanchez-Vives, and Mel Slater. “Immersive Journalism: Immersive Virtual Reality for the First Person Experience of News,” Presence: Teleoperators and Virtual Environments 19, no. 4 (2010): 291 – 301
2. Maria V Sanchez-Vives, Mel Slater (2005) From presence to consciousness through virtual reality. Nat Rev Neurosci 6: 4. 332-339
3. Mel Slater (2009) Place illusion and plausibility can lead to realistic behaviour in immersive virtual environments. Philos Trans R Soc Lond B Biol Sci 364: 1535. 3549-3557
4. Peggy Weil and Nonny de la Peña, “Avatar Mediated Cinema” ACM International Conference Proceeding Series; 352 (2008): 209-212
5. gonegitmo.blogspot.com
6. "Cap & Trade – an Immersive Journalism Experience | Immersive Journalism". Archived from the original on August 26, 2011. Retrieved October 24, 2011.
7. Shachtman, Noah. "Somali Piracy: The Game". Wired. ISSN   1059-1028 . Retrieved 2022-09-01.
8. Tracy Fullerton, “Documentary Games: Putting the Player in the Path of History,” in Playing the Past: Nostalgia in Video Games and Electronic Literature, ed. Zach Whalen and Laurie Taylor (Vanderbilt University Press, September 2008).
9. Keith Halper, "Kuma Games Press Release," http://ww.kumagames.com/press%5B%5D.
10. "Towards Immersive Journalism: The IPSRESS Experience". YouTube. 2009-07-15. Retrieved 2013-08-02.
11. "Excellence in Immersive Storytelling". Online Journalism Awards.
12. "Excellence in Immersive Storytelling a new category in ONA's 2017 Online Journalism Awards". Immersive Shooter. 2017-05-17. Retrieved 2022-01-07.