Cinematic virtual reality

Last updated

Cinematic virtual reality (Cine-VR) is an immersive experience where the audience can look around in 360 degrees while hearing spatialized audio specifically designed to reinforce the belief that the audience is actually in the virtual environment rather than watching it on a two-dimensional screen. [1] Cine-VR is different from traditional Virtual Reality which uses computer generated worlds and characters more akin to interactive gaming engines, while cine-VR uses live images captured thorough a camera which makes it more like film. [2]

Contents

When storytellers began working in cine-VR, they applied many of the same cinematic narrative rules, but the technology demonstrated that VR can offer different possibilities that go beyond "traditional" cinema which will require new techniques and practices. [3] Harrison Weber, journalist of Venturebeat , described cine-VR like this: "It's a lot like film, only it puts the audience inside your story. With it, you can create entire worlds for your audience but none of the original rules of cinema apply. How do you create your art when all of your tools have changed?" [4]

The Human Interface Technology (HIT) Lab at the University of Canterbury differentiates cine-VR from other content created with 360-degree cameras based on the content, likening the prefix "cinematic" to that of "narrative". The HIT Lab requires cine-VR to be "narrative-based, instead of purely for novelty, entertainment, exploration, etc."; the cine-VR experience can be a drama, a documentary, or a hybrid as long as the story contains a beginning, a middle, and an end. [5] According to Ohio University's Game Research and Immersive Design (GRID) Lab, a cine-VR project differentiates itself from 360-degree video by using cinematic production techniques such as lighting design, sound design, scenic design, and blocking techniques (the latter two in the case of dramatic work). [6]

The concepts of "immersion" and "presence" are central to cine-VR. [7]  The term presence is defined as "a sense of being there" [8] and described as "a feeling of actually being on location in a story rather than experiencing it from the outside". [9] Scholar Christian Roth differentiates immersion from presence by defining immersion as an objective criterion which depends on hardware and software, while presence is defined as the more subjective, psychological sense of being in the environment, and mainly influenced by the content of that world (e.g. story, characters, and location). [10] Immersion could be seen as a quality of the medium, in this case a cine-VR experience, while presence is a characteristic of the user experience; hence, higher immersion may lead to or result in deeper presence. [11] Immersion has objective components that can be advanced by technical considerations like image quality and sound quality, while presence is affected by individual users' subjective variations but is aided by the technical aspects that foster immersion. [12]

Cine-VR provides a more photorealistic user experience than traditional virtual reality, but current technology does not allow the audience to move around in video. In some ways, cine-VR is a trade-off, as fully computer-generated VR looks less realistic than cine-VR but is more interactive. [12] The ability to look around inside of a virtual reality space is known as three degrees of freedom (3 DOF), while being able to move around inside a virtual environment is known as six degrees of freedom (6 DOF). 6 DOF should ideally enhance the user's sense of presence. [7] The 3 DOF of cine-VR are defined as yaw (rotating your head left or right), pitch (tilting your head up or down) and roll (tilting your head on its axis upside down or right-side up). [3] Since experiencing a story using 3 DOF is quite different from watching a traditional film, television show or stage play, storytellers have recognized a need to develop a new creative language for cine-VR. [13] The key element that differentiates cinema and cinematic VR is the new role of the audience. Technologically speaking, this requires the storyteller to embrace the concept of immersion. [3]

With the ability to use 3 DOF, the cine-VR audience can freely choose the viewing direction when they experience the story. Therefore, traditional filmmaking techniques for guiding the viewers' attention cannot be used: techniques such as panning the camera or cutting to a close-up shot are no longer available to the filmmaker; instead it is the viewer who decides where to look. [14] Subsequently, in cine-VR, the storyteller has to rely more on lighting, sound design, and how the characters and sets are arranged to best tell the story. [13] Famous filmmakers have been attempting to do this at least since the mid-2010s when Kathryn Bigelow directed the cine-VR piece The Protectors (2016), Doug Liman directed Invisible (2017) and Alejandro Gonzalez Iñárritu, debuted Carne y Arena / Flesh & Sand at the Cannes Film Festival in 2017. [2]

Equipment

Cameras

A variety of cameras can be used to create cine-VR images, including traditional cinema cameras in conjunction with a panorama tripod head. Most commonly 360° cameras are used, allowing the storyteller to capture the entire 360° space at one time. 360° cameras use multiple lens combinations to capture all portions of the 360° image simultaneously. Those disparate images are then combined into one 360° panoramic image using a process called "stitching". If a single lens faces in one direction, the image is referred to as monoscopic. If two lenses are used for a single direction, the image is referred to as stereoscopic. [15]

Stereoscopic imaging creates a 3D effect. This technique leverages the parallax difference between multiple lenses to achieve the illusion of depth. Stereoscopic content is generally contained within one media file with the images stacked above and below each other or in a side-by side-fashion. [7]

Ambisonic microphones

Ambisonics is a method for recording, mixing and playing 360-degree audio. It was invented in the 1970s but was never commercially adopted until the development of the virtual reality industry (including cine-VR) which requires 360° audio to match with the 360° images. [16]  Audio designer Simon Goodwin describes ambisonics, as "a generalized way of representing a soundfield—the distribution of sounds from all directions around a listener." [17] Ambisonic audio recreates the soundfield spherically and is uniquely suited for VR applications because it provides motion-tracked variations of audio signals and enables sounds to be positioned anywhere around a user—up/down, front/back, left/right. Properly implemented, Ambisonic audio allows users to move their heads and bodies around in the soundfield just as they might turn to look for the source of a sound in real life. As users glance around, the headset uses motion tracking to alter sound direction and quality. [7]

Ambisonic techniques are needed which guide the attention of the cine-VR spectator toward important visual information in the scene. Attention guidance using ambisonics improves the general viewing experience, since viewers will be less afraid to miss something when watching a cine-VR story. [14]  A level of aural realism is achievable by combining ambisonic recordings of environments with dialogue captured through a traditional microphone (usually a lavalier) along with added sound effects generated through foley work.  Ambisonic audio, in combination with traditional microphones and sound effects, plays an important in creating a sense of immersion for the user experience. [18]

Head mounted displays

Cine-VR should ideally be played on a head mounted display (or HMD) with headphones. While in such a headset, most surrounding distractions are visually blocked out. HMDs have built-in hardware sensors called gyroscopes and accelerometers to move the images in concert with the audience moving their head. Gyroscopes track how much something is tilting and help to smooth the graphical playback to prevent videos from shaking. Accelerometers measure the actual movement in space. The combination of these two can precisely track the device's position and orientation. Along with optical or infrared tracking, gyroscopes and accelerometers are integral parts of the VR headsets' tracking capabilities and increase immersion. [19] In contrast with the other mediums, "immersion" is still the main experiential value that the experts remark in cine-VR.  This is achieved primarily through head mounted displays used to view its content. [3]

According to John Bowditch, director of Ohio University's GRID Lab, "The VR industry is trending towards wireless HMDs due to consumer demand and ease of use; however, the overall quality does not currently match the performance of headsets wired to a PC. Some wireless headsets support wired-connections to computers (usually with a USB cable) for more processor extensive applications. Wireless headsets are generally less expensive because they do not require a PC to operate. Most wireless headsets can be initially configured with a smartphone and then run independently by downloading or streaming content through a Wi-Fi connection. Wireless headsets tend to be more comfortable, easier to transport, and work well for both seated and standing experiences. Most cine-VR playback is either seated or standing and will not require walking around. Swivel office chairs that rotate 360° with ease are our preferred furniture. However, any furniture that doesn't restrict your audience's movements is usable." [20]

See also

Related Research Articles

<span class="mw-page-title-main">Virtual reality</span> Computer-simulated experience

Virtual reality (VR) is a simulated experience that employs pose tracking and 3D near-eye displays to give the user an immersive feel of a virtual world. Applications of virtual reality include entertainment, education and business. VR is one of the key technologies in the reality-virtuality continuum. As such, it is different from other digital visualization solutions, such as augmented virtuality and augmented reality.

<span class="mw-page-title-main">Augmented reality</span> View of the real world with computer-generated supplementary features

Augmented reality (AR) is an interactive experience that combines the real world and computer-generated content. The content can span multiple sensory modalities, including visual, auditory, haptic, somatosensory and olfactory. AR can be defined as a system that incorporates three basic features: a combination of real and virtual worlds, real-time interaction, and accurate 3D registration of virtual and real objects. The overlaid sensory information can be constructive, or destructive. As such, it is one of the key technologies in the reality-virtuality continuum.

<span class="mw-page-title-main">Ambisonics</span> Full-sphere surround sound format

Ambisonics is a full-sphere surround sound format: in addition to the horizontal plane, it covers sound sources above and below the listener.

<span class="mw-page-title-main">Head-mounted display</span> Type of display device

A head-mounted display (HMD) is a display device, worn on the head or as part of a helmet, that has a small display optic in front of one or each eye. An HMD has many uses including gaming, aviation, engineering, and medicine. Virtual reality headsets are HMDs combined with IMUs. There is also an optical head-mounted display (OHMD), which is a wearable display that can reflect projected images and allows a user to see through it.

<span class="mw-page-title-main">360-degree video</span> Visual arts technique

360-degree videos, also known as surround video, or immersive videos or spherical videos, are video recordings where a view in every direction is recorded at the same time, shot using an omnidirectional camera or a collection of cameras. The term 360x180 can be used to indicate 360° of azimuth and 180° from nadir to zenith. During playback on normal flat display the viewer has control of the viewing direction like a panorama. It can also be played on a display or projectors arranged in a sphere or some part of a sphere.

<span class="mw-page-title-main">Immersion (virtual reality)</span> Perception of being physically present in a non-physical world

Immersion into virtual reality (VR) is a perception of being physically present in a non-physical world. The perception is created by surrounding the user of the VR system in images, sound or other stimuli that provide an engrossing total environment.

In computing, 3D interaction is a form of human-machine interaction where users are able to move and perform interaction in 3D space. Both human and machine process information where the physical position of elements in the 3D space is relevant.

<span class="mw-page-title-main">Oculus Rift</span> Virtual reality headsets by Oculus VR

Oculus Rift is a discontinued line of virtual reality headsets developed and manufactured by Oculus VR, a virtual reality company founded by Palmer Luckey that is widely credited with reviving the virtual reality industry. It was the first virtual reality headset to provide a realistic experience at an accessible price, utilizing novel technology to increase quality and reduce cost by orders of magnitude compared to earlier systems. The first headset in the line was the Oculus Rift DK1, released on March 28, 2013. The last was the Oculus Rift S, discontinued in April 2021.

<span class="mw-page-title-main">Google Cardboard</span> Discontinued virtual reality platform

Google Cardboard is a discontinued virtual reality (VR) platform developed by Google. Named for its fold-out cardboard viewer into which a smartphone is inserted, the platform was intended as a low-cost system to encourage interest and development in VR applications. Users can either build their own viewer from simple, low-cost components using specifications published by Google, or purchase a pre-manufactured one. To use the platform, users run Cardboard-compatible mobile apps on their phone, place it into the back of the viewer, and view content through the lenses.

<span class="mw-page-title-main">Virtual reality headset</span> Head-mounted device that provides virtual reality for the wearer

A virtual reality headset is a head-mounted display that provides a virtual reality environment for the wearer. VR headsets are widely used with VR video games but they are also used in other applications, including simulators and trainers. VR headsets typically include a stereoscopic display, stereo sound, and sensors like accelerometers and gyroscopes for tracking the pose of the user's head to match the orientation of the virtual camera with the user's eye positions in the real world.

<span class="mw-page-title-main">Virtual reality sex</span> VR technology

Virtual reality sex is a technology that allows the user to receive tactile sensations from remote participants, or fictional characters through the use of computer-controlled sex toys. Usually the user also wears a virtual reality headset so they can see and interact with the partner. The very first VR Porn experiences were filmed by an award-winning studio called VR Bangers. Two entrepreneurs decided to leave their jobs and create an adult studio that films virtual reality sex. They have opened their site vrbangers.com in 2014 and released 21 films that became the very first VR Porn experience in the world. Since then, the studio has been pushing VR technology to another level with more camera and sound innovations for virtual reality.

<span class="mw-page-title-main">Pose tracking</span>

In virtual reality (VR) and augmented reality (AR), a pose tracking system detects the precise pose of head-mounted displays, controllers, other objects or body parts within Euclidean space. Pose tracking is often referred to as 6DOF tracking, for the six degrees of freedom in which the pose is often tracked.

Virtual reality (VR) is a computer application which allows users to experience immersive, three dimensional visual and audio simulations. According to Pinho (2004), virtual reality is characterized by immersion in the 3D world, interaction with virtual objects, and involvement in exploring the virtual environment. The feasibility of the virtual reality in education has been debated due to several obstacles such as affordability of VR software and hardware. The psychological effects of virtual reality are also a negative consideration. However, recent technological progress has made VR more viable and promise new learning models and styles for students. These facets of virtual reality have found applications within the primary education sphere in enhancing student learning, increasing engagement, and creating new opportunities for addressing learning preferences.

<span class="mw-page-title-main">Virtual reality game</span> Video game played in virtual reality

A virtual reality game or VR games is a video game played on virtual reality (VR) hardware. Most VR games are based on player immersion, typically through head-mounted display unit or headset with stereoscopic displays and one or more controllers.

Volumetric capture or volumetric video is a technique that captures a three-dimensional space, such as a location or performance. This type of volumography acquires data that can be viewed on flat screens as well as using 3D displays and VR goggles. Consumer-facing formats are numerous and the required motion capture techniques lean on computer graphics, photogrammetry, and other computation-based methods. The viewer generally experiences the result in a real-time engine and has direct input in exploring the generated volume.

<span class="mw-page-title-main">Virtual reality applications</span> Overview of the various applications that make use of virtual reality

Virtual reality applications are applications that make use of virtual reality (VR), an immersive sensory experience that digitally simulates a virtual environment. Applications have been developed in a variety of domains, such as education, architectural and urban design, digital marketing and activism, engineering and robotics, entertainment, virtual communities, fine arts, healthcare and clinical therapies, heritage and archaeology, occupational safety, social science and psychology.

<span class="mw-page-title-main">Eric R. Williams</span> American screenwriter

Eric R. Williams is an American screenwriter, professor, cinematic virtual reality director, and new media storyteller. He is known for developing alternative narrative and documentary techniques that take advantage of digital technologies.

Immersive learning is a learning method which students being immersed into a virtual dialogue, the feeling of presence is used as an evidence of getting immersed. The virtual dialogue can be created by two ways, the usage of virtual technics, and the narrative like reading a book. The motivations of using virtual reality (VR) for teaching contain: learning efficiency, time problems, physical inaccessibility, limits due to a dangerous situation and ethical problems.

<span class="mw-page-title-main">PlayStation VR2</span> Virtual reality headset developed by Sony Interactive Entertainment

The PlayStation VR2 is a virtual reality headset for the PlayStation 5 home video game console developed by Sony Interactive Entertainment released on February 22, 2023.

References

  1. ↑ Mateer, John (2 January 2017). "Directing for Cinematic Virtual Reality: how the traditional film director's craft applies to immersive environments and notions of presence" (PDF). Journal of Media Practice. 18 (1): 14–25. doi:10.1080/14682753.2017.1305838. S2CID   149242071.
  2. 1 2 Ross, Miriam; Munt, Alex (1 June 2018). "Cinematic virtual reality: Towards the spatialized screenplay". Journal of Screenwriting. 9 (2): 191–209. doi:10.1386/josc.9.2.191_1. hdl: 10453/127306 . S2CID   191658470.
  3. 1 2 3 4 Pérez, Júlia Martínez (2016). Cinematic Virtual Reality: A Definition and Classification Proposal (PDF) (Thesis). S2CID   54916725.
  4. ↑ Harrison Weber. 2016. "How filmmakers are inventing the language of VR." https://venturebeat.com/2016/05/04/how-filmmakersare-inventing-the-language-of-vr/  Retrieved 2 January 2022.
  5. ↑ Tong, Lingwei; Lindeman, Robert W.; Regenbrecht, Holger (18 May 2021). "Viewer's Role and Viewer Interaction in Cinematic Virtual Reality". Computers. 10 (5): 66. doi: 10.3390/computers10050066 .
  6. ↑ Williams, Eric R.; Love, Carrie; Love, Matt; Durado, Adonis (2021). "Cine-VR: A new medium". Virtual Reality Cinema. pp. 1–19. doi:10.4324/9781003028284-1. ISBN   978-1-00-302828-4. S2CID   234089129.
  7. 1 2 3 4 Love, Matt; Linscott, Charles P. ("Chip") (2021). "Cine-VR Production Overview". The Power of Virtual Reality Cinema for Healthcare Training. pp. 11–20. doi:10.4324/9781003168683-2. ISBN   978-1-00-316868-3. S2CID   238655342.
  8. ↑ Riva, Giuseppe; Mantovani, Fabrizia; Capideville, Claret Samantha; Preziosa, Alessandra; Morganti, Francesca; Villani, Daniela; Gaggioli, Andrea; Botella, Cristina; Alcañiz, Mariano (1 February 2007). "Affective Interactions Using Virtual Reality: The Link between Presence and Emotions". CyberPsychology & Behavior. 10 (1): 45–56. doi:10.1089/cpb.2006.9993. PMID   17305448.
  9. ↑ Henrikson, Rorik; Araujo, Bruno; Chevalier, Fanny; Singh, Karan; Balakrishnan, Ravin (2016). "Multi-Device Storyboards for Cinematic Narratives in VR". Proceedings of the 29th Annual Symposium on User Interface Software and Technology (PDF). pp. 787–796. doi:10.1145/2984511.2984539. ISBN   978-1-4503-4189-9. S2CID   16214242.
  10. ↑ Roth, P. C. H. (2016). Experiencing Interactive Storytelling (Thesis). hdl:1871/53840.
  11. ↑ Vosmeer, Mirjam; Schouten, Ben (14 June 2017). "Project Orpheus A Research Study into 360° Cinematic VR". Proceedings of the 2017 ACM International Conference on Interactive Experiences for TV and Online Video. pp. 85–90. doi:10.1145/3077548.3077559. ISBN   978-1-4503-4529-3. S2CID   24856146.
  12. 1 2 "Immersion, Presence, and Reality Trade-Offs". The VR Book. 2015. pp. 45–52. doi:10.1145/2792790.2792796. ISBN   978-1-970001-12-9.
  13. 1 2 Kobylinski, Paweł; Pochwatko, Grzegorz (2020). "Detection of Strong and Weak Moments in Cinematic Virtual Reality Narration with the Use of 3D Eye Tracking" (PDF). ACHI 2020, The Thirteenth International Conference on Advances in Computer-Human Interactions. pp. 280–284. ISBN   978-1-61208-761-0.
  14. 1 2 Rothe, Sylvia; Buschek, Daniel; Hußmann, Heinrich (19 March 2019). "Guidance in Cinematic Virtual Reality-Taxonomy, Research Status and Challenges". Multimodal Technologies and Interaction. 3 (1): 19. doi: 10.3390/mti3010019 .
  15. ↑ Williams, Eric R.; Love, Carrie; Love, Matt; Durado, Adonis (2021). "Shooting and editing in cine-VR". Virtual Reality Cinema. pp. 77–93. doi:10.4324/9781003028284-5. ISBN   978-1-00-302828-4. S2CID   234090868.
  16. ↑ "Ambisonics Explained: A Guide for Sound Engineers: Waves." waves.com, October 10, 2017. https://www.waves.com/ambisonics-explained-guide-for-sound-engineers .
  17. ↑ Goodwin, Simon N. (2019). Beep to Boom: The Development of Advanced Runtime Sound Systems for Games and Extended Reality. Routledge. ISBN   978-1-351-00552-4.[ page needed ]
  18. ↑ Linscott, Charles "Chip" (2021). "Audio Considerations for Cine-VR". The Power of Virtual Reality Cinema for Healthcare Training. pp. 121–130. doi:10.4324/9781003168683-12. ISBN   978-1-00-316868-3. S2CID   238675063.
  19. ↑ Bowditch, John (2021). "Distribution Models". The Power of Virtual Reality Cinema for Healthcare Training. pp. 217–226. doi:10.4324/9781003168683-21. ISBN   978-1-00-316868-3. S2CID   243419839.
  20. ↑ Bowditch, John; Love, Matt (2021). "Choosing Cameras and Head-Mounted Displays". The Power of Virtual Reality Cinema for Healthcare Training. pp. 65–74. doi:10.4324/9781003168683-7. ISBN   978-1-00-316868-3. S2CID   238697118.
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.