On-set virtual production

Last updated

On-set virtual production (OSVP), also known as virtual production (VP), or In-Camera Visual Effects (ICVFX), and often called The Volume, is an entertainment technology for television and film production in which LED panels are used as a backdrop for a set, on which video or computer-generated imagery can be displayed in real-time. The use of OSVP became widespread after its use in the first season of The Mandalorian (2019), which used Unreal Engine, developed by Epic Games.

Contents

History

Australian film director Clayton Jacobson first had the idea of improving the green screen that was then in use, when filming a TV ad for detergent in 2003. Watching his son playing videogames and seeing the 3D technology used in them gave him the idea. Eventually, in 2016, Jacobson and his son made one of the first prototypes for a virtual production stage in their shed, using a set of LED screens. However, he could not get anyone to take an interest in developing the technology further, so gave up on it. Other filmmakers had also caught on to the idea though, and in 2018 an Australian cinematographer, Greig Fraser, used the technology to film the Star Wars franchise spin-off series, The Mandalorian (released 2019). Instead of using the green screen during the filming stage, the team combined post-production with the production stage of the series. They installed huge LED walls linked to powerful computers which ran Unreal Engine gaming software (used for Fortnite, among others). They called this soundstage "the volume", a term already used to refer to a stage where visual effects techniques take place. [1]

Since its inventive use in The Mandalorian, which used ILM's StageCraft, the technology has become increasingly popular. Miles Perkins, industry manager of film and TV for Epic Games and maker of the Unreal Engine, estimated that there were around 300 stages by October 2022, increased from only three in 2019. [2] Most of these were built during or after the COVID-19 pandemic, when lockdowns meant that production studios had to find ways to produce films without travelling to other locations. [1]

In March 2023, the world's largest virtual production stage was opened at the Docklands Studios Melbourne, in the city of Melbourne, Australia. [1]

Terminology

On-set virtual production (OSVP) is also known as virtual production (VP), In-Camera Visual Effects (ICVFX), [3] or The Volume.[ citation needed ]

Technology

With careful adjustment and calibration, an OSVP set can be made to closely approximate the appearance of a real set or outdoor location. [4] OSVP can be viewed as an application of extended reality. OSVP contrasts with virtual studio technology, in which a green screen backdrop surrounds the set, and the virtual surroundings are composited into the green screen plate downstream from the camera, in that in OSVP the virtual world surrounding the set is visible to the camera, actors, and crew, and objects on set are illuminated by light from the LED screen, creating realistic interactive lighting effects, and that the virtual background and foreground are captured directly in camera, complete with natural subtle cues like lens distortion, depth of field effects, bokeh and lens flare. This makes it a far more natural experience that more closely approximates location shooting, making the film-making process faster and more intuitive than can be achieved on a virtual set.[ citation needed ]

To render parallax depth cues correctly from the viewpoint of a moving camera, the system requires the use of match moving of the background imagery based on data from low-latency real-time motion capture technology to track the camera.[ citation needed ]

Industry organizations including SMPTE, the Academy of Motion Picture Arts and Sciences and the American Society of Cinematographers have started initiatives to support the development of OSVP. [4] [5] [6] [7]

Examples

Stages that use OSVP include the various StageCraft stages, Pixomondo's Toronto-based LED stage, which has a longterm lease from CBS, ZeroSpace's Volume in New York City, [8] or Lux Machina various stages. [2] In Japan, the LED wall and virtual production were used by Toei Company for its Super Sentai shows Avataro Sentai Donbrothers [9] and Ohsama Sentai King-Ohger , with the latter also being produced in collaboration with Sony PCL Inc. [10]

As of March 2023 the largest OSVP is at Docklands Studios Melbourne in Australia. [1]

Productions using the technologies

Television series

Feature films

Short films

See also

Related Research Articles

Industrial Light & Magic (ILM) is an American motion picture visual effects company that was founded on May 26, 1975 by George Lucas. It is a division of the film production company Lucasfilm, which Lucas founded, and was created when he began production on the original Star Wars, now the fourth episode of the Skywalker Saga.

<span class="mw-page-title-main">Sound stage</span> Soundproof, hangar-like structure, building, or room, used for theatrical film-making

A sound stage is a large, soundproof structure, building or room with large doors and high ceilings, used for the production of theatrical film-making and television productions, usually located on a secured movie or television studio property.

<span class="mw-page-title-main">Virtual cinematography</span> CGI essentially

Virtual cinematography is the set of cinematographic techniques performed in a computer graphics environment. It includes a wide variety of subjects like photographing real objects, often with stereo or multi-camera setup, for the purpose of recreating them as three-dimensional objects and algorithms for the automated creation of real and simulated camera angles. Virtual cinematography can be used to shoot scenes from otherwise impossible camera angles, create the photography of animated films, and manipulate the appearance of computer-generated effects.

Pixomondo (PXO) is an Academy and Emmy award-winning international visual effects and virtual production company with studios in Los Angeles, Frankfurt, Stuttgart, Vancouver, Toronto, Montreal, and London. The company provides virtual production and visual effects for feature films, television, and commercials. As of 2019, PXO employs over 655 people worldwide.

DNEG is a British visual effects, computer animation and stereo conversion studio that was founded in 1998 in London, and rebranded as DNEG in 2014 after a merger with Indian VFX company Prime Focus; it was named after the letters "D" and "Neg" from their former name.

<span class="mw-page-title-main">Cinesite</span> Visual effects studio

Cinesite is an independent, multinational business which provides services to the media and entertainment industries. Its head office in London opened for business in 1994, initially offering services in visual effects for film and television, subsequently expanding to include animation.

<span class="mw-page-title-main">Craig Barron</span> American visual effects artist (born 1961)

Craig Barron is an American visual effects artist and creative director at Magnopus, a media company that produces visual development and virtual production services for motion pictures, television, museums and multimedia platforms.

<i>Ex Machina</i> (film) 2014 film by Alex Garland

Ex Machina is a 2014 science fiction psychological thriller film written and directed by Alex Garland in his directorial debut. A co-production between the United Kingdom and the United States, it stars Domhnall Gleeson, Alicia Vikander, and Oscar Isaac. It follows a programmer who is invited by his CEO to administer the Turing test to an intelligent humanoid robot.

Dane Allan Smith is a Creative Producer specializing in digital media. 20th Century Fox invited Smith to join a mentoring Program as a result of his inclusion on a Producers 'Watch": list He is known for his work on Dune (2022), Rings of Power (2022), Avengers Endgame (2021), Star Wars: Episode VIII - The Last Jedi (2017), Shang Chi and the Legend of the Ten Rings (2021), The Mandalorian and Jungle Cruise 2021. Dane currently manages daily operations as founding partner of Daneiam Inc.

<span class="mw-page-title-main">Milk VFX</span>

Milk VFX is a British independent visual effects studio. It was founded in London in 2013 and is known for creating complex and innovative sequences for high-end television and feature films.

The Mandalorian is an American space Western television series created by Jon Favreau for the streaming service Disney+. It is the first live-action series in the Star Wars franchise and begins five years after the events of the film Return of the Jedi (1983). It follows a lone bounty hunter who protects a Force-sensitive child, Grogu, from remnant Imperial forces.

<span class="mw-page-title-main">Chapter 1: The Mandalorian</span> 1st episode of the 1st season of The Mandalorian

"Chapter 1: The Mandalorian" is the first episode of the first season and series premiere of the American streaming television series The Mandalorian. It was written by the series' showrunner Jon Favreau and directed by Dave Filoni. The episode takes place in the Star Wars universe five years after the events of Return of the Jedi (1983). In the episode, it follows a lone bounty hunter only known as the Mandalorian, who accepts a mission given by a mysterious Client who tasks him to bring alive a fifty-year-old target. He is helped by vapor farmer Kuiil in exchange of ending the constant bounty hunting activity in his area.

The Visual Effects Society Award for Outstanding Animated Character in an Animated Feature is one of the annual awards given by the Visual Effects Society starting from 2002. Since its inception, the award's title has gone through six title changes, and one major category shift. First awarded in 2002, the award was titled "Best Character Animation in an Animated Motion Picture" and given to the best character animation in an animated film, with no specific character cited. This would change in 2004, when the category was re-titled "Outstanding Performance by an Animated Character in an Animated Motion Picture", and given to visual effects artists for work on a specified character. as well as the voice actor for the character. The category was again re-titled in 2008, this time to "Outstanding Animated Character in an Animated Feature Motion Picture". In 2015, it was titled "Outstanding Animated Performance in an Animated Feature", but changed in 2017 to "Outstanding Animated Character in an Animated Feature", its current title.

The Visual Effects Society Award for Outstanding Model in a Photoreal or Animated Project is one of the annual awards given by the Visual Effects Society. The award goes to artists whose work in models, miniatures, have been deemed worthy of recognition. Originally, the award was given separately to artists in both film and television, with the categories "Best Models and Miniatures in a Motion Picture" and "Best Models and Miniatures in a Televised Program, Music Video or Commercial". In 2004, there was only one category, recognizing only work in motion pictures with "Outstanding Models and Miniatures in a Motion Picture". In 2005, television was honored, once again, with "Outstanding Models and Miniatures in a Broadcast Program, Commercial or Music Video". Television series and/or televised content would be honored, intermittently, until 2015, when the category was redesigned to honor any motion media project. It has continued to do so since then.

The Visual Effects Society Award for Outstanding Virtual Cinematography in a CG Project is one of the annual awards given by the Visual Effects Society. The award goes to artists whose work in virtual cinematography. It was first awarded in 2003 and 2004 for, under the title "Best Visual Effects Photography in a Motion Picture", before being scrapped from the ceremony. The award was given separately to artists in live-action film, animated film, and commercials/television. These categories were first awarded in 2012. The following year, the category specifically awarding animated film was dropped, with only live-action film and commercial/television being awarded. In 2015, only live-action films were awarded. The following year, the category's title changed to "Outstanding Virtual Cinematography in a Photoreal Project". It has held its current title since 2020.

The Visual Effects Society Award for Outstanding Supporting Visual Effects in a Photoreal Episode is one of the annual awards given by the Visual Effects Society, starting in 2004. While the award's title has changed several time within this period, the recipients have been television episodes and/or movies or specials with less prominent, more subtle visual effects work.

<i>The Mandalorian</i> season 1 Season of television series

The first season of the American television series The Mandalorian stars Pedro Pascal as the title character, a lone bounty hunter hired to retrieve "The Child". It is part of the Star Wars franchise, set five years after the events of Return of the Jedi (1983). The season was produced by Lucasfilm, Fairview Entertainment, and Golem Creations, with Jon Favreau serving as showrunner.

StageCraft is an on-set virtual production visual effects technology composed of a video wall designed by Industrial Light & Magic (ILM). Initially developed for the Disney+ series The Mandalorian, it has since been used in other productions and has been cited as a revolutionary visual effects technology. The soundstage in which StageCraft is implemented is called The Volume.

<span class="mw-page-title-main">Jellyfish Pictures</span> Film and television production company

Jellyfish Pictures is a British VFX and animation company specializing in the creation of digital visual effects and animation.

Robert D. Bredow is an American visual effects artist. He was born in California, where he grew up in La Habra. He was nominated for an Academy Award in the category Best Visual Effects for the film Solo: A Star Wars Story.

References

  1. 1 2 3 4 Purtill, James (22 March 2023). "Virtual production studios are replacing green screens as world's largest opens in Melbourne". ABC News. ABC Science. Australian Broadcasting Corporation . Retrieved 24 March 2023.
  2. 1 2 Giardina, Carolyn (19 October 2022). "Too Much Volume? The Tech Behind 'Mandalorian' and 'House of the Dragon' Faces Growing Pains". The Hollywood Reporter . Retrieved 4 February 2023.
  3. ↑ "In-Camera Visual Effects (ICVFX)". www.nepgroup.com. Retrieved 4 February 2023.
  4. 1 2 "2021: The On-Set Virtual Production Initiative". www.smpte.org. Retrieved 22 February 2022.
  5. ↑ "Partnering to Develop Advisory Group of the On-Set Virtual Production (OSVP) Initiative". systemscontractor. 18 January 2022. Retrieved 22 February 2022.
  6. ↑ "SMPTE Hollywood To Explore Lighting and Color Science For Virtual Production". SHOOTonline. Retrieved 22 February 2022.
  7. ↑ "AMPAS Science & Technology Council Presents Virtual Production Primer - The American Society of Cinematographers". theasc.com. Retrieved 22 February 2022.
  8. ↑ Abram, Cleo. "We Tested Sci-Fi Movie Tech (MKBHD)". YouTube. Retrieved 12 March 2023.
  9. 1 2 VFXアナトミー インカメラVFXによる特撮をTVシリーズで実現！　『暴太郎戦隊ドンブラザーズ』
  10. 1 2 新番組「王様戦隊キングオージャー」の背景CGがハイクオリティーと話題に　グリーンバックを使わない最新撮影技術とは？
  11. ↑ Gartenberg, Chaim (20 February 2020). "How The Mandalorian teamed up with Fortnite creator Epic Games to create its digital sets". The Verge . Archived from the original on 20 February 2020. Retrieved 20 February 2020.
  12. ↑ Lane, Carly (9 June 2021). "Exclusive: Here's When 'The Mandalorian' Season 3 Is Filming; New Details on 'Book of Boba Fett' Connection". Collider . Archived from the original on 9 June 2021. Retrieved 9 June 2021.
  13. 1 2 3 4 5 6 Giardina, Carolyn (19 October 2022). "Too Much Volume? The Tech Behind 'Mandalorian' and 'House of the Dragon' Faces Growing Pains". The Hollywood Reporter. Retrieved 4 February 2023.
  14. ↑ "ILM STAGECRAFT HONORED WITH ENGINEERING, SCIENCE & TECHNOLOGY EMMY® AWARD". Industrial Light & Magic . 29 July 2022. Retrieved 29 July 2022.
  15. ↑ Peterson, Karen M. (29 April 2022). "How 'Our Flag Means Death' Designers Mixed History With Broad Theatricality". Variety. Retrieved 16 July 2022.
  16. ↑ Bui, Hoai-Tran (17 June 2020). "The Obi-Wan Kenobi Disney+ Series Will Use the Same Technology as 'The Mandalorian'". /Film . Archived from the original on 18 June 2020. Retrieved 18 June 2020.
  17. 1 2 3 "StageCraft | Industrial Light & Magic". www.ilm.com. Retrieved 4 February 2023.
  18. ↑ Dominguez, Noah (14 November 2022). "Andor Actually Used StageCraft Technology Despite Early Reports". ComicBook.com. Retrieved 25 November 2022.
  19. ↑ "HOUSE OF THE DRAGON MAY LOOK TOTALLY DIFFERENT IN SEASON 2 FOR ONE CONTROVERSIAL REASON". Inverse. 19 October 2022. Retrieved 25 October 2022.
  20. ↑ "Inside Dark Bay, the spinning LED volume at the heart of Netflix's upcoming '1899'". TechCrunch. 10 February 2022. Retrieved 20 November 2022.
  21. ↑ Giardina, Carolyn (28 June 2022). "'Percy Jackson and the Olympians' Takes ILM Virtual Production Route (Exclusive)". The Hollywood Reporter . Archived from the original on 28 June 2022. Retrieved 28 June 2022.
  22. ↑ "MoTA = @cary_fukunaga". www.instagram.com. Retrieved 4 February 2023.
  23. ↑ Fisher, Jacob (24 June 2021). "'The Acolyte' Aiming To Begin Filming In February In London". DiscussingFilm. Archived from the original on 30 June 2021. Retrieved 30 June 2021.
  24. ↑ "Oblivion". Lux Machina. Retrieved 4 February 2023.
  25. ↑ Bardini, Julio (14 August 2022). "Industrial Light & Magic's Digital StageCraft Technology: What We Know About the Volume". Collider. Retrieved 4 February 2023.
  26. ↑ "Lux Machina Consulting on Instagram: "We had a great time working with Director of Photography Jess Hall on Ghost in the Shell to provide LED green screen replacement for some of the driving scenes in the movie. Contact us for help with any film or television projects! We specialize in capturing realistic environments in camera as part of the visual effects workflow! Pictured: Pilou Asbaek and Scarlett Johansson in Ghost in the Shell from Paramount Pictures and Dreamworks Pictures."". Instagram. Retrieved 4 February 2023.
  27. ↑ Desowitz, Bill (15 October 2018). "Beyond Christopher Nolan: 'First Man' Redefines In-Camera VFX". IndieWire. Archived from the original on 3 November 2018. Retrieved 2 November 2018.
  28. ↑ "How Cutting-Edge ILM Technology Brought 'The Mandalorian' to Life". No Film School. 5 February 2020. Retrieved 26 June 2021.
  29. ↑ "The Irishman". Lux Machina. Retrieved 4 February 2023.
  30. ↑ "Lux Machina Consulting on Instagram: "Congrats to @starwarsriseofskywalker and @theirishmanfilm for the many Oscar nominations this week, including for Visual Effects! We are so grateful to have been a part of both productions. Looking forward to the big night, Feb 5 live at 8e | 5p! . . . . . . #virtualproduction #incameravfx #vfx #film #technology #videoproduction #led #projection #videoscreens #luxmc #luxmachina #consultants #engineers #programmers"". Instagram. Retrieved 4 February 2023.
  31. ↑ Mike Seymour (21 January 2021). "The Midnight Sky's Virtual Production using StageCraft". Fxguide . Retrieved 26 June 2021.
  32. ↑ Filippo Magnifico (23 December 2020). "In Vacanza su Marte: il film con Boldi e De Sica realizzato con una tecnologia simile a quella di The Mandalorian". Blog Screenweek . Retrieved 30 May 2022.
  33. ↑ "The Next Generation of Extended Reality Sounds Stages". Lux Machina. Retrieved 4 February 2023.
  34. ↑ Giardina, Carolyn (19 October 2020). "'The Batman' Using 'Mandalorian' Virtual Production Techniques". The Hollywood Reporter . Archived from the original on 19 October 2020. Retrieved 19 October 2020.
  35. ↑ Carolyn Giardina (10 September 2020). "Industrial Light & Magic Expands Virtual Production Services, Supports 'Thor 4'". The Hollywood Reporter . Retrieved 26 June 2021.
  36. ↑ "Lux Machina Consulting on Instagram: "Congratulations to @topgunmovie "Maverick" on its six @theacademy nominations, including Best Visual Effects and Best Picture! It was an honor for @luxmc_ to be part of such an iconic project by providing in-camera virtual backgrounds and working closely with "Maverick's" overall VFX supervisor Ryan Tudhope and Director of Photography @claudiomirandadop We are thrilled at how much the fans, the critics, and now the Academy of Motion Picture Arts and Sciences have responded to the film. The Oscars will be handed out Sunday, March 12, 2023. Get your popcorn ready! (@luxmc_ is part of @nep_group) #vfx #visualeffects #luxmc #awards #oscars #nominations #academy #topgun #NEP #NEPVS #topgunmaverick #sequel #ICFVX #film #photography #directorofphotography"". Instagram. Retrieved 4 February 2023.
  37. ↑ Villaverde, Noah (8 June 2022). "'Black Adam' Director Talks Using Lucasfilm's Volume & Other New Tech". Heroic Hollywood. Retrieved 19 July 2022.
  38. ↑ Kadner, Noah (24 April 2023). "Virtual Production—When The Fabelmans Met The Mandalorian". Frame.io Insider. Retrieved 26 April 2023.
  39. ↑ "Lux Machina Consulting on Instagram: "Wishing Eddie Alcazar the best of luck as he heads to #Sundance to premier his newest film "Divinity." A portion of the film was shot using the #LuxMC LED volume, and Junaid Baig, VP of Research & Development for #NEPVS, used AI to provide slow motion footage of already animated stop motion sequences on the film."". Instagram. Retrieved 4 February 2023.
  40. ↑ Erao, Math (17 May 2021). "Ant-Man and the Wasp: Quantumania Director Shares Photo From The Mandalorian's Virtual Set". Comic Book Resources . Archived from the original on 17 May 2021. Retrieved 17 May 2021.
  41. ↑ "Christy Hall Shot Daddio Entirely Inside a Taxi Cab". 12 September 2023.
  42. ↑ Couch, Kim Masters, Scott Feinberg, Aaron; Masters, Kim; Feinberg, Scott; Couch, Aaron (9 January 2023). "Francis Ford Coppola's 'Megalopolis' in Peril Amid Ballooning Budget, Crew Exodus (Exclusive)". The Hollywood Reporter. Retrieved 4 February 2023.{{cite web}}: CS1 maint: multiple names: authors list (link)
  43. ↑ "Lux Machina Consulting on Instagram: "What does it take to make an award winning film? We're glad you asked! Watch the new behind the scenes video and find out how the creative artists were able to seamlessly blend live action video with stop-motion for @eddiealcazar 's #VimeoBestOfTheYear award-winning short film "The Vandal." In "The Making of The Vandal," you'll see how we used our #LUXMC LED stage and interactive lighting sources for the miniature photography. Watch "The Making of The Vandal" via the link in our bio."". Instagram. Retrieved 4 February 2023.
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.