Projection mapping

Last updated
Projection mapping of a pattern onto curved surfaces Had to wait for 5ENSES to close to avoid spoilers but nice projection mapping by @DoloresJoya @CODA ME codame (photo by j bizzie).jpg
Projection mapping of a pattern onto curved surfaces

Projection mapping, similar to video mapping and spatial augmented reality, is a projection technique [1] [2] used to turn objects, often irregularly shaped, into display surfaces for video projection. The objects may be complex industrial landscapes, such as buildings, small indoor objects, or theatrical stages. Using specialized software, a two- or three-dimensional object is spatially mapped on the virtual program which mimics the real environment it is to be projected on. The software can then interact with a projector to fit any desired image onto the surface of that object. [3] The technique is used by artists and advertisers who can add extra dimensions, optical illusions, and notions of movement onto previously static objects. The video is commonly combined with or triggered by audio to create an audiovisual narrative. In recent years the technique has also been widely used in the context of cultural heritage, as it has proved to be an excellent edutainment [4] tool.

Contents

History

Although the term "projection mapping" is relatively new, the technique dates back to the mid-20th century. Josef Svoboda, a Czech scenographer, debuted Lanterna Magika at Expo 58, the 1958 Brussels World Fair. [5] Lanterna Magika was a live performance that featured singers, dancers, and musicians performing against a backdrop of projected film footage. Günther Schneider-Siemssen was a German set designer, who coined the phrase, "painting with light. [6] He used Pani projectors to create large-scale projections for operas. In the late 1960s, audiences referred to projection mapping as, "the Madame Leota effect, [7] " based on the use of the technique in Disney's Haunted Mansion. Disneyland debuted a projection effect in 1969, when they opened their Haunted Mansion attraction, which featured singing three-dimensional busts. The singers' faces were filmed on 16mm film and projected onto busts of their faces to make the busts appear animated. [8] Another early example of projection mapping was in the 1967 TV movie Magical Mystery Tour during the Blue Jay Way scene, where images were projected onto George Harrison, including a cat's face and a headless male torso with the words "Magical Mystical Boy" written on its chest. [9]

The next record of projection mapping is from 1980, when installation artist Michael Naimark filmed people interacting with objects in a living room and then projected it in the room, creating illusions as if the people interacting with the objects were really there. [10] [11]

In 1984 the Stephen Sondheim original Broadway production of Sunday in the Park With George , written and directed by James Lapine, was the first known use of projection mapping in a Broadway musical or play. [12] It was used at the end of Act II, in the Chromolume #7 special effects sequence designed by Bran Ferren [13] to project geometrically-correct moving cinematic images onto the surface of the 4' diameter sphere topping the Chromolume device. Due to the brightness limitations of video projection at the time, the images were projected using 7000w xenon-illuminated 35mm motion picture film (at 48 frames per second). The film's images were digitally pre-distorted to map correctly onto the sphere from the high projection angle in the Booth theater.

The first time the concept of projection mapping was investigated academically [14] was at the University of North Carolina at Chapel Hill in the late 1990s, where a team led by Ramesh Raskar worked on a project called Office of the Future, to connect offices in different locations by projecting people into an office space as if they were really there. [15] By 2001, more artists began using projection mapping in artwork, and groups such as Microsoft began experimenting with it as a means of technological advancement.[ citation needed ]

Methods

The Sydney Opera House during the 2013 Vivid Sydney projection display Vivid Sydney - Opera House sails (9002375891).jpg
The Sydney Opera House during the 2013 Vivid Sydney projection display

After the object which will be projected on is chosen or created, software is used to map the corners of the video to the surfaces. First, one must choose the images or video to project. Each video is then placed on its designated surface. Alternatively, one may map the entire scene in 3D and attempt to project and mask the image back onto its framework. The next step is masking, using opacity templates to actually "mask" the exact shapes and positions of the different elements of the building or space of projection.

In 3D mapping, coordinates are defined for where the object is placed in relation to the projector. The projector's XYZ orientation, position, and lens specification result in a determined virtual scene. [16]

A projection mapping test grid projected onto a building Moscow, VDNKh, test grid projected onto the Cosmos pavilion (21549825834).jpg
A projection mapping test grid projected onto a building

Adjustments are commonly made by manually adjusting either the physical or virtual scene for best results. Large projectors with 20,000 lumens output or greater are used for large-scale projections such as on city skyscrapers. Due to the scale and brightness some projects require, often large arrays of powerful projectors are combined into a single image through a method known as "edge blending" or "stacking". [17] The result is a much brighter projection that maintains its seamless look. This technique is used for most large projection-mapping shows, and requires skill and patience to be set up with specialised software. For smaller productions, smaller, lower-output projectors are sufficient. In most cases, a 2200-lumen projector is adequate for projections under indoor light or theatrical lighting.

Video mapping software can be used in projects like these, though Adobe Photoshop, Adobe After Effects and other packages can also be used by creative artists. Extensible open-source software frameworks are also available.

Projection mapping can be separated into four categories:

Productions, advertisement and art

Projection mapping at Fete des Lumieres by P. Warrener in Lyon (2008) Cath. St Jean (2).JPG
Projection mapping at Fête des Lumières by P. Warrener in Lyon (2008)
Projection mapping on Cinderella Castle at Walt Disney World Celebrate the Magic.jpg
Projection mapping on Cinderella Castle at Walt Disney World

Projection mapping first came to prominence through guerrilla advertising campaigns and video jockeys for electronic musicians. [18] Large companies such as Nokia, Samsung, Unilever Pakistan, Pakistan Tobacco company, Bank Alfalah, Brighto Paints, Benson & Hedges, John Players Gold Leaf and BMW have since used video projections in marketing campaigns in cities across the world, commonly using mapping techniques to project scenes onto the sides of buildings. Projection mapping can also be interactive: Nokia Ovi Maps did a project where projections mimicked people's movements. [19] Projection mapping has been used at conferences as a means of decoration or immersing audience members in an experienced theme. Images can be projected onto a flat surface, or onto an unusual object such as a car or a chair. The festival Fête des Lumières in Lyon, a festival to honour the Virgin Mary, has recently incorporated 3D mapping in their productions, creating the illusion of a giant pinball machine on the side of a building. Common techniques for these performances include 3D mapping and 3D projection to create the illusion of depth, as well as motion, such as crumbling buildings.

It is also being used in technology such as Domes , where it is combined with virtual reality and augmented reality to create 360-degree projections for a more immersive experience.

The use of projection mapping in TV and films is becoming more popular. For the science-fiction film Oblivion (2013), the filmmakers used projection mapping to create an immersive environment. [20] For a 2016 TV commercial, Audi used projection mapping to showcase the technology of the Audi Q7 car. The ad, "Projection of Greatness", [21] was filmed with no CGI and used only content that was caught live in camera. [22]

In the electronic dance music (EDM) community, it is becoming increasingly common for DJs to accompany their music with synced visuals, which can be either prerecorded or played live by a video jockey (VJ). Though normal projection screens are commonly used, some visual artists create special 3D installations to project onto. [23] Many EDM artists employ projection mapping techniques. Visual artists also use projection mapping for creative expression, sometimes to enhance existing creative media such as painting and drawing. [24]

Artists may use it as an avant garde form of expression as it is new technology that can turn their creative ideas into 3D projections, connecting with audiences in a new way. [25] Video projections have appeared in urban centres such as New York City and London, where artists have used guerilla projections in public without any necessary approval. This way, artists can show their work in any location as anything and anywhere can be a canvas. Often people also use it as a means of activism; the group Occupy Wall Street used it to project onto the Verizon Wireless building in New York City as a means to visually spread the word that Occupy Wall Street is still alive. [26] The Japanese theatre play Mysteries of Yoshitsune I&II (2012–14) by Gackt is notable for the first major use of projection mapping in Japanese theatre stage play. [27] [28] [29]

Projection mapping is frequently used by Walt Disney Imagineering and Walt Disney Creative Entertainment in the Disney Parks. Examples include The Magic, the Memories and You, Disney Dreams! , Celebrate the Magic , Once Upon a Time , Disneyland Forever , Halloween Screams , Believe... In Holiday Magic , Remember... Dreams Come True , Happily Ever After [30] and most recently Sunset Seasons Greetings at Disney's Hollywood Studios.

When Paul Oakenfold became the first DJ to perform live at Stonehenge, projection mapping was used to transform the prehistoric monument into a spectacular light show. [31] To avoid damage to the ancient stones, only 50 of Oakenfold's close friends were invited, and were required to wear noise-cancelling headphones to hear the music above the nearby A303 road.

Activism

Projection mapping has also been used as a way to highlight political and social causes by groups such as Greenpeace and Led By Donkeys. [32] [33] The White Cliffs of Dover are frequently used to project messages of protest, [34] [35] support [36] and memorials. [37]

See also

Related Research Articles

<span class="mw-page-title-main">Augmented reality</span> View of the real world with computer-generated supplementary features

Augmented reality (AR) is an interactive experience that combines the real world and computer-generated content. The content can span multiple sensory modalities, including visual, auditory, haptic, somatosensory and olfactory. AR can be defined as a system that incorporates three basic features: a combination of real and virtual worlds, real-time interaction, and accurate 3D registration of virtual and real objects. The overlaid sensory information can be constructive, or destructive. As such, it is one of the key technologies in the reality-virtuality continuum.

2.5D perspective refers to gameplay or movement in a video game or virtual reality environment that is restricted to a two-dimensional (2D) plane with little or no access to a third dimension in a space that otherwise appears to be three-dimensional and is often simulated and rendered in a 3D digital environment.

<span class="mw-page-title-main">Cave automatic virtual environment</span> Immersive virtual reality environment

A cave automatic virtual environment is an immersive virtual reality environment where projectors are directed to between three and six of the walls of a room-sized cube. The name is also a reference to the allegory of the Cave in Plato's Republic in which a philosopher contemplates perception, reality, and illusion.

A volumetric display device is a display device that forms a visual representation of an object in three physical dimensions, as opposed to the planar image of traditional screens that simulate depth through a number of different visual effects. One definition offered by pioneers in the field is that volumetric displays create 3D imagery via the emission, scattering, or relaying of illumination from well-defined regions in (x,y,z) space.

<span class="mw-page-title-main">Real-time computer graphics</span> Sub-field of computer graphics

Real-time computer graphics or real-time rendering is the sub-field of computer graphics focused on producing and analyzing images in real time. The term can refer to anything from rendering an application's graphical user interface (GUI) to real-time image analysis, but is most often used in reference to interactive 3D computer graphics, typically using a graphics processing unit (GPU). One example of this concept is a video game that rapidly renders changing 3D environments to produce an illusion of motion.

In the fields of computing and computer vision, pose represents the position and orientation of an object, usually in three dimensions. Poses are often stored internally as transformation matrices. The term “pose” is largely synonymous with the term “transform”, but a transform may often include scale, whereas pose does not.

<span class="mw-page-title-main">3D rendering</span> Process of converting 3D scenes into 2D images

3D rendering is the 3D computer graphics process of converting 3D models into 2D images on a computer. 3D renders may include photorealistic effects or non-photorealistic styles.

Shader lamps is a computer graphic technique used to change the appearance of physical objects. The still or moving objects are illuminated, using one or more video projectors, by static or animated texture or video stream. The method was invented at University of North Carolina at Chapel Hill by Ramesh Raskar, Greg Welch, Kok-lim Low and Deepak Bandyopadhyay in 1999 as a follow on to Spatial Augmented Reality also invented at University of North Carolina at Chapel Hill in 1998 by Ramesh Raskar, Greg Welch and Henry Fuchs.

<span class="mw-page-title-main">3D computer graphics</span> Graphics that use a three-dimensional representation of geometric data

3D computer graphics, sometimes called CGI, 3-D-CGI or three-dimensional computer graphics, are graphics that use a three-dimensional representation of geometric data that is stored in the computer for the purposes of performing calculations and rendering digital images, usually 2D images but sometimes 3D images. The resulting images may be stored for viewing later or displayed in real time.

A projection augmented model is an element sometimes employed in virtual reality systems. It consists of a physical three-dimensional model onto which a computer image is projected to create a realistic looking object. Importantly, the physical model is the same geometric shape as the object that the PA model depicts.

A structured-light 3D scanner is a 3D scanning device for measuring the three-dimensional shape of an object using projected light patterns and a camera system.

<span class="mw-page-title-main">Computer graphics</span> Graphics created using computers

Computer graphics deals with by generating images and art with the aid of computers. Today, computer graphics is a core technology in digital photography, film, video games, digital art, cell phone and computer displays, and many specialized applications. A great deal of specialized hardware and software has been developed, with the displays of most devices being driven by computer graphics hardware. It is a vast and recently developed area of computer science. The phrase was coined in 1960 by computer graphics researchers Verne Hudson and William Fetter of Boeing. It is often abbreviated as CG, or typically in the context of film as computer generated imagery (CGI). The non-artistic aspects of computer graphics are the subject of computer science research.

In computing, 3D interaction is a form of human-machine interaction where users are able to move and perform interaction in 3D space. Both human and machine process information where the physical position of elements in the 3D space is relevant.

A variety of computer graphic techniques have been used to display video game content throughout the history of video games. The predominance of individual techniques have evolved over time, primarily due to hardware advances and restrictions such as the processing power of central or graphics processing units.

A holographic screen is a two-dimensional display technology that uses coated glass media for the projection surface of a video projector. "Holographic" refers not to a stereoscopic effect, but to the coating that bundles light using formed microlenses. The lens design and attributes match the holographic area. The lenses may appear similar to the Fresnel lenses used in overhead projectors. The resulting effect is that of a free-space display, because the image carrier appears very transparent. Additionally, the beam manipulation by the lenses can be used to make the image appear to be floating in front of or behind the glass, rather than directly on it. However, this display is only two-dimensional and not true three-dimensional. It is unclear if such a technology will be able to provide acceptable three-dimensional images in the future.

<span class="mw-page-title-main">IllumiRoom</span> Microsoft research Project

IllumiRoom is a Microsoft Research project that augments a television screen with images projected onto the wall and surrounding objects. The current proof-of-concept uses a Kinect sensor and video projector. The Kinect sensor captures the geometry and colors of the area of the room that surrounds the television, and the projector displays video around the television that corresponds to a video source on the television, such as a video game or movie.

Image Geometry Correction is the process of digitally manipulating image data such that the image’s projection precisely matches a specific projection surface or shape. Image geometry correction compensates for the distortion created by off-axis projector or screen placement or non-flat screen surface, by applying a pre-compensating inverse distortion to that image in the digital domain.

<i>Mysteries of Yoshitsune I & II</i> 2014 soundtrack album by Gackt

Moon Saga: Mysteries of Yoshitsune I & II is a duology of theatre stage plays written, composed and directed by Japanese recording artist Gackt. The original productions are notable for the first major use of projection mapping in theatre stage plays. The original soundtrack was released on October 1, 2014 and compiles music from the plays. It consists of two music CDs, one for each chapter, the first of which was performed in 2012, while the second was performed in 2014.

Commercial augmented reality (CAR) describes augmented reality (AR) applications that support various B2B (Business-to-Business) and B2C (Business-to-Consumer) commercial activities, particularly for the retail industry. The use of CAR started in 2010 with virtual dressing rooms for E-commerce.

The Augmented Reality Sandtable (ARES) is an interactive, digital sand table that uses augmented reality (AR) technology to create a 3D battlespace map. It was developed by the Human Research and Engineering Directorate (HRED) at the Army Research Laboratory (ARL) to combine the positive aspects of traditional military sand tables with the latest digital technologies to better support soldier training and offer new possibilities of learning. It uses a projector to display a topographical map on top of the sand in a regular sandbox as well as a motion sensor that keeps track of changes in the layout of the sand to appropriately adjust the computer-generated terrain display.

References

  1. ↑ Maniello, Donato (2014). Augmented Reality in public spaces. Basic Techniques for video mapping. Vol. I. Brienza: Le Penseur. ISBN   978-8895315348.
  2. ↑ Maniello, Donato (2018). Advanced video mapping techniques. Spatial Augmented Reality applied to cultural heritage. Vol. II. Brienza: Le Penseur. ISBN   978-8895315584.
  3. ↑ "What is projection mapping" . Retrieved 13 February 2015.
  4. ↑ Maniello, Donato (2018). Spatial Augmented Reality. The development of edutainment for augmented digital spaces. Vol. III. Brienza: Le Pensuer. ISBN   978-8895315591.
  5. ↑ Kapic (http://www.kapik.cz), (c) 2011 Martin Novák (http://www u2cms cz), Tomáš. "Laterna magika concept | Josef Svoboda - scénograf a architekt". [TR#Josef Svoboda]. Retrieved 2023-02-15.{{cite web}}: CS1 maint: multiple names: authors list (link) CS1 maint: numeric names: authors list (link)
  6. ↑ https://austria-forum.org, Austria-Forum |. "Schneider-Siemssen, Günther". Austria-Forum (in German). Retrieved 2023-02-15.{{cite web}}: External link in |last= (help)
  7. ↑ "The Secret Tech Behind Disney's Haunted Mansion Illusions". Gizmodo. 31 October 2015. Retrieved 2020-04-17.
  8. ↑ "The Illustrated History of Projection Mapping" . Retrieved 25 March 2013.
  9. ↑ "The Beatles: Blue Jay Way Is a Hidden Masterpiece". Den of Geek. Retrieved 2020-01-16.
  10. ↑ Naimark, Michael. "Spatial Correspondence in Motion Picture Display" (PDF). SPIE Proceedings, vol. 462, Optics and Entertainment, Los Angeles, 1984.
  11. ↑ Naimark, Michael. "Two Unusual Projection Spaces". Presence, Special Issue on Projection, MIT Press, 14.5, October 2005.
  12. ↑ "Sunday in the Park, Isolating with Mandy and Bernadette, While Overlooking the Isle of Toronto – Times Square Chronicles". 29 March 2020. Retrieved 2020-04-17.
  13. ↑ "Sunday In the Park with Geoge". IBDB.
  14. ↑ Raskar, Ramesh. "Spatially Augmented Reality" (PDF). Office of the Future. IWAR. Retrieved 21 September 2016.
  15. ↑ Towles, Herman. "Office of the Future". UNC.
  16. ↑ "Virtual replica of the real scene" . Retrieved 11 July 2012.
  17. ↑ "Edge Blending and Stacking in Projection Mapping" . Retrieved 26 June 2020.
  18. ↑ Sigrist, Peter. "A short history of 3D projection mapping". Archived from the original on 17 April 2011. Retrieved 3 July 2012.
  19. ↑ O'Neill, Megan. "Comcast is looking for a Executive Director, XFTV Programming. WEB VIDEO 3D Projection Mapping Taking The Advertising World By Storm".
  20. ↑ "Projection Mapping in Oblivion (starring Tom Cruise)".
  21. ↑ "Projection of Greatness: The all-new Audi Q7". YouTube .
  22. ↑ "Illuminating Audi's latest advancements". 17 November 2016.
  23. ↑ "deadmau5 lights up London with amazing 4D projection- Nokia". YouTube .
  24. ↑ "How To Use Projection Mapping In Events".
  25. ↑ "Mission / Artist Statement". Integrated Visions.
  26. ↑ Day, Adrienne (26 January 2012). "When Skyscrapers are your Screen". The New York Times.
  27. ↑ "Moon Saga 〜義経秘伝〜 Vol.2 Live Report" (in Japanese). Excite Japan. 2014-08-08. Retrieved August 28, 2014.
  28. ↑ "Gacktが紡ぎだす歴史ファンタジーの世界感を 日本の舞台史上最大規模のプロジェクションマッピングで演出する Moon Saga－義経秘伝－第二章". Theatrix (in Japanese). Entertainment Plus. 2014-07-24. Retrieved August 28, 2014.
  29. ↑ "原作、脚本、音楽、演出、主演をGacktがプロデュースする舞台『Moon Saga－義経秘伝－第二章』、絶賛上演中！". Theatrix (in Japanese). Entertainment Plus. 2014-08-18. Retrieved August 28, 2014.
  30. ↑ Mine, Mark R.; van Baar, Jeroen; Grundhöfer, Anselm; Rose, David; Yang, Bei (July 2012). "Projection-Based Augmented Reality in Disney Theme Parks". Computer. 45 (7): 32–40. doi:10.1109/MC.2012.154. S2CID   501354.
  31. ↑ "Projection Mapping Lights Up Stonehenge".
  32. ↑ "Projection artists bring light to social issues with attention-grabbing protests".
  33. ↑ "Projection Powers Protest Movements".
  34. ↑ "'Refugees Welcome' has been projected on to the White Cliffs of Dover ahead of a far-right protest".
  35. ↑ "War veterans project love letter to Europe on White Cliffs of Dover".
  36. ↑ "Baddiel, Skinner and Lightning Seeds message projected on white cliffs of Dover".
  37. ↑ "Projection on the White Cliffs of Dover in honour of Dame Vera Lynn".
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.