Digital puppetry

Last updated

Digital puppetry is the manipulation and performance of digitally animated 2D or 3D figures and objects in a virtual environment that are rendered in real-time by computers. It is most commonly used in filmmaking and television production but has also been used in interactive theme park attractions and live theatre.

Contents

The exact definition of what is and is not digital puppetry is subject to debate among puppeteers and computer graphics designers, but it is generally agreed that digital puppetry differs from conventional computer animation in that it involves performing characters in real-time, rather than animating them frame by frame.

Digital puppetry is closely associated with character animation, motion capture technologies, and 3D animation, as well as skeletal animation. Digital puppetry is also known as virtual puppetry, performance animation, living animation, aniforms, live animation and real-time animation (although the latter also refers to animation generated by computer game engines). Machinima is another form of digital puppetry, and Machinima performers are increasingly being identified as puppeteers.

History and usage

Early experiments

One of the earliest pioneers of digital puppetry was Lee Harrison III. He conducted experiments in the early 1960s that animated figures using analog circuits and a cathode ray tube. Harrison rigged up a body suit with potentiometers and created the first working motion capture rig, animating 3D figures in real-time on his CRT screen. He made several short films with this system, which he called ANIMAC. [1] Among the earliest examples of digital puppets produced with the system included a character called "Mr. Computer Image" who was controlled by a combination of the ANIMAC's body control rig and an early form of voice-controlled automatic lip sync. [2]

Waldo C. Graphic

Perhaps the first truly commercially successful example of a digitally animated figure being performed and rendered in real-time is Waldo C. Graphic, a character created in 1988 by Jim Henson and Pacific Data Images for the Muppet television series The Jim Henson Hour . Henson had used the Scanimate system to generate a digital version of his Nobody character in real-time for the television series Sesame Street as early as 1970 [3] and Waldo grew out of experiments Henson conducted to create a computer generated version of his character Kermit the Frog [4] in 1985. [5]

Waldo's strength as a computer-generated puppet was that he could be controlled by a single puppeteer (Steve Whitmire [6] ) in real-time in concert with conventional puppets. The computer image of Waldo was mixed with the video feed of the camera focused on physical puppets so that all of the puppeteers in a scene could perform together. (It was already standard Muppeteering practice to use monitors while performing, so the use of a virtual puppet did not significantly increase the complexity of the system.) Afterward, in post-production, PDI re-rendered Waldo in full resolution, adding a few dynamic elements on top of the performed motion. [7]

Waldo C. Graphic can be seen today in Jim Henson's Muppet*Vision 3D at Disney's Hollywood Studios in Lake Buena Vista, Florida.

Mike Normal

Another significant development in digital puppetry in 1988 was Mike Normal, which Brad DeGraf and partner Michael Wahrman developed to show off the real-time capabilities of Silicon Graphics' then-new 4D series workstations. Unveiled at the 1988 SIGGRAPH convention, it was the first live performance of a digital character. Mike was a sophisticated talking head driven by a specially built controller that allowed a single puppeteer to control many parameters of the character's face, including mouth, eyes, expression, and head position. [8]

The system developed by deGraf/Wahrman to perform Mike Normal was later used to create a representation of the villain Cain in the motion picture RoboCop 2 , which is believed to be the first example of digital puppetry being used to create a character in a full-length motion picture.

Trey Stokes was the puppeteer for both Mike Normal's SIGGRAPH debut and Robocop II.

Sesame Street: Elmo's World

One of the most widely seen successful examples of digital puppetry in a TV series is Sesame Street's "Elmo's World" segment. A set of furniture characters were created with CGI, to perform simultaneously with Elmo and other real puppets. They were performed in real-time on set, simultaneously with live puppet performances. As with the example of Henson's Waldo C. Graphic above, the digital puppets' video feed was seen live by both the digital and physical puppet performers, allowing the digital and physical characters to interact. [9]

Disney theme parks

Walt Disney Imagineering has also been an important innovator in the field of digital puppetry, developing new technologies to enable visitors to Disney theme parks to interact with some of the company's famous animated characters. [10] In 2004, they used digital puppetry techniques to create the Turtle Talk with Crush attractions at Epcot and Disney California Adventure Park. In the attraction, a hidden puppeteer performs and voices a digital puppet of Crush, the laid-back sea turtle from Finding Nemo , on a large rear-projection screen. To the audience, Crush appears to be swimming inside an aquarium and engages in unscripted, real-time conversations with theme park guests.

Disney Imagineering continued its use of digital puppetry with the Monsters Inc. Laugh Floor, a new attraction in Tomorrowland at Walt Disney World's Magic Kingdom, which opened in the spring of 2007. Guests temporarily enter the "monster world" introduced in Disney and Pixar's 2001 film, Monsters, Inc. , where they are entertained by Mike Wazowski and other monster comedians who are attempting to capture laughter, which they convert to energy. Much like Turtle Talk, the puppeteers interact with guests in real time, just as a real-life comedian would interact with his/her audience.

Disney also uses digital puppetry techniques in Stitch Encounter, which opened in 2006 at the Hong Kong Disneyland park. Disney has another version of the same attraction in Disneyland Resort Paris called Stitch Live!

Military Simulation & Training

Since 2014, the United States Army's Program Executive Office for Simulation, Training, Research, and Instrumentation (PEO STRI), a division of US Army Simulation and Training Technology Center (STTC), has been experimenting with digital puppetry as a method of teaching advanced situational awareness for infantry squads. [11] A single improvisor using motion capture technology from Organic Motion Inc interacted with squads through the medium of several different life-sized avatars of varying ages and genders that were projected onto multiple walls throughout an urban operations training facility. The motion capture technology was paired with real-time voice shifting to achieve the effect. [12]

Types of digital puppetry

Waldo puppetry

A digital puppet is controlled onscreen in real-time by a puppeteer who uses a telemetric input device known as a Waldo (after the short story "Waldo" by Robert A. Heinlein which features a man who invents and uses such devices), connected to the computer. The X-Y-Z axis movement of the input device causes the digital puppet to move correspondingly.

Computer facial animation

Computer facial animation is primarily an area of computer graphics that encapsulates methods and techniques for generating and animating images or models of a character's face. The importance of human faces in verbal and non-verbal communication and advances in computer graphics hardware and software have caused considerable scientific, technological, and artistic interests in computer facial animation.

Motion capture puppetry/performance animation

An object (puppet) or human body is used as a physical representation of a digital puppet and manipulated by a puppeteer. The movements of the object or body are matched correspondingly by the digital puppet in real time. Motion capture puppetry is commonly used, for example, by VTubers, who rig digital avatars to correspond to the movements of their heads.

Virtual human

Virtual human (or digital human) are simulations of human beings on computers. The research domain is concerned with their representation, movement, and behavior, and also show that the human-like appearance of virtual human shows higher message credibility than anime-like virtual human in an advertising context. A particular case of a virtual human is the virtual actor, which is a virtual human (avatar or autonomous) representing an existing personality and acting in a film or a series.

Aniforms

Aniforms is a two-dimensional cartoon character operated like a puppet, to be displayed to live audiences or in visual media. The concept was invented by Morey Bunin with his spouse Charlotte, Bunin being a puppeteer who had worked with string marionettes and hand puppets. The distinctive feature of an Aniforms character is that it displays a physical form that appears "animated" on a real or simulated television screen. The technique was used in television production.

Machinima

A production technique that can be used to perform digital puppets. Machinima involves creating computer-generated imagery (CGI) using the low-end 3D engines in video games. Players act out scenes in real-time using characters and settings within a game and the resulting footage is recorded and later edited into a finished film. [13]

Related Research Articles

<span class="mw-page-title-main">Animation</span> Method of creating moving pictures

Animation is a filmmaking technique by which still images are manipulated to create moving images. In traditional animation, images are drawn or painted by hand on transparent celluloid sheets (cels) to be photographed and exhibited on film. Animation has been recognized as an artistic medium, specifically within the entertainment industry. Many animations are computer animations made with computer-generated imagery (CGI). Stop motion animation, in particular claymation, has continued to exist alongside these other forms.

<span class="mw-page-title-main">Computer animation</span> Art of creating moving images using computers

Computer animation is the process used for digitally generating animations. The more general term computer-generated imagery (CGI) encompasses both static scenes and dynamic images, while computer animation only refers to moving images. Modern computer animation usually uses 3D computer graphics. The animation's target is sometimes the computer itself, while other times it is film.

<span class="mw-page-title-main">Machinima</span> Use of real-time computer graphics engines to create a cinematic production

Machinima, originally machinema, is the use of real-time computer graphics engines to create a cinematic production. The word "machinima" is a portmanteau of the words machine and cinema. According to Guinness World Records, machinima is the art of making animated narrative films from computer graphics, most commonly using the engines found in video games.

<span class="mw-page-title-main">Puppeteer</span> Person who manipulates a puppet

A puppeteer is a person who manipulates an inanimate object called a puppet to create the illusion that the puppet is alive. The puppet is often shaped like a human, animal, or legendary creature. The puppeteer may be visible to or hidden from the audience.

Visual effects is the process by which imagery is created or manipulated outside the context of a live-action shot in filmmaking and video production. The integration of live-action footage and other live-action footage or CGI elements to create realistic imagery is called VFX.

<span class="mw-page-title-main">Skeletal animation</span> Computer animation technique

Skeletal animation or rigging is a technique in computer animation in which a character is represented in two parts: a surface representation used to draw the character and a hierarchical set of interconnected parts, a virtual armature used to animate the mesh. While this technique is often used to animate humans and other organic figures, it only serves to make the animation process more intuitive, and the same technique can be used to control the deformation of any object—such as a door, a spoon, a building, or a galaxy. When the animated object is more general than, for example, a humanoid character, the set of "bones" may not be hierarchical or interconnected, but simply represent a higher-level description of the motion of the part of mesh it is influencing.

<span class="mw-page-title-main">Lance Williams (graphics researcher)</span> American graphics researcher

Lance J. Williams was a prominent graphics researcher who made major contributions to texture map prefiltering, shadow rendering algorithms, facial animation, and antialiasing techniques. Williams was one of the first people to recognize the potential of computer graphics to transform film and video making.

<span class="mw-page-title-main">Paul Debevec</span> American computer graphics professional

Paul Ernest Debevec is a researcher in computer graphics at the University of Southern California's Institute for Creative Technologies. He is best known for his work in finding, capturing and synthesizing the bidirectional scattering distribution function utilizing the light stages his research team constructed to find and capture the reflectance field over the human face, high-dynamic-range imaging and image-based modeling and rendering.

<span class="mw-page-title-main">Virtual cinematography</span> CGI essentially

Virtual cinematography is the set of cinematographic techniques performed in a computer graphics environment. It includes a wide variety of subjects like photographing real objects, often with stereo or multi-camera setup, for the purpose of recreating them as three-dimensional objects and algorithms for the automated creation of real and simulated camera angles. Virtual cinematography can be used to shoot scenes from otherwise impossible camera angles, create the photography of animated films, and manipulate the appearance of computer-generated effects.

Facial motion capture is the process of electronically converting the movements of a person's face into a digital database using cameras or laser scanners. This database may then be used to produce computer graphics (CG), computer animation for movies, games, or real-time avatars. Because the motion of CG characters is derived from the movements of real people, it results in a more realistic and nuanced computer character animation than if the animation were created manually.

William Todd-Jones is a Welsh puppeteer, puppet designer, performer, director, movement consultant and writer for film, television and theatre in the UK and abroad.

Interactive skeleton-driven simulation is a scientific computer simulation technique used to approximate realistic physical deformations of dynamic bodies in real-time. It involves using elastic dynamics and mathematical optimizations to decide the body-shapes during motion and interaction with forces. It has various applications within realistic simulations for medicine, 3D computer animation and virtual reality.

<span class="mw-page-title-main">3D computer graphics</span> Graphics that use a three-dimensional representation of geometric data

3D computer graphics, sometimes called CGI, 3-D-CGI or three-dimensional computer graphics, are graphics that use a three-dimensional representation of geometric data that is stored in the computer for the purposes of performing calculations and rendering digital images, usually 2D images but sometimes 3D images. The resulting images may be stored for viewing later or displayed in real time.

Gregory Peter Panos is an American writer, futurist, educator, strategic planning consultant, conference / event producer, and technology evangelist in augmented reality, virtual reality, human simulation, motion capture, performance animation, 3D character animation, human-computer interaction, and user experience design.

<span class="mw-page-title-main">Computer graphics</span> Graphics created using computers

Computer graphics deals with by generating images and art with the aid of computers. Today, computer graphics is a core technology in digital photography, film, video games, digital art, cell phone and computer displays, and many specialized applications. A great deal of specialized hardware and software has been developed, with the displays of most devices being driven by computer graphics hardware. It is a vast and recently developed area of computer science. The phrase was coined in 1960 by computer graphics researchers Verne Hudson and William Fetter of Boeing. It is often abbreviated as CG, or typically in the context of film as computer generated imagery (CGI). The non-artistic aspects of computer graphics are the subject of computer science research.

iClone is a real-time 3D animation and rendering software program. Real-time playback is enabled by using a 3D videogame engine for instant on-screen rendering.

The history of computer animation began as early as the 1940s and 1950s, when people began to experiment with computer graphics – most notably by John Whitney. It was only by the early 1960s when digital computers had become widely established, that new avenues for innovative computer graphics blossomed. Initially, uses were mainly for scientific, engineering and other research purposes, but artistic experimentation began to make its appearance by the mid-1960s – most notably by Dr. Thomas Calvert. By the mid-1970s, many such efforts were beginning to enter into public media. Much computer graphics at this time involved 2-D imagery, though increasingly as computer power improved, efforts to achieve 3-D realism became the emphasis. By the late 1980s, photo-realistic 3-D was beginning to appear in film movies, and by mid-1990s had developed to the point where 3-D animation could be used for entire feature film production.

<span class="mw-page-title-main">Computer-generated imagery</span> Application of computer graphics to create or contribute to images

Computer-generated imagery (CGI) is a specific-technology or application of computer graphics for creating or improving images in art, printed media, simulators, videos and video games. These images are either static or dynamic. CGI both refers to 2D computer graphics and 3D computer graphics with the purpose of designing characters, virtual worlds, or scenes and special effects. The application of CGI for creating/improving animations is called computer animation, or CGI animation.

<span class="mw-page-title-main">Virtual human</span> Computer simulation of a person

A virtual human is a software fictional character or human being. Virtual human have been created as tools and artificial companions in simulation, video games, film production, human factors and ergonomic and usability studies in various industries, clothing industry, telecommunications (avatars), medicine, etc. These applications require domain-dependent simulation fidelity. A medical application might require an exact simulation of specific internal organs; film industry requires highest aesthetic standards, natural movements, and facial expressions; ergonomic studies require faithful body proportions for a particular population segment and realistic locomotion with constraints, etc.

An Aniform is a two-dimensional cartoon character operated like a puppet, to be displayed to live audiences or in visual media. The concept was invented by Morey Bunin with his spouse Charlotte, Bunin being a puppeteer who had worked with string marionettes and hand puppets. The distinctive feature of an Aniforms character is that it displays a physical form that appears "animated" on a real or simulated television screen. The technique was used in television production.

References

  1. ↑ A Critical History of Computer Graphics and Animation: Analog approaches, non-linear editing, and compositing Archived 2007-03-28 at the Wayback Machine , accessed April 28, 2007
  2. ↑ Mr. Computer Image Demo (video). 1968.
  3. ↑ Jim Henson's Red Book Entry, accessed October 10, 2014
  4. ↑ Finch, Christopher. Jim Henson: The Works (New York: Random House, 1993)
  5. ↑ Sturman, David J. A Brief History of Motion Capture for Computer Character Animation Archived October 12, 2012, at the Wayback Machine , accessed February 9, 2007
  6. ↑ Henson.com Featured Creature: Waldo C. Graphic (archive.org), accessed February 9, 2007
  7. ↑ Walters, Graham. The story of Waldo C. Graphic. Course Notes: 3D Character Animation by Computer, ACM SIGGRAPH '89, Boston, July 1989, pp. 65-79
  8. ↑ Barbara Robertson, Mike, the talking head Computer Graphics World, July 1988, pp. 15-17.
  9. ↑ Yilmaz, Emre. Elmo's World: Digital Puppetry on Sesame Street. Conference Abstracts and Applications, SIGGRAPH '2001, Los Angeles, August 2001, p. 178
  10. ↑ Kleczek, Jakub (2015). "Digital Puppeteering". Theatr Lalek. POLUNIMA (119).
  11. ↑ Gregory, Rick (July 2014). "Squad Overmatch Study Looks to Build Resilience on the Battlefield" (PDF). Inside PEO STRI. United States Army. Archived from the original (PDF) on October 20, 2016.
  12. ↑ Thuermer, Karen (December 15, 2015). "Avatars for Training". Military Training International. Defense House Publishing.
  13. ↑ Hancock, Hugh (2007). Machinima For Dummies. John Wiley & Sons, Inc. ISBN   978-0-470-19583-3.
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.