 | This article has multiple issues. Please help improve it or discuss these issues on the talk page . (Learn how and when to remove these template messages)
|
A projection augmented model (PA model) is an element sometimes employed in virtual reality systems. It consists of a physical three-dimensional model onto which a computer image is projected to create a realistic looking object. Importantly, the physical model is the same geometric shape as the object that the PA model depicts.
Spatially augmented reality (SAR) renders virtual objects directly within or on the user's physical space. [1] A key benefit of SAR is that the user does not need to wear a head-mounted display. Instead, with the use of spatial displays, wide field of view and possibly high-resolution images of virtual objects can be integrated directly into the environment. For example, the virtual objects can be realized by using digital light projectors to paint 2D/3D imagery onto real surfaces, or by using built-in flat panel displays.
Real objects can be physically handled and naturally manipulated to be viewed from any direction, which is essential for ergonomic evaluation and provides a strong sense of palpability. [2] Although simulated haptic feedback devices enable some aspects of computer-generated objects to be touched, they can not match this level of functionality. [3] It is, therefore, unsurprising that physical objects are still used for many applications, such as product design. [4] However, computer-generated objects have a key advantage; they provide a level of flexibility that cannot be matched by physical objects. Therefore, a display is needed that somehow joins the real physical world and computer-generated objects together, thus enabling them to be experienced simultaneously. [5]
Tangible user interfaces (TUI) and augmented reality both aim to address this issue. TUI systems use real physical objects to both represent and also interact with computer-generated information (Figure 1). However, while TUIs create a physical link between real and computer-generated objects, they do not create the illusion that the computer-generated objects are actually in a user's real environment. That is the aim of augmented reality.
Figure 1 Continuum of advanced computer interfaces, based on Milgram and Kishino (1994).
Unlike virtual reality (VR), which immerses a user in a computer-generated environment, augmented reality (AR) joins together physical and virtual spaces by creating the illusion that computer-generated objects are actually real objects in a user's environment [6] (Figure 1). Furthermore, head-mounted-display based AR and VR systems can directly incorporate physical objects. Thus, as a user reaches out to a computer-generated object that they can see, they touch an equivalent physical model that is placed at the same spatial location. [7] Such systems enable the computer-generated visual appearance of the object to be dynamically altered, while the physical model provides haptic feedback for the object's underlying form. However, head-mounted-display based systems require users to wear equipment, which limits the number of people who can simultaneously use the display.
A variant of the AR paradigm that does not suffer from these limitations is spatially augmented reality (Figure 1). [8] Spatially augmented reality displays project computer-generated information directly into the user's environment. [9] Although there are several possible display configurations, the most natural type is the projection augmented model.
Figure 2 The Projection Augmented model concept
A projection augmented model (PA model) consists of a physical three-dimensional model, onto which a computer image is projected to create a realistic looking object (Figure 2). Importantly, the physical model is the same geometric shape as the object that the PA model depicts. For example, the image projected onto the objects shown in Figure 3 provides colour and visual texture, which makes them appear to be made from different materials.
Figure 3 An example of a Projection Augmented model (inset - with the projection off).
PA models use a unique combination of physical objects and computer-generated information, and hence they inherit advantages from both. “The human interface to a physical model is the essence of ‘intuitive’. There are no widgets to manipulate, no sliders to move, and no displays to look through (or wear). Instead, we walk around objects, moving in and out to zoom, gazing and focusing on interesting components, all at very high visual, spatial, and temporal fidelity”. [10] PA models combine the high level of intuitiveness of physical models with the flexibility and functionality of computer graphics, such as the ability to be quickly altered, animated, saved and updated (Jacucci, Oulasvirta, Psik, Salovaara & Wagner, 2005). Thus, a PA model essentially gives a physical form to a computer-generated object, which a user can touch and grasp with their bare hands. It is therefore unsurprising that user studies, which compared PA models to other Virtual and Augmented Reality displays, found PA models to be a natural and intuitive type of display (Nam & Lee, 2003; Stevens et al., 2002).
However, the PA model concept is not new. In fact, one of the first PA model type displays was created over twenty years ago when Naimark built the ‘Displacements’ art installation (Naimark, 1984) and more recently in the “Haunted Mansion” attraction in Disney World (Liljegren & Foster, 1990). At the time technology did not exist for a PA model to be much more than an artistic statement. However, given the technology available today and a little “unfettered imagination”, exploring novel projection displays is now “potentially boundless”. [11]
The growth in PA model technology has been marked by the recent recreation of Naimark's ‘Displacements’ installation at SIGGRAPH (Displacements, 2005). Specifically, new technology has been developed that semi-automates the process of both creating and aligning the physical model and projected image. This supports multiple projectors, which enables a PA model to be illuminated from every direction. Furthermore, powerful projectors (2000-3000 lumens) can be used to allow a PA model to be located in a well-lit room (Nam, 2005; Umemoro, Keller & Stappers, 2003). However, whilst this technology enables a PA model to be a viable and useful type of display, it does not address its main aim.
A PA model aims to create the illusion of actually being the object that it depicts. For example, when used for a product design application, it is important that a PA model provides a convincing perceptual impression of actually being the final product (Nam, 2006; Saakes, 2006; Verlinden, Horváth & Edelenbos, 2006; Keller & Stappers, 2001). Similarly, when used for a museum display application to create a replica of an artefact, a PA model aims to create the illusion of being the real artefact (Hirooka & Satio, 2006; Senckenberg Museum, 2006; Bimber, Gatesy, Witmer, Raskar & Encarnacao, 2002; Museum of London, 1999).
However, no previous research has specifically considered this illusion. Therefore, this thesis defines the ‘Projection Augmented model illusion’ as the situation in which a PA model is perceived to actually be the object that it depicts. For example, this illusion occurs when a user perceives the PA model in Figure 3 to be real bricks, flower pots, and pieces of wood, as opposed to white models with an image projected onto them. However, the essence of this illusion does not involve deceiving the user. A user can perceive a PA model to be the object that it depicts, whilst knowing that it is actually a white model and a projected image.
Technology has been developed to enhance this illusion by increasing the physical similarity between the PA model and the object that it depicts, or in other words, increasing the fidelity of the PA model. For example, the way in which the specular highlights on an object move as the viewer changes position can be dynamically simulated. This enables a PA model to appear to be made from a wide range of materials. For example, a dull clay vase can appear to be made from a shiny plastic material.
However, whether or not the PA model illusion occurs is entirely dependent on a user's subjective perceptual impression. Therefore, increasing the fidelity of different aspects of a PA model may each have a different effect on the strength of the illusion. This is essentially the same as the way in which increasing the fidelity of different aspects of a computer-generated photorealistic image, may each have a different effect on the degree to which the image is perceived to be a real photograph (Longhurst, Ledda & Chalmers, 2003; Rademacher, Lengyel, Cutrell, & Whitted, 2001). For example, increasing the fidelity of the textures in the image may typically be more important than increasing the fidelity of the shadows. It cannot therefore be assumed that increasing the fidelity of any aspect of a PA model will automatically strengthen the PA model illusion, and similarly it cannot be assumed that decreasing the fidelity of any aspect will automatically weaken it. Therefore, given that no previous research has investigated this illusion, it is difficult to determine the success of the technology that aims to enhance it, and difficult to make informed decisions when developing new technology. The capabilities of the human perceptual system should guide the development of any advanced interface (Stanney et al., 2004), hence this issue needs to be addressed.
Note: Projection Augmented models are sometimes referred to as 'Shader Lamps' (Raskar, Welch, Low & Bandyopadhyay, 2001, p. 89).
Rendering or image synthesis is the process of generating a photorealistic or non-photorealistic image from a 2D or 3D model by means of a computer program. The resulting image is referred to as the render. Multiple models can be defined in a scene file containing objects in a strictly defined language or data structure. The scene file contains geometry, viewpoint, textures, lighting, and shading information describing the virtual scene. The data contained in the scene file is then passed to a rendering program to be processed and output to a digital image or raster graphics image file. The term "rendering" is analogous to the concept of an artist's impression of a scene. The term "rendering" is also used to describe the process of calculating effects in a video editing program to produce the final video output.
Virtual reality (VR) is a simulated experience that employs 3D near-eye displays and pose tracking to give the user an immersive feel of a virtual world. Applications of virtual reality include entertainment, education and business. VR is one of the key technologies in the reality-virtuality continuum. As such, it is different from other digital visualization solutions, such as augmented virtuality and augmented reality.
Augmented reality (AR) is an interactive experience that combines the real world and computer-generated content. The content can span multiple sensory modalities, including visual, auditory, haptic, somatosensory and olfactory. AR can be defined as a system that incorporates three basic features: a combination of real and virtual worlds, real-time interaction, and accurate 3D registration of virtual and real objects. The overlaid sensory information can be constructive, or destructive. As such, it is one of the key technologies in the reality-virtuality continuum.
A cave automatic virtual environment is an immersive virtual reality environment where projectors are directed to between three and six of the walls of a room-sized cube. The name is also a reference to the allegory of the Cave in Plato's Republic in which a philosopher contemplates perception, reality, and illusion.
A light field is a vector function that describes the amount of light flowing in every direction through every point in a space. The space of all possible light rays is given by the five-dimensional plenoptic function, and the magnitude of each ray is given by its radiance. Michael Faraday was the first to propose that light should be interpreted as a field, much like the magnetic fields on which he had been working. The term light field was coined by Andrey Gershun in a classic 1936 paper on the radiometric properties of light in three-dimensional space.
Mixed reality (MR) is a term used to describe the merging of a real-world environment and a computer-generated one. Physical and virtual objects may co-exist in mixed reality environments and interact in real time.
A 3D display is a display device capable of conveying depth to the viewer. Many 3D displays are stereoscopic displays, which produce a basic 3D effect by means of stereopsis, but can cause eye strain and visual fatigue. Newer 3D displays such as holographic and light field displays produce a more realistic 3D effect by combining stereopsis and accurate focal length for the displayed content. Newer 3D displays in this manner cause less visual fatigue than classical stereoscopic displays.
Real-time computer graphics or real-time rendering is the sub-field of computer graphics focused on producing and analyzing images in real time. The term can refer to anything from rendering an application's graphical user interface (GUI) to real-time image analysis, but is most often used in reference to interactive 3D computer graphics, typically using a graphics processing unit (GPU). One example of this concept is a video game that rapidly renders changing 3D environments to produce an illusion of motion.
The Electronic Visualization Laboratory (EVL) is an interdisciplinary research lab and graduate studies program at the University of Illinois at Chicago, bringing together faculty, students and staff primarily from the Art and Computer Science departments of UIC. The primary areas of research are in computer graphics, visualization, virtual and augmented reality, advanced networking, and media art. Graduates of EVL either earn a Masters or Doctoral degree in Computer Science.
3D computer graphics, sometimes called CGI, 3-D-CGI or three-dimensional computer graphics, are graphics that use a three-dimensional representation of geometric data that is stored in the computer for the purposes of performing calculations and rendering digital images, usually 2D images but sometimes 3D images. The resulting images may be stored for viewing later or displayed in real time.
Immersion into virtual reality (VR) is the perception of being physically present in a non-physical world. The perception is created by surrounding the user of the VR system in images, sound or other stimuli that provide an engrossing total environment.
In computer vision and computer graphics, 3D reconstruction is the process of capturing the shape and appearance of real objects. This process can be accomplished either by active or passive methods. If the model is allowed to change its shape in time, this is referred to as non-rigid or spatio-temporal reconstruction.
Lawrence Jay Rosenblum is an American mathematician, and Program Director for Graphics and Visualization at the National Science Foundation.
Computer graphics deals with by generating images and art with the aid of computers. Today, computer graphics is a core technology in digital photography, film, video games, digital art, cell phone and computer displays, and many specialized applications. A great deal of specialized hardware and software has been developed, with the displays of most devices being driven by computer graphics hardware. It is a vast and recently developed area of computer science. The phrase was coined in 1960 by computer graphics researchers Verne Hudson and William Fetter of Boeing. It is often abbreviated as CG, or typically in the context of film as computer generated imagery (CGI). The non-artistic aspects of computer graphics are the subject of computer science research.
In computing, 3D interaction is a form of human-machine interaction where users are able to move and perform interaction in 3D space. Both human and machine process information where the physical position of elements in the 3D space is relevant.
Computer-generated imagery (CGI) is a specific-technology or application of computer graphics for creating or improving images in art, printed media, simulators, videos and video games. These images are either static or dynamic. CGI both refers to 2D computer graphics and 3D computer graphics with the purpose of designing characters, virtual worlds, or scenes and special effects. The application of CGI for creating/improving animations is called computer animation, or CGI animation.
Projection mapping, similar to video mapping and spatial augmented reality, is a projection technique used to turn objects, often irregularly shaped, into display surfaces for video projection. The objects may be complex industrial landscapes, such as buildings, small indoor objects, or theatrical stages. Using specialized software, a two- or three-dimensional object is spatially mapped on the virtual program which mimics the real environment it is to be projected on. The software can then interact with a projector to fit any desired image onto the surface of that object. The technique is used by artists and advertisers who can add extra dimensions, optical illusions, and notions of movement onto previously static objects. The video is commonly combined with or triggered by audio to create an audiovisual narrative. In recent years the technique has also been widely used in the context of cultural heritage, as it has proved to be an excellent edutainment tool.
Oliver Bimber is a German computer scientist. He is professor for computer graphics at the Johannes Kepler University Linz, Austria where he heads the Institute of Computer Graphics.
Visual computing is a generic term for all computer science disciplines dealing with images and 3D models, such as computer graphics, image processing, visualization, computer vision, virtual and augmented reality and video processing. Visual computing also includes aspects of pattern recognition, human computer interaction, machine learning and digital libraries. The core challenges are the acquisition, processing, analysis and rendering of visual information. Application areas include industrial quality control, medical image processing and visualization, surveying, robotics, multimedia systems, virtual heritage, special effects in movies and television, and computer games.
Industrial augmented reality (IAR) is related to the application of augmented reality (AR) and heads-up displays to support an industrial process. The use of IAR dates back to the 1990s with the work of Thomas Caudell and David Mizell about the application of AR at Boeing. Since then several applications of this technique over the years have been proposed showing its potential in supporting some industrial processes. Although there have been several advances in technology, IAR is still considered to be at an infant developmental stage.
Azuma, R., Baillot, Y., Behringer, R., Feiner, S., Julier, S., & MacIntyre, B. (2001). Recent Advances in Augmented Reality. IEEE Computer Graphics and Applications, 21(6), 34-47.
Baradaran, H., & Stuerzlinger, W. (2005). A Comparison of Real and Virtual 3D Construction Tools with Novice Users. In Proceedings of International Conference on Computer Graphics & Virtual Reality – CGVR’06 – part of 2006 World Congress in Computer Science, Computer Engineering, and Applied Computing - WORLDCOMP'06. World Academy of Science.
Billingshurst, M., Grasset, R., & Looser, J. (2005). Designing Augmented Reality Interfaces [ dead link ]. In Proceedings of Annual Conference on Computer Graphics and Interactive Techniques – SIGGRAPH’05 (pp. 17–22). New York: ACM Press.
Bimber, O., Gatesy, S., Witmer, L., Raskar, R., & Encarnacao, L. (2002). Merging Fossil Specimens with Computer-Generated Information. IEEE Computer, 35(9), 25-30.
Bimber, O., & Raskar, R. (2005). Spatial Augmented Reality: A Modern Approach to Augmented Reality. In Proceedings of Annual Conference on Computer Graphics and Interactive Techniques - SIGGRAPH’05. New York: ACM Press.
Borst, C., & Volz, R. (2005). Evaluation of a Haptic Mixed Reality System for Interactions with a Virtual Control Panel. Presence: Teleoperators and Virtual Environments, 14(6), 677-696.
Brooks, F. (1999). What's real about virtual reality? IEEE Computer Graphics and Applications, 19(6), 16-27.
Burdea, G., & Coffet, P. (2003). Virtual Reality Technology, 2nd Edition. Washington: Wiley-IEEE Press.
Cruz-Neira, C., Sandin, D., & DeFanti, T. (1993). Surround-screen projection-based virtual reality: the design and implementation of the CAVE. In Proceedings of Annual Conference on Computer Graphics and Interactive Techniques - SIGGRAPH’93 (pp. 135–142). New York: ACM Press.
"Michael Naimark: Interactive and Immersive Film Environments, 1977–1997. An Exhibition at Annual Conference on Computer Graphics and Interactive Techniques – SIGGRAPH'05". 2005. Archived from the original on 2015-11-05. Retrieved 2024-03-05.
Drettakis, G., Roussou, M., Tsingos, N., Reche, A., & Gallo, E. (2004). Image-based Techniques for the Creation and Display of Photorealistic Interactive Virtual Environments. In Proceedings of the 10th Eurographics Symposium on Virtual Environments – EGVE’04 (pp. 157–166).
Dutson, A., & Wood, K. (2005). Using rapid prototypes for functional evaluation of evolutionary product designs. Rapid Prototyping Journal, 11 (3), 125-11.
Evans, M., Wallace, D., Cheshire, D., & Sener, B. (2005). An evaluation of haptic feedback modelling during industrial design practice. Design Studies, 26,487-508.
"CAVE: The Most Widely Installed Fully Immersive Visualization System in the World". FakeSpace. 2006. Archived from the original on 2008-01-08. Retrieved 2006-09-20.
Fischer, J., Bartz, D., & Straßer, W. (2006). Enhanced Visual Realism by Incorporating Camera Image Effects. In Proceedings of International Symposium on Mixed and Augmented Reality - ISMAR’06. Washington: IEEE Computer Society Press.
Gibson, I., Gao, Z., & Campbell, I. (2004). A Comparative Study of Virtual prototyping and Physical Prototyping. International Journal of Manufacturing Technology and Management, 6(6), 503-522.
Hirooka, S., & Saito, H. (2006). Calibration Free Virtual Display System Using Video Projector onto Real Object Surface. IEICE-Transactions on Info and Systems - Special Section on Artificial Reality and Telexistence, E89-D(1), 88-97.
Hoffman, H., Garcia-Palacios, A., Carlin, C., Furness, T., Botella-Arbona, C. (2003). Interfaces that heal: Coupling real and virtual objects to cure spider phobia. International Journal of Human-Computer Interaction, 16, 283-300.
Ichida, H., Itoh, Y., Kitamura, Y., & Kishino, F. (2004). ActiveCube and its 3D Applications. In Proceedings of IEEE Virtual Reality Conference – VR’04. Washington: IEEE Computer Society Press.
Ishii, H., & Ullmer, B. (1997). Tangible Bits: Towards Seamless Interfaces between People, Bits and Atoms. In Proceedings Conference on Human Factors in Computing Systems – CHI-97 (pp. 234–241). New York: ACM Press.
Ishii, H., & Ullmer, B. (2001). Emerging Framework for Tangible User Interfaces. In J. Carroll (Eds.), Human-Computer Interaction in the New Millennium (pp. 579–601). Addison-Wesley.
Jacucci, G., Oulasvirta, A., Psik, T., Salovaara, A., & Wagner, I. (2005). Augmented reality painting and collage: Evaluating tangible interaction in a field study. In Proceedings of Tenth IFIP-TC13 International Conference on Human-Computer Interaction INTERACT'05 (pp. 43–56).
Keller, I., & Stappers, P. (2001). TRI: Inspiration Support for a design studio environment. International Journal of Design Computing, 3, 1-17.
Khoudja M., Hafez M., & Kheddar A. (2004). Tactile Interfaces. A State of the Art Survey. In Proceedings of 35th International Symposium on Robotics (pp. 721–726).
Kölsch, M., Bane, R., Höllerer, T., & Turk, M. (2006). Multimodal interaction with a wearable augmented reality system. IEEE Computer Graphics and Applications, 26(3), 62 -71.
Lee, S., Chen, T., Kim, J., Han, S., & Pan, Z. (2004). Affective Property Evaluation of Virtual Product Designs. In Proceedings of IEEE Virtual Reality Conference – VR’04 (pp. 207–216). Washington: IEEE Computer Society Press.
Lee, W., & Park, J. (2006) Augmented Foam: Touchable and Graspable Augmented Reality for Product Design Simulation. Bulletin of Japanese Society for the Design Science, 52(6), 17-26.
Liljegren, G., & Foster, E. (1990). Figure with Back Projected Image Using Fibre Optics. US Patent # 4,978.216, Walt Disney Company, Burbank California, USA, December 18, 1990.
Longhurst, P., Ledda, P., & Chalmers, A. (2003). Psychophysically based artistic techniques for increased perceived realism of virtual environments, In Proceedings of Proceedings of the 4th International Conference on Computer Graphics, Virtual Reality, Visualisation and Interaction in Africa - AFRIGRAPH '03 (pp. 123–132). New York: ACM Press.
Milgram, P., & Kishino, F. (1994). A taxonomy of mixed reality visual displays. IEICE Transactions on Information and Systems Special Issue on Networked Reality (E77D), 12, 1321-1329.
Naimark, M. (2005). Two Unusual Projection Spaces. Presence: Teleoperators and Virtual Environments, Special Issue on Projection, 14(5), 597-506.
Naimark, M. (1984). "'Displacements'. An exhibit at San Francisco Museum of Modern Art". www.naimark.net. Retrieved 2006-09-20.
Nam, T. (2005). Sketch-Based Rapid Prototyping Platform for Hardware-Software Integrated Interactive Products. In Proceedings of the Third Symposium on Applied Perception in Graphics and Visualization at SIGGRAPH – APGV’05 (pp. 1689–1692). New York: ACM Press.
Nam. T. (2006). Sketching for Hardware Software Integrated Interactive Product Design. In Proceedings Conference on Human Factors in Computer Systems - CHI’06, Workshop on “Sketching" Nurturing Creativity: Commonalities in Art, Design, Engineering and Research. New York: ACM Press.
Nam, T., & Lee, W. (2003). Integrating hardware and software: augmented reality based on prototyping method for digital products. In Proceedings of Conference on Human Factors in Computing Systems CHI’03 (pp. 956–957). New York: ACM Press.
Ni, T., Schmidt, G., Staadt, O., Livingston, M., Ball, R., & May, R. (2006). A Survey of Large High-Resolution Display Technologies, Techniques, and Applications. In Proceedings of IEEE Virtual Reality Conference – VR’06 (pp. 223–236). Washington: IEEE Computer Society Press.
Rademacher, P., Lengyel, J., Cutrell, E., & Whitted, T. (2001). Measuring the perception of visual realism in images. In Proceedings of the 12th Eurographics Workshop on Rendering Techniques (pp. 235–248). Springer.
Raskar, R., Welch, G., Low K., & Bandyopadhyay, D. (2001). Shader Lamps: Animating Real Objects With Image-Based Illuminations. In Proceedings of the 12th Eurographics Workshop on Rendering Techniques (pp. 89–102). Springer.
Saakes, D. (2006). Material light: exploring expressive materials. Personal Ubiquitous Computing, 10(2), 144-147.
"Dinosaur Fossil Exhibit" (PDF). Senckenberg Museum. 2006. Archived from the original (PDF) on 2006-05-17. Retrieved 2006-09-20.
Stanney, K., Samman, S., Reeves, L., Hale, K., Buff, W., Bowers, C., Goldiez, B., Nicholson, D., & Lackey, S. (2004). A paradigm shift in interactive computing: Deriving multimodal design principles from behavioural and neurological foundations. International Journal of Human-Computer Interaction, 17(2), 229-257.
Stevens, B., Jerrams-Smith, J., Heathcote, D., & Callear, D. (2002). Putting the Virtual into Reality: Assessing Object-Presence with Projection-Augmented Models. Presence: Teleoperators and Virtual Environments, 11(1), 79-92.
Umemoro, H., Keller, I., & Stappers, P. (2003). More light on your table: Table-sized Sketchy VR in support of fluid collaboration. In Proceedings of the 6th Asian Design International Conference.
Verlinden, J., Horváth, I., & Edelenbos, E. (2006). Treatise of technologies for interactive augmented prototyping. Proceedings of the 7th International Symposium on Tools and Methods of Competitive Engineering – TMCE’06. Rotterdam: Millpress.
Whitton, M., Lok, B., Insko, B., & Brooks, F. (2005). Integrating Real and Virtual Objects in Virtual Environments – Invited Paper. In Proceedings of HCI International Conference.
Bennett, E., & Stevens, B. (2006). The effect that the visual and haptic problems associated with touching a Projection Augmented model have on object-presence. Journal of Presence: Teleoperators and Virtual Environments, special edition of the best papers from the International Presence Conference, 15(4), 419-437, MIT Press.
Bennett, E., & Stevens, B. (2006). The ‘Detection, Perception and Object-Presence framework’: A unified structure for investigating illusory representations of reality. In Proceedings of SIGGRAPH's Computer Graphics and Applied Perception Symposium.
