Computer-mediated reality

Last updated
The MannGlas "Digital Eye Glass" welding helmet uses cameras and high-dynamic-range imaging to augment the user's view in dark areas and diminish it in bright areas. MannGlass welding helmet.jpg
The MannGlas "Digital Eye Glass" welding helmet uses cameras and high-dynamic-range imaging to augment the user's view in dark areas and diminish it in bright areas.
Art installation illustrating the mediated reality concept. First we display what's really there, and then this allows a computer to be inserted into the "reality stream" to modify it. MediatedReality videocam with tv on easel at niagara falls.jpg
Art installation illustrating the mediated reality concept. First we display what's really there, and then this allows a computer to be inserted into the "reality stream" to modify it.
Mediated Reality application running on Apple iPhone MediatedReality on iPhone2009 07 13 21 33 39.jpg
Mediated Reality application running on Apple iPhone

Computer-mediated reality refers to the ability to add to, subtract information from, or otherwise manipulate one's perception of reality through the use of a wearable computer or hand-held device [1] such as a smartphone.

Contents

Mediated reality is a proper superset of mixed reality, augmented reality, and virtual reality, as it also includes, for example, diminished reality. [2]

Typically, it is the user's visual perception of the environment that is mediated. This is done through the use of some kind of electronic device, such as an EyeTap device or smart phone, which can act as a visual filter between the real world and what the user perceives. Computer-mediated reality has been used to enhance visual perception as an aid to the visually impaired. [3] This example achieves a mediated reality by altering a video input stream light that would have normally reached the user's eyes, and computationally altering it to filter it into a more useful form. It has also been used for interactive computer interfaces. [4]

The use of computer-mediated reality to diminish perception, by the removal or masking of visual data, has been used for architectural applications, and is an area of ongoing research. [5]

The long-term effects of altering perceived reality have not been thoroughly studied, and negative side effects of long-term exposure might be possible.[ citation needed ] Short term effects have been demonstrated with the eyestrain caused by computers. [6]

As a seeing aid

In the 1970s and 1980s, Steve Mann introduced the Generation-1 and Generation-2 "Digital Eye Glass", initially as a vision aid to help people see better, as a welding helmet, [7] [8] [9] [10] and as a general-purpose seeing aid for everyday life as outlined in IEEE Technology & Society 31(3) [11] and the supplemental material entitled "GlassEyes". [12]

In this sense, mediated reality is a proper superset of mixed reality, augmented reality, and virtual reality, as it also includes, for example, diminished reality. [13]

Window managers

One common window manager in mediated reality is the "Reality Window Manager". [14]

Wireless mediated reality

With wireless communications, mediated reality can also become a communications medium among different communities. [15] For example, Bluetooth devices are often used with mediated reality. [16] With the use of EyeTap, such interaction is called "seeing eye-to-eye". [17]

Applications

Applications of mediated reality include devices that help people see better, as well as devices for gaming and equipment repair, telemedicine, remote expert advice interfaces, and wayfinding. Mediated reality is also used in robotics [18] and drawing applications such as the "Loose and Sketchy" drawing package. [19]

One key application of computer-mediated reality is healthcare and medicine, which has become a popular research area, specifically beginning in the 1990s with the field growing larger over time. [20] Common research topics include applications of computer-mediated reality in surgery, diagnosing diseases generally, and aiding care of neurodegenerative diseases like Parkinson's. [20] In surgery, studies have shown that the use of virtual reality simulations can be used to reduce error, improve efficiency, and be used generally as training. [20] Other applications in surgery include modeling, which allows for more extensive planning of surgeries prior to the procedure that can also be more personalized to the patient, and image-guided surgery, in which machines and overlays would allow for more accuracy in the surgical processes. [20] There have even been complete telesurgeries, where the surgeon operates on a 3D model of the patient while a robot executes the actions. [20] This has been used for basic surgeries on swine and more advanced surgeries on swine and humans. [20] With neurodegenerative diseases, virtual reality has been used to simulate situations that train memory but aren't reproduce in a standard treatment environment. [20] Virtual reality may also be used for interfacing with a patient in their home, with data sent directly to the physician, or creating games that would encourage these exercises. [20] Virtual reality has also been used to aid those with a fear of heights, anxiety, depression, and autism. [20] It has also been used to reduce patients' pain. [20]

A widely implemented development of computer-mediated reality in medicine is the invention of electronic consultation, such as services like Teladoc. [21]

Within video games, the most common form of computer-mediated reality is virtual reality headsets, such as Oculus Rift. [22] These are devices that attach to the user's head and immerse their field of vision solely within the game world. [22] Controllers used for virtual reality games range from traditionally designed controllers to controllers designed to track the motions of the user. [22] However, alongside the uses previously mention in medical fields, [20] virtual reality has gained notoriety with Meta, formerly known as Facebook, [23] and the introduction of their "metaverse." [24] The metaverse is intended to be designed in a similar fashion to massive multiplayer online games, where users can interact with each other in an interconnected virtual space. [23] It has already been subject to controversy, as a woman claimed she was assaulted in the Metaverse. [25] In addition to virtual reality, augmented reality has also been used for video games. The most prominent example is Pokémon Go , an augmented reality game for mobile devices where Pokémon creatures are displayed through the phone to appear as if they are part of the real world. [26] Other examples of AR games include Harry Potter: Wizards Unite and Ingress , which was developed by the same developer behind Pokémon Go.

Augmented reality technology has similarly been applied with social media applications such as Snapchat. These lenses range from adding 3D digital objects to pictures to altering the appearance of those in the photos. [27] [28] [29] One of the most common types are beauty filters, which digitally alter the user's appearance to make the user appear more "beautiful." [30] Beauty filters have been correlated to lower self esteem. [31]

Mixed reality and augmented reality are special cases of mediated reality. Viraugmixmodmediated reality.png
Mixed reality and augmented reality are special cases of mediated reality.

Mediated reality is related to other concepts such as augmented reality (which is a special case of mediated reality), virtual reality, mixed reality, etc. [32]

See also

Related Research Articles

<span class="mw-page-title-main">Virtual reality</span> Computer-simulated experience

Virtual reality (VR) is a simulated experience that employs 3D near-eye displays and pose tracking to give the user an immersive feel of a virtual world. Applications of virtual reality include entertainment, education and business. VR is one of the key technologies in the reality-virtuality continuum. As such, it is different from other digital visualization solutions, such as augmented virtuality and augmented reality.

<span class="mw-page-title-main">Augmented reality</span> View of the real world with computer-generated supplementary features

Augmented reality (AR) is an interactive experience that combines the real world and computer-generated 3D content. The content can span multiple sensory modalities, including visual, auditory, haptic, somatosensory and olfactory. AR can be defined as a system that incorporates three basic features: a combination of real and virtual worlds, real-time interaction, and accurate 3D registration of virtual and real objects. The overlaid sensory information can be constructive, or destructive. As such, it is one of the key technologies in the reality-virtuality continuum.

<span class="mw-page-title-main">Mixed reality</span> Merging of real and virtual worlds to produce new environments

Mixed reality (MR) is a term used to describe the merging of a real-world environment and a computer-generated one. Physical and virtual objects may co-exist in mixed reality environments and interact in real time.

<span class="mw-page-title-main">Head-mounted display</span> Type of display device

A head-mounted display (HMD) is a display device, worn on the head or as part of a helmet, that has a small display optic in front of one or each eye. HMDs have many uses including gaming, aviation, engineering, and medicine.

<span class="mw-page-title-main">Virtual keyboard</span> Software component

A virtual keyboard is a software component that allows the input of characters without the need for physical keys. Interaction with a virtual keyboard happens mostly via a touchscreen interface, but can also take place in a different form when in virtual or augmented reality.

<span class="mw-page-title-main">Immersion (virtual reality)</span> Perception of being physically present in a non-physical world

In virtual reality (VR), immersion is the perception of being physically present in a non-physical world. The perception is created by surrounding the user of the VR system in images, sound or other stimuli that provide an engrossing total environment.

A projection augmented model is an element sometimes employed in virtual reality systems. It consists of a physical three-dimensional model onto which a computer image is projected to create a realistic looking object. Importantly, the physical model is the same geometric shape as the object that the PA model depicts.

Metaio GmbH was a privately held augmented reality (AR) company that was acquired by Apple Inc. in May 2015. Headquartered in Munich, Germany, with subsidiaries in San Francisco, California, New York City, New York, and Dallas, Texas, Metaio provided a software development kit (SDK) for programming PC, web, mobile applications, and custom offline augmented reality applications. Additionally, Metaio was the creator of Junaio, a free mobile AR browser available for Android and iOS devices.

<span class="mw-page-title-main">Smartglasses</span> Wearable computers glasses

Smartglasses or smart glasses are eye or head-worn wearable computers. Many smartglasses include displays that add information alongside or to what the wearer sees. Alternatively, smartglasses are sometimes defined as glasses that are able to change their optical properties, such as smart sunglasses that are programmed to change tint by electronic means. Alternatively, smartglasses are sometimes defined as glasses that include headphone functionality.

Dieter Schmalstieg is an Austrian computer scientist, full professor, and head of the Institute of Computer Graphics and Vision (ICG) at Graz University of Technology. In 1993 he received a master of science diploma and in 1997 the degree of doctor of technical sciences. Currently he has over 300 peer-reviewed works which were cited over 20,000 times which brought him an h-index of 70.

<span class="mw-page-title-main">Windows Mixed Reality</span> Mixed reality platform

Windows Mixed Reality (WMR) is a discontinued platform by Microsoft which provides augmented reality and virtual reality experiences with compatible head-mounted displays.

<span class="mw-page-title-main">Microsoft HoloLens</span> Mixed reality smartglasses

Microsoft HoloLens is an augmented reality (AR)/mixed reality (MR) headset developed and manufactured by Microsoft. HoloLens runs the Windows Mixed Reality platform under the Windows 10 operating system. Some of the positional tracking technology used in HoloLens can trace its lineage to the Microsoft Kinect, an accessory for Microsoft's Xbox 360 and Xbox One game consoles that was introduced in 2010.

VPL Research was one of the first companies that developed and sold virtual reality products. It was founded by computer scientist Jaron Lanier in 1984. "VPL" stood for "Virtual Programming Languages". In 1990, VPL Research filed for bankruptcy and in 1998 all of its patents were bought by Sun Microsystems.

<span class="mw-page-title-main">Virtual reality headset</span> Head-mounted device that provides virtual reality for the wearer

A virtual reality headset is a head-mounted device that uses 3D near-eye displays and positional tracking to provide a virtual reality environment for the user. VR headsets are widely used with VR video games, but they are also used in other applications, including simulators and trainers. VR headsets typically include a stereoscopic display, stereo sound, and sensors like accelerometers and gyroscopes for tracking the pose of the user's head to match the orientation of the virtual camera with the user's eye positions in the real world. Augmented reality (AR) headsets are VR headsets that enable the user to see and interact with the outside world. Examples of AR headsets include the Apple Vision Pro and Meta Quest 3.

<span class="mw-page-title-main">Industrial augmented reality</span>

Industrial augmented reality (IAR) is related to the application of augmented reality (AR) and heads-up displays to support an industrial process. The use of IAR dates back to the 1990s with the work of Thomas Caudell and David Mizell about the application of AR at Boeing. Since then several applications of this technique over the years have been proposed showing its potential in supporting some industrial processes. Although there have been several advances in technology, IAR is still considered to be at an infant developmental stage.

<span class="mw-page-title-main">Pose tracking</span> Detection of a virtual/augmented reality users pose

In virtual reality (VR) and augmented reality (AR), a pose tracking system detects the precise pose of head-mounted displays, controllers, other objects or body parts within Euclidean space. Pose tracking is often referred to as 6DOF tracking, for the six degrees of freedom in which the pose is often tracked.

<span class="mw-page-title-main">Dancing Hot Dog</span> Virtual character

The Dancing Hot Dog is the name often used to refer to a character and an Internet meme that originated in 2017, after the Snapchat mobile app released an augmented reality camera lens that includes an animated rendering of a dancing anthropomorphic hot dog.

GoMeta is an American software company headquartered in San Diego. The company was founded by Dmitry Shapiro, Sean Thielen, and Jonathan Miller in September 2016.

<span class="mw-page-title-main">Filter (social media)</span> Effects used to alter the appearance of a digital image

Filters are digital image effects often used on social media. They initially simulated the effects of camera filters, and they have since developed with facial recognition technology and computer-generated augmented reality. Social media filters—especially beauty filters—are often used to alter the appearance of selfies taken on smartphones or other similar devices. While filters are commonly associated with beauty enhancement and feature alterations, there is a wide range of filters that have different functions. From adjusting photo tones to using face animations and interactive elements, users have access to a range of tools. These filters allow users to enhance photos and allow room for creative expression and fun interactions with digital content.

<span class="mw-page-title-main">Snapchat dysmorphia</span> Psychological phenomenon

Snapchat dysmorphia, also known as "selfie dysmorphia", is a trending phenomenon used to describe patients who seek out plastic surgery in order to replicate and appear like their filtered selfies or altered images of themselves. The increasing availability and variety of filters used on social media apps, such as Snapchat or Instagram, allow users to edit and apply filters to their photos in an instant – blemish the skin, narrow the nose, enlarge the eyes, and numerous other edits to one's facial features. These heavily edited images create unrealistic and unnatural expectations of one's appearance, showing users a "perfected" view of themselves. The disconnection between one's real-life appearance and the highly filtered versions of oneself manifest into body insecurity and dysmorphia. The distorted perception of oneself can potentially evolve into an obsessive preoccupation with perceived flaws in one's appearance, a mental disorder known as body dysmorphic disorder. BDD has been classified as part of the obsessive-compulsive spectrum and it is currently affecting one in 50 Americans.

References

  1. ↑ "Wearable, Tetherless, Computer-Mediated Reality", Technical Report #260, M.I.T. Medial Lab Perceptual Computing Section, Cambridge, Massachusetts, 1994
  2. ↑ Mann, S., & Fung, J. (2001). Videoorbits on EyeTap devices for deliberately diminished reality or altering the visual perception of rigid planar patches of a real world scene. Proceedings of the Second IEEE International Symposium on Mixed Reality, pp 48-55, March 14–15, 2001.
  3. ↑ Video mediation
  4. ↑ Grasset, R.; Gascuel, J. -D.; Schmalstieg (2003). "Interactive Mediated Reality". The Second IEEE and ACM International Symposium on Mixed and Augmented Reality, 2003. Proceedings. ISMAR 2003. pp. 302–303. CiteSeerX   10.1.1.109.7275 . doi:10.1109/ISMAR.2003.1240731. ISBN   978-0-7695-2006-3. S2CID   2007012.
  5. ↑ See, for instance, here and here Archived 2005-11-30 at the Wayback Machine
  6. ↑ "How to Avoid Eye Strain Caused by Increased Screen Time Amid the COVID-19 Pandemic". healthcare.utah.edu. Retrieved 2022-06-03.
  7. ↑ Quantigraphic camera promises HDR eyesight from Father of AR, by Chris Davies, SlashGear, Sep 12th 2012
  8. ↑ IEEE Spectrum
  9. ↑ IEEE Computer
  10. ↑ A magical welding helmet that lets you see the world in HDR–in real-time
  11. ↑ Through the Glass, Lightly, IEEE Technology & Society, Volume 31, Number 3, Fall 2012, pages 10-14
  12. ↑ "GlassEyes": The Theory of EyeTap Digital Eye Glass, supplemental material for "Through the Glass, Lightly", IEEE Technology and Society, Vol. 31, No. 3, Fall 2012
  13. ↑ Mann, S., & Fung, J. (2001). Videoorbits on EyeTap devices for deliberately diminished reality or altering the visual perception of rigid planar patches of a real world scene. Proceedings of the Second IEEE International Symposium on Mixed Reality, pp 48-55, March 14–15, 2001.
  14. ↑ Mediated Reality, by Steve Mann, Linux Journal, Article No5, Issue59, 1999march01, ISSN   1075-3583
  15. ↑ Guo, Z.; Zhu, J. J. H.; Chen, H. (2001). "Mediated Reality Bites: Comparing Direct and Indirect Experience as Sources of Perceptions Across Two Communities in China". Int. J. Public Opin. Res. 13 (4): 398–418. doi:10.1093/ijpor/13.4.398.
  16. ↑ Mediated Reality Bluetooth Device Locator Archived 2011-07-06 at the Wayback Machine by S Parikh et al., IEEE CSIDC 2002
  17. ↑ Tang, F.; Aimone, C.; Fung, J.; Marjan, A.; Mann, S. (2002). "Seeing Eye to Eye: a shared mediated reality using EyeTap devices and the VideoOrbits Gyroscopic Head Tracker" (PDF). Proceedings. International Symposium on Mixed and Augmented Reality. ISMAR’02. pp. 267–268. CiteSeerX   10.1.1.134.7981 . doi:10.1109/ISMAR.2002.1115106. ISBN   978-0-7695-1781-0. S2CID   707231. Archived from the original (PDF) on 2016-03-04. Retrieved 2015-09-04.
  18. ↑ Daniel Suarez: Daemon: Bot-mediated Reality [usurped] (discussion of Mediated and Augmented Reality, bots, etc.)
  19. ↑ Haller, M.; Landerl, F.; Billinghurst, M. (2005). "A Loose and Sketchy Approach in a Mediated Reality Environment". Proceedings of the 3rd international conference on Computer graphics and interactive techniques in Australasia and South East Asia. GRAPHITE '05. pp. 371–379. CiteSeerX   10.1.1.61.6181 . doi:10.1145/1101389.1101463. ISBN   978-1-59593-201-3. S2CID   2899320.
  20. 1 2 3 4 5 6 7 8 9 10 11 Yeung, A., Tosevska, A., Klager, E., Eibensteiner, F., Laxar, D., Stoyanov, J., Glisic, M., Zeiner, S., Kulnik, S. T., Crutzen, R., Kimberger, O., Kletecka-Pulker, M., Atanasov, A. G., & Willschke, H (10 February 2021). "Virtual and Augmented Reality Applications in Medicine: Analysis of the Scientific Literature". Journal of Medical Internet Research. 23 (2): e25499. doi: 10.2196/25499 . PMC   7904394 . PMID   33565986.{{cite journal}}: CS1 maint: multiple names: authors list (link)
  21. ↑ "Telemedicine Technology _ Medicwell Telemedicine Platform". Medicwell: Reducing Coronavirus Spread Blog!. 2021-06-10. Retrieved 2022-06-01.
  22. 1 2 3 "The complete guide to virtual reality – everything you need to get started". the Guardian. 2016-11-10. Retrieved 2022-06-01.
  23. 1 2 "Facebook announces name change to Meta in rebranding effort". the Guardian. 2021-10-28. Retrieved 2022-06-01.
  24. ↑ Ravenscraft, Eric. "What Is the Metaverse, Exactly?". Wired. ISSN   1059-1028 . Retrieved 2022-06-01.
  25. ↑ Sparks, Hannah (2021-12-17). "Woman claims she was virtually 'groped' in Meta's VR metaverse". New York Post. Retrieved 2022-06-03.
  26. ↑ "How "Pokemon Go" Took Augmented Reality Mainstream". Knowledge at Wharton. Retrieved 2022-06-01.
  27. ↑ Newton, Casey (2017-04-18). "Snapchat adds world lenses to further its push into augmented reality". The Verge. Retrieved 2022-06-01.
  28. ↑ "Snapchat introduces World Lenses – live filters for just about anything". TechCrunch. Retrieved 2022-06-01.
  29. ↑ Lavanya (2021-08-30). "An Ultimate Guide on Snapchat Lenses | How to Enable & Use Lenses On Snapchat? List of Best Snapchat Lenses Codes". Version Weekly. Retrieved 2022-06-01.
  30. ↑ Shein, Esther (November 2021). "Filtering for beauty". Communications of the ACM. 64 (11): 17–19. doi:10.1145/3484997. ISSN   0001-0782. S2CID   239770270.
  31. ↑ "Is that really me? The ugly truth about beauty filters". the Guardian. 2022-01-01. Retrieved 2022-06-01.
  32. ↑ Mediated Reality with implementations for everyday life Archived 2017-08-02 at the Wayback Machine , Presence Connect, MIT Press journal PRESENCE: Teleoperators and Virtual Environments, Date Posted: 2002 August 6
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.