Microsoft Holoportation

Last updated

Microsoft Holoportation [1] [2] is a project from Microsoft Research that demonstrates real-time holographic communications [3] [4] [5] [6] [7] [8] with the Microsoft Hololens. Holoportation is described as "a new type of 3D capture technology that allows high-quality 3D models of people to be reconstructed, compressed and transmitted anywhere in the world in real time. This allows users wearing virtual or augmented reality displays to see, hear and interact with remote participants in 3D, almost as if they were present in the same physical space. From an audio-visual perspective, communicating and interacting with remote users edges closer to face-to-face communication." [9] [10] The project was launched by Shahram Izadi and his Microsoft team in 2016. In March 2016, Alex Kipman performed a live demonstration [11] [12] [13] [14] of the technology at the TED conference as part of his talk. [15] In 2020, Microsoft Mesh was launched which offered Holoportation capabilities to "project yourself as your most lifelike, photorealistic self in mixed reality to interact as if you are there in person" [16] [17] [18]

See also

Related Research Articles

Augmented reality View of the real world with computer-generated supplementary features

Augmented reality (AR) is an interactive experience of a real-world environment where the objects that reside in the real world are enhanced by computer-generated perceptual information, sometimes across multiple sensory modalities, including visual, auditory, haptic, somatosensory and olfactory. AR can be defined as a system that incorporates three basic features: a combination of real and virtual worlds, real-time interaction, and accurate 3D registration of virtual and real objects. The overlaid sensory information can be constructive, or destructive. This experience is seamlessly interwoven with the physical world such that it is perceived as an immersive aspect of the real environment. In this way, augmented reality alters one's ongoing perception of a real-world environment, whereas virtual reality completely replaces the user's real-world environment with a simulated one. Augmented reality is related to two largely synonymous terms: mixed reality and computer-mediated reality.

Holodeck Star Trek device

The Holodeck is a fictional device from the television franchise Star Trek which uses "holograms" to create a realistic 3D simulation of a real or imaginary setting, in which participants can freely interact with the environment as well as objects and characters, and sometimes a predefined narrative.

Mixed reality Merging of real and virtual worlds to produce new environments

Mixed reality (MR) is the merging of real and virtual worlds to produce new environments and visualizations, where physical and digital objects co-exist and interact in real time. Mixed reality does not exclusively take place in either the physical world or virtual world, but is a hybrid of augmented reality and virtual reality. To mark the difference: Augmented reality takes place in the physical world, with information or objects added virtually like an overlay; Virtual Reality immerged you in a fully virtual world without the intervention of the physical world.

Don Box

Don Box is a former Microsoft Technical Fellow.

Microsoft engineering groups are the operating divisions of Microsoft. Starting in April 2002, Microsoft organised itself into seven groups, each an independent financial entity. In September 2005, Microsoft announced a reorganization of its then seven groups into three. In July 2013, Microsoft announced another reorganization into five engineering groups and six corporate affairs groups. A year later, in June 2015, Microsoft reformed into three engineering groups. In September 2016, a new group was created to focus on artificial intelligence and research. On March 29, 2018, a new structure merged all of these into three.

A holographic display is a type of display that utilizes light diffraction to create a virtual three-dimensional image. Holographic displays are distinguished from other forms of 3D displays in that they do not require the aid of any special glasses or external equipment for a viewer to see the image.

A virtual touch screen (VTS) is a user interface system that augments virtual objects into reality either through a projector or optical display using sensors to track a person's interaction with the object. For instance, using a display and a rear projector system a person could create images that look three-dimensional and appear to float in midair. Some systems utilize an optical head-mounted display to augment the virtual objects onto the transparent display utilizing sensors to determine visual and physical interactions with the virtual objects projected.

Optical head-mounted display Type of wearable device

An optical head-mounted display (OHMD) is a wearable device that has the capability of reflecting projected images as well as allowing the user to see through it, similar to augmented reality technology. OHMD technology has existed since 1997 in various forms, but despite a number of attempts from industry, has yet to be commercialised.

Holography is often used as a plot device in science fiction, appearing in a wide range of books, films, television series, animation and video games. Probably the first reference is by Isaac Asimov in his sci-fi novel series “the Foundation Trilogy” in 1951.

Magic Leap American augmented reality startup

Magic Leap, Inc. is an American startup company that released a head-mounted virtual retinal display, called Magic Leap One, which superimposes 3D computer-generated imagery over real world objects, by "projecting a digital light field into the user's eye", involving technologies potentially suited to applications in augmented reality and computer vision. It is attempting to construct a light-field chip using silicon photonics.

Windows Mixed Reality Mixed reality platform

Windows Mixed Reality is a platform introduced as part of the Windows 10 operating system, which provides augmented reality and mixed reality experiences with compatible head-mounted displays.

Microsoft HoloLens Mixed reality smartglasses

Microsoft HoloLens, known under development as Project Baraboo, are a pair of mixed reality smartglasses developed and manufactured by Microsoft. HoloLens was the first head-mounted display running the Windows Mixed Reality platform under the Windows 10 computer operating system. The tracking technology used in HoloLens can trace its lineage to Kinect, an add-on for Microsoft's Xbox game console that was introduced in 2010.

Sidekick is a project developed by NASA and Microsoft to provide virtual aid for astronauts using the Microsoft HoloLens augmented-reality headset. As of December 2015, Sidekick is deployed at the International Space Station.

Holoverse is an amusement attraction on the Gold Coast, Australia, that opened in June 2016.

Remix 3D

Remix 3D was a website created by Microsoft for users to browse, distribute, and download 3D objects. The website featured a free library containing thousands of 3D objects for use in virtual reality and augmented reality applications. Remix 3D was released first to Windows Insiders and then as part of the Windows 10 Fall Creators Update, alongside Paint 3D. The service also directly integrated with and/or supported 3D Viewer, as well as Paint 3D, SketchUp, Minecraft, and PowerPoint.

HoloLens 2 2019 mixed reality smartglasses

Microsoft HoloLens 2 is a pair of mixed reality smartglasses developed and manufactured by Microsoft. It is the successor to the pioneering Microsoft HoloLens. On February 24, 2019 the HoloLens 2 enterprise edition debuted as the first variant of the device, followed by a developer edition that was announced on May 2, 2019. It was subsequently released in limited numbers on November 7, 2019.

The Azure Kinect DK is a developer kit and PC peripheral which employs the use of artificial intelligence sensors for computer vision and speech models, and is connected to the Microsoft Azure cloud. It is the successor to the Microsoft Kinect line of sensors.

Alex Kipman Brazilian engineer (born 1979)

Alex Kipman is a Brazilian engineer. He is the lead developer of the Microsoft HoloLens smartglasses and helped develop the Xbox Kinect. He was born in Curitiba in 1979, and grew up in Natal, Brazil.

Integrated Visual Augmentation System United States Army program

The Integrated Visual Augmentation System (IVAS) is a United States Army program to provide infantry with a mixed reality headset, which provides a wide variety of capabilities to soldiers including, but not limited to, digital wide-angle multi-spectral image fusion of solid-state low-light, thermal, and daylight sensing, object outlining emphasis, 3D map and waypoint overlays, a picture-in-picture display of weapons sights, and a Squad Immersive Virtual Trainer that allows them to train against virtual enemies overlaid upon their real-world field of vision.

Shahram Izadi is a computer scientist known for his work on Augmented Reality, Computer Vision, Artificial Intelligence and Human-Computer Interaction. He is currently at Google as a senior director of engineering where he is reported to be working on an augmented reality headset while also managing Google's ARCore software toolkit.

References

  1. ↑ "Holoportation". Microsoft Research. Retrieved 2020-12-31.
  2. ↑ "holoportation: virtual 3D teleportation in real-time (Microsoft Research) - YouTube". www.youtube.com. Retrieved 2020-12-31.
  3. ↑ "Microsoft Researchers Hacked Together a HoloLens and 3D Cameras to Achieve 'Holoportation'". Wired. ISSN   1059-1028 . Retrieved 2020-12-31.
  4. ↑ Bohn, Dieter (2016-03-26). "Microsoft has created Star Wars-style holographic communication". The Verge. Retrieved 2020-12-31.
  5. ↑ Foley, Mary Jo. "How Microsoft's HoloLens could change communication via 'Holoportation'". ZDNet. Retrieved 2020-12-31.
  6. ↑ "It's 2026 and Anyone in This Room Could Be a Hologram". Gizmodo. Retrieved 2020-12-31.
  7. ↑ "Microsoft Demos Star Wars-Style 'Holoportation'". PCMAG. Retrieved 2020-12-31.
  8. ↑ Shammas, John (2016-03-27). "Microsoft creates amazing Star Wars holograph so you can be in 2 places at once". mirror. Retrieved 2020-12-31.
  9. ↑ "Holoportation". Microsoft Research. Retrieved 2021-01-01.
  10. ↑ Orts-Escolano, Sergio; Rhemann, Christoph; Fanello, Sean; Chang, Wayne; Kowdle, Adarsh; Degtyarev, Yury; Kim, David; Davidson, Philip L.; Khamis, Sameh; Dou, Mingsong; Tankovich, Vladimir (2016-10-16). "Holoportation: Virtual 3D Teleportation in Real-time". Proceedings of the 29th Annual Symposium on User Interface Software and Technology. UIST '16. Tokyo, Japan: Association for Computing Machinery: 741–754. doi:10.1145/2984511.2984517. ISBN   978-1-4503-4189-9.
  11. ↑ "The dawn of the age of holograms | Alex Kipman - YouTube". www.youtube.com. Retrieved 2021-01-01.
  12. ↑ "TED 2016: HoloLens unveils 'teleportation' to Mars". BBC News. 2016-02-19. Retrieved 2020-12-31.
  13. ↑ Schwartz, Ariel. "The creator of Microsoft's HoloLens just showed the first-ever 'real-life holographic teleportation'". Business Insider. Retrieved 2020-12-31.
  14. ↑ "HoloLens TED Talk shows what augmented reality can do". Engadget. Retrieved 2020-12-31.
  15. ↑ Blog, Microsoft Devices (2016-03-25). "TED2016: It's a phenomenal time to be human". Microsoft Devices Blog. Retrieved 2020-12-31.
  16. ↑ Foley, Mary Jo. "Microsoft's latest holoportation demo shows off its mixed reality, AI, translation technologies". ZDNet. Retrieved 2020-12-31.
  17. ↑ Foley, Mary Jo. "Holoportation is still on Microsoft's mixed-reality radar". ZDNet. Retrieved 2020-12-31.
  18. ↑ "Introducing Microsoft Mesh | Here can be anywhere". www.microsoft.com. Retrieved 2022-02-27.
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.