Volumetric display

A volumetric display device is a display device that forms a visual representation of an object in three physical dimensions, as opposed to the planar image of traditional screens that simulate depth through a number of different visual effects. One definition offered by pioneers in the field is that volumetric displays create 3D imagery via the emission, scattering, or relaying of illumination from well-defined regions in (x,y,z) space.

A true volumetric display produces in the observer a visual experience of a material object in three-dimensional space, even though no such object is present. The perceived object displays characteristics similar to an actual material object by allowing the observer to view it from any direction, to focus a camera on a specific detail, and to see perspective – meaning that the parts of the image closer to the viewer appear larger than those further away.

Volumetric 3D displays are technically not autostereoscopic, even though they create three-dimensional imagery visible to the unaided eye. This is because the displays do not generate stereoscopic images; they naturally provide focally-accurate holographic wavefronts to the eyes. Due to this, they have accurate characteristics of material objects such as focal depth, motion parallax, and vergence.

Volumetric displays are one of several kinds of 3D displays. Other types are stereoscopes, view-sequential displays, ^[1] electro-holographic displays, ^[2] "two view" displays, ^{[3] [4]} and panoramagrams.

Although first postulated in 1912, and a staple of science fiction, volumetric displays are not widely used in everyday life. There are numerous potential markets for volumetric displays with use cases including medical imaging, mining, education, advertising, simulation, video games, communication and geophysical visualisation. When compared to other 3D visualisation tools such as virtual reality, volumetric displays offer an inherently different mode of interaction, providing the opportunity for a group of people to gather around the display and interact in a natural manner without having to don 3D glasses or other head gear.

Types

Many different attempts have been made to produce volumetric imaging devices. ^[5] There is no officially accepted "taxonomy" of the variety of volumetric displays, an issue which is complicated by the many permutations of their characteristics. For example, illumination within a volumetric display can either reach the eye directly from the source or via an intermediate surface such as a mirror or glass; likewise, this surface, which need not be tangible, can undergo motion such as oscillation or rotation. One categorization is as follows:

Swept-volume display

Swept-surface (or "swept-volume") volumetric 3D displays rely on the human persistence of vision to fuse a series of slices of the 3D object into a single 3D image. ^[6] A variety of swept-volume displays have been created.

For example, the 3D scene is computationally decomposed into a series of "slices", which can be rectangular, disc-shaped, or helically cross-sectioned, whereupon they are projected onto or from a display surface undergoing motion. The image on the 2D surface (created by projection onto the surface, LEDs embedded in the surface, or other techniques) changes as the surface moves or rotates. Due to the persistence of vision, humans perceive a continuous volume of light. The display surface can be reflective, transmissive, or a combination of both.

Another type of 3D display that is a candidate member of the class of swept-volume 3D displays is the varifocal mirror architecture. One of the first references to this type of system is from 1966, in which a vibrating mirrored drumhead reflects a series of patterns from a high-frame-rate 2D image source, such as a vector display, to a corresponding set of depth surfaces.

An example of a commercially available Swept-volume display is the Voxon VX1 from Voxon Photonics. This display has a volume area that is 18 cm × 18 cm × 8 cm (7.1 in × 7.1 in × 3.1 in) deep and can render up to 500 million voxels per second. Content for the VX1 can be created using Unity or using standard 3D file types such as OBJ, STL and DICOM for medical imaging.

A Voxon VX1 volumetric display showing DICOM medical data

Static volume

So-called "static-volume" volumetric 3D displays create imagery without any macroscopic moving parts in the image volume. ^[7] It is unclear whether the rest of the system must remain stationary for membership in this display class to be viable.

This is probably the most "direct" form of volumetric display. In the simplest case, an addressable volume of space is created out of active elements that are transparent in the off state but are either opaque or luminous in the on state. When the elements (called voxels) are activated, they show a solid pattern within the space of the display.

Several static-volume volumetric 3D displays use laser light to encourage visible radiation in a solid, liquid, or gas. For example, some researchers have relied on two-step upconversion within a rare-earth-doped material when illuminated by intersecting infrared laser beams of the appropriate frequencies. ^{[8] [9]}

Recent advances have focused on non-tangible (free-space) implementations of the static-volume category, which might eventually allow direct interaction with the display. For instance, a fog display using multiple projectors can render a 3D image in a volume of space, resulting in a static-volume volumetric display. ^{[10] [11]}

A technique presented in 2006 does away with the display medium altogether, using a focused pulsed infrared laser (about 100 pulses per second; each lasting a nanosecond) to create balls of glowing plasma at the focal point in normal air. The focal point is directed by two moving mirrors and a sliding lens, allowing it to draw shapes in the air. Each pulse creates a popping sound, so the device crackles as it runs. Currently it can generate dots anywhere within a cubic metre. It is thought that the device could be scaled up to any size, allowing 3D images to be generated in the sky. ^{[12] [13]}

Later modifications such as the use of an neon/argon/xenon/helium gas mix similar to a plasma globe and a rapid gas recycling system employing a hood and vacuum pumps could allow this technology to achieve two-colour (R/W) and possibly RGB imagery by changing the pulse width and intensity of each pulse to tune the emission spectra of the luminous plasma body.

In 2017, a new display known as the "3D Light PAD" was published. ^[14] The display's medium consists of a class of photoactivatable molecules (known as spirhodamines) and digital light-processing (DLP) technology to generate structured light in three dimensions. The technique bypasses the need to use high-powered lasers and the generation of plasma, which alleviates concerns for safety and dramatically improves the accessibility of the three-dimensional displays. UV-light and green-light patterns are aimed at the dye solution, which initiates photoactivation and thus creates the "on" voxel. The device is capable of displaying a minimal voxel size of 0.68 mm³, with 200 μm resolution, and good stability over hundreds of on–off cycles.

Human–computer interfaces

The unique properties of volumetric displays, which may include 360-degree viewing, agreement of vergence and accommodation cues, and their inherent "three-dimensionality", enable new user interface techniques. There is recent work investigating the speed and accuracy benefits of volumetric displays, ^[15] new graphical user interfaces, ^[16] and medical applications enhanced by volumetric displays. ^{[17] [18]}

Also, software platforms exist that deliver native and legacy 2D and 3D content to volumetric displays. ^[19]

Artistic use

Hologlyphics: artistic use of volumetric displays, involving lasers and lissajous curves.

An artform called Hologlyphics has been explored since 1994, combining elements of holography, music, video synthesis, visionary film, sculpture and improvisation. Whilst this type of display may render visual data in a volume, it is not an addressable display and capable of only lissajous figures, such at those generated by bouncing a laser off a galvo or speaker cone.

Technical challenges

Known volumetric display technologies also have several drawbacks that are exhibited depending on trade-offs chosen by the system designer.

It is often claimed that volumetric displays are incapable of reconstructing scenes with viewer-position-dependent effects, such as occlusion and opacity. This is a misconception; a display whose voxels have non-isotropic radiation profiles are indeed able to depict position-dependent effects. To-date, occlusion-capable volumetric displays require two conditions: (1) the imagery is rendered and projected as a series of "views", rather than "slices", and (2) the time-varying image surface is not a uniform diffuser. For example, researchers have demonstrated spinning-screen volumetric displays with reflective and/or vertically diffuse screens whose imagery exhibits occlusion and opacity. One system ^{[20] [21]} created HPO 3D imagery with a 360-degree field of view by oblique projection onto a vertical diffuser; another ^[22] projects 24 views onto a rotating controlled-diffusion surface; and another ^[23] provides 12-view images utilizing a vertically oriented louver.

So far, the ability to reconstruct scenes with occlusion and other position-dependent effects have been at the expense of vertical parallax, in that the 3D scene appears distorted if viewed from locations other than those the scene was generated for.

One other consideration is the very large amount of bandwidth required to feed imagery to a volumetric display. For example, a standard 24 bits per pixel, 1024×768 resolution, flat/2D display requires about 135 MB/s to be sent to the display hardware to sustain 60 frames per second, whereas a 24 bits per voxel, 1024×768×1024 (1024 "pixel layers" in the Z axis) volumetric display would need to send about three orders of magnitude more (135 GB/s) to the display hardware to sustain 60 volumes per second. As with regular 2D video, one could reduce the bandwidth needed by simply sending fewer volumes per second and letting the display hardware repeat frames in the interim, or by sending only enough data to affect those areas of the display that need to be updated, as is the case in modern lossy-compression video formats such as MPEG. Furthermore, a 3D volumetric display would require two to three orders of magnitude more CPU and/or GPU power beyond that necessary for 2D imagery of equivalent quality, due at least in part to the sheer amount of data that must be created and sent to the display hardware. However, if only the outer surface of the volume is visible, the number of voxels required would be of the same order as the number of pixels on a conventional display. This would only be the case if the voxels do not have "alpha" or transparency values.

See also

• Holography
• Volumetric Haptic Display
• Volumetric video
• Volumetric printing
• Virtual retinal display
• Display device
• 3D display
• Zebra Imaging
• Autostereoscopy
• Multiscopy
• Vergence-Accommodation Conflict

References

Footnotes

1. Cossairt, Oliver; Moller, Christian; Benton, Steve; Travis, Adrian (January 2004). "Cambridge-MIT View Sequential Display". Northwestern University. Archived from the original on 2 August 2022.
2. Lucente, Mark (November 1994). "Electronic Holography: The Newest". Massachusetts Institute of Technology. Archived from the original on 19 September 2006. Retrieved 1 August 2022.
3. Habib, Maged S; Lowell, James A; Holliman, Nick S; Hunter, Andrew; Vaideanu, Daniella; Hildreth, Anthony; Steel, David HW (December 2008). "Assessment of stereoscopic optic disc images using an autostereoscopic screen – experimental study". BMC Ophthalmology. 8 (1): 13. doi: 10.1186/1471-2415-8-13 . PMC   2496897 . PMID   18651983.
4. Pickering, Mark R. (2014). "Stereoscopic and Multi-View Video Coding". Academic Press Library in signal Processing - Image and Video Compression and Multimedia. Vol. 5. pp. 119–153. doi:10.1016/B978-0-12-420149-1.00004-1. ISBN   978-0-12-420149-1.
5. US Patent Office
6. Gately, Matthew, et al. "A three-dimensional swept volume display based on LED arrays." Journal of Display Technology 7.9 (2011): 503-514.
7. Blundell, Barry G., and Adam J. Schwarz. "The classification of volumetric display systems: characteristics and predictability of the image space." IEEE Transactions on Visualization and Computer Graphics 8.1 (2002): 66-75.
8. Joseph A. Matteo (16 March 2001). "Volumetric Display". Lecture notes for the Applied Vision and Imaging Systems class at Stanford University . Archived from the original on 2005-09-09.
9. Downing, Elizabeth; Hesselink, Lambertus; Ralston, John; Macfarlane, Roger (1996). "A Three-Color, Solid-State, Three-Dimensional Display". Science . 273 (5279): 1185–1189. Bibcode:1996Sci...273.1185D. doi:10.1126/science.273.5279.1185. S2CID   136426473.
10. 3D Multi-Viewpoint Fog Projection Display
11. Tim Stevens (17 March 2011). "3D fog projection display brings purple bunnies to life, just in time to lay chocolate eggs (video)". Engadget .
12. David Hambling (27 February 2006). "3D plasma shapes created in thin air". New Scientist .
13. "Japanese Device Uses Laser Plasma to Display 3D Images in the Air". Physorg.com. 27 February 2006.
14. Patel, S. K.; Cao, J.; Lippert, A. R. "A Volumetric 3D Photoactivatable Dye Display". Nature Commun. 2017, in press.
15. van Orden, K. F. and Broyles, J. W. (2000, March). Visuospatial task performance as a function of two- and three-dimensional display presentation techniques, Displays, 21(1), 17-24. PDF: Mirror, with permission
16. Grossman, T., Wigdor, D., and Balakrishnan, R. (2004). "Multi-finger gestural interaction with 3D volumetric displays", Proceedings of UIST, ACM Symposium on User Interface Software and Technology, (pp. 61–70). PDF at author site
17. "Exploring Cutting-Edge 3D Imaging System for Cancer Treatment Planning, Rush University Medical Center", Medical News Today, (29 Apr 05).
18. Wang, A.S.; Girish Narayan; Kao, D.; Liang, D. (2005). "An evaluation of using real-time volumetric display of 3D ultrasound data for intracardiac catheter manipulation tasks". Fourth International Workshop on Volume Graphics, 2005. pp. 41–45. doi:10.1109/VG.2005.194095. ISBN   3-905673-26-6.
19. Chun, W.-S., Napoli, J., Cossairt, O. S., Dorval, R. K., Hall, D. M., Purtell II, T. J., Schooler, J. F., Banker, Y., and Favalora, G. E. (2005). Spatial 3-D Infrastructure: Display-Independent Software Framework, High-Speed Rendering Electronics, and Several New Displays. In Stereoscopic Displays and Virtual Reality Systems XII, ed. Andrew J. Woods, Mark T. Bolas, John O. Merritt, and Ian E. McDowall, Proc. SPIE-IS&T Electronic Imaging, SPIE Vol. 5664, (pp. 302–312). San Jose, California: SPIE-IS&T.
20. Cossairt, O. S. and Napoli, J. (2004), Radial multiview three-dimensional displays, U.S. Pat. App. 2005/0180007 A1. Provisional (Jan. 16, 2004). Nonprovisional (Jan. 14, 2005). Published (Aug. 18, 2005)
21. Favalora, G. E. (2005, 4 Aug.). "The Ultimate Display: What Will It Be?", presented at ACM SIGGRAPH, Los Angeles, California.
22. Otsuka, Rieko; Hoshino, Takeshi; Horry, Youichi (10 November 2004). "Transpost: all-around display system for 3D solid image": 187–194. doi:10.1145/1077534.1077576.{{cite journal}}: Cite journal requires |journal= (help)
23. Tanaka, Kenji; Aoki, Soko (2 February 2006). "A method for the real-time construction of a full parallax light field". In Woods, Andrew J.; Dodgson, Neil A.; Merritt, John O.; Bolas, Mark T.; McDowall, Ian E. (eds.). Stereoscopic Displays and Virtual Reality Systems XIII. Vol. 6055. p. 605516. doi:10.1117/12.643597.

Further reading

• Blundell, B.G., (2011). "About 3D Volumetric Displays", Walker & Wood Ltd. ISBN   9780473193768. (http://www.barrygblundell.com, PDF file).
• Blundell, B.G., (2011). "3D Displays and Spatial Interaction: Exploring the Science, Art, Evolution, and Use of 3D Technologies,Volume I: From Perception to Technologies", Walker & Wood Ltd. ISBN   9780473177003. (http://www.barrygblundell.com, PDF file).
• Blundell, B.G. and Schwarz, A J (2007). "Enhanced Visualization: Making Space for 3D Images", John Wiley & Sons. ISBN   0-471-78629-2.
• Blundell, B.G. and Schwarz, A J (2006). Creative 3-D Displays and Interaction Interfaces: A Transdisciplinary Approach, John Wiley & Sons. ISBN   0-471-23928-3. (http://www.barrygblundell.com, PDF file).
• Blundell, B. G. and Schwarz, A. J. (2000). Volumetric Three-Dimensional Display Systems, John Wiley & Sons. ISBN   0-471-23928-3 (http://www.barrygblundell.com, PDF file).
• Favalora, G. E. (2005, Aug.). "Volumetric 3D Displays and Application Infrastructure", Computer, 38(8), 37–44. Illustrated technical survey of contemporary and historic volumetric 3-D displays. IEEE citation via ACM
• Funk, W. (2008). "Hologlyphics: Volumetric image synthesis performance system", Proc. SPIE, vol. 6803, SPIE — Int'l Soc. for Optical Eng., Stereoscopic Displays and Applications XIX. PDF at author site
• Halle, M. (1997). "Autostereoscopic displays and computer graphics", Computer Graphics, ACM SIGGRAPH, vol. 31, no. 2, (pp. 58–62). A thoughtful and concise overview of the field of 3-D display technologies, particularly non-volumetric displays. HTML and PDF
• Hartwig, R. (1976). Vorrichtung zur Dreidimensionalen Abbildung in Einem Zylindersymmetrischen Abbildungstraum, German patent DE2622802C2, filed 1976, issued 1983. One of the earliest patent references for the rotating helix 3-D display.
• Honda, T. (2000). Three-Dimensional Display Technology Satisfying 'Super Multiview Condition.' In B. Javidi and F. Okano (Eds.), Proc. Three-Dimensional Video and Display: Devices and Systems, vol. CR76, SPIE Press, (pp. 218–249). ISBN   0-8194-3882-0
• Langhans, K., Bezecny, D., Homann, D., Bahr, D., Vogt, C., Blohm, C., and Scharschmidt, K.-H.(1998). "New Portable FELIX 3D Display", Proc. SPIE, vol. 3296, SPIE — Int'l Soc. for Optical Eng., (pp. 204–216). Includes a thorough literature review of volumetric displays.
• Lewis, J. D., Verber, C. M., and McGhee, R. B. (1971). A True Three-Dimensional Display, IEEE Trans. Electron Devices, 18, 724–732. An early investigation into so-called solid-state 3-D displays.
• Roth, E. (2006). Volumetric Display based on Inkjet-Technology, PDF (Archived 03-14-2012: )

External links

• Dragon O - a commercially available Interactive Volumetric LED Display composed of 50cmx50cmx3m plugin modules. Positioned for audiovisual interactive experiences and installations
• Volumetric Motion Picture and 3D Digital Film Forum
• VisualCube — a small volumetric display composed of 6x6x6 voxels, each represented by a 2-color LED
• Voxon Photonics — a commercially available swept-volume based volumetric display positioned for gaming and entertainment applications
• Volumetric Displays — Summary of history, practical issues, and state of the art up until March 1996
• The Return of the 3D Crystal Ball — A comprehensive article on Actuality Systems' Volumetric technology including an interview, pictures and a movie
• Felix3D Display — Some examples for volumetric displays
• Interactive 360° Light Field Display Archived 2009-10-27 at the Wayback Machine — by USC Institute for Creative Technologies
• QinetiQ Autostereo 3D Display Wall — Press Release from 2004, perhaps discontinued as no further references found
• SPIE / IS&T Stereoscopic Displays and Virtual Reality Applications annual global conference
• Esna Ashari, Zhila; Kavehvash, Zahra; Mehrany, Khashayar (July 2014). "Diffraction Influence on the Field of View and Resolution of Three-Dimensional Integral Imaging". Journal of Display Technology. 10 (7): 553–559. arXiv: 1711.01033 . Bibcode:2014JDisT..10..553A. doi:10.1109/JDT.2014.2307959.