Structured-light 3D scanner

Last updated

A structured-light 3D scanner is a 3D scanning device for measuring the three-dimensional shape of an object using projected light patterns and a camera system. [1]

Contents

The light source from the scanner head projects a series of parallel patterns onto the scan target. When light projects onto the object's surface, the patterns become distorted. The cameras capture these images and send them to the 3D scanning software for processing.

Principle

Projecting a narrow band of light onto a three-dimensionally shaped surface produces a line of illumination that appears distorted from other perspectives than that of the projector, and can be used for geometric reconstruction of the surface shape (light section).

A faster and more versatile method is the projection of patterns consisting of many stripes at once, or of arbitrary fringes, as this allows for the acquisition of a multitude of samples simultaneously. Seen from different viewpoints, the pattern appears geometrically distorted due to the surface shape of the object.

Although many other variants of structured light projection are possible, patterns of parallel stripes are widely used. The picture shows the geometrical deformation of a single stripe projected onto a simple 3D surface. The displacement of the stripes allows for an exact retrieval of the 3D coordinates of any details on the object's surface.

Generation of light patterns

Fringe pattern recording system with 2 cameras (avoiding obstructions) 3-proj2cam.svg
Fringe pattern recording system with 2 cameras (avoiding obstructions)

Two major methods of stripe pattern generation have been established: Laser interference and projection.

The laser interference method works with two wide planar laser beam fronts. Their interference results in regular, equidistant line patterns. Different pattern sizes can be obtained by changing the angle between these beams. The method allows for the exact and easy generation of very fine patterns with unlimited depth of field. Disadvantages are high cost of implementation, difficulties providing the ideal beam geometry, and laser typical effects like speckle noise and the possible self interference with beam parts reflected from objects. Typically, there is no means of modulating individual stripes, such as with Gray codes.

The projection method uses incoherent light and basically works like a video projector. Patterns are usually generated by passing light through a digital spatial light modulator, typically based on one of the three currently most widespread digital projection technologies, transmissive liquid crystal, reflective liquid crystal on silicon (LCOS) or digital light processing (DLP; moving micro mirror) modulators, which have various comparative advantages and disadvantages for this application. Other methods of projection could be and have been used, however.

Patterns generated by digital display projectors have small discontinuities due to the pixel boundaries in the displays. Sufficiently small boundaries however can practically be neglected as they are evened out by the slightest defocus.

A typical measuring assembly consists of one projector and at least one camera. For many applications, two cameras on opposite sides of the projector have been established as useful.

Invisible (or imperceptible) structured light uses structured light without interfering with other computer vision tasks for which the projected pattern will be confusing. Example methods include the use of infrared light or of extremely high framerates alternating between two exact opposite patterns. [2]

Calibration

A 3D scanner in a library. Calibration panels can be seen on the right. Library 3D Scanner.jpg
A 3D scanner in a library. Calibration panels can be seen on the right.

Geometric distortions by optics and perspective must be compensated by a calibration of the measuring equipment, using special calibration patterns and surfaces. A mathematical model is used for describing the imaging properties of projector and cameras. Essentially based on the simple geometric properties of a pinhole camera, the model also has to take into account the geometric distortions and optical aberration of projector and camera lenses. The parameters of the camera as well as its orientation in space can be determined by a series of calibration measurements, using photogrammetric bundle adjustment.

Analysis of stripe patterns

There are several depth cues contained in the observed stripe patterns. The displacement of any single stripe can directly be converted into 3D coordinates. For this purpose, the individual stripe has to be identified, which can for example be accomplished by tracing or counting stripes (pattern recognition method). Another common method projects alternating stripe patterns, resulting in binary Gray code sequences identifying the number of each individual stripe hitting the object. An important depth cue also results from the varying stripe widths along the object surface. Stripe width is a function of the steepness of a surface part, i.e. the first derivative of the elevation. Stripe frequency and phase deliver similar cues and can be analyzed by a Fourier transform. Finally, the wavelet transform has recently been discussed for the same purpose.

In many practical implementations, series of measurements combining pattern recognition, Gray codes and Fourier transform are obtained for a complete and unambiguous reconstruction of shapes.

Another method also belonging to the area of fringe projection has been demonstrated, utilizing the depth of field of the camera. [3]

It is also possible to use projected patterns primarily as a means of structure insertion into scenes, for an essentially photogrammetric acquisition.

Precision and range

The optical resolution of fringe projection methods depends on the width of the stripes used and their optical quality. It is also limited by the wavelength of light.

An extreme reduction of stripe width proves inefficient due to limitations in depth of field, camera resolution and display resolution. Therefore, the phase shift method has been widely established: A number of at least 3, typically about 10 exposures are taken with slightly shifted stripes. The first theoretical deductions of this method relied on stripes with a sine wave shaped intensity modulation, but the methods work with "rectangular" modulated stripes, as delivered from LCD or DLP displays as well. By phase shifting, surface detail of e.g. 1/10 the stripe pitch can be resolved.

Current optical stripe pattern profilometry hence allows for detail resolutions down to the wavelength of light, below 1 micrometer in practice or, with larger stripe patterns, to approx. 1/10 of the stripe width. Concerning level accuracy, interpolating over several pixels of the acquired camera image can yield a reliable height resolution and also accuracy, down to 1/50 pixel.

Arbitrarily large objects can be measured with accordingly large stripe patterns and setups. Practical applications are documented involving objects several meters in size.

Typical accuracy figures are:

As the method can measure shapes from only one perspective at a time, complete 3D shapes have to be combined from different measurements in different angles. This can be accomplished by attaching marker points to the object and combining perspectives afterwards by matching these markers. The process can be automated, by mounting the object on a motorized turntable or CNC positioning device. Markers can as well be applied on a positioning device instead of the object itself.

The 3D data gathered can be used to retrieve CAD (computer aided design) data and models from existing components (reverse engineering), hand formed samples or sculptures, natural objects or artifacts.

Challenges

As with all optical methods, reflective or transparent surfaces raise difficulties. Reflections cause light to be reflected either away from the camera or right into its optics. In both cases, the dynamic range of the camera can be exceeded. Transparent or semi-transparent surfaces also cause major difficulties. In these cases, coating the surfaces with a thin opaque lacquer just for measuring purposes is a common practice. A recent method handles highly reflective and specular objects by inserting a 1-dimensional diffuser between the light source (e.g., projector) and the object to be scanned. [4] Alternative optical techniques have been proposed for handling perfectly transparent and specular objects. [5]

Double reflections and inter-reflections can cause the stripe pattern to be overlaid with unwanted light, entirely eliminating the chance for proper detection. Reflective cavities and concave objects are therefore difficult to handle. It is also hard to handle translucent materials, such as skin, marble, wax, plants and human tissue because of the phenomenon of sub-surface scattering. Recently, there has been an effort in the computer vision community to handle such optically complex scenes by re-designing the illumination patterns. [6] These methods have shown promising 3D scanning results for traditionally difficult objects, such as highly specular metal concavities and translucent wax candles. [7]

Speed

Although several patterns have to be taken per picture in most structured light variants, high-speed implementations are available for a number of applications, for example:

Motion picture applications have been proposed, for example the acquisition of spatial scene data for three-dimensional television.

Applications

Software

See also

Related Research Articles

<span class="mw-page-title-main">Computer vision</span> Computerized information extraction from images

Computer vision tasks include methods for acquiring, processing, analyzing and understanding digital images, and extraction of high-dimensional data from the real world in order to produce numerical or symbolic information, e.g. in the forms of decisions. Understanding in this context means the transformation of visual images into descriptions of the world that make sense to thought processes and can elicit appropriate action. This image understanding can be seen as the disentangling of symbolic information from image data using models constructed with the aid of geometry, physics, statistics, and learning theory.

<span class="mw-page-title-main">Interferometry</span> Measurement method using interference of waves

Interferometry is a technique which uses the interference of superimposed waves to extract information. Interferometry typically uses electromagnetic waves and is an important investigative technique in the fields of astronomy, fiber optics, engineering metrology, optical metrology, oceanography, seismology, spectroscopy, quantum mechanics, nuclear and particle physics, plasma physics, biomolecular interactions, surface profiling, microfluidics, mechanical stress/strain measurement, velocimetry, optometry, and making holograms.

<span class="mw-page-title-main">Machine vision</span> Technology and methods used to provide imaging-based automatic inspection and analysis

Machine vision (MV) is the technology and methods used to provide imaging-based automatic inspection and analysis for such applications as automatic inspection, process control, and robot guidance, usually in industry. Machine vision refers to many technologies, software and hardware products, integrated systems, actions, methods and expertise. Machine vision as a systems engineering discipline can be considered distinct from computer vision, a form of computer science. It attempts to integrate existing technologies in new ways and apply them to solve real world problems. The term is the prevalent one for these functions in industrial automation environments but is also used for these functions in other environment vehicle guidance.

<span class="mw-page-title-main">Thermography</span> Use of thermograms to study heat distribution in structures or regions

Infrared thermography (IRT), thermal video and/or thermal imaging, is a process where a thermal camera captures and creates an image of an object by using infrared radiation emitted from the object in a process, which are examples of infrared imaging science. Thermographic cameras usually detect radiation in the long-infrared range of the electromagnetic spectrum and produce images of that radiation, called thermograms. Since infrared radiation is emitted by all objects with a temperature above absolute zero according to the black body radiation law, thermography makes it possible to see one's environment with or without visible illumination. The amount of radiation emitted by an object increases with temperature; therefore, thermography allows one to see variations in temperature. When viewed through a thermal imaging camera, warm objects stand out well against cooler backgrounds; humans and other warm-blooded animals become easily visible against the environment, day or night. As a result, thermography is particularly useful to the military and other users of surveillance cameras.

A volumetric display device is a display device that forms a visual representation of an object in three physical dimensions, as opposed to the planar image of traditional screens that simulate depth through a number of different visual effects. One definition offered by pioneers in the field is that volumetric displays create 3D imagery via the emission, scattering, or relaying of illumination from well-defined regions in (x,y,z) space.

<span class="mw-page-title-main">Coordinate-measuring machine</span> Device for measuring the geometry of objects

A coordinate measuring machine (CMM) is a device that measures the geometry of physical objects by sensing discrete points on the surface of the object with a probe. Various types of probes are used in CMMs, the most common being mechanical and laser sensors, though optical and white light sensor do exist. Depending on the machine, the probe position may be manually controlled by an operator or it may be computer controlled. CMMs typically specify a probe's position in terms of its displacement from a reference position in a three-dimensional Cartesian coordinate system. In addition to moving the probe along the X, Y, and Z axes, many machines also allow the probe angle to be controlled to allow measurement of surfaces that would otherwise be unreachable.

<span class="mw-page-title-main">3D scanning</span> Scanning of an object or environment to collect data on its shape

3D scanning is the process of analyzing a real-world object or environment to collect three dimensional data of its shape and possibly its appearance. The collected data can then be used to construct digital 3D models.

<span class="mw-page-title-main">Profilometer</span> Measuring instrument for surface profile and roughness

A profilometer is a measuring instrument used to measure a surface's profile, in order to quantify its roughness. Critical dimensions as step, curvature, flatness are computed from the surface topography.

The following are common definitions related to the machine vision field.

<span class="mw-page-title-main">Structured light</span>

Structured light is the process of projecting a known pattern on to a scene. The way that these deform when striking surfaces allows vision systems to calculate the depth and surface information of the objects in the scene, as used in structured light 3D scanners.

InspecVision Ltd. is a UK engineering company based in Mallusk, Northern Ireland, established in 2003. It is a manufacturing company that produces computer vision inspection systems. The company is one of several local companies created as spinoffs or inspired by research conducted at the Queen's University of Belfast.

As a measuring instrument, HoloVID is a tool originally developed by Mark Slater for the holographic dimensional measurement of the internal isogrid structural webbing of the Delta family of launch vehicles in 1981.

ISO 25178: Geometrical Product Specifications (GPS) – Surface texture: areal is an International Organization for Standardization collection of international standards relating to the analysis of 3D areal surface texture.

<span class="mw-page-title-main">3D reconstruction</span> Process of capturing the shape and appearance of real objects

In computer vision and computer graphics, 3D reconstruction is the process of capturing the shape and appearance of real objects. This process can be accomplished either by active or passive methods. If the model is allowed to change its shape in time, this is referred to as non-rigid or spatio-temporal reconstruction.

<span class="mw-page-title-main">Time-of-flight camera</span> Range imaging camera system

A time-of-flight camera, also known as time-of-flight sensor, is a range imaging camera system for measuring distances between the camera and the subject for each point of the image based on time-of-flight, the round trip time of an artificial light signal, as provided by a laser or an LED. Laser-based time-of-flight cameras are part of a broader class of scannerless LIDAR, in which the entire scene is captured with each laser pulse, as opposed to point-by-point with a laser beam such as in scanning LIDAR systems. Time-of-flight camera products for civil applications began to emerge around 2000, as the semiconductor processes allowed the production of components fast enough for such devices. The systems cover ranges of a few centimeters up to several kilometers.

<span class="mw-page-title-main">Structured light plethysmography</span>

Structured Light Plethysmography (SLP) technology is a noninvasive method for collecting accurate representations of chest and abdominal wall movement. A checkerboard pattern of light is projected from a light projector onto the chest of an individual. Movements of the grid are viewed by two digital cameras, digitalised, and processed to form a 3D model and can be interrogated to assess lung function. The system has been tested on over 70 adults. SLP is simple to use, accurate and cost effective, is self-calibrating and does not require the use of plastic consumables, reducing cost, risk of cross infection and the device's carbon footprint. In conjunction with the Cambridge Veterinary School, proof of concept studies have indicated that the device is sensitive enough to noninvasively pick up respiratory movements in domestic animals.

Computer stereo vision is the extraction of 3D information from digital images, such as those obtained by a CCD camera. By comparing information about a scene from two vantage points, 3D information can be extracted by examining the relative positions of objects in the two panels. This is similar to the biological process of stereopsis.

<span class="mw-page-title-main">White light scanner</span>

A white light scanner (WLS) is a device for performing surface height measurements of an object using coherence scanning interferometry (CSI) with spectrally-broadband, "white light" illumination. Different configurations of scanning interferometer may be used to measure macroscopic objects with surface profiles measuring in the centimeter range, to microscopic objects with surface profiles measuring in the micrometer range. For large-scale non-interferometric measurement systems, see structured-light 3D scanner.

<span class="mw-page-title-main">3D reconstruction from multiple images</span> Creation of a 3D model from a set of images

3D reconstruction from multiple images is the creation of three-dimensional models from a set of images. It is the reverse process of obtaining 2D images from 3D scenes.

Cultural property imaging is a necessary part of long term preservation of cultural heritage. While the physical conditions of objects will change over time, imaging serves as a way to document and represent heritage in a moment in time of the life of the item. Different methods of imaging produce results that are applicable in various circumstances. Not every method is appropriate for every object, and not every object needs to be imaged by multiple methods. In addition to preservation and conservation-related concerns, imaging can also serve to enhance research and study of cultural heritage.

References

  1. Borko Furht (2008). Encyclopedia of Multimedia (2nd ed.). Springer. p. 222. ISBN   978-0-387-74724-8.
  2. Fofi, David; T. Sliwa; Y. Voisin (January 2004). "A Comparative Survey on Invisible Structured Light" (PDF). SPIE Electronic Imaging — Machine Vision Applications in Industrial Inspection XII. San Jose, USA. pp. 90–97.
  3. "Tiefenscannende Streifenprojektion (DSFP) mit 3D-Kalibrierung". University of Stuttgart (in German). Archived from the original on 9 April 2013.
  4. Shree K. Nayar and Mohit Gupta, Diffuse Structured Light, Proc. IEEE International Conference on Computational Photography, 2012
  5. Eron Steger & Kiriakos N. Kutulakos (2008). "A Theory of Refractive and Specular 3D Shape by Light-Path Triangulation". Int. J. Computer Vision, vol. 76, no. 1.
  6. Mohit Gupta, Amit Agrawal, Ashok Veeraraghavan and Srinivasa G. Narasimhan (2011). "Measuring Shape in the Presence of Inter-reflections, Sub-surface Scattering and Defocus". Proc. CVPR.{{cite news}}: CS1 maint: multiple names: authors list (link)
  7. Mohit Gupta; Shree K. Nayar (2012). "Micro Phase Shifting". Proc. CVPR.
  8. "ATOS – Industrial 3D Scanning Technology". GOM GmbH. Retrieved 9 July 2018.
  9. Walecki, W. J.; Szondy, F.; Hilali, M. M. (2008). "Fast in-line surface topography metrology enabling stress calculation for solar cell manufacturing for throughput in excess of 2000 wafers per hour". Measurement Science and Technology. 19 (2): 025302. doi:10.1088/0957-0233/19/2/025302. S2CID   121768537.
  10. "3D vision system enables DHL's e-fulfillment robot". The Robot Report. December 12, 2018.
  11. Kyriakos Herakleous & Charalambos Poullis (2014). "3DUNDERWORLD-SLS: An Open-Source Structured-Light Scanning System for Rapid Geometry Acquisition". arXiv: 1406.6595 [cs.CV].
  12. Hesam H. (2015). "DIY 3D scanner based on structured light and stereo vision in Python language".
  13. Wilm, Jakob; Olesen, Oline V.; Larsen, Rasmus (2014). "SLStudio: Open-source framework for real-time structured light". 2014 4th International Conference on Image Processing Theory, Tools and Applications (IPTA). pp. 1–4. doi:10.1109/IPTA.2014.7002001. ISBN   978-1-4799-6463-5. S2CID   206932100.

Sources

Further reading