Point cloud

Last updated
A point cloud image of a torus Point cloud torus.gif
A point cloud image of a torus
Geo-referenced point cloud of Red Rocks, Colorado (by DroneMapper) Geo-Referenced Point Cloud.JPG
Geo-referenced point cloud of Red Rocks, Colorado (by DroneMapper)

A point cloud is a discrete set of data points in space. The points may represent a 3D shape or object. Each point position has its set of Cartesian coordinates (X, Y, Z). [1] [2] Points may contain data other than position such as RGB colors, [2] normals, [3] timestamps [4] and others. Point clouds are generally produced by 3D scanners or by photogrammetry software, which measure many points on the external surfaces of objects around them. As the output of 3D scanning processes, point clouds are used for many purposes, including to create 3D computer-aided design (CAD) or geographic information systems (GIS) models for manufactured parts, for metrology and quality inspection, and for a multitude of visualizing, animating, rendering, and mass customization applications.

Contents

Alignment and registration

When scanning a scene in real world using LiDar, the captured point clouds contain snippets of the scene, which requires alignment to generate a full map of the scanned environment.

Point clouds are often aligned with 3D models or with other point clouds, a process termed point set registration.

The Iterative closest point (ICP) algorithm can be used to align two point clouds that have an overlap between them, and are separated by a rigid transform. [5] Point clouds with elastic transforms can also be aligned by using a non-rigid variant of the ICP (NICP). [6] With advancements in machine learning in recent years, point cloud registration may also be done using end-to-end neural networks. [7]

For industrial metrology or inspection using industrial computed tomography, the point cloud of a manufactured part can be aligned to an existing model and compared to check for differences. Geometric dimensions and tolerances can also be extracted directly from the point cloud.

Conversion to 3D surfaces

An example of a 1.2 billion data point cloud render of Beit Ghazaleh, a heritage site in danger in Aleppo (Syria) Extract Video Beit Ghazaleh Orthophoto Survey AG&P 2017.gif
An example of a 1.2 billion data point cloud render of Beit Ghazaleh, a heritage site in danger in Aleppo (Syria)
Generating or reconstructing 3D shapes from single or multi-view depth maps or silhouettes and visualizing them in dense point clouds Synthesizing 3D Shapes via Modeling Multi-View Depth Maps and Silhouettes With Deep Generative Networks.png
Generating or reconstructing 3D shapes from single or multi-view depth maps or silhouettes and visualizing them in dense point clouds

While point clouds can be directly rendered and inspected, [10] [11] point clouds are often converted to polygon mesh or triangle mesh models, non-uniform rational B-spline (NURBS) surface models, or CAD models through a process commonly referred to as surface reconstruction.

There are many techniques for converting a point cloud to a 3D surface. [12] Some approaches, like Delaunay triangulation, alpha shapes, and ball pivoting, build a network of triangles over the existing vertices of the point cloud, while other approaches convert the point cloud into a volumetric distance field and reconstruct the implicit surface so defined through a marching cubes algorithm. [13]

In geographic information systems, point clouds are one of the sources used to make digital elevation model of the terrain. [14] They are also used to generate 3D models of urban environments. [15] Drones are often used to collect a series of RGB images which can be later processed on a computer vision algorithm platform such as on AgiSoft Photoscan, Pix4D, DroneDeploy or Hammer Missions to create RGB point clouds from where distances and volumetric estimations can be made.[ citation needed ]

Point clouds can also be used to represent volumetric data, as is sometimes done in medical imaging. Using point clouds, multi-sampling and data compression can be achieved. [16]

MPEG Point Cloud Compression

MPEG began standardizing point cloud compression (PCC) with a Call for Proposal (CfP) in 2017. [17] [18] [19] Three categories of point clouds were identified: category 1 for static point clouds, category 2 for dynamic point clouds, and category 3 for LiDAR sequences (dynamically acquired point clouds). Two technologies were finally defined: G-PCC (Geometry-based PCC, ISO/IEC 23090 part 9) [20] for category 1 and category 3; and V-PCC (Video-based PCC, ISO/IEC 23090 part 5) [21] for category 2. The first test models were developed in October 2017, one for G-PCC (TMC13) and another one for V-PCC (TMC2). Since then, the two test models have evolved through technical contributions and collaboration, and the first version of the PCC standard specifications was expected to be finalized in 2020 as part of the ISO/IEC 23090 series on the coded representation of immersive media content. [22]

See also

Related Research Articles

<span class="mw-page-title-main">Moving Picture Experts Group</span> Alliance of working groups to set standards for multimedia coding

The Moving Picture Experts Group (MPEG) is an alliance of working groups established jointly by ISO and IEC that sets standards for media coding, including compression coding of audio, video, graphics, and genomic data; and transmission and file formats for various applications. Together with JPEG, MPEG is organized under ISO/IEC JTC 1/SC 29 – Coding of audio, picture, multimedia and hypermedia information.

MPEG-4 is a group of international standards for the compression of digital audio and visual data, multimedia systems, and file storage formats. It was originally introduced in late 1998 as a group of audio and video coding formats and related technology agreed upon by the ISO/IEC Moving Picture Experts Group (MPEG) under the formal standard ISO/IEC 14496 – Coding of audio-visual objects. Uses of MPEG-4 include compression of audiovisual data for Internet video and CD distribution, voice and broadcast television applications. The MPEG-4 standard was developed by a group led by Touradj Ebrahimi and Fernando Pereira.

<span class="mw-page-title-main">JPEG 2000</span> Image compression standard and coding system

JPEG 2000 (JP2) is an image compression standard and coding system. It was developed from 1997 to 2000 by a Joint Photographic Experts Group committee chaired by Touradj Ebrahimi, with the intention of superseding their original JPEG standard, which is based on a discrete cosine transform (DCT), with a newly designed, wavelet-based method. The standardized filename extension is .jp2 for ISO/IEC 15444-1 conforming files and .jpx for the extended part-2 specifications, published as ISO/IEC 15444-2. The registered MIME types are defined in RFC 3745. For ISO/IEC 15444-1 it is image/jp2.

X3D is a set of royalty-free ISO/IEC standards for declaratively representing 3D computer graphics. X3D includes multiple graphics file formats, programming-language API definitions, and run-time specifications for both delivery and integration of interactive network-capable 3D data. X3D version 4.0 has been approved by Web3D Consortium, and is under final review by ISO/IEC as a revised International Standard (IS).

<span class="mw-page-title-main">Volume rendering</span> Representing a 3D-modeled object or dataset as a 2D projection

In scientific visualization and computer graphics, volume rendering is a set of techniques used to display a 2D projection of a 3D discretely sampled data set, typically a 3D scalar field.

Registration authorities (RAs) exist for many standards organizations, such as ANNA, the Object Management Group, W3C, and others. In general, registration authorities all perform a similar function, in promoting the use of a particular standard through facilitating its use. This may be by applying the standard, where appropriate, or by verifying that a particular application satisfies the standard's tenants. Maintenance agencies, in contrast, may change an element in a standard based on set rules – such as the creation or change of a currency code when a currency is created or revalued. The Object Management Group has an additional concept of certified provider, which is deemed an entity permitted to perform some functions on behalf of the registration authority, under specific processes and procedures documented within the standard for such a role.

<span class="mw-page-title-main">Motion estimation</span> Process used in video coding/compression

In computer vision and image processing, motion estimation is the process of determining motion vectors that describe the transformation from one 2D image to another; usually from adjacent frames in a video sequence. It is an ill-posed problem as the motion happens in three dimensions (3D) but the images are a projection of the 3D scene onto a 2D plane. The motion vectors may relate to the whole image or specific parts, such as rectangular blocks, arbitrary shaped patches or even per pixel. The motion vectors may be represented by a translational model or many other models that can approximate the motion of a real video camera, such as rotation and translation in all three dimensions and zoom.

<span class="mw-page-title-main">Iterative closest point</span> Algorithm

Iterative closest point (ICP) is an algorithm employed to minimize the difference between two clouds of points. ICP is often used to reconstruct 2D or 3D surfaces from different scans, to localize robots and achieve optimal path planning, to co-register bone models, etc.

<span class="mw-page-title-main">3D scanning</span> Scanning of an object or environment to collect data on its shape

3D scanning is the process of analyzing a real-world object or environment to collect three dimensional data of its shape and possibly its appearance. The collected data can then be used to construct digital 3D models.

DSSP stands for digital shape sampling and processing. It is an alternative and often preferred way of describing "reverse engineering" software and hardware. The term originated in a 2005 Society of Manufacturing Engineers' "Blue Book" on the topic, which referenced numerous suppliers of both scanning hardware and processing software.

The Video Coding Experts Group or Visual Coding Experts Group is a working group of the ITU Telecommunication Standardization Sector (ITU-T) concerned with standards for compression coding of video, images, audio signals, biomedical waveforms, and other signals. It is responsible for standardization of the "H.26x" line of video coding standards, the "T.8xx" line of image coding standards, and related technologies.

Image-based meshing is the automated process of creating computer models for computational fluid dynamics (CFD) and finite element analysis (FEA) from 3D image data. Although a wide range of mesh generation techniques are currently available, these were usually developed to generate models from computer-aided design (CAD), and therefore have difficulties meshing from 3D imaging data.

<span class="mw-page-title-main">3D modeling</span> Form of computer-aided engineering

In 3D computer graphics, 3D modeling is the process of developing a mathematical coordinate-based representation of a surface of an object in three dimensions via specialized software by manipulating edges, vertices, and polygons in a simulated 3D space.

<span class="mw-page-title-main">Point Cloud Library</span> Open-source algorithm library

The Point Cloud Library (PCL) is an open-source library of algorithms for point cloud processing tasks and 3D geometry processing, such as occur in three-dimensional computer vision. The library contains algorithms for filtering, feature estimation, surface reconstruction, 3D registration, model fitting, object recognition, and segmentation. Each module is implemented as a smaller library that can be compiled separately. PCL has its own data format for storing point clouds - PCD, but also allows datasets to be loaded and saved in many other formats. It is written in C++ and released under the BSD license.

<span class="mw-page-title-main">3D reconstruction from multiple images</span> Creation of a 3D model from a set of images

3D reconstruction from multiple images is the creation of three-dimensional models from a set of images. It is the reverse process of obtaining 2D images from 3D scenes.

<span class="mw-page-title-main">Digital outcrop model</span> Digital 3D representation of the outcrop surface

A digital outcrop model (DOM), also called a virtual outcrop model, is a digital 3D representation of the outcrop surface, mostly in a form of textured polygon mesh.

<span class="mw-page-title-main">CloudCompare</span>

CloudCompare is a 3D point cloud processing software. It can also handle triangular meshes and calibrated images.

Volumetric capture or volumetric video is a technique that captures a three-dimensional space, such as a location or performance. This type of volumography acquires data that can be viewed on flat screens as well as using 3D displays and VR goggles. Consumer-facing formats are numerous and the required motion capture techniques lean on computer graphics, photogrammetry, and other computation-based methods. The viewer generally experiences the result in a real-time engine and has direct input in exploring the generated volume.

JPEG XS is an interoperable, visually lossless, low-latency and lightweight image and video coding system used in professional applications. Applications of the standard include streaming high-quality content for virtual reality, drones, autonomous vehicles using cameras, gaming, and broadcasting. It was the first ISO codec ever designed for this specific purpose. JPEG XS, built on core technology from both intoPIX and Fraunhofer IIS, is formally standardized as ISO/IEC 21122 by the Joint Photographic Experts Group with the first edition published in 2019. Although not official, the XS acronym was chosen to highlight the eXtra Small and eXtra Speed characteristics of the codec. Today, the JPEG committee is still actively working on further improvements to XS, with the second edition scheduled for publication and initial efforts being launched towards a third edition.

Gaussian Splatting is a volume rendering technique that deals with the direct rendering of volume data without converting the data into surface or line primitives. The technique was originally introduced as splatting by Lee Westover in the early 1990s. With advancements in computer graphics, newer methods such as 3D and 4D Gaussian splatting have been developed to offer real-time radiance field rendering and dynamic scene rendering respectively.

References

  1. ↑ "What are Point Clouds". Tech27.
  2. 1 2 "What is a Point Cloud? - GIGABYTE Global". GIGABYTE. Retrieved 2024-06-26.
  3. ↑ Simsangcheol (2023-02-21). "Estimate normals in Point Cloud". Medium. Retrieved 2024-06-26.
  4. ↑ "Defra Data Services Platform". environment.data.gov.uk. Retrieved 2024-06-26.
  5. ↑ "Continuous ICP (CICP)". www.cs.cmu.edu. Retrieved 2024-06-26.
  6. ↑ Li, Hao; Sumner, Robert W.; Pauly, Mark (July 2008). "Global Correspondence Optimization for Non-Rigid Registration of Depth Scans". Computer Graphics Forum. 27 (5): 1421–1430. doi:10.1111/j.1467-8659.2008.01282.x. ISSN   0167-7055.
  7. ↑ Lu, Weixin; Wan, Guowei; Zhou, Yao; Fu, Xiangyu; Yuan, Pengfei; Song, Shiyu (2019). "DeepVCP: An End-to-End Deep Neural Network for Point Cloud Registration": 12–21.{{cite journal}}: Cite journal requires |journal= (help)
  8. ↑ English: Image from a very high precision 3D laser scanner survey (1.2 billion data points) of Beit Ghazaleh -- a heritage site in danger in Aleppo Syria. This was a collaborative scientific work for the study, safeguarding and emergency consolidation of remains of the structure., 2017-11-02, retrieved 2018-06-11
  9. ↑ "Soltani, A. A., Huang, H., Wu, J., Kulkarni, T. D., & Tenenbaum, J. B. Synthesizing 3D Shapes via Modeling Multi-View Depth Maps and Silhouettes With Deep Generative Networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 1511-1519)". GitHub . 27 January 2022.
  10. ↑ Levoy, M. and Whitted, T., "The use of points as a display primitive".. Technical Report 85-022, Computer Science Department, University of North Carolina at Chapel Hill, January, 1985
  11. ↑ Rusinkiewicz, S. and Levoy, M. 2000. QSplat: a multiresolution point rendering system for large meshes. In Siggraph 2000. ACM, New York, NY, 343–352. DOI= http://doi.acm.org/10.1145/344779.344940
  12. ↑ Berger, M., Tagliasacchi, A., Seversky, L. M., Alliez, P., Guennebaud, G., Levine, J. A., Sharf, A. and Silva, C. T. (2016), A Survey of Surface Reconstruction from Point Clouds. Computer Graphics Forum.
  13. ↑ Meshing Point Clouds A short tutorial on how to build surfaces from point clouds
  14. ↑ From Point Cloud to Grid DEM: A Scalable Approach
  15. ↑ K. Hammoudi, F. Dornaika, B. Soheilian, N. Paparoditis. Extracting Wire-frame Models of Street Facades from 3D Point Clouds and the Corresponding Cadastral Map. International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences (IAPRS), vol. 38, part 3A, pp. 91–96, Saint-Mandé, France, 1–3 September 2010.
  16. ↑ Sitek; et al. (2006). "Tomographic Reconstruction Using an Adaptive Tetrahedral Mesh Defined by a Point Cloud". IEEE Trans. Med. Imaging. 25 (9): 1172–9. doi:10.1109/TMI.2006.879319. PMID   16967802. S2CID   27545238.
  17. ↑ "MPEG Point Cloud Compression" . Retrieved 2020-10-22.
  18. ↑ Schwarz, Sebastian; Preda, Marius; Baroncini, Vittorio; Budagavi, Madhukar; Cesar, Pablo; Chou, Philip A.; Cohen, Robert A.; Krivokuća, Maja; Lasserre, Sébastien; Li, Zhu; Llach, Joan; Mammou, Khaled; Mekuria, Rufael; Krivokuća, Maja; Nakagami, Ohji; Siahaan, Ernestasia; Tabatabai, Ali; Tourapis, Alexis M.; Zakharchenko, Vladyslav (2018-12-10). "Emerging MPEG Standards for Point Cloud Compression". IEEE Journal on Emerging and Selected Topics in Circuits and Systems. 9 (1): 133–148. doi: 10.1109/JETCAS.2018.2885981 .
  19. ↑ Graziosi, Danillo; Nakagami, Ohji; Kuma, Satoru; Zaghetto, Alexandre; Suzuki, Teruhiko; Tabatabai, Ali (2020-04-03). "An overview of ongoing point cloud compression standardization activities: video-based (V-PCC) and geometry-based (G-PCC)". APSIPA Transactions on Signal and Information Processing. 9: 1–17. doi: 10.1017/ATSIP.2020.12 .
  20. ↑ "ISO/IEC DIS 23090-9". ISO. Retrieved 2020-06-07.
  21. ↑ "ISO/IEC DIS 23090-5". ISO. Retrieved 2020-10-21.
  22. ↑ "Immersive Media Architectures | MPEG". mpeg.chiariglione.org. Retrieved 2020-06-07.
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.