This article may rely excessively on sources too closely associated with the subject , potentially preventing the article from being verifiable and neutral.(October 2014) |
Original author(s) | Willow Garage |
---|---|
Initial release | March 2010 [1] [2] |
Stable release | |
Repository | |
Operating system | Cross-platform |
Type | Library |
License | BSD license |
Website | pointclouds |
The Point Cloud Library (PCL) is an open-source library of algorithms for point cloud processing tasks and 3D geometry processing, such as occur in three-dimensional computer vision. The library contains algorithms for filtering, feature estimation, surface reconstruction, 3D registration, [5] model fitting, object recognition, and segmentation. Each module is implemented as a smaller library that can be compiled separately (for example, libpcl_filters, libpcl_features, libpcl_surface, ...). PCL has its own data format for storing point clouds - PCD (Point Cloud Data), but also allows datasets to be loaded and saved in many other formats. It is written in C++ and released under the BSD license.
These algorithms have been used, for example, for perception in robotics to filter outliers from noisy data, stitch 3D point clouds together, segment relevant parts of a scene, extract keypoints and compute descriptors to recognize objects in the world based on their geometric appearance, and create surfaces from point clouds and visualize them. [6] [ failed verification ]
PCL requires several third-party libraries to function, which must be installed. Most mathematical operations are implemented using the Eigen library. The visualization module for 3D point clouds is based on VTK. Boost is used for shared pointers and the FLANN library for quick k-nearest neighbor search. Additional libraries such as Qhull, OpenNI, or Qt are optional and extend PCL with additional features.
PCL is cross-platform software that runs on the most commonly used operating systems: Linux, Windows, macOS and Android. The library is fully integrated with the Robot Operating System (ROS) and provides support for OpenMP and Intel Threading Building Blocks (TBB) libraries for multi-core parallelism. [7] [8]
The library is constantly updated and expanded, and its use in various industries is constantly growing. For example, PCL participated in the Google Summer of Code 2020 initiative with three projects. One was the extension of PCL for use with Python using Pybind11. [9]
A large number of examples and tutorials are available on the PCL website, either as C++ source files or as tutorials with a detailed description and explanation of the individual steps.
Point cloud library is widely used in many different fields, here are some examples:
PCL requires for its installation several third-party libraries, which are listed below. Some libraries are optional and extend PCL with additional features. The PCL library is built with the CMake build system (http://www.cmake.org/) at least in version 3.5.0. [10] [8]
Mandatory libraries:
Optional libraries that enable some additional features:
The PCD (Point Cloud Data) is a file format for storing 3D point cloud data. It was created because existing formats did not support some of the features provided by the PCL library. PCD is the primary data format in PCL, but the library also offers the ability to save and load data in other formats (such as PLY, IFS, VTK, STL, OBJ, X3D). However, these other formats do not have the flexibility and speed of PCD files. One of the PCD advantages is the ability to store and process organized point cloud datasets. Another is very fast saving and loading of points that are stored in binary form. [11] [12]
The PCD version is specified with the numbers 0.x (e.g., 0.5, 0.6, etc.) in the header of each file. The official version in 2020 is PCD 0.7 (PCD_V7). The main difference compared to version 0.6 is that a new header - VIEWPOINT has been added. It specifies the information about the orientation of the sensor relative to the dataset. [13]
The PCD file is divided into two parts - header and data. The header has a precisely defined format and contains the necessary information about the point cloud data that are stored in it. The header must be encoded in ASCII, however, the data can be stored in ASCII or binary format. Thanks to the fact that the ASCII format is more human readable, it can be opened in standard software tools and easily edited.
In version 0.7 the version of the PCD file is at the beginning of the header, followed by the name, size, and type of each dimension of the stored data. It also shows a number of points (height*width) in the whole cloud and information about whether the point cloud dataset is organized or unorganized. The data type specifies in which format the point cloud data are stored (ASCII or binary). The header is followed by a set of points. Each point can be stored on a separate line (unorganized point-cloud) or they are stored in an image-like organized structure (organized point-cloud). [11] More detailed information about header entries can be found in documentation. Below is an example of a PCD file. The order of header entries is important!
# .PCD v.7 - Point Cloud Data file format VERSION .7 FIELDS x y z rgb SIZE 4 4 4 4 TYPE F F F F COUNT 1 1 1 1 WIDTH 213 HEIGHT 1 VIEWPOINT 0 0 0 1 0 0 0 POINTS 213 DATA ascii 0.93773 0.33763 0 4.2108e+06 0.90805 0.35641 0 4.2108e+06 0.81915 0.32 0 4.2108e+06 0.97192 0.278 0 4.2108e+06 ... ...
The development of the Point Cloud Library started in March 2010 at Willow Garage. The project initially resided on a sub domain of Willow Garage then moved to a new website www.pointclouds.org in March 2011. [1] PCL's first official release (Version 1.0) was released two months later in May 2011. [2]
PCL is divided into several smaller code libraries that can be compiled separately. Some of the most important modules and their functions are described below. [14] [15]
When scanning a 3D point cloud, errors and various deviations can occur, which causes noise in the data. This complicates the estimation of some local point cloud characteristics, such as surface normals. These inaccuracies can lead to significant errors in further processing and it is therefore advisable to remove them with a suitable filter. The pcl_filters library provides several useful filters for removing outliers and noise and also downsampling the data. Some of them use simple criteria to trim points, others use statistical analysis.
The pcl_features library contains algorithms and data structures for 3D feature estimation. Mostly used local geometric features are the point normal and underlying surface's estimated curvature. The features describe geometrical patterns at a certain point based on selected k-neighborhood (data space selected around the point). The neighborhood can be selected by determining a fixed number of points in the closest area or defining a radius of a sphere around the point.
One of the easiest implemented methods for estimating the surface normal is an analysis of the eigenvectors and eigenvalues of a covariance matrix created from the neighborhood of the point. Point Feature Histograms (or faster FPFH) descriptors are an advanced feature representation and depend on normal estimations at each point. It generalizes the mean curvature around the point using a multidimensional histogram of values. Some of other descriptors in the library are Viewpoint Feature Histogram (VFH) descriptor, NARF descriptors, Moment of inertia and eccentricity based descriptors, Globally Aligned Spatial Distribution (GASD) descriptors, and more.
The pcl_segmentation library contains algorithms for segmenting a point cloud into different clusters. Clustering is often used to divide the cloud into individual parts, that can be further processed. There are implemented several classes, that support various segmentation methods:
The pcl_visualization library is used to quickly and easily visualize 3D point cloud data. The package makes use of the VTK library for 3D rendering of clouds and range images. The library offers:
Registration is the problem of aligning various point cloud datasets acquired from different views into a single point cloud model. The pcl_registration library implements number of point cloud registration algorithms for both organized and unorganized datasets. The task is to identify the corresponding points between the data sets and find a transformation that minimizes their distance.
The iterative closest point algorithm minimizes the distances between the points of two pointclouds. It can be used for determining if one PointCloud is just a rigid transformation of another. Normal Distributions Transform (NDT) is a registration algorithm that can be used to determine a rigid transformation between two point clouds that have over 100,000 points.
The sample_consensus library holds SAmple Consensus (SAC) methods like RANSAC and models to detect specific objects in point clouds. Some of the models implemented in this library include plane models that are often used to detect interior surfaces such as walls and floors. Next models are the lines, 2D and 3D circles in a plane, sphere, cylinder, cone, a model for determining a line parallel with a given axis, a model for determining a plane perpendicular to a user-specified axis, plane parallel to a user-specified axis, etc. These can be used to detect objects with common geometric structures (e.g., fitting a cylinder model to a mug).
Robust sample consensus estimators that are available in the library:
Several algorithms for surface reconstruction of 3D point clouds are implemented in the pcl_surface library. There are several ways to reconstruct the surface. One of the most commonly used is meshing, and the PCL library has two algorithms: very fast triangulation of original points and slower networking, which also smooths and fills holes. If the cloud is noisy, it is advisable to use surface smoothing using one of the implemented algorithms.
The Moving Least Squares (MLS) surface reconstruction method is a resampling algorithm that can reconstruct missing parts of a surface. Thanks to higher order polynomial interpolations between surrounding data points, MLS can correct and smooth out small errors caused by scanning.
Greedy Projection Triangulation implements an algorithm for fast surface triangulation on an unordered PointCloud with normals. The result is a triangle mesh that is created by projecting the local neighborhood of a point along the normal of the point. It works best if the surface is locally smooth and there are smooth transitions between areas with different point densities. Many parameters can be set that are taken into account when connecting points (how many neighbors are searched, the maximum distance for a point, minimum and maximum angle of a triangle).
The library also implements functions for creating a concave or convex hull polygon for a plane model, Grid projection surface reconstruction algorithm, marching cubes, ear clipping triangulation algorithm, Poisson surface reconstruction algorithm, etc.
The io_library allows you to load and save point clouds to files, as well as capture clouds from various devices. It includes functions that allow you to concatenate the points of two different point clouds with the same type and number of fields. The library can also concatenate fields (e.g., dimensions) of two different point clouds with same number of points.
Starting with PCL 1.0 the library offers a new generic grabber interface that provides easy access to different devices and file formats. The first devices supported for data collection were OpenNI compatible cameras (tested with Primesense Reference Design, Microsoft Kinect and Asus Xtion Pro cameras). As of PCL 1.7, point cloud data can be also obtained from the Velodyne High Definition LiDAR (HDL) system, which produces 360 degree point clouds. PCL supports both the original HDL-64e and HDL-32e. There is also a new driver for Dinast Cameras (tested with IPA-1110, Cyclopes II and IPA-1002 ng T-Less NG). PCL 1.8 brings support for IDS-Imaging Ensenso cameras, DepthSense cameras (e.g. Creative Senz3D, DepthSense DS325), and davidSDK scanners.
The pcl_kdtree library provides the kd-tree data-structure for organizing a set of points in a space with k dimensions. Used to find the K nearest neighbors (using FLANN) of a specific point or location.
The pcl_octree library implements the octree hierarchical tree data structure for point cloud data. The library provides nearest neighbor search algorithms, such as “Neighbors within Voxel Search”, “K Nearest Neighbor Search” and “Neighbors within Radius Search”. There are also several octree types that differ by their leaf node's properties. Each leaf node can hold a single point or a list of point indices, or it does not store any point information. The library can be also used for detection of spatial changes between multiple unorganized point clouds by recursive comparison of octet tree structures.
The pcl_search library implements methods for searching for nearest neighbors using different data structures, that can be found in other modules, such as KdTree, Octree, or specialized search for organized datasets.
The range_image library contains two classes for representing and working with range images whose pixel values represent a distance from the sensor. The range image can be converted to a point cloud if the sensor position is specified or the borders can be extracted from it.
The pcl_keypoints library contains implementations of point cloud keypoint detection algorithms (AGAST corner point detector, Harris detector, BRISK detector, etc.).
The pcl_common library contains the core data structures for point cloud, types for point representation, surface normals, RGB color values, etc. There are also implemented useful methods for computing distances, mean values and covariance, geometric transformations, and more. The common library is mainly used by other PCL modules.
Computer vision tasks include methods for acquiring, processing, analyzing and understanding digital images, and extraction of high-dimensional data from the real world in order to produce numerical or symbolic information, e.g. in the forms of decisions. Understanding in this context means the transformation of visual images into descriptions of the world that make sense to thought processes and can elicit appropriate action. This image understanding can be seen as the disentangling of symbolic information from image data using models constructed with the aid of geometry, physics, statistics, and learning theory.
A point cloud is a discrete set of data points in space. The points may represent a 3D shape or object. Each point position has its set of Cartesian coordinates. Point clouds are generally produced by 3D scanners or by photogrammetry software, which measure many points on the external surfaces of objects around them. As the output of 3D scanning processes, point clouds are used for many purposes, including to create 3D computer-aided design (CAD) or geographic information systems (GIS) models for manufactured parts, for metrology and quality inspection, and for a multitude of visualizing, animating, rendering, and mass customization applications.
In scientific visualization and computer graphics, volume rendering is a set of techniques used to display a 2D projection of a 3D discretely sampled data set, typically a 3D scalar field.
The scale-invariant feature transform (SIFT) is a computer vision algorithm to detect, describe, and match local features in images, invented by David Lowe in 1999. Applications include object recognition, robotic mapping and navigation, image stitching, 3D modeling, gesture recognition, video tracking, individual identification of wildlife and match moving.
Iterative closest point (ICP) is an algorithm employed to minimize the difference between two clouds of points. ICP is often used to reconstruct 2D or 3D surfaces from different scans, to localize robots and achieve optimal path planning, to co-register bone models, etc.
JT is an openly-published ISO-standardized 3D CAD data exchange format used for product visualization, collaboration, digital mockups, and other purposes. It was developed by Siemens.
In computer graphics, marching squares is an algorithm that generates contours for a two-dimensional scalar field. A similar method can be used to contour 2D triangle meshes.
Materialise Mimics is an image processing software for 3D design and modeling, developed by Materialise NV, a Belgian company specialized in additive manufacturing software and technology for medical, dental and additive manufacturing industries. Materialise Mimics is used to create 3D surface models from stacks of 2D image data. These 3D models can then be used for a variety of engineering applications. Mimics is an acronym for Materialise Interactive Medical Image Control System. It is developed in an ISO environment with CE and FDA 510k premarket clearance. Materialise Mimics is commercially available as part of the Materialise Mimics Innovation Suite, which also contains Materialise3-matic, a design and meshing software for anatomical data. The current version is 24.0(released in 2021), and it supports Windows 10, Windows 7, Vista and XP in x64.
MeshLab is a 3D mesh processing software system that is oriented to the management and processing of unstructured large meshes and provides a set of tools for editing, cleaning, healing, inspecting, rendering, and converting these kinds of meshes. MeshLab is free and open-source software, subject to the requirements of the GNU General Public License (GPL), version 2 or later, and is used as both a complete package and a library powering other software. It is well known in the more technical fields of 3D development and data handling.
Nearest neighbor search (NNS), as a form of proximity search, is the optimization problem of finding the point in a given set that is closest to a given point. Closeness is typically expressed in terms of a dissimilarity function: the less similar the objects, the larger the function values.
Livewire, is a segmentation technique which allows a user to select regions of interest to be extracted quickly and accurately, using simple mouse clicks. It is based on the lowest cost path algorithm, by Edsger W. Dijkstra. Firstly convolve the image with a Sobel filter to extract edges. Each pixel of the resulting image is a vertex of the graph and has edges going to the 4 pixels around it, as up, down, left, right. The edge costs are defined based on a cost function. In 1995, Eric N. Mortensen and William A. Barrett made some extension work on livewire segmentation tool, which is known as Intelligent Scissors.
PLY is a computer file format known as the Polygon File Format or the Stanford Triangle Format. It was principally designed to store three-dimensional data from 3D scanners. The data storage format supports a relatively simple description of a single object as a list of nominally flat polygons. A variety of properties can be stored, including color and transparency, surface normals, texture coordinates and data confidence values. The format permits one to have different properties for the front and back of a polygon.
ParaView is an open-source multiple-platform application for interactive, scientific visualization. It has a client–server architecture to facilitate remote visualization of datasets, and generates level of detail (LOD) models to maintain interactive frame rates for large datasets. It is an application built on top of the Visualization Toolkit (VTK) libraries. ParaView is an application designed for data parallelism on shared-memory or distributed-memory multicomputers and clusters. It can also be run as a single-computer application.
3D Slicer (Slicer) is a free and open source software package for image analysis and scientific visualization. Slicer is used in a variety of medical applications, including autism, multiple sclerosis, systemic lupus erythematosus, prostate cancer, lung cancer, breast cancer, schizophrenia, orthopedic biomechanics, COPD, cardiovascular disease and neurosurgery.
ELKI is a data mining software framework developed for use in research and teaching. It was originally created by the database systems research unit at the Ludwig Maximilian University of Munich, Germany, led by Professor Hans-Peter Kriegel. The project has continued at the Technical University of Dortmund, Germany. It aims at allowing the development and evaluation of advanced data mining algorithms and their interaction with database index structures.
A digital outcrop model (DOM), also called a virtual outcrop model, is a digital 3D representation of the outcrop surface, mostly in a form of textured polygon mesh.
IMOD is an open-source, cross-platform suite of modeling, display and image processing programs used for 3D reconstruction and modeling of microscopy images with a special emphasis on electron microscopy data. IMOD has been used across a range of scales from macromolecule structures to organelles to whole cells and can also be used for optical sections. IMOD includes tools for image reconstruction, image segmentation, 3D mesh modeling and analysis of 2D and 3D data.
CloudCompare is a 3D point cloud processing software. It can also handle triangular meshes and calibrated images.
Vaa3D is an Open Source visualization and analysis software suite created mainly by Hanchuan Peng and his team at Janelia Research Campus, HHMI and Allen Institute for Brain Science. The software performs 3D, 4D and 5D rendering and analysis of very large image data sets, especially those generated using various modern microscopy methods, and associated 3D surface objects. This software has been used in several large neuroscience initiatives and a number of applications in other domains. In a recent Nature Methods review article, it has been viewed as one of the leading open-source software suites in the related research fields. In addition, research using this software was awarded the 2012 Cozzarelli Prize from the National Academy of Sciences.
Geological structure measurement by LiDAR technology is a remote sensing method applied in structural geology. It enables monitoring and characterisation of rock bodies. This method's typical use is to acquire high resolution structural and deformational data for identifying geological hazards risk, such as assessing rockfall risks or studying pre-earthquake deformation signs.