Deep Learning with Point Clouds
Lidar sensors record point cloud data that provides rich 3-D geometric information of their surroundings. You can process this data to get a better understanding of an environment, and use it for various applications in driving, robotics, medicine, forestry, construction, urban planning, and oceanography.
Point cloud data is highly unordered and sparse as it stores points in a 3-D space without any discretion. Additionally, factors such as sensor range, occlusions, and uneven sampling of points also affect the nature of point cloud data. These factors make point cloud processing a challenging task.
Deep learning addresses various challenges in processing point cloud data. It is easier to perform complex point cloud processing tasks such as segmentation, detection, and tracking, by training deep learning networks.
Segmentation clusters points in a point cloud and assigns class labels such as
buildingto those clusters.
Detection identifies and locates objects in a 3-D point cloud scene.
Tracking tracks the state of objects across different point cloud frames.
These are the common steps for any point cloud processing workflow using deep learning.
Lidar Toolbox™ provides functions to import and read raw point cloud data from several file formats. For more information, see I/O.
The toolbox enables you to divide this data into training and test data sets, and store
them as datastore objects. For example, you can store point cloud files by using the
fileDatastore object. For more information on datastore objects, see Datastores for Deep Learning (Deep Learning Toolbox).
The Import Point Cloud Data For Deep Learning example shows you how to import a large point cloud data set, and then configure and load a datastore.
Augment and Preprocess Data
In this step you prepare training data by labeling, augmenting, and preprocessing it.
You need a large, labeled data set to train a deep learning network. If you have an unlabeled data set, you can use the Lidar Labeler app to label your training data. For information on how to use the app, see Get Started with the Lidar Labeler.
You can preprocess your data before training the network. Lidar Toolbox provides function to perform various preprocessing tasks.
Denoise, downsample, and filter point clouds.
Convert unorganized data into the organized format.
Divide aerial point cloud data into blocks to perform block-by-block processing.
For more information on preprocessing, see Lidar Processing Applications (Deep Learning Toolbox).
Data augmentation adds variety to the existing training data. The robustness of a network to data transformations increases when you train it on a data set with a lot of variety.
Augmentation techniques reduce overfitting problems and enable the network to better learn and infer features.
For more information on how to perform data augmentation on point clouds, see Data Augmentations for Lidar Object Detection Using Deep Learning.
Every network has a unique architecture specific to the task you design it for. You define the network architecture based on the network input and the layers. Most deep learning networks either encode the point cloud into an image-like format, voxelize the point cloud, or operate directly on individual points.
Lidar Toolbox provides functions to create deep learning networks, as well as functions to use them for specific workflows.
To programmatically create a custom network layer-by-layer, use the functions specified in List of Deep Learning Layers (Deep Learning Toolbox). You can also interactively create the network by using the Deep Network Designer (Deep Learning Toolbox) app.
To visualize the network structure, use the
analyzeNetwork (Deep Learning Toolbox)
Test and Evaluate Results
Run the trained network on your test data set and evaluate the performance of the network. Depending on the task, you must use different functions to evaluate the results.
This functionality requires Deep Learning Toolbox™ licence.
- Aerial Lidar Semantic Segmentation Using PointNet++ Deep Learning
- Code Generation for Aerial Lidar Semantic Segmentation Using PointNet++ Deep Learning
- Lidar 3-D Object Detection Using PointPillars Deep Learning
- Code Generation for Lidar Object Detection Using PointPillars Deep Learning