Getting Started with PointNet++

PointNet++ is a popular neural network used for semantic segmentation of unorganized lidar point clouds. Semantic segmentation associates each point in a 3-D point cloud with a class label, such as car, truck, ground, or vegetation.

PointNet++ network partitions the input points into a set of clusters and then extracts the features using a multi-layer perceptron (MLP) network. The network applies PointNet recursively on the nested, partitioned inputs to extract multi-scale features for accurate semantic segmentation.

Applications of PointNet++ include:

• Tree segmentation for digital forestry applications.

• Extracting a digital terrain model from aerial lidar data.

• Perception for indoor navigation in robotics.

• 3-D city modelling from aerial lidar data.

PointNet++ Network

The PointNet++ network contains an encoder with set abstraction modules and a decoder with feature propagation modules.

The set abstraction module processes and extracts a set of points to produce a new set with fewer elements. Each set abstraction module contains a sampling and grouping layer followed by a mini-PointNet network.

• The sampling and grouping layer performs sampling by identifying the centroids of local regions. It then performs grouping by constructing local region sets of the neighboring points around the centroids.

• The mini-PointNet network contains a shared MLP network with a series of convolution, normalization, relu layers followed by a max pooling layer. It encodes the local region patterns into feature vectors.

The feature propagation module interpolates the subsampled points and then concatenates them with the point features from the set abstraction modules. The network then passes these features through the unit PointNet network.

The sampling & grouping layer of the set abstraction module and the interpolation layer of the feature propagation module in this network are implemented using the functionLayer (Deep Learning Toolbox) function.

Create PointNet++ Network

Use the pointnetplusLayers function to create a PointNet++ network for segmenting point cloud data.

Train PointNet++ Network

To learn how to train a PointNet++ network for segmenting point cloud data, see Aerial Lidar Semantic Segmentation Using PointNet++ Deep Learning.

Code Generation

To learn how to generate CUDA® code for a PointNet++ network, see Code Generation For Aerial Lidar Semantic Segmentation Using PointNet++ Deep Learning.

References

[1] Qi, Charles R., Li Yi, Hao Su, and Leonidas J. Guibas. ‘PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space’. ArXiv:1706.02413 [Cs], 7 June 2017. https://arxiv.org/abs/1706.02413.

[2] Varney, Nina, Vijayan K. Asari, and Quinn Graehling. ‘DALES: A Large-Scale Aerial LiDAR Data Set for Semantic Segmentation’. ArXiv:2004.11985 [Cs, Stat], 14 April 2020. https://arxiv.org/abs/2004.11985.