Autonomous systems use multiple sensors such as camera, IMU, and radar in their sensor suite for environmental perception. Lidars can overcome some of the drawbacks of other sensors by providing highly accurate, structural, and 3D information of the surroundings. This advantage contributed to the introduction of lidar sensors into the mainstream perception market.

This market adoption of lidars is driven by three key reasons:

1. Low-cost lidars

The introduction of low-cost lidars, with enhancement in characteristics such as range, size, and robustness, have increased the availability of the technology for comparatively low-revenue industrial applications.

2. Accurate 3D data

Lidars gather high-density 3D information of the surroundings as point clouds with a higher accuracy than other range sensors such as radars and sonars. This, in turn, improves the accuracy of the 3D reconstruction.

3. Lidar processing algorithms

The recent developments in lidar processing workflows such as semantic segmentation, object detection and tracking, lidar camera data fusion, and lidar SLAM has enabled the industry to add lidars into their development workflows. You can use tools such as MATLAB to develop and apply lidar processing algorithms.

Lidars are used across a wide range of industries from automated driving to to geoscience. We can broadly classify these applications into three groups based on the platform they are mounted on.

1. Aerial lidars
2. Ground lidars
3. Indoor lidars

Aerial Lidars

Aerial lidars are lidar sensors mounted on unmanned aerial vehicles (UAV) or aircrafts. Aerial lidars capture 3D point cloud data of a large terrain that can be used for lidar mapping, feature extraction, terrain classification, and other use cases.

Examples of aerial lidar applications include:

• Agriculture: Lidar technology is extensively used in agriculture for mapping vegetation area and identifying the exact terrain of the farm and the water catchment area.
  
• Urban planning: Lidars are used in creating digital surface models (DSMs) or even digital city models (DCMs) of an area, which can help design a city or build new infrastructures in an existing city.
  
• Geological mapping: Lidars can be used to create 3D maps of the Earth’s surface, which can be further used in applications such as mining, precision forestry, and oil and gas exploration.
• Aerial navigation and path planning: Lidars are now being used in UAVs to gather live 3D data to navigate autonomously through the surroundings.

See examples of using MATLAB^® for aerial lidar processing:

• Aerial Lidar SLAM
• Terrain classification
• Aerial navigation and path planning

Ground Lidars

There are two types of ground lidars: stationary terrestrial lidars and mobile lidars.

• Stationary terrestrial lidars are lidars mounted on a stationary platform. They are commonly used for land surveys, road surveys, topological mapping, creating digital elevation maps (DEMs), agriculture, and other applications. These are more suited for applications where detailed and closer data capturing is needed.
• Mobile lidars are ground lidars attached on a mobile platform such as a car or a truck. The most important mobile lidar application is autonomous driving. Lidars mounted on the vehicles capture 3D point cloud data of the surroundings and they are further used in perception and navigation workflows. These workflows will be described in detail in the next section. 

See examples of using MATLAB for ground lidar processing:

• Track Vehicles Using Lidar: From Point Cloud to Track List
• Build a Map with Lidar Odometry and Mapping (LOAM) Using Unreal Engine Simulation

Indoor Lidars

Lidars are widely used in indoor robotics applications by mounting them on mobile robots. Apart from 3D lidars, 2D lidars or laser scanners are also used in indoor robotics applications like lidar scanning and mapping. They collect depth information of the surroundings and are then further processed based on the use cases.

Common uses of indoor lidars include:

• Lidar mapping and SLAM: You can use 2D or 3D lidars to create 2D or 3D SLAM and mapping, respectively.
• Obstacle detection, collision warning, and avoidance: 2D lidars are widely used to detect obstacles. This data can be further used to create collision warnings or to avoid obstacles.

See examples of using MATLAB for ground lidar processing:

1. Collision Warning Using 2D Lidar
2. Build Indoor Map from Lidar Scans Using SLAM

MATLAB and Lidar Toolbox™ simplify lidar processing tasks. With dedicated tools and functions, MATLAB helps you overcome common challenges in processing lidar data like 3D data types, sparsity of data, invalid points in the data, and high noises.

You can import live and recorded lidar data into MATLAB, implement lidar processing workflows, and create C/C++ and CUDA^® code to deploy into production.

Some of the important capabilities MATLAB provides in processing lidar point clouds are described in the following sections.

Streaming, Reading, and Writing Lidar Data

The first step in processing any sensor data in MATLAB is to get the data into the MATLAB workspace. You can:

• Stream live data from Velodyne sensors using the Velodyne Lidar Hardware Support Package and from Ouster sensors using the Ouster Lidar Hardware Support Package.
• Read stored point clouds in different file formats such as PCD, PLY, PCAP (Velodyne, Ouster, and Hesai Pandar), Ibeo data container, LAS, and LAZ.
• Write point clouds in different file formats such as PCD, PLY, LAS, and LAZ.
• Simulate lidar data, allowing you to test your algorithms and workflows before deploying to a real-world system. In MATLAB, you can synthesize 3D or 2D lidar data in simulation environments by defining sensor parameters for testing your processing algorithms. Lidar Toolbox, UAV Toolbox, and Automated Driving Toolbox provides lidar sensor models to simulate lidar point clouds.

Lidar Data Processing

You can preprocess lidar data to improve the quality of data and extract basic information from it. Lidar Toolbox™ provides functionality for downsampling, median filtering, aligning, transforming, and extracting features from point clouds.

Lidar Camera Calibration

MATLAB enables lidar camera calibration to estimate lidar-camera transforms for fusing camera and lidar data. You can further fuse color information in lidar point clouds and estimate 3D bounding boxes in lidar using 2D bounding boxes from a co-located camera.

Deep Learning for Lidar

With MATLAB, you can apply deep learning algorithms for object detection and semantic segmentation on lidar data.

• With just a few lines of code in MATLAB, you can import pretrained semantic segmentation models, including PointSeg and SqueezeSegV2 on lidar data. You can also train, evaluate, and deploy your own deep learning models.
• MATLAB enables designing, training, and evaluating robust detectors such as PointPillars and ComplexYolo-V4 networks. You can detect and fit oriented bounding boxes around objects in lidar point clouds.
• The Lidar Labeler app in Lidar Toolbox simplifies point cloud labeling tasks. You can manually label point clouds for object detection and semantic segmentation, apply built-in or custom algorithms to automate lidar point cloud labeling, and evaluate automation algorithm performance.

Object Tracking on Point Clouds

MATLAB can unify multiple domains that feed into an end-to-end object tracking workflow. This enables you to read lidar data, preprocess it, apply deep learning to detect objects, track these objects using a predefined tracker, and deploy this on target hardware.

Point Cloud Registration and SLAM

MATLAB provides functions to register lidar point clouds and build 3D maps from ground and aerial lidar data using SLAM algorithms. This includes:

• Feature-based map building
• Map building with the lidar odometry and mapping (LOAM) method
• Segment matching

For more information about lidar processing, see Lidar Toolbox and Computer Vision Toolbox™.

Examples and How To

Software Reference

See also: Automated Driving Toolbox™, Navigation Toolbox™, Robotics System Toolbox™, MATLAB and Simulink for automated driving systems, MATLAB and Simulink for robotics and autonomous systems, computer vision