Labeler Enhancements: Rotated rectangle ROI label support
This table describes enhancements for these labeling apps:
| Feature | Image Labeler | Video Labeler | Ground Truth Labeler | Lidar Labeler | Medical Image Labeler |
|---|---|---|---|---|---|
| New Image Labeler interface | Yes | No | No | No | No |
| Rotated rectangle ROI support | Yes | Yes | Yes | No | No |
| Rotated rectangle ROI support for temporal interpolation and point tracker automation algorithms | No | Yes | Yes | No | No |
| Merge ground truth objects | Yes | No | No | No | Yes |
Load pointCloud objects from workspace | No | No | No | Yes | No |
| Visualize point cloud intensity using colormap value | No | No | No | Yes | No |
| Cluster point cloud using DBSCAN clustering algorithm | No | No | No | Yes | No |
| Use burst mode to label static objects in all point cloud frames at once | No | No | No | Yes | No |
| Use pretrained deep learning models to generate automated labels in a point cloud | No | No | No | Yes | No |
| Use a custom automation function to generate labels | Yes | Yes | Yes | Yes | Yes |
| Specify image display window level and window width in the app toolstrip | No | No | No | No | Yes |
| New Preferences dialog box for image display settings | No | No | No | No | Yes |
Driving Scenario to RoadRunner Scenario: Export driving scenario to RoadRunner scenario using Driving Scenario Designer app
You can now interactively export the road network, static and dynamic actors in a driving scenario to a RoadRunner scenario using the Driving Scenario Designer app. For more information, see the Export Driving Scenario to RoadRunner Scenario Simulation example.
lidarPointCloudGenerator Enhancements: Performance
improvements using new syntax without actor pose inputs
You can now generate point cloud measurements from a lidarPointCloudGenerator
System object™ without manually specifying actor pose inputs. First, use the
addSensors object function of the
drivingScenario object to register a
lidarPointCloudGenerator object as a sensor to a driving
scenario. Then, call the lidarPointCloudGenerator object
without any inputs.
lidar = lidarPointCloudGenerator; [ptCloud,isValidTime] = lidar();
The lidarPointCloudGenerator object shows improved
performance when operating without actor pose inputs. For example, this code
simulates a driving scenario with 100 vehicles, each represented by the
driving.scenario.carMesh. It calls a lidar sensor object
without inputs and is about 2x faster as compared to using actor pose
inputs:
% Create a driving scenario and add a road
scenario = drivingScenario;
road(scenario,[-100 0 0; 900 0 0],50);
% Use driving,scenario.carMesh as the vehicle mesh
myCarMesh = driving.scenario.carMesh;
% Create 100 vehicles including the ego-vehicle
for i = 1:100
v = vehicle(scenario,'Mesh',myCarMesh);
v.Position = [10*i -5+10*mod(i,2) 0];
v.Velocity = [-20 0 0];
end
egoVehicle = scenario.Actors(1);
egoVehicle.Velocity = [0 0 0];
egoVehicle.Position = [0 0 0];
ap = actorProfiles(scenario);
% Create a lidar sensor object
lidar = lidarPointCloudGenerator;
lidar.ActorProfiles = ap;
% Register the lidar sensor to the driving scenario
addSensors(senario,lidar,egoVehicle.ActorID);
scenario.SampleTime = 0.1;
scenario.StopTime = 15;
tic;
while advance(scenario)
for i = 1:numel(scenario.Actors)
scenario.Actors(i).Position = scenario.Actors(i).Position + scenario.Actors(i).Velocity*scenario.SampleTime;
end
% Lidar with inputs
% tgts = targetPoses(egoVehicle);
% rdmesh = roadMesh(egoVehicle);
% ptCloud = lidar(tgts,rdmesh,scenario.SimulationTime);
% Lidar with no inputs
ptCloud = lidar();
end
toc;The approximate execution times are:
With actor pose inputs — 25.7s
Without actor pose inputs — 12.8s
The code was timed on a Windows 10, Intel® Xeon® W-2133 CPU @ 3.6GHz test system. When the 100 vehicles in the driving scenario above were represented by more complex meshes, performance improvements up to 10x have been observed.
OpenStreetMap Import Enhancements: Import lane specifications from OpenStreetMap file to driving scenario
You can now import lane specifications for driving scenarios created using
OpenStreetMap® files. The OpenStreetMap file contains lane-level information in the
lanes and lanes:backward tags. For
more information, see OpenStreetMap Import Limitations.
Simulation 3D Scene Configuration Block: Import RoadRunner scene as Unreal Engine scene
The Simulation 3D Scene Configuration block now provides the
option to simulate custom scenes built in RoadRunner. Use the Scene source parameter to specify the source of
scene as RoadRunner. Then, in the Project parameter,
specify the Filmbox (.fbx) file corresponding to the desired RoadRunner scene. You can obtain the Filmbox (.fbx) file by exporting
the scene from RoadRunner.
For an example on how to import a RoadRunner scene in to Unreal Engine® simulation environment, see Import RoadRunner Scene into Unreal Engine Using Simulink.
Simulation 3D Scene Configuration Block: Camera view improvements, including controls for visualizing the virtual world
The Simulation 3D Scene Configuration block now provides the option to view the
virtual world scene from a camera positioned at a custom viewpoint. To set the custom
viewpoint, set Scene view parameter to Custom and
specify the Initial viewer translation and Initial viewer
rotation to position the camera at the specified location. When the simulation
starts, you can view the virtual world in a free camera mode at the specified
viewpoint.
When the Scene view parameter is set to Scene Origin
or Custom, you can use keyboard shortcuts and mouse controls for camera
features, including to record camera viewpoints, attach camera to actors, and orbit around
the actor. You can also control the camera when the simulation is paused.
Simulation 3D Probabilistic Radar Block Enhancement: Default elevation resolution parameter updated to match the field of view
The default value of the Elevation resolution of radar
(deg) parameter of the Simulation 3D Probabilistic Radar block is now 5
degrees to match the default elevation value of the Field of view
(deg) parameter. If the you specify azimuth or elevation
resolution values that exceed the corresponding field of view values, the block
returns an error.
getAllScenarioVariables Function: Retrieve the names and
values of all scenario variables
Use the getAllScenarioVariables function to retrieve the names and
values of all variables in a scenario.
setScenarioVariable Function: value
now accepts logical, integer, and double-precision floating point inputs
Previously, the value argument of the setScenarioVariable function accepted only string scalars and
character vectors. As of R2023b, you can now specify logical, double-precision
floating point, or integer values when assigning variable values with
setScenarioVariable.
importScenario Function: Specify additional import
options for CSV files with importoptions
You can now specify additional properties when importing trajectories from CSV
files by using the importoptions argument. Use a
csvTrajectoryImportOptions import options configuration
object to specify options for the "CSV Trajectory" format.
The object has properties specified as one or more name-value arguments. If a
property value is "auto", the RoadRunner application
determines what value to use, and sets the property to that value. For more
information, see the importoptions argument of the importScenario function.
Sensor Simulation for RoadRunner Scenario Enhancements: Simulate RoadRunner Scenario with lidar sensor model defined in MATLAB
You can now use the SensorSimulation object to view and control lidar sensors in a
RoadRunner scenario. Configure the lidar sensor model using the lidarPointCloudGenerator System object in MATLAB®, and then use the addSensors function to add it to a vehicle in your RoadRunner scenario. You can then obtain point cloud measurements from the
sensor, and visualize them by using the birdsEyePlot object.
Sensor Simulation for RoadRunner Scenario Enhancements: Simulate RoadRunner Scenario with lidar sensor model defined in Simulink, and use Bird's-Eye Scope to visualize sensor measurements from RoadRunner Scenario
You can now use the SensorSimulation object to view and control lidar sensors in a
RoadRunner scenario. Use the Lidar Point Cloud Generator block to configure the lidar sensor
model in a Simulink® model that defines custom behavior for a RoadRunner actor. To add the lidar sensor to a vehicle in your RoadRunner scenario, use the addSensors function and specify the block path to the Lidar Point Cloud Generator block. For more information, see the
Add Sensors to RoadRunner Scenario Using Simulink example.
You can now use the Bird's-Eye Scope app to visualize lane markings, road boundaries, sensor measurements and detections from RoadRunner scenarios. Bird's-Eye Scope supports visualizing detections and measurements from these sensor model blocks:
RoadRunner HD Map Enhancements: Create new types of lanes, and add user-defined metadata for lanes to RoadRunner HD Map scene model
You can now represent these types of lanes in a RoadRunner HD Map scene model using the LaneType property of the roadrunner.hdmap.Lane object: Entry,
Exit, OffRamp,
OnRamp, Stop,
Median, and None.
For example, if you want to specify a lane of type OnRamp,
enter this
command:
aLane = roadrunner.hdmap.Lane(LaneType="OnRamp");
You can also add user-defined metadata for lanes to a RoadRunner HD Map scene model. For more information, see the Metadata property of the roadrunner.hdmap.Lane object.
RoadRunner HD Map Examples: Build test track 3D scene and build RoadRunner scene with intersection and static objects
The Build Test Track 3D Scene in RoadRunner Using RoadRunner HD Map example shows how to create a RoadRunner HD Map for a test track from a Keyhole Markup Language (KML) file containing its latitude-longitude coordinates and a GeoTIFF file containing its elevation. You can add lanes, lane boundaries, lane markings, and barriers to the RoadRunner HD Map and import it into RoadRunner. Then, you can build a 3D scene of the test track in RoadRunner.
The Build RoadRunner Scene with Intersection and Static Objects Using RoadRunner HD Map example shows how create a RoadRunner HD Map from a Keyhole Markup Language (KML) file containing the latitude-longitude coordinates of a road intersection. You can import the RoadRunner HD Map data into RoadRunner and use it to build a 3D scene containing the road intersection and surrounding static objects, such as trees and buildings.
To build scenes in RoadRunner, you must have a RoadRunner Scene Builder license.
Lateral Offset Follower Example: Design and simulate lateral offset action follower
The Lateral Offset Follower with High-Fidelity Vehicle Dynamics example shows how to design and implement a lateral offset action follower in Simulink and cosimulate it with RoadRunner Scenario. The lateral offset action follower enables the ego vehicle to shift away from the center of its lane. This example enables you to use either a 3DOF or 14DOF vehicle model to navigate the ego vehicle along the path.
Unreal Engine Visualization: Visualize RoadRunner Scenario simulation in Unreal Engine 3D viewer
You can use the sim3d.scenario.ActorBehavior (Automated Driving Toolbox) class to define a RoadRunner Scenario actor behavior that opens an Unreal Engine 3D visualization. You can create custom camera perspectives and 3D actors in this environment. For more information, see Simulate RoadRunner Scenarios with Actors Modeled in MATLAB and View in Unreal Engine 3D Viewer (Automated Driving Toolbox).
Functionality being removed or changed
Updated release compatibility when cosimulating MATLAB or Simulink with RoadRunner
Before R2023b, cosimulation between one release of MATLAB or Simulink and a different release of RoadRunner did not return an error but could produce unexpected results.
Starting in R2023b, when cosimulating RoadRunner with MATLAB or Simulink, the major release of MATLAB and Simulink must be the same as the major release of RoadRunner. Cosimulating between different releases results causes MATLAB to return an error.
For example, if you are using RoadRunner R2023b, you must use MATLAB or Simulink R2023b to run cosimulation.
For information on how to download the latest release of RoadRunner, see Install and Activate RoadRunner (RoadRunner).
New Example: Extract 3D vehicle information from camera data
The Extract 3D Vehicle Information from Recorded Monocular Camera Data for Scenario Generation example shows how to extract vehicle position, orientation, and dimension information to generate a RoadRunner scenario from camera data. You use a semi-automated workflow that uses a pretrained 3D object detector and a custom app to visually inspect the detections and filter noisy or unwanted detections.
Scenario Builder Enhancements: Feature enhancements (October 2023, Version 23.2.1)
The detect object function of the laneBoundaryDetector object can now return the confidence scores
of detected lanes in images. The scores output argument
applies to only the syntaxes used to detect lane boundaries in an image
coordinate system.
When using the initialized laneBoundaryTracker
System object, you can now smooth lane boundary detections by specifying the
additional UseSmoother name-value argument.
Scenario Builder Enhancement and Examples: Feature improvement, and new examples (December 2023, Version 23.2.2)
roadrunnerStaticObjectInfo Function
EnhancementThe roadrunnerStaticObjectInfo function can now generate sign
and sign type information in RoadRunner HD Map format from parametric cuboid models of signs.
The Extract Key Scenario Events from Recorded Sensor Data example shows how to extract key scenario events by using recorded sensor data. In this example, you extract events such as left turns, right turns, accelerations, decelerations, lane changes, and cut ins from ego and target actor trajectories.
The Generate RoadRunner Scene with Traffic Signs Using Recorded Sensor Data example shows how to generate a road scene containing traffic signs extracted from labeled camera and lidar sensor data.
New Example: Generate RoadRunner Scene Using Aerial Lidar Data (January 2024, Version 23.2.3)
The Generate RoadRunner Scene Using Aerial Lidar Data example shows how to generate a RoadRunner scene containing roads and static objects by using aerial lidar and OpenStreetMap data. In this example, you extract trees, buildings, and road elevation information from lidar data, and reconstruct a road network by using the OpenStreetMap data.
Scenario Variant Generator Enhancements and Examples: Feature improvement, and new examples
getScenario Function EnhancementThe getScenario function has a new syntax to create a
RoadRunner
ScenarioSimulation object from a
ScenarioDescriptor object. When you specify the
simulator as "RoadRunner", you must also specify the
SimulatorInstance argument as a roadrunner object to return a RoadRunner
ScenarioSimulation object.
The Generate Scenario Variants for Emergency Lane Keeping Testing example shows how to generate scenario variants from a seed scenario to use for emergency lane keeping (ELK) testing, in accordance with the European New Car Assessment Programme (Euro NCAP®) test protocol standards.
The Create Routes Similar to Seed Actor Trajectory in Scenes example shows how to create routes, in multiple target scenes, that are similar to an actor trajectory in the seed scenario.
The Generate Scenario Variants Using Excel Spreadsheets example shows how to generate variants of a seed scenario by specifying variation parameters such as speed, dimensions, waypoints, yaw, and collision, as tables in Microsoft® Excel® spreadsheets.
identifyActorOfInterest Function: Identify ego and target
actors in scenario (October 2023, Version 23.2.1)
Use the identifyActorOfInterest function to identify the IDs of ego and
target actors in a seed scenario descriptor.
Object Detection: Evaluate object detection results
Use the new evaluateObjectDetection function in Computer Vision Toolbox™ to evaluate the quality of object detection results using these
metrics: confusion matrix, normalized confusion matrix, average precision, and
mean average precision. You can specify for the function to return additional
metrics, such as MR (miss rate), LAMR (log-average miss rate), FPPI (false
positives per image), and AOS (average orientation similarity, for rotated
bounding boxes only). The objectDetectionMetrics object stores the metrics.
Lidar Lane Detection: Detect lanes in lidar point clouds using LLDN-GFC deep learning network
The Automated Driving Toolbox™ Model for Lidar Lane Detection offers functions to detect lanes in lidar point clouds using a lidar lane detection network utilizing global feature correlation (LLDN-GFC) deep learning network.
You can create a pretrained lidar lane detector by using the lidarLaneDetector object. The pretrained network has been trained
on the K-Lane data set, and it can detect up to six lanes from lidar point
clouds. Alternatively, you can train the network on your data set by using the
trainLidarLaneDetector function.
To detect lanes in point clouds using the trained detector, use the detect function. Use the evaluate function to evaluate the performance of lane detection
results across these metrics: classification accuracy, F1-score for
classification, F1-score for detection, precision, and recall.
To get started with the Automated Driving Toolbox Model for Lidar Lane Detection, see the Get Started with Lidar Lane Detection Using Deep Learning and Lane Detection in Lidar Data Using Deep Learning example.
To use these functions and the example, you must have the Automated Driving Toolbox Model for Lidar Lane Detection support package. You can download the Automated Driving Toolbox Model for Lidar Lane Detection support package from the Add-On Explorer. For more information about installing add-ons, see Get and Manage Add-Ons.
Functionality being removed or changed
evaluateDetectionPrecision function being removed in
future release
Warns
The evaluateDetectionPrecision function in Computer Vision Toolbox will be removed in a future release. Use the evaluateObjectDetection function to evaluate object
detection results with metrics such as average precision, instead.
evaluateDetectionMissRate function being removed in
future release
Warns
The evaluateDetectionMissRate function in Computer Vision Toolbox will be removed in a future release. Use the evaluateObjectDetection function to evaluate object
detection results with metrics such as the log-average miss rate,
instead.
evaluateDetectionAOS function being removed in
future release
Warns
The evaluateDetectionAOS function in Computer Vision Toolbox will be removed in a future release. Use the evaluateObjectDetection function to evaluate object
detection results with metrics such as the AOS, instead.
Monocular vSLAM: Implement complete feature-based monocular visual SLAM
workflow with the monovslam object
The new monovslam object in Computer Vision Toolbox and its supporting object functions enable you to implement a
complete visual simultaneous localization and mapping (vSLAM) workflow.
The monovslam object functions enable you to extract ORB features from
images, track the features to estimate camera poses, identify key frames, and
reconstruct a 3-D environment. The vSLAM algorithm also searches for loop
closures using the bag-of-features algorithm and optimizes the camera poses
using pose graph optimization. The monovslam object supports C/C++ code generation for both host and
non-host target platforms, which you can use for hardware deployment of vSLAM
code.
To use the monovslam object, you must have a Navigation Toolbox™ license.
Live Parking Spot Detection Example: Develop a live parking spot detection system using deep learning and visual SLAM
The Perception Based Live Parking Spot Detection Using Unreal Engine Simulation example shows how to develop a live parking spot detection system in Simulink using Unreal Engine simulation. The example uses deep learning to segment parking lane markings and vehicles from images. It then shows how to use visual simultaneous localization and mapping (SLAM) to build a local map and use the local map to detect empty parking spots while the vehicle moves around a parking lot.
Occupancy Map Example: Build an occupancy map using ultrasonic sensor data
The Build Occupancy Map Using Simulation 3D Ultrasonic Sensor example
shows how to build an occupancy map using ultrasonic sensor data in Simulink using the US City Block
Unreal Engine scene. The example uses the synthetic range measurements from the
Simulation 3D Ultrasonic Sensor mounted on a vehicle, and computes
the occupancy map along the vehicle trajectory as an occupancyMap (Navigation Toolbox) object.
Code Generation: Generate C/C++ code for coordinate conversions using MATLAB Coder
The latlon2local and local2latlon functions now support code generation.
PIL Verification Example: Verify generated code of lidar SLAM algorithm for map creation using processor-in-the-loop simulation
The PIL Verification of Generated Code for Map Creation Using Lidar SLAM example shows how to design a lidar SLAM algorithm for building a map, generate embedded code of the algorithm, and verify it using processor-in-the-loop (PIL) simulation on NVIDIA® Jetson™ hardware.
Real-Time Testing Using ASAM XIL Example: Deploy and test highway lane-following controller in real-time using ASAM XIL
The Automate Real-Time Testing of Highway Lane Following Controller Using ASAM XIL shows how to deploy and test a lane-following controller on a Speedgoat® real-time machine using the ASAM® XIL standard. The example enables you to reuse the desktop simulation test cases and automate the regression testing of the deployed application.
Truck Lane-Keeping System Example: Design and cosimulate lane-keeping system for self-driving trucks with RoadRunner Scenario and Simulink
The Lane-Keeping System for Self-Driving Trucks example shows how to design a lane-keeping system for self-driving trucks in Simulink and simulate it with RoadRunner Scenario. The lane-keeping system uses probabilistic vision and radar sensors and integrates sensor fusion, decision logic, and longitudinal and lateral controls for lane-following with tractor-trailer dynamics.
C++ Integration Example: Integrate C++ code of sensor fusion algorithm in Simulink
The Integrate and Verify C++ Code of Sensor Fusion Algorithm in Simulink example shows how to integrate C++ code of a sensor fusion algorithm into the Forward Vehicle Sensor Fusion test bench model and verify the performance of the sensor fusion algorithm using the generalized optimal subpattern assignment (GOSPA) metric. The example tests the algorithm in a 3D simulation environment that uses the Unreal Engine from Epic Games®.
AEB with RoadRunner Scenario Example Update: Obtain ground truth measurements from RoadRunner Scenario Reader and automate testing of scenario variants using Simulink Test
The Autonomous Emergency Braking with RoadRunner Scenario example now integrates vision and radar sensors with RoadRunner Scenario Reader blocks. These sensors enable the ego vehicle to directly obtain the poses of the targets, within sensor coverage, from RoadRunner Scenario.
The example also enables you to perform scripted iterative testing, using Simulink Test™, to automate testing of scenario variants for the European New Car Assessment Programme (Euro NCAP) Car-to-Car Front Turn-Across-Path (CCFtap) test scenario.
Automated Parking Valet Example Update: Plan global path using hybrid A* algorithm and generate local trajectory using time elastic band (TEB) algorithm
The Automated Parking Valet example now uses an optimal hybrid A*
algorithm with plannerHybridAStar (Navigation Toolbox) object to plan the initial
global path for the vehicle to an empty parking spot. The example then uses the
time elastic band algorithm with controllerTEB (Navigation Toolbox) object to generate a local
trajectory for the planned global path. Before R2023b, the example used the
rapidly exploring random tree (RRT*) algorithm for global path planning and
cubic spline smoothing for local trajectory generation.