R2022a

New Features, Bug Fixes, Compatibility Considerations
expand all

Apps and Visualization

Experiment Manager: Offload experiments as batch jobs in a cluster

Starting in R2022a, Experiment Manager supports offloading experiments as batch jobs in a cluster. You can configure the cluster to run multiple trials at the same time or to run a single trial at a time on multiple parallel workers. While the experiment is running in the cluster, you can run other experiments, close the app and continue using MATLAB®, or close your MATLAB session. Batch execution of experiments requires Parallel Computing Toolbox™. For more information, see Offload Experiments as Batch Jobs to Cluster.

Experiment Manager: Manage experiments using new context menu options

In R2022a, initiate actions directly in the Experiment Browser pane and in the results table:

  • To add a new experiment to a project, in the Experiment Browser pane, right-click the name of the project and select New Experiment.

  • To create a copy of an experiment, in the Experiment Browser pane, right-click the name of the experiment and select Duplicate.

  • To stop a running trial, cancel a queued trial, or restart a stopped or canceled trial, in the results table, right-click the row for the trial and select Stop, Cancel, or Restart. Alternatively, click the Stop, Cancel, or Restart buttons in the Actions column of the results table.

  • To export the training information or trained network for a stopped or completed trial, in the results table, right-click the row for the trial and select Export Training Information or Export Trained Network.

Experiment Manager: Specify hyperparameters using character vectors

You can now specify hyperparameter values as cell arrays of character vectors. Before R2022a, Experiment Manager only supported hyperparameter specifications using scalars and vectors with numeric, logical, or string values. Now, these are valid hyperparameter specifications:

  • 0.01

  • 0.01:0.01:0.05

  • [0.01 0.02 0.04 0.08]

  • ["sgdm" "rmsprop" "adam"]

  • {'squeezenet' 'googlenet' 'resnet18'}

Experiment Manager: View stopping reasons in results table

When an experiment trial ends, the Status column of the results table now displays one of these reasons for stopping:

  • Max epochs completed

  • Met validation criterion

  • Stopped by OutputFcn

  • Training loss is NaN

For an example of an experiment that displays multiple stopping reasons, see Experiment with Weight Initializers for Transfer Learning.

Experiment Manager: Export results table to MATLAB workspace

Starting in R2022a, you can save the contents of the results table as a table array in the MATLAB workspace. On the Experiment Manager toolstrip, select Export > Results Table. For an example that shows how to export the results table as part of evaluating an experiment, see Evaluate Deep Learning Experiments by Using Metric Functions.

Experiment Manager: Sort your experiment annotations

You can now sort annotations. By default, annotations are sorted from oldest to newest creation time. To sort annotations, use the Sort By list. You can sort by creation time or trial number. For more information, see Sort, Filter, and Annotate Experiment Results.

Deep Network Designer: Create deep learning experiments suitable for Experiment Manager

Use Deep Network Designer to create deep learning experiments suitable for hyperparameter sweeping in Experiment Manager. After you train a network using Deep Network Designer, create an experiment by clicking Export > Create Experiment. Deep Network Designer generates an experiment setup function using your network and the imported data. You can use the generated setup function as a starting point for an Experiment Manager experiment. For example, sweep through a range of hyperparameter values or use Bayesian optimization to find optimal training options. For more information, see Generate Experiment Using Deep Network Designer.

Deep Network Designer: Access pretrained audio networks

You can now access the following pretrained audio networks from the Deep Network Designer Start Page:

These networks require Deep Learning Toolbox™ and Audio Toolbox™.

You can use Deep Network Designer to visualize, edit, and train using the pretrained audio networks. For an example showing how to retrain a pretrained audio classification network for a new task, see Transfer Learning with Pretrained Audio Networks in Deep Network Designer.

Deep Network Designer: Export training plot as image

Export the deep learning training progress plot as an image using Deep Network Designer. After training, you can save the training progress plot by clicking Export Training Plot in the Training tab. You can save the plot as a PNG, JPEG, or TIFF file. You can also save the individual plots of loss, accuracy, and root mean squared error using the axes toolbar.

Network Analyzer: View dimension labels and total number of learnables

View the activation dimension labels and the total number of learnable parameters of a network using Network Analyzer. To analyze your network, use the analyzeNetwork function or click Analyze in Deep Network Designer.

Each activation dimension has one of the following labels:

  • S — Spatial

  • C — Channel

  • B — Batch observations

  • T — Time or sequence

  • U — Unspecified

View the dimension labels to understand how data propagates through the network and how the layers modify the size and layout of activations.

Training Progress Plot: Export training plot as image

Export the deep learning training progress plot as an image or PDF. You can view the training progress plot when using trainNetwork by setting the Plots training option to 'training-progress'. To save the training progress plot, click Export Training Plot in the training window. You can save the plot as a PNG, JPEG, TIFF, or PDF file. You can also save the individual plots of loss, accuracy, and root mean squared error using the axes toolbar.

Shallow Neural Networks: Improved visual design of network diagram and training window

The view function and the network training window have been updated with new visual designs. The training window opens by default when using any of the shallow neural network training functions (for example, trainlm, trainscg, trainbr). For example, use the train function to train a simple feedforward network and view the updated training window.

[x,t] = simplefit_dataset;
net = feedforwardnet(10);
net = train(net,x,t);

 Functionality being removed or changed

nntraintool has been removed

Errors

nntraintool has been removed. To train a shallow neural network and open the training window, use train instead.

nntool has been removed

Errors

nntool has been removed. Use nnstart instead.

Algorithms

1-D Convolutional Networks: Create and train networks with 1-D transposed convolution for sequence and time series data

Create and train deep learning networks with 1-D transposed convolution layers for sequence and time series data.

Create a 1-D transposed convolution layer using the transposedConv1dLayer function. The function returns a TransposedConvolution1DLayer object.

The dimension that the layer upsamples depends on the layer input:

  • For time series and vector sequence input (data with three dimensions corresponding to the channels, observations, and time steps, respectively), the layer upsamples the time dimension.

  • For 1-D image input (data with three dimensions corresponding to the spatial pixels, channels, and observations, respectively), the layer upsamples the spatial dimension.

  • For 1-D image sequence input (data with four dimensions corresponding to the spatial pixels, channels, observations, and time steps, respectively), the layer upsamples the spatial dimension.

For custom training loop workflows, you can also apply the 1-D transposed convolution operation to dlarray data using the dltranspconv function by specifying formatted weights with a "T" (time) dimension or using the WeightsFormat option. For more information, use the command help dlarray/dltranspconv.

For an example showing how to do anomaly detection using transposed convolutions, see Time Series Anomaly Detection Using Deep Learning.

Batch Normalization: Normalize mini-batches of 1-D images and 1-D, 2-D, and 3-D image sequence input

Normalize mini-batches of 1-D images and 1-D, 2-D, and 3-D image sequence input using BatchNormalizationLayer objects. To create a batch normalization layer, use batchNormalizationLayer.

Spatio-Temporal Convolution and Pooling: Apply 2-D and 3-D convolution and pooling to sequences of images

Apply 2-D convolutions and pooling to sequences of 1-D or 2-D images using Convolution2DLayer and GlobalAveragePooling2DLayer objects.

The dimensions that the layer convolves over depend on the layer input:

  • For 2-D image input (data with four dimensions corresponding to pixels in two spatial dimensions, the channels, and the observations), the layer convolves or pools over the spatial dimensions.

  • For 2-D image sequence input (data with five dimensions corresponding to the pixels in two spatial dimensions, the channels, the observations, and the time steps), the layer convolves or pools over the two spatial dimensions.

  • For 1-D image sequence input (data with four dimensions corresponding to the pixels in one spatial dimension, the channels, the observations, and the time steps), the layer convolves or pools over the spatial and time dimensions.

Apply 3-D convolutions and pooling to sequences of 2-D or 3-D images using Convolution3DLayer, MaxPooling3DLayer, AveragePooling3DLayer, and GlobalAveragePooling3DLayer objects.

The dimensions that the layer convolves or pools over depends on the layer input:

  • For 3-D image input (data with five dimensions corresponding to pixels in three spatial dimensions, the channels, and the observations), the layer convolves or pools over the spatial dimensions.

  • For 3-D image sequence input (data with six dimensions corresponding to the pixels in three spatial dimensions, the channels, the observations, and the time steps), the layer convolves or pools over the spatial dimensions.

  • For 2-D image sequence input (data with five dimensions corresponding to the pixels in two spatial dimensions, the channels, the observations, and the time steps), the layer convolves or pools over the spatial and time dimensions.

Network Training: Specify checkpoint frequency

The CheckpointPath training option enables you to save networks as MAT files periodically during training. This periodic saving is especially useful when you have a large network or a large data set, and training takes a long time. If the training is interrupted for some reason, you can resume training from the last saved checkpoint network.

You can now specify how often the software saves checkpoint networks. To specify the number of iterations or epochs between saving checkpoints, use the CheckpointFrequency training option. To specify the checkpoint frequency unit, use the CheckpointFrequencyUnit training option.

Network Training: Train networks with sequence input in parallel

The trainNetwork function supports training networks with sequence input in parallel. To train a network in parallel, set the ExecutionEnvironment training option to 'parallel' or 'multi-gpu' using the trainingOptions function.

To train a networks with lstmLayer, bilstmLayer, or gruLayer objects in parallel, the SequenceLength training option must be 'longest' or 'shortest'.

Multi-Input Networks: Train networks with mixtures of image, sequence, or feature inputs

The trainNetwork function now supports layer graphs with mixtures of ImageInputLayer, Image3DInputLayer, SequenceInputLayer, and FeatureInputLayer layer objects. When using the trainNetwork function, the network must have at most one sequence input layer.

When training a multi-input network with a sequence input layer, the SequenceLength training option must be 'longest' or 'shortest'.

When making network predictions using the predict, classify, predictAndUpdateState, classifyAndUpdateState, or activations functions, the SequenceLength option must be 'longest' or 'shortest' and the ExecutionEnvironment option must be 'auto', 'gpu', or 'cpu'.

To specify training data with multiple inputs, use a transformed or combined datastore. To transform the outputs of a datastore, use the transform function. To create a datastore that outputs multiple values by combining datastores, use the combine function.

For an example showing how to train a network with both image and feature input, see Train Network on Image and Feature Data.

Deep Learning Model Hub: Discover pretrained models for deep learning in MATLAB

To find the latest pretrained models for deep learning in MATLAB, see MATLAB Deep Learning Model Hub.

You can find models suitable for a range of deep learning applications, such as lidar point cloud processing, audio speech to text, and pose estimation from images. For example:

  • Find transformer models, such as GPT-2, BERT, and FinBERT, suitable for natural language processing tasks.

  • Find models such as YOLO v4 and Mask R-CNN, suitable for object detection tasks.

Flatten Layer: Use flatten layers in networks with image or feature input

Networks with image or feature input now support FlattenLayer objects. Use flatten layers to collapse the spatial dimensions of the layer input into the channel dimension.

Function Layer: Generate code for function layer

Function layers with unformatted inputs support code generation and GPU code generation when the predict function is a named function on the path.

To create a function layer that applies a specified function to it's input, use functionLayer.

For an example showing how to generate code for an image classification network, see Code Generation for Deep Learning Networks.

Custom Layers: Accelerate custom layer functions

When you create a custom layer, if you do not specify a backward function, then the software automatically determines the gradients using automatic differentiation.

When training a network with a custom layer without a backward function, the software traces each input dlarray object of the custom layer forward function to determine the computation graph used for automatic differentiation. This tracing process can take some time and can spend time recomputing the same trace. By optimizing, caching, and reusing the traces, you can speed up gradient computation when training a network. The software can also reuse these traces to speed up network predictions after training.

To indicate that the custom layer supports acceleration, also inherit from the nnet.layer.Acceleratable class when defining the custom layer. When a custom layer inherits from nnet.layer.Acceleratable, the software automatically caches traces when passing data through a dlnetwork object.

For more information, see Custom Layer Function Acceleration. To learn more about defining custom layers, see Define Custom Deep Learning Layers.

Recurrent Layers: Recurrent layers in dlnetwork objects support inputs without time dimensions

When included in a dlnework object, LSTMLayer, BiLSTMLayer, and GRULayer objects automatically infer a singleton time dimension when the input data does not specify a time dimension. In these cases, the layer output does not contain time dimensions.

For more information, see the Layer Input and Output Formats sections of lstmLayer, bilstmLayer, and gruLayer.

Custom Training Loops: dlnetwork objects support TransposedConvolution1DLayer objects

Create dlnetwork objects containing TransposedConvolution1DLayer objects. To create a 1-D transposed convolution layer, use the transposedConv1dLayer function.

Custom Training Loops: Apply transposed convolution over time dimension of dlarray

For custom training loop workflows, you can apply the transposed convolution operation to dlarray data over the time dimension using the dltranspconv function by specifying formatted weights with a "T" (time) dimension or using the WeightsFormat option. For more information, use the command help dlarray/dltranspconv.

Custom Training Loops: Reset state parameters of dlnetwork objects

The resetState function now supports dlnetwork input.

The resetState function has an effect only if the input network has state parameters (for example, a network with at least one recurrent layer, such as an LSTM layer). If the input network does not have state parameters, then the function has no effect and returns the input network.

Custom Training Loops: Plot dlarray objects

The plot function and the addpoints function for animatedLine objects support dlarray data as input.

Plotting functions do not support tracing dlarray objects.

For more information about support for dlarray objects, see List of Functions with dlarray Support.

fullyconnect Function: Improved single-precision performance with GPUs using the TensorFloat-32 (TF32) compute mode

The fullyconnect function shows improved single-precision performance with GPUs using the TF32 compute mode, such as NVIDIA® Ampere architecture GPUs. For example, computing the weighted sum of input data using the following test is about 3.4x faster than in the previous release:

function timeFullyConnect
    
% Set up random single-precision input data, weights, and biases
inputSize = 10000;
observations = 100000;
outputFeatures = 5000;
        
X  = dlarray(gpuArray(rand(inputSize,observations,'single')),'SB');
weights = gpuArray(dlarray(rand(outputFeatures,inputSize,'single')));
bias    = gpuArray(dlarray(rand(outputFeatures,1,'single')));
    

% Time fullyconnect
f = @() fullyconnect(X,weights,bias);
gputimeit(f)

end

The approximate execution times are:

R2021b: 0.88 seconds

R2022a: 0.26 seconds

The code was timed on a Windows® 10, Intel® Xeon® W-2133 @ 3.60 GHz test system with NVIDIA RTX A5000 GPU by calling the function timeFullyConnect.

max Function: Improved performance within a dlgradient call

The max function shows improved performance when used within a dlgradient call. The min function shows a similar performance improvement when used with a dlgradient call. For example, evaluating gradients containing a max function in the following test is about 4.9x faster than in the previous release:

function timeMax 

% Prepare input data and gradient function 
x = dlarray(rand(220, 200, 3, 100)-0.5); 
fcn = @(x) max(x,0); 
gradFcn = @(x) dlgradient(sum(fcn(x), 'all'), x); 

% Warm up iterations 
for i=1:5 
    dlfeval(gradFcn, x); 
end 

 
% Timed iterations 
tic 
for i=1:10 
    dlfeval(gradFcn, x); 
end 
toc 

end

The approximate execution times are:

R2021b: 2.26 seconds

R2022a: 0.46 seconds

The code was timed on a Windows 10, Intel Xeon W-2133 @ 3.60 GHz test system with NVIDIA RTX A5000 GPU by calling the function timeMax.

Accelerated Functions: Improved performance of accelerated functions that use parentheses indexing with non-repeated indices

Accelerated functions that use parentheses indexing with non-repeated indices show improved performance. For example, evaluating an accelerated convolution operation in the following test is about 7.0x faster than in the previous release:

function timeAccFcn

% Prepare input data
inputSize = [256,256,64,90];
X = dlarray(gpuArray(randn(inputSize)));
W = dlarray(gpuArray(randn(3,3,1,1,64)));
bias = dlarray(gpuArray(randn(1)));

% Prepare convolution function
fcn = @(x, W, bias) dlconv(double(squeeze(x(:, :, :, 2, :)-x(:, :, :, 1, :))), W, bias, ...
    'Stride', [1 1], 'Padding', 'same', 'DataFormat', 'SSCB');
accFcn = dlaccelerate(fcn);
clearCache(accFcn);

% Warm up iterations
for i = 1:6
    dlfeval(accFcn,X, W, bias);
end

% Timed iterations
gd = gpuDevice;
tic
for i = 1:200
    dlfeval(accFcn,X, W, bias);
    wait(gd);
end
toc

end

The approximate execution times are:

R2021b: 2.23 seconds

R2022a: 0.32 seconds

The code was timed on a Windows 10, Intel Xeon W-2133 @ 3.60 GHz test system with NVIDIA RTX A5000 GPU by calling the function timeAccFcn.

dlnetwork State Updates: Improved performance when updating the network state of a dlnetwork

Updating the State property of a dlnetwork shows improved performance. For example, updating the network state of the Resnet-50 network in the following test is about 5.5x faster than in the previous release:

function timeStateUpdate

% Prepare network and input variables
lgraph=layerGraph(resnet50);
lgraph = removeLayers(lgraph, {lgraph.Layers(end-2:end).Name});
net = dlnetwork(lgraph);
x = gpuArray(dlarray(rand(10, 10, 3, 10)-0.5, 'SSCB'));

% Calculate a new state
[~, newState] = forward(net, x);

% Timed iterations
tic
for i=1:10
    net.State = newState;
end
toc

end

The approximate execution times are:

R2021b: 1.2 seconds

R2022a: 0.22 seconds

The code was timed on a Windows 10, Intel Xeon W-2133 @ 3.60 GHz test system with NVIDIA RTX A5000 GPU by calling the function timeStateUpdate.

forward Function: Reduced GPU memory usage when accelerated

Computing the output of a dlnetwork using the forward function and specifying the acceleration input as 'auto' uses less GPU memory.

Interoperability

ONNX Version Support: Updated support for ONNX intermediate representation and operator sets

The importONNXNetwork, importONNXLayers, and exportONNXNetwork functions now support ONNX™ intermediate representation version 7 and ONNX operator sets 6 to 14.

TensorFlow-Keras and ONNX Code Generation: Additional Keras and ONNX built-in layers support code generation

You can use MATLAB Coder™ or GPU Coder™ together with Deep Learning Toolbox to generate MEX, standalone CPU, CUDA® MEX, or standalone CUDA code for an imported network. You can generate code for any imported network whose layers support code generation. For lists of the layers that support code generation with MATLAB Coder and GPU Coder, see Supported Layers (MATLAB Coder) and Supported Layers (GPU Coder), respectively.

You can now generate generic C or C++ code for the following Keras and ONNX built-in layers:

  • nnet.keras.layer.GlobalAveragePooling2dLayer

  • nnet.keras.layer.FlattenCStyleLayer

  • nnet.keras.layer.ZeroPadding2dLayer

  • nnet.onnx.layer.ElementwiseAffineLayer

  • nnet.onnx.layer.FlattenLayer

You can now generate MEX, standalone CPU, CUDA MEX, or standalone CUDA code for the following Keras and ONNX built-in layers:

  • nnet.keras.layer.ClipLayer

  • nnet.keras.layer.PreluLayer

  • nnet.keras.layer.TimeDistributedFlattenCStyleLayer

  • nnet.onnx.layer.ClipLayer

  • nnet.onnx.layer.GlobalAveragePooling2dLayer

  • nnet.onnx.layer.PreluLayer

  • nnet.onnx.layer.SigmoidLayer

  • nnet.onnx.layer.TanhLayer

ONNX Export Support: Specify batch size of exported network

You can now specify a dynamic or fixed batch size for a trained MATLAB deep learning network that you export to the ONNX model format. To do so, use the BatchSize name-value argument of exportONNXNetwork.

ONNX Import Layer Support: Import networks that include 1-D convolution and pooling layers

You can now import an ONNX network that includes 1-D convolution and pooling layers (convolution1dLayer, maxPooling1dLayer, averagePooling1dLayer, globalMaxPooling1dLayer, and globalAveragePooling1dLayer) by using importONNXNetwork and importONNXLayers. For a list of supported layers, see ONNX Operators Supported for Conversion into Built-In MATLAB Layers.

TensorFlow Operator Support: Import networks that include ExpandDims operators

You can now import a TensorFlow™ network that includes ExpandDims operators by using the importTensorFlowNetwork and importTensorFlowLayers functions. For a list of the TensorFlow operators that the functions support for conversion into MATLAB functions with dlarray support, see Supported TensorFlow Operators.

Deployment

Acceleration Modes: Use accelerator and rapid accelerator modes to speedup Simulink simulations

Deep learning Simulink® models now support the accelerator and rapid accelerator modes to speed up the execution of your model.

The accelerator modes use the Intel MKL-DNN library to perform acceleration. In the Model Configuration Parameters, on the Simulation Target pane, set the Language to C++ and the Target library to MKL-DNN. Then on the Simulation tab, in the Simulate section, select Accelerator or Rapid Accelerator from the drop-down menu and start the simulation.

For information on improving simulation speed with accelerator and rapid accelerator modes, Acceleration for Simulink Deep Learning Models.

For an example showing how to speed up the execution of your model, see Lane and Vehicle Detection in Simulink Using Deep Learning.

Quantization: Quantize neural networks without a specified target

With MATLAB, you can quantize your neural networks without generating code or committing to a specific target for code deployment. This can be useful if you:

  • Do not have access to your target hardware.

  • Want to inspect your quantized network without generating code.

Your quantized network implements int8 data instead of single data. It keeps the same layers and connections as the original network, and it has the same inference behavior as it would when running on hardware.

Once you have quantized your network, you can use the quantizationDetails function to inspect your quantized network. Additionally, you also have the option to deploy the code to a GPU target.

For an example showing how to quantize your neural network with MATLAB, see Emulate Target Agnostic Quantized Network.

Quantization: Estimate neural network layer metrics

With the estimateNetworkMetrics function, you can estimate the metrics for each layer of your neural network. For more information, see estimateNetworkMetrics.

Quantization: Validate the performance of the optimized network for a CPU target

You can now use the dlquantizer object and the validate function to quantize a network and generate code for CPU targets. Additionally, the Deep Network Quantizer app now fully supports the workflow for a CPU target.

Taylor Pruning: Prune dlnetwork object to compress the model

You can prune a dlnetwork object by using the first-order Taylor approximation algorithm. This process identifies and removes filters from the prunable layers of the dlnetwork object. Pruning is an optimization that returns another dlnetwork object that is a compressed version of your original model and that consumes less computational resources.

Use the taylorPrunableNetwork function to convert a dlnetwork object to another representation that is suitable for pruning using the Taylor pruning algorithm.

To perform pruning, use these object functions:

  • forward and predict: Perform training and inference respectively.

  • updateScore and updatePrunables: Compute and update first-order Taylor scores. Remove filters from prunable layers.

  • dlnetwork: Extract the compressed dlnetwork object back from the pruned TaylorPrunableNetwork object.

For examples showing how to prune the convolutional filters of a network using Taylor pruning, see:

TensorFlow Lite: Generate C++ code for pretrained models and deploy on Linux platforms

Use the loadTFLiteModel function to load a pretrained TensorFlow Lite model into a TFLiteModel object. Use this object with the predict function in your MATLAB code to perform inference in MATLAB execution, code generation, or inside MATLAB Function blocks in Simulink models.

To use this functionality, you must install the Deep Learning Toolbox Interface for TensorFlow Lite. For more information, see Prerequisites for Deep Learning with TensorFlow Lite Models. For an example, see Generate Code for TensorFlow Lite Model and Deploy on Raspberry Pi.

Application Examples

Deep Learning Workflows: New and updated examples and topics

New and updated examples and topics help you progress with deep learning:

Import and Export: New Examples

Image Processing and Computer Vision: New and updated examples

Lidar Processing: New and updated examples

Audio Processing: New examples

Signal Processing: New examples

New examples for signal processing tasks include:

Wireless Communications: New examples

Computational Finance: New examples

Simulink: New examples