Main Content

importONNXFunction

Import pretrained ONNX network as a function

    Description

    example

    params = importONNXFunction(modelfile,NetworkFunctionName) imports an ONNX™ (Open Neural Network Exchange) network from the file modelfile and returns an ONNXParameters object (params) that contains the network parameters. The function also creates a model function with the name specified by NetworkFunctionName that contains the network architecture. For more information about the network function, see Imported ONNX Model Function.

    Use the ONNXParameters object and the NetworkFunctionName model function to perform common deep learning tasks, such as image and sequence data classification, transfer learning, object detection, and image segmentation. importONNXFunction is useful when you cannot import the network using the importONNXNetwork function (for example, importONNXFunction can import YOLOv3) or if you want to define your own custom training loop (for more details, see Train Network Using Custom Training Loop).

    This function requires the Deep Learning Toolbox™ Converter for ONNX Model Format support package. If this support package is not installed, then the function provides a download link.

    Examples

    collapse all

    Import an ONNX network as a function. The network contains ONNX operators that are not supported by Deep Learning Toolbox layers. You can use the imported model function for deep learning tasks, such as prediction and transfer learning.

    Download and install the Deep Learning Toolbox Converter for ONNX Model Format support package. You can enter importONNXFunction at the command line to check if the support package is installed. If it is not installed, then the function provides a link to the required support package in the Add-On Explorer. To install the support package, click the link, and then click Install.

    Specify the file to import as shufflenet with operator set 9 from the ONNX Model Zoo. shufflenet is a convolutional neural network that is trained on images from the ImageNet database.

    modelfile = 'shufflenet-9.onnx';

    A recommended practice is to try to import the network by using importONNXNetwork. If importONNXNetwork is unable to import the network because some of the network layers are not supported, you can import the network as layers by using importONNXLayers, or as a function by using importONNXFunction.

    Import the shufflenet network as layers. The software generates placeholder layers in place of the unsupported layers.

    lgraph = importONNXLayers(modelfile,'OutputLayerType','classification');
    Warning: Unable to import some ONNX operators, because they are not supported. They have been replaced by placeholder layers. To find these layers, call the function findPlaceholderLayers on the returned object.
    
    4 operators(s)	:	Average pooling layer in ONNX file does not include padding in the average. This may cause small numeric differences between the ONNX and MATLAB network outputs.
    32 operators(s)	:	The Reshape operator is supported only when it performs a flattening operation.
    16 operators(s)	:	The operator 'Transpose' is not supported.
    
    To import the ONNX network as a function, which can support most ONNX operators, call importONNXFunction.
    

    Find the placeholder layers and display the number of placeholder layers.

    indPlaceholderLayers = findPlaceholderLayers(lgraph);
    numel(indPlaceholderLayers)
    ans = 48
    

    You must replace the 48 placeholder layers to use lgraph for deep learning tasks, such as prediction.

    Instead, import the network as a function to generate a model function that you can readily use for deep learning tasks.

    params = importONNXFunction(modelfile,'shufflenetFcn')
    OpsetVersion = 9
    A function 'shufflenetFcn' containing the imported ONNX network has been saved to the current directory.
    To learn how to use this function, type: help shufflenetFcn
    
    params = 
      ONNXParameters with properties:
    
                 Learnables: [1×1 struct]
              Nonlearnables: [1×1 struct]
                      State: [1×1 struct]
              NumDimensions: [1×1 struct]
        NetworkFunctionName: 'shufflenetFcn'
    
    

    importONNXFunction returns the ONNXParameters object params, which contains the network parameters, and the model function shufflnetFcn, which contains the network architecture. importONNXFunction saves shufflenetFcn in the current folder. You can open the model function to view or edit the network architecture by using open shufflenetFcn.

    Import an ONNX network as a function, and use the pretrained network to predict the class label of an input image.

    Specify the file to import as shufflenet with operator set 9 from the ONNX Model Zoo. shufflenet is a convolutional neural network that is trained on more than a million images from the ImageNet database. As a result, the network has learned rich feature representations for a wide range of images. The network can classify images into 1000 object categories, such as keyboard, mouse, pencil, and many animals.

    modelfile = 'shufflenet-9.onnx';

    Import the pretrained ONNX network as a function by using importONNXFunction, which returns an ONNXParamaters object that contains the network parameters. The function also creates a new model function in the current folder that contains the network architecture. Specify the name of the model function as shufflenetFcn.

    params = importONNXFunction(modelfile,'shufflenetFcn');
    OpsetVersion = 9
    A function 'shufflenetFcn' containing the imported ONNX network has been saved to the current directory.
    To learn how to use this function, type: help shufflenetFcn
    

    Read the image you want to classify and display the size of the image. The image is 792-by-1056 pixels and has three color channels (RGB).

    I = imread('peacock.jpg');
    size(I)
    ans = 1×3
    
             792        1056           3
    
    

    Resize the image to the input size of the network. Show the image.

    I = imresize(I,[224 224]);
    imshow(I)

    The inputs to shufflenet require further preprocessing (for more details, see ShuffleNet in ONNX Model Zoo). Rescale the image. Normalize the image by subtracting the training images mean and dividing by the training images standard deviation.

    I = rescale(I,0,1);
    
    meanIm = [0.485 0.456 0.406];
    stdIm = [0.229 0.224 0.225];
    I = (I - reshape(meanIm,[1 1 3]))./reshape(stdIm,[1 1 3]);
    
    imshow(I)

    Import the class names from squeezenet, which is also trained with images from the ImageNet database.

    net = squeezenet;
    ClassNames = net.Layers(end).ClassNames;

    Calculate the class probabilities by specifying the image to classify I and the ONNXParameters object params as input arguments to the model function shufflenetFcn.

    scores = shufflenetFcn(I,params);

    Find the class index with the highest probability. Display the predicted class for the input image and the corresponding classification score.

    indMax = find(scores==max(scores));
    ClassNames(indMax)
    ans = 1×1 cell array
        {'peacock'}
    
    
    scoreMax = scores(indMax)
    scoreMax = 0.7517
    

    Import the alexnet convolution neural network as a function and fine-tune the pretrained network with transfer learning to perform classification on a new collection of images.

    This example uses several helper functions. To view the code for these functions, see Helper Functions.

    Unzip and load the new images as an image datastore. imageDatastore automatically labels the images based on folder names and stores the data as an ImageDatastore object. An image datastore enables you to store large image data, including data that does not fit in memory, and efficiently read batches of images during training of a convolutional neural network. Specify the mini-batch size.

    unzip('MerchData.zip');
    miniBatchSize = 8;
    imds = imageDatastore('MerchData', ...
        'IncludeSubfolders',true, ...
        'LabelSource','foldernames',...
        'ReadSize', miniBatchSize);

    This data set is small, containing 75 training images. Display some sample images.

    numImages = numel(imds.Labels);
    idx = randperm(numImages,16);
    figure
    for i = 1:16
        subplot(4,4,i)
        I = readimage(imds,idx(i));
        imshow(I)
    end

    Extract the training set and one-hot encode the categorical classification labels.

    XTrain = readall(imds);
    XTrain = single(cat(4,XTrain{:}));
    YTrain_categ = categorical(imds.Labels);
    YTrain = onehotencode(YTrain_categ,2)';

    Determine the number of classes in the data.

    classes = categories(YTrain_categ);
    numClasses = numel(classes)
    numClasses = 5
    

    AlexNet is a convolutional neural network that is trained on more than a million images from the ImageNet database. As a result, the network has learned rich feature representations for a wide range of images. The network can classify images into 1000 object categories, such as keyboard, mouse, pencil, and many animals.

    Import the pretrained alexnet network as a function.

    alexnetONNX()
    params = importONNXFunction('alexnet.onnx','alexnetFcn')
    A function containing the imported ONNX network has been saved to the file alexnetFcn.m.
    To learn how to use this function, type: help alexnetFcn.
    
    params = 
      ONNXParameters with properties:
    
                 Learnables: [1×1 struct]
              Nonlearnables: [1×1 struct]
                      State: [1×1 struct]
              NumDimensions: [1×1 struct]
        NetworkFunctionName: 'alexnetFcn'
    
    

    params is an ONNXParameters object that contains the network parameters. alexnetFcn is a model function that contains the network architecture. importONNXFunction saves alexnetFcn in the current folder.

    Calculate the classification accuracy of the pretrained network on the new training set.

    accuracyBeforeTraining = getNetworkAccuracy(XTrain,YTrain,params);
    fprintf('%.2f accuracy before transfer learning\n',accuracyBeforeTraining);
    0.01 accuracy before transfer learning
    

    The accuracy is very low.

    Display the learnable parameters of the network. These parameters, for example the weights (W) and bias (B) of convolution and fully connected layers, are updated by the network during training. Nonlearnable parameters remain constant during training.

    params.Learnables
    ans = struct with fields:
        data_Mean: [227×227×3 dlarray]
          conv1_W: [11×11×3×96 dlarray]
          conv1_B: [96×1 dlarray]
          conv2_W: [5×5×48×256 dlarray]
          conv2_B: [256×1 dlarray]
          conv3_W: [3×3×256×384 dlarray]
          conv3_B: [384×1 dlarray]
          conv4_W: [3×3×192×384 dlarray]
          conv4_B: [384×1 dlarray]
          conv5_W: [3×3×192×256 dlarray]
          conv5_B: [256×1 dlarray]
            fc6_W: [6×6×256×4096 dlarray]
            fc6_B: [4096×1 dlarray]
            fc7_W: [1×1×4096×4096 dlarray]
            fc7_B: [4096×1 dlarray]
            fc8_W: [1×1×4096×1000 dlarray]
            fc8_B: [1000×1 dlarray]
    
    

    The last two learnable parameters of the pretrained network are configured for 1000 classes. The parameters fc8_W and fc8_B must be fine-tuned for the new classification problem. Transfer the parameters to classify 5 classes by initializing them.

    params.Learnables.fc8_B = rand(5,1);
    params.Learnables.fc8_W = rand(1,1,4096,5);

    Freeze all the parameters of the network to convert them to nonlearnable parameters. Because you do not need to compute the gradients of the frozen layers, freezing the weights of many initial layers can significantly speed up network training.

    params = freezeParameters(params,'all');

    Unfreeze the last two parameters of the network to convert them to learnable parameters.

    params = unfreezeParameters(params,'fc8_W');
    params = unfreezeParameters(params,'fc8_B');

    Now the network is ready for training. Initialize the training progress plot.

    plots = "training-progress";
    if plots == "training-progress"
        figure
        lineLossTrain = animatedline;
        xlabel("Iteration")
        ylabel("Loss")
    end

    Specify the training options.

    velocity = [];
    numEpochs = 5;
    miniBatchSize = 16;
    numObservations = size(YTrain,2);
    numIterationsPerEpoch = floor(numObservations./miniBatchSize);
    initialLearnRate = 0.01;
    momentum = 0.9;
    decay = 0.01;

    Train the network.

    iteration = 0;
    start = tic;
    executionEnvironment = "cpu"; % Change to "gpu" to train on a GPU.
    
    % Loop over epochs.
    for epoch = 1:numEpochs
        
        % Shuffle data.
        idx = randperm(numObservations);
        XTrain = XTrain(:,:,:,idx);
        YTrain = YTrain(:,idx);
        
        % Loop over mini-batches.
        for i = 1:numIterationsPerEpoch
            iteration = iteration + 1;
            
            % Read mini-batch of data.
            idx = (i-1)*miniBatchSize+1:i*miniBatchSize;
            X = XTrain(:,:,:,idx);        
            Y = YTrain(:,idx);
            
            % If training on a GPU, then convert data to gpuArray.
            if (executionEnvironment == "auto" && canUseGPU) || executionEnvironment == "gpu"
                X = gpuArray(X);         
            end
            
            % Evaluate the model gradients and loss using dlfeval and the
            % modelGradients function.
            [gradients,loss,state] = dlfeval(@modelGradients,X,Y,params);
            params.State = state;
            
            % Determine learning rate for time-based decay learning rate schedule.
            learnRate = initialLearnRate/(1 + decay*iteration);
            
            % Update the network parameters using the SGDM optimizer.
            [params.Learnables,velocity] = sgdmupdate(params.Learnables,gradients,velocity);
            
            % Display the training progress.
            if plots == "training-progress"
                D = duration(0,0,toc(start),'Format','hh:mm:ss');
                addpoints(lineLossTrain,iteration,double(gather(extractdata(loss))))
                title("Epoch: " + epoch + ", Elapsed: " + string(D))
                drawnow
            end
        end
    end

    Calculate the classification accuracy of the network after fine-tuning.

    accuracyAfterTraining = getNetworkAccuracy(XTrain,YTrain,params);
    fprintf('%.2f accuracy after transfer learning\n',accuracyAfterTraining);
    0.99 accuracy after transfer learning
    

    Helper Functions

    This section provides the code of the helper functions used in this example.

    The getNetworkAccuracy function evaluates the network performance by calculating the classification accuracy.

    function accuracy = getNetworkAccuracy(X,Y,onnxParams)
    
    N = size(X,4);
    Ypred = alexnetFcn(X,onnxParams,'Training',false);
    
    [~,YIdx] = max(Y,[],1);
    [~,YpredIdx] = max(Ypred,[],1);
    numIncorrect = sum(abs(YIdx-YpredIdx) > 0);
    accuracy = 1 - numIncorrect/N;
    
    end

    The modelGradients function calculates the loss and gradients.

    function [grad, loss, state] = modelGradients(X,Y,onnxParams)
    
    [y,state] = alexnetFcn(X,onnxParams,'Training',true);
    loss = crossentropy(y,Y,'DataFormat','CB');
    grad = dlgradient(loss,onnxParams.Learnables);
    
    end

    The alexnetONNX function generates an ONNX model of the alexnet network. You need Deep Learning Toolbox Model for AlexNet Network support to access this model.

    function alexnetONNX()
        
    exportONNXNetwork(alexnet,'alexnet.onnx');
    
    end
    

    Input Arguments

    collapse all

    Name of the ONNX model file containing the network, specified as a character vector or string scalar. The file must be in the current folder or a folder on the MATLAB® path, or you must include a full or relative path to the file.

    Example: 'shufflenet.onnx'

    Name of the model function, specified as a character vector or string scalar. The function NetworkFunctionName contains the architecture of the imported ONNX network. The file is saved in an M-file in the current folder, or you must include a full or relative path to the file. The NetworkFunctionName file is required for using the network. For more information, see Imported ONNX Model Function.

    Example: 'shufflenetFcn'

    Output Arguments

    collapse all

    Network parameters, returned as an ONNXParameters object. params contains the network parameters of the imported ONNX model. Use dot notation to reference properties of params. For example, params.Learnables displays the network learnable parameters, such as the weights of the convolution layers.

    More About

    collapse all

    Imported ONNX Model Function

    importONNXFunction creates a model function that contains the network architecture of the imported ONNX model. Specify the name NetworkFunctionName as an input argument to importONNXFunction.

    Syntax

    Use the following syntaxes to interface with the imported ONNX model function (NetworkFunctionName):

    • [Y,state] = NetworkFunctionName(X,params) returns the output data Y and the updated network state for the input data X.

    • [Y,state] = NetworkFunctionName(X,params,Name,Value) uses additional options specified by one or more name-value pair arguments.

    • [Y1,Y2,...,Yn,state] = NetworkFunctionName(X1,X2,...,Xn,params) returns multiple output data (Y1,Y2,...,Yn) and the updated network state for the multiple input data (X1,X2,...,Xn).

    • [Y1,Y2,...,Yn,state] = NetworkFunctionName(X1,X2,...,Xn,params,Name,Value) uses additional options specified by one or more name-value pair arguments for multiple inputs and outputs.

    Input Arguments
    ArgumentDescription
    XInput data, specified as an array or dlarray.
    paramsNetwork parameters, specified as an ONNXParameters object.
    Name-Value Pair Arguments
    Argument nameDescription
    'Training'

    Training option, specified as 'false' (default) or 'true'.

    'InputDataPermutation'

    Permutation applied to the dimension ordering of input X, specified as 'auto' (default), 'none', a numeric vector, or a cell array.

    Assign a value to the name-value pair argument 'InputDataPermutation' to permute the input data into the dimension ordering required by the imported ONNX model.

    • Assign the value 'auto' to apply an automatic permutation based on assumptions about common input data X. For more details, see Automatic Input Data Permutation.

    • Assign the value 'none' to pass X in the original ordering.

    • Assign a numeric vector value to customize the input dimension ordering; for example, [4 3 1 2].

    • Assign a cell array value for multiple inputs; for example, {[3 2 1],'none'}.

    'OutputDataPermutation'

    Permutation applied to the dimension ordering of output Y, specified as 'auto' (default), 'none', a numeric vector, or a cell array.

    Assign a value to the name-value pair argument 'OutputDataPermutation' to match the dimension ordering of the imported ONNX model.

    • Assign the value 'auto' to return Y in Deep Learning Toolbox ordering. For more details, see Automatic Output Data Permutation.

    • Assign the value 'none' to return Y in ONNX ordering.

    • Assign a numeric vector value to customize the output dimension ordering; for example, [3 4 2 1].

    • Assign a cell array value for multiple outputs; for example, {[3 2 1],'none'}.

    Output Arguments
    ArgumentDescription
    Y

    Output data, returned as an array or dlarray.

    • If X is an array or you use ONNXFunction to predict, Y is a array.

    • If X is a dlarray or you use ONNXFunction for training, Y is a dlarray.

    state

    Updated network state, specified as a structure.

    The network state contains information remembered by the network between iterations and updated across multiple training batches.

    The interpretation of input argument X and output argument Y can differ between models. For more information about the model input and output arguments, refer to help for the imported model function NetworkFunctionName, or refer to the ONNX documentation [1].

    Automatic Permutation for Imported Model Function

    By default, NetworkFunctionName automatically permutes input and output data to facilitate image classification tasks. Automatic permutation might be unsuitable for other tasks, such as object detection and time series classification.

    Automatic Input Data Permutation

    To automatically permute the input, NetworkFunctionName assumes the following based on the input dimensions specified by the imported ONNX network.

    Number of ONNX Model Input DimensionsInterpretation of Input DataONNX Standard Dimension OrderingDeep Learning Toolbox Standard Dimension OrderingAutomatic Permutation of Input
    42-D image

    NCHW

    H, W, and C correspond to the height, width, and number of channels of the image, respectively, and N is the number of observations.

    HWCN

    H, W, and C correspond to the height, width, and number of channels of the image, respectively, and N is the number of observations.

    [ 4 3 1 2 ]

    If the size of the input dimensions is a number other than 4, NetworkFunctionName specifies the input argument 'InputDataPermutation' as 'none'.

    Automatic Output Data Permutation

    To automatically permute the output, NetworkFunctionName assumes the following based on the output dimensions specified by the imported ONNX network.

    Number of ONNX Model Output DimensionsInterpretation of Output DataONNX Standard Dimension OrderingDeep Learning Toolbox Standard Dimension OrderingAutomatic Permutation of Output
    22-D image classification scores

    NK

    K is the number of classes and N is the number of observations.

    KN

    K is the number of classes and N is the number of observations.

    [ 2 1 ]
    42-D image pixel classification scores

    NKHW

    H and W correspond to the height and width of the image, respectively, K is the number of classes, and N is the number of observations.

    HWKN

    H and W correspond to the height and width of the image, respectively, K is the number of classes, and N is the number of observations.

    [3 4 2 1]

    If the size of the output dimensions is a number other than 2 or 4, NetworkFunctionName specifies the input argument 'OutputDataPermutation' as 'none'.

    Supported ONNX Layers

    importONNXFunction supports the following ONNX layers, with some limitations. Compare these layers with the layers supported by importONNXNetwork.

    ONNX Layers Supported by importONNXFunctionimportONNXNetwork Support
    AbsNo

    Add

    Yes
    AndNo
    ArgMaxNo

    AveragePool

    Yes

    BatchNormalization

    Yes
    CastNo
    CeilNo

    Clip

    Yes
    CompressNo

    Concat

    Yes

    Constant

    Yes
    ConstantOfShapeNo

    Conv

    Yes

    ConvTranspose

    Yes
    DepthToSpaceNo

    Div

    Yes

    Dropout

    Yes
    EqualNo
    ExpNo
    ExpandNo

    Flatten

    Yes
    FloorNo
    GatherNo

    Gemm

    Yes

    GlobalAveragePool

    Yes

    Greater

    Yes
    HardmaxNo

    Identity

    Yes
    IfNo
    InstanceNormalizationYes

    LeakyRelu

    Yes
    LessNo
    LogNo
    LoopNo

    LRN

    Yes

    LSTM

    Yes

    MatMul

    Yes

    MaxPool

    Yes

    Mul

    Yes
    NonMaxSuppressionNo
    NonZeroNo
    NotNo
    OneHotNo
    OrNo
    PadNo
    PowNo

    PRelu

    Yes
    RandomUniformNo
    RangeNo
    ReciprocalNo
    ReduceMaxNo
    ReduceMeanNo
    ReduceMinNo
    ReduceProdNo
    ReduceSumNo

    Relu

    Yes

    Reshape

    Yes
    ResizeYes
    RoiAlignNo
    RoundNo
    ScanNo
    ScatterNo
    ScatterElementsNo
    SequenceAtNo
    ShapeNo

    Sigmoid

    Yes
    SliceNo

    Softmax

    Yes
    SpaceToDepthYes
    SplitNo
    SplitToSequenceNo
    SqrtNo
    SqueezeNo

    Sub

    Yes

    Sum

    Yes

    Tanh

    Yes
    TileNo
    TopKNo
    TransposeNo
    UnsqueezeNo
    UpsampleNo
    WhereNo

    Tips

    • Refer to the ONNX documentation for each model to see the required preprocessing of the network inputs. For example, you need to resize (using imresize), rescale, and normalize the input images to networks trained with the ImageNet dataset (such as AlexNet, GoogleNet, ShuffleNet, and SqueezeNet).

    • importONNXFunction supports ONNX operator sets 7, 8, 9, 10, and 11.

    Alternative Functionality

    importONNXFunction is useful when you cannot import a pretrained ONNX network by using importONNXNetwork. If you want to generate code for a pretrained network, use importONNXLayers. Find and replace the generated placeholder layers by using findPlaceholderLayers and replaceLayer, respectively. Then, use assembleNetwork to return a DAGNetwork object. You can generate code for a trained DAGNetwork.

    References

    [1] Open Neural Network Exchange. https://github.com/onnx/.

    [2] ONNX. https://onnx.ai/.

    Introduced in R2020b