importTensorFlowLayers

Import layers from TensorFlow network

collapse all in page

    Syntax

    lgraph = importTensorFlowLayers(modelFolder)
    lgraph = importTensorFlowLayers(modelFolder,Name,Value)

    Description

    example

    lgraph = importTensorFlowLayers(modelFolder) returns the layers of a TensorFlow™ network from the folder modelFolder, which contains the model in the saved model format (compatible only with TensorFlow 2). The function imports the layers defined in the saved_model.pb file and the learned weights contained in the variables subfolder, and returns lgraph as a LayerGraph object.

    importTensorFlowLayers requires the Deep Learning Toolbox™ Converter for TensorFlow Models support package. If this support package is not installed, then importTensorFlowLayers provides a download link.

    Note

    importTensorFlowLayers tries to generate a custom layer when you import a custom TensorFlow layer or when the software cannot convert a TensorFlow layer into an equivalent built-in MATLAB® layer. For a list of layers for which the software supports conversion, see TensorFlow-Keras Layers Supported for Conversion into Built-In MATLAB Layers.

    importTensorFlowLayers saves the generated custom layers and the associated TensorFlow operators in the package +modelFolder.

    importTensorFlowLayers does not automatically generate a custom layer for each TensorFlow layer that is not supported for conversion into built-in MATLAB layer. For more information on how to handle unsupported layers, see Tips.

    example

    lgraph = importTensorFlowLayers(modelFolder,Name,Value) imports the layers and weights from a TensorFlow network with additional options specified by one or more name-value arguments. For example, 'OutputLayerType','classification' appends a classification output layer to the end of the imported network architecture.

    Examples

    collapse all

    This example uses:

    Open Live Script

    Import a pretrained TensorFlow network in the saved model format as a LayerGraph object. Then, assemble the imported layers into a DAGNetwork object, and use the assembled network to classify an image.

    Specify the model folder.

    if ~exist('digitsDAGnet','dir')
    unzip('digitsDAGnet.zip')
end
modelFolder = './digitsDAGnet';

    Specify the class names.

    classNames = {'0','1','2','3','4','5','6','7','8','9'};

    Import the layers and weights of a TensorFlow network in the saved model format. By default, importTensorFlowLayers imports the network as a LayerGraph object compatible with a DAGNetwork object. Specify the output layer type for an image classification problem. 

    lgraph = importTensorFlowLayers(modelFolder,'OutputLayerType','classification')
    Importing the saved model...
Translating the model, this may take a few minutes...
Finished translation

    lgraph = 
  LayerGraph with properties:

         Layers: [13×1 nnet.cnn.layer.Layer]
    Connections: [13×2 table]
     InputNames: {'input_1'}
    OutputNames: {'ClassificationLayer_activation_1'}

    Display the last layer of the imported network. The output shows that importTensorFlowLayers appends a ClassificationOutputLayer to the end of the network architecture.

    lgraph.Layers(end)
    ans = 
  ClassificationOutputLayer with properties:

            Name: 'ClassificationLayer_activation_1'
         Classes: 'auto'
    ClassWeights: 'none'
      OutputSize: 'auto'

   Hyperparameters
    LossFunction: 'crossentropyex'

    The classification layer does not contain the classes, so you must specify these before assembling the network. If you do not specify the classes, then the software automatically sets the classes to 1, 2, ..., N, where N is the number of classes.

    The classification layer has the name 'ClassificationLayer_activation_1'. Set the classes to classNames and then replace the imported classification layer with the new one.

    cLayer = lgraph.Layers(end);
cLayer.Classes = classNames;
lgraph = replaceLayer(lgraph,'ClassificationLayer_activation_1',cLayer);

    Assemble the layer graph using assembleNetwork to return a DAGNetwork object.

    net = assembleNetwork(lgraph)
    net = 
  DAGNetwork with properties:

         Layers: [13×1 nnet.cnn.layer.Layer]
    Connections: [13×2 table]
     InputNames: {'input_1'}
    OutputNames: {'ClassificationLayer_activation_1'}

    Read the image you want to classify.

    digitDatasetPath = fullfile(toolboxdir('nnet'),'nndemos','nndatasets','DigitDataset');
I = imread(fullfile(digitDatasetPath,'5','image4009.png'));

    Classify the image using the imported network.

    label = classify(net,I);

    Display the image and the classification result.

    imshow(I)
title(['Classification result ' classNames{label}])

    This example uses:

    Open Live Script

    Import a pretrained TensorFlow network in the saved model format as a LayerGraph object compatible with a dlnetwork object. Then, convert the LayerGraph object to a dlnetwork to classify an image.

    Specify the model folder.

    if ~exist('digitsDAGnet','dir')
    unzip('digitsDAGnet.zip')
end
modelFolder = './digitsDAGnet';

    Specify the class names.

    classNames = {'0','1','2','3','4','5','6','7','8','9'};

    Import the TensorFlow network as layers compatible with a dlnetwork object.

    lgraph = importTensorFlowLayers(modelFolder,'TargetNetwork','dlnetwork')
    Importing the saved model...
Translating the model, this may take a few minutes...
Finished translation

    lgraph = 
  LayerGraph with properties:

         Layers: [12×1 nnet.cnn.layer.Layer]
    Connections: [12×2 table]
     InputNames: {'input_1'}
    OutputNames: {1×0 cell}

    Read the image you want to classify and display the size of the image. The image is a grayscale (one-channel) image with size 28-by-28 pixels.

    digitDatasetPath = fullfile(toolboxdir('nnet'),'nndemos','nndatasets','DigitDataset');
I = imread(fullfile(digitDatasetPath,'5','image4009.png'));
size(I)
    ans = 1×2

    28    28

    Convert the imported layer graph to a dlnetwork object.

    dlnet = dlnetwork(lgraph);

    Display the input size of the network. In this case, the image size matches the network input size. If they do not match, you must resize the image by using imresize(I, netInputSize(1:2)).

    dlnet.Layers(1).InputSize
    ans = 1×3

    28    28     1

    Convert the image to a dlarray. Format the images with the dimensions 'SSCB' (spatial, spatial, channel, batch). In this example, the batch size is 1 and you can omit it ('SSC').

    I_dlarray = dlarray(single(I),'SSCB');

    Classify the sample image and find the predicted label.

    prob = predict(dlnet,I_dlarray);
[~,label] = max(prob);

    Display the image and the classification result.

    imshow(I)
title(['Classification result ' classNames{label}])

    This example uses:

    Open Live Script

    Import a pretrained TensorFlow network in the saved model format as a LayerGraph object. Then, assemble the imported layers into a DAGNetwork object. The imported network contains layers that are not supported for conversion into built-in MATLAB layers. The software automatically generates custom layers when you import these layers.

    This example uses the helper function findCustomLayers. To view the code for this function, see Helper Function.

    Specify the model folder.

    if ~exist('digitsDAGnetwithnoise','dir')
    unzip('digitsDAGnetwithnoise.zip')
end
modelFolder = './digitsDAGnetwithnoise';

    Specify the class names.

    classNames = {'0','1','2','3','4','5','6','7','8','9'};

    Import the layers and weights of a TensorFlow network in the saved model format. By default, importTensorFlowLayers imports the network as a LayerGraph object compatible with a DAGNetwork object. Specify the output layer type for an image classification problem.

    lgraph = importTensorFlowLayers(modelFolder,'OutputLayerType','classification');
    Importing the saved model...
Translating the model, this may take a few minutes...
Finished translation

    If the imported network contains layers not supported for conversion into built-in MATLAB layers, then importTensorFlowLayers can automatically generate custom layers in place of these layers. importTensorFlowLayers saves each generated custom layer to a separate .m file in the package +digitsDAGnetwithnoise in the current folder.

    Find the indices of the automatically generated custom layers, using the helper function findCustomLayers, and display the custom layers.

    ind = findCustomLayers(lgraph.Layers,'+digitsDAGnetwithnoise');
lgraph.Layers(ind)
    ans = 
  2×1 Layer array with layers:

     1   'gaussian_noise_1'   GaussianNoise   digitsDAGnetwithnoise.kGaussianNoise1Layer3766
     2   'gaussian_noise_2'   GaussianNoise   digitsDAGnetwithnoise.kGaussianNoise2Layer3791

    The classification layer does not contain the classes, so you must specify these before assembling the network. If you do not specify the classes, then the software automatically sets the classes to 1, 2, ..., N, where N is the number of classes.

    The classification layer has the name 'ClassificationLayer_activation_1'. Set the classes to classNames and then replace the imported classification layer with the new one.

    cLayer = lgraph.Layers(end);
cLayer.Classes = classNames;
lgraph = replaceLayer(lgraph,'ClassificationLayer_activation_1',cLayer);

    Assemble the layer graph using assembleNetwork. The function returns a DAGNetwork object that is ready to use for prediction.

    net = assembleNetwork(lgraph)
    net = 
  DAGNetwork with properties:

         Layers: [15×1 nnet.cnn.layer.Layer]
    Connections: [15×2 table]
     InputNames: {'input_1'}
    OutputNames: {'ClassificationLayer_activation_1'}

    Helper Function

    This section provides the code of the helper function findCustomLayers used in this example. findCustomLayers returns the indices of the custom layers that importTensorFlowNetwork automatically generates.

    function indices = findCustomLayers(layers,PackageName)

s = what(['.\' PackageName]);

indices = zeros(1,length(s.m));
for i = 1:length(layers)
    for j = 1:length(s.m)
        if strcmpi(class(layers(i)),[PackageName(2:end) '.' s.m{j}(1:end-2)])
            indices(j) = i;
        end
    end
end

end

    Input Arguments

    collapse all

    Name of the folder containing the TensorFlow model, specified as a character vector or string scalar. modelFolder must be in the current folder, or you must include a full or relative path to the folder. modelFolder must contain the file saved_model.pb, and the subfolder variables. It can also contain the subfolders assets and assets.extra.

    • The file saved_model.pb contains the layer graph architecture and training options (for example, optimizer, losses, and metrics).

    • The subfolder variables contains the weights learned by the pretrained TensorFlow network. By default, importTensorFlowLayers imports the weights.

    • The subfolder assets contains supplementary files (for example, vocabularies), which the layer graph can use. importTensorFlowLayers does not import the files in assets.

    • The subfolder assets.extra contains supplementary files (for example, information for users), which coexist with the layer graph.

    Example: 'MobileNet'

    Example: './MobileNet'

    Name-Value Pair Arguments

    Specify optional comma-separated pairs of Name,Value arguments. Name is the argument name and Value is the corresponding value. Name must appear inside quotes. You can specify several name and value pair arguments in any order as Name1,Value1,...,NameN,ValueN.

    Example: importTensorFlowLayers(modelFolder,'TargetNetwork','dagnetwork','OutputLayerType','classification') imports the network layers and weights from modelFolder, saves the automatically generated custom layers in the package +modelFolder in the current folder, specifies that the imported layers are compatible with a DAGNetwork object, and appends a classification output layer to the end of the imported layers.

    Name of the package in which importTensorFlowLayers saves custom layers, specified as a character vector or string scalar. importTensorFlowLayers saves the custom layers package +PackageName in the current folder. If you do not specify 'PackageName', then importTensorFlowLayers saves the custom layers in a package named +modelFolder in the current folder. For more information on packages, see Packages Create Namespaces.

    importTensorFlowLayers tries to generate a custom layer when you import a custom TensorFlow layer or when the software cannot convert a TensorFlow layer into an equivalent built-in MATLAB layer. importTensorFlowLayers saves each generated custom layer to a separate .m file in +PackageName. To view or edit a custom layer, open the associated .m file. For more information on custom layers, see Deep Learning Custom Layers.

    The package +PackageName can also contain the subpackage +ops, which contains TensorFlow operators that are used in the automatically generated custom layers. importTensorFlowLayers saves the associated MATLAB function for each operator in a separate .m file in the subpackage +ops. The object functions of dlnetwork, such as the predict function, use these operators when interacting with the custom layers.

    Example: 'PackageName','MobileNet'

    Example: 'PackageName','CustomLayers'

    Target type of Deep Learning Toolbox network for imported network architecture, specified as 'dagnetwork' or 'dlnetwork'.

    • If you specify 'TargetNetwork' as 'dagnetwork', the imported network architecture is compatible with a DAGNetwork object. In this case, the imported lgraph must include an output layer specified by the TensorFlow saved model loss function or the name-value argument 'OutputLayerType'.

    • If you specify 'TargetNetwork' as 'dlnetwork', the imported network architecture is compatible with a dlnetwork object. In this case, the imported lgraph does not include an output layer.

    Example: 'TargetNetwork','dlnetwork' imports a LayerGraph object compatible with a dlnetwork object.

    Type of output layer that importTensorFlowLayers appends to the end of the imported network architecture, specified as 'classification', 'regression', or 'pixelclassification'. Appending a pixelClassificationLayer (Computer Vision Toolbox) object requires Computer Vision Toolbox™.

    • If you specify 'TargetNetwork' as 'dagnetwork' and the saved model in modelFolder does not specify a loss function, you must assign a value to the name-value argument 'OutputLayerType'. A DAGNetwork object must have an output layer.

    • If you specify 'TargetNetwork' as 'dlnetwork', importTensorFlowLayers ignores the name-value argument 'OutputLayerType'. A dlnetwork object does not have an output layer.

    Example: 'OutputLayerType','classification'

    Size of the input images for the network, specified as a vector of two or three numerical values corresponding to [height,width] for grayscale images and [height,width,channels] for color images, respectively. The network uses this information when the saved_model.pb file in modelFolder does not specify the input size.

    Example: 'ImageInputSize',[28 28]

    Indicator to display import progress information in the command window, specified as a numeric or logical 1 (true) or 0 (false).

    Example: 'Verbose','true'

    Output Arguments

    collapse all

    Network architecture, returned as a LayerGraph object.

    To use the imported layer graph for prediction, you must convert the LayerGraph object to a DAGNetwork or dlnetwork object. Specify the name-value argument 'TargetNetwork' as 'dagnetwork' or 'dlnetwork' depending on the intended workflow.

    • Convert a LayerGraph to a DAGNetwork by using assembleNetwork. On the DAGNetwork object, you then predict class labels using the classify function.

    • Convert a LayerGraph to a dlnetwork by using dlnetwork. On the dlnetwork object, you then predict class labels using the predict function. Specify the input data as a dlarray using the correct data format (for more information, see the fmt argument of dlarray).

    Limitations

    • importTensorFlowLayers supports TensorFlow versions v2.0, v2.1, v.2.2, and v2.3.

    More About

    collapse all

    TensorFlow-Keras Layers Supported for Conversion into Built-In MATLAB Layers

    importTensorFlowLayers supports the following TensorFlow-Keras layer types for conversion into built-in MATLAB layers, with some limitations.

    TensorFlow-Keras LayerCorresponding Deep Learning Toolbox Layer
    AddadditionLayer

    Activation, with activation names:

    • 'elu'

    • 'relu'

    • 'linear'

    • 'softmax'

    • 'sigmoid'

    • 'swish'

    • 'tanh'

    Layers:

    Advanced activations:

    • ELU

    • Softmax

    • ReLU

    • LeakyReLU

    • PReLu*

    Layers:

    AveragePooling2DaveragePooling2dLayer with PaddingValue specified as 'mean'
    BatchNormalizationbatchNormalizationLayer
    Bidirectional(LSTM(__))bilstmLayer
    ConcatenatedepthConcatenationLayer
    Conv2Dconvolution2dLayer
    Conv2DTransposetransposedConv2dLayer
    CuDNNGRUgruLayer
    CuDNNLSTMlstmLayer
    DensefullyConnectedLayer
    DepthwiseConv2DgroupedConvolution2dLayer
    DropoutdropoutLayer
    EmbeddingwordEmbeddingLayer (Text Analytics Toolbox)
    Flattennnet.keras.layer.FlattenCStyleLayer
    GlobalAveragePooling2DglobalAveragePooling2dLayer
    GlobalMaxPooling2DglobalMaxPooling2dLayer
    GRUgruLayer
    InputimageInputLayer, sequenceInputLayer, or featureInputLayer
    LSTMlstmLayer
    MaxPooling2DmaxPooling2dLayer
    MultiplymultiplicationLayer
    SeparableConv2DgroupedConvolution2dLayer or convolution2dLayer
    TimeDistributedsequenceFoldingLayer before the wrapped layer, and sequenceUnfoldingLayer after the wrapped layer
    UpSampling2Dresize2dLayer (Image Processing Toolbox)
    UpSampling3Dresize3dLayer (Image Processing Toolbox)
    ZeroPadding2Dnnet.keras.layer.ZeroPadding2DLayer

    * For a PReLU layer, importTensorFlowLayers replaces a vector-valued scaling parameter with the average of the vector elements. You can change the parameter back to a vector after import. For an example, see Import Keras PReLU Layer.

    Supported TensorFlow-Keras Loss Functions

    importTensorFlowLayers supports the following Keras loss functions:

    • mean_squared_error

    • categorical_crossentropy

    • sparse_categorical_crossentropy

    • binary_crossentropy

    Tips

    • If the imported network contains a layer not supported for conversion into a built-in MATLAB layer (see TensorFlow-Keras Layers Supported for Conversion into Built-In MATLAB Layers) and importTensorFlowLayers does not automatically generate a custom layer, then importTensorFlowLayers inserts a placeholder layer in place of the unsupported layer. To find the names and indices of the unsupported layers in the network, use the findPlaceholderLayers function. You then can replace a placeholder layer with a new layer that you define. To replace a layer, use replaceLayer.

    • importTensorFlowLayers can import TensorFlow networks created with the TensorFlow-Keras sequential or functional API.

    • To use a pretrained network for prediction or transfer learning on new images, you must preprocess your images in the same way the images that were used to train the imported model were preprocessed. The most common preprocessing steps are resizing images, subtracting image average values, and converting the images from BGR images to RGB.

      • To resize images, use imresize. For example, imresize(image,[227,227,3]).

      • To convert images from RGB to BGR format, use flip. For example, flip(image,3).

      For more information on preprocessing images for training and prediction, see Preprocess Images for Deep Learning.

    • The members of the package +PackageName (custom layers and TensorFlow operators) are not accessible if the package parent folder is not on the MATLAB path. For more information, see Packages and the MATLAB Path.

    Alternative Functionality

    Use importTensorFlowNetwork or importTensorFlowLayers to import a TensorFlow network in the saved model format [2]. Alternatively, if the network is in HDF5 or JSON format, use importKerasNetwork or importKerasLayers to import the network.

    References

    [2] Using the SavedModel format. https://www.tensorflow.org/guide/saved_model.

    See Also

    | | | | | | | | | | | |

    Topics

    Introduced in R2021a

    Deep Learning Toolbox Documentation

    Support