error compiling ResNet50 Deep Learning HDL Toolbox Support Package

10 views (last 30 days)
Hello,
I have a pretrained ResNet50 imported to a DAGNetwork object. The NN is working properly in matlab. However, I'm trying to implement it in a FPGA and I get the followong error while compiling. NN was imported using importTensorFlowNetwork and I'm trying to generate HDL with Deep Learning HDL Toolbox Support Package For Intel FPGA And SoC Devices (I was able to implement a sequential NN using a SeriesNetwork object and this tool):
Error in dnnfpga.apis.Workflow/compile
Error in generate_HDL_from_NN (line 18)
hW.compile;
Caused by:
Layer 'conv2_block1_3_conv': Input size mismatch. Size of input to this layer is different from the expected input size.
Inputs to this layer:
from layer 'conv2_block1_0_conv' (size 8(S) × 8(S) × 256(C) × 1(B))
Layer 'conv2_block1_add': Unconnected input. Each layer input must be connected to the output of another layer.
Layer 'conv2_block2_add': Unconnected input. Each layer input must be connected to the output of another layer.
Layer 'conv2_block3_add': Unconnected input. Each layer input must be connected to the output of another layer.
Layer 'conv3_block1_add': Unconnected input. Each layer input must be connected to the output of another layer.
Layer 'conv3_block2_add': Unconnected input. Each layer input must be connected to the output of another layer.
Layer 'conv3_block3_add': Unconnected input. Each layer input must be connected to the output of another layer.
Layer 'conv3_block4_add': Unconnected input. Each layer input must be connected to the output of another layer.
Layer 'conv4_block1_add': Unconnected input. Each layer input must be connected to the output of another layer.
Layer 'conv4_block2_add': Unconnected input. Each layer input must be connected to the output of another layer.
Layer 'conv4_block3_add': Unconnected input. Each layer input must be connected to the output of another layer.
Layer 'conv4_block4_add': Unconnected input. Each layer input must be connected to the output of another layer.
Layer 'conv4_block5_add': Unconnected input. Each layer input must be connected to the output of another layer.
Layer 'conv4_block6_add': Unconnected input. Each layer input must be connected to the output of another layer.
Layer 'conv5_block1_add': Unconnected input. Each layer input must be connected to the output of another layer.
Layer 'conv5_block2_add': Unconnected input. Each layer input must be connected to the output of another layer.
Layer 'conv5_block3_add': Unconnected input. Each layer input must be connected to the output of another layer.
Do you have any suggestion to solve this issue?
Thank you in advance
---------------------------
Software version:
Deep Learning HDL Toolbox Support Package For Intel FPGA And SoC Devices 21.2.4
Matlab R2021b Update 6 (9.11.0.2207237)
Ubuntu 20.04
---------------------------

Answers (2)

Wang Chen
Wang Chen on 16 Apr 2023
Edited: Wang Chen on 16 Apr 2023
Hi Ruben,
For Resnet50 network, have you tried to download the Deep Learning Toolbox Model for ResNet-50 Network support pacakge?The Resnet50 network in this support package works together with Deep Learning HDL Toolbox:
There is also a ResNet-18 network support pacakge:
For the network you imported using importTensorFlowNetwork, could you contact MathWorks customer support and shared the network with customer support? It looks like the network is not correctly constructed in MATLAB. The error message on the "Unconnected input. Each layer input must be connected to the output of another layer" are generic MATLAB network connection errors. They are not specific to FPGA targeting.
Thanks,
Wang
  6 Comments
Rubén
Rubén on 17 Apr 2023
analyzeNetwork does not give neither warnings nor errors. In addtion nn imported seems working properly, it classify correctly.
My NN uses ResNet50, that is standard and is supported by HDL toolbox, and keras flatten, reLu and softmax layers, that all of them are also supported.
I don't see the point to train Resnet50 out of Matlab and inclue additional layers in Matlab because my model need the additional layers and the training will be different.
Wang Chen
Wang Chen on 17 Apr 2023
Hi Ruben,
It will be great if you can share your imported network to us, so we can take a look at it and see what is the cause of the issue. Could you contact MathWorks customer support and shared the network with customer support?
Alternatively, as Sivylla suggested, you could add the additional desired layers to the MATLAB built-in ResNet50 or ResNet18, using MATLAB tools like Deep Network Designer:
Thanks,
Wang

Sign in to comment.


Rubén
Rubén on 17 Apr 2023
The size of the NN is over the 5MB limit. Please, find the "NN.mat" in the following download link: NN.mat
  5 Comments
Wang Chen
Wang Chen on 28 Apr 2023
Hi Ruben, the root cause of this bug is the pattern of zero padding layer (nnet.keras.layer.ZeroPadding2dLayer) followed by a group convolution layer, please see following bug report: https://www.mathworks.com/support/bugreports/2661508
This bug is fixed in R2022b.
If you cannot upgrade to R2022b, the work-around is to merge the zero padding into the group convolution layer. These two layers can be merged because the group convolution layer also has the padding options.
Thanks,
Wang
Rubén
Rubén on 3 May 2023
Hello @Wang Chen,
I have imported the SaveModel as a layerGraph ("layers" in this example) object and I tried to merge both layers in this way:
%% Merge 0 padding layer into groupedConvolution2dLayer layer
% Copy conv layer and change Padding size
conv = layers.Layers(3);
conv.PaddingSize = [layers.Layers(2).Top layers.Layers(2).Bottom layers.Layers(2).Left layers.Layers(2).Right];
% Replace conv layer and remove padding layer
layers = replaceLayer(layers, layers.Layers(3).Name, conv);
layers = removeLayers(layers, layers.Layers(2).Name);
% Connect input layer with conv layer
layers = connectLayers(layers, layers.Layers(1).Name, layers.Layers(2).Name)
NN = assembleNetwork(layers);
after doing this, I have built the DAGNetwork object using assembleNetwork and after analising it with analyzeNetwork there is no error/warning.
However, I get the following error whe I try to compile with hW.compile as before:
Error using dnnfpga.compiler.codegenfpga
Invalid network.
Error in dnnfpga.apis.Workflow/compileNetwork
Error in dnnfpga.apis.Workflow/compile
Error in generate_HDL_from_NN (line 18)
hW.compile;
Error in importTensorFlowNN (line 36)
hW = generate_HDL_from_NN(nn, ToolName, ToolPath, Vendor, Platform, ip, Interface, User, Pass);
Caused by:
Layer 'conv2_block1_3_conv': Input size mismatch. Size of input to this layer is different from the expected input size.
Inputs to this layer:
from layer 'conv2_block1_0_conv' (size 8(S) × 8(S) × 256(C) × 1(B))
Layer 'conv2_block1_add': Unconnected input. Each layer input must be connected to the output of another layer.
Layer 'conv2_block2_add': Unconnected input. Each layer input must be connected to the output of another layer.
Layer 'conv2_block3_add': Unconnected input. Each layer input must be connected to the output of another layer.
Layer 'conv3_block1_add': Unconnected input. Each layer input must be connected to the output of another layer.
Layer 'conv3_block2_add': Unconnected input. Each layer input must be connected to the output of another layer.
Layer 'conv3_block3_add': Unconnected input. Each layer input must be connected to the output of another layer.
Layer 'conv3_block4_add': Unconnected input. Each layer input must be connected to the output of another layer.
Layer 'conv4_block1_add': Unconnected input. Each layer input must be connected to the output of another layer.
Layer 'conv4_block2_add': Unconnected input. Each layer input must be connected to the output of another layer.
Layer 'conv4_block3_add': Unconnected input. Each layer input must be connected to the output of another layer.
Layer 'conv4_block4_add': Unconnected input. Each layer input must be connected to the output of another layer.
Layer 'conv4_block5_add': Unconnected input. Each layer input must be connected to the output of another layer.
Layer 'conv4_block6_add': Unconnected input. Each layer input must be connected to the output of another layer.
Layer 'conv5_block1_add': Unconnected input. Each layer input must be connected to the output of another layer.
Layer 'conv5_block2_add': Unconnected input. Each layer input must be connected to the output of another layer.
Layer 'conv5_block3_add': Unconnected input. Each layer input must be connected to the output of another layer.
Find the nn int this link NN_without_padding.mat
Are both layers merged properly?

Sign in to comment.

Products


Release

R2021b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!