How to combine two pre trained deep learning models (Mobilenetv2 and nasnet_mobile)

Hi

how can we create a hybrid model based on two pretrained models (Mobilenetv2 and nasnet_mobile) ?

Previously I did for mobilenetv2 and Resnet using a deep network analyzer app by replacing input layer and classification layers of both models with single input layer and classification layer with the help of a depth concatination layer. it worrks fine.

but when i try to do with the these two models it is not working..

any link available for proper way of combining models available?

the model it generted after the procedure is attached herewith

Thank You

[Vidhi Agarwal]

Hi Ravi, Can you please provide the data regarding the network parameter?

Answer 1

[Milan Bansal]

Hi Ravi,

I understand that you are trying to create a hybrid model based on two pretrained models (Mobilenetv2 and nasnet_mobile) and are facing issues with the workflow of creating a single input and classification layer using "depthConcatenationLayer".

Encountered issues might be due to differences in the input sizes, layer names, or the way features are extracted and concatenated. For this particualar case where "MobileNet-v2" and "NasNet-Mobile" needs to be combinen, Deep Network Designer can be used. Please follow the below steps to combine the two models.

1. Open Deep Network Designer App by entering the command "deepNetworkDesigner" in the command window.
2. Click on the "New" button at the top left corner.
3. Select "NasNet-Mobile" from Image Networks.
4. Repeat the previous step to add "MobileNet-v2" to same canvas as "NasNet-Mobile".
5. Both the models expects an input image of size 224x224x3. Remove the image input layer from one of MobileNet-v2 and connect the remaining part to the image input layer of the NasNet-Mobile.
6. Both the network have a layer with the name "global_average_pooling_2d_1"(3rd layer from the last in both the network), change the name of layers with this name as all the layer names must be unique in a Neural Network. Change the name to "global_average_pooling_2d_1_NasNet" for NasNet-Mobile and "global_average_pooling_2d_1_mobileNetv2" for MobileNet-v2.
7. Remove the softmax prediction layer and the corresponding fully connected layer from both the networks.
8. Add a "depthConcatenationLayer" and connect the output of "global_average_pooling_2d_1_NasNet" and "global_average_pooling_2d_1_mobileNetv2" to it.
9. Finally add a "fullyConnectedLayer" and prediction layer to the "depthConcatenationLayer" added in the previous step.
10. Analyze the network and check for errors.

Please refer to the following documentation link to learn more about Deep Network Designer App.

https://www.mathworks.com/help/deeplearning/gs/get-started-with-deep-network-designer.html

Hope this helps.