Clear Filters
Clear Filters

how to install a network structure with two input,one is imageinput,the other is sequence input?

2 views (last 30 days)
I want to improve the performance of a CNN network. So I add a sequencial input(which is match with the CNN image input), and training it with LSTM network ,then join them together in their fullyconnected layers, and use softmax to classify. But the deepNetworkDesigner don't support these two kinds of input. How can I solve this problem?

Answers (1)

Krishna
Krishna on 30 May 2024
Hi Zhengyun,
It seems you're looking to enhance the performance of a CNN network by incorporating a sequential layer for your image data, which will then be fed into an LSTM network. The output from this would subsequently be merged and passed through a fully connected layer for classification purposes.
You can do this in MATLAB using ‘dlnetwork’. You can pass the data as sequence to an LSTM architecture and as an image to CNN architecture which are two different networks then combine using ‘additionLayer.’ Make sure your dimension of the output are same for both the network. Then pass this combined output through fully connected layer and then a SoftMax function. This is how the architecture should like.
Please go through the following documentation to learn more,
Also please go through this example to learn more about using CNN and RNN together,
Also, for this type of combined networks you need to use Datastores to train the network in MATLAB, please go through the following documentation to learn more,
Hope this helps.

Categories

Find more on Deep Learning Toolbox in Help Center and File Exchange

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!