how to install a network structure with two input,one is imageinput,the other is sequence input?
4 views (last 30 days)
Show older comments
I want to improve the performance of a CNN network. So I add a sequencial input(which is match with the CNN image input), and training it with LSTM network ,then join them together in their fullyconnected layers, and use softmax to classify. But the deepNetworkDesigner don't support these two kinds of input. How can I solve this problem?
0 Comments
Answers (1)
Krishna
on 30 May 2024
Hi Zhengyun,
It seems you're looking to enhance the performance of a CNN network by incorporating a sequential layer for your image data, which will then be fed into an LSTM network. The output from this would subsequently be merged and passed through a fully connected layer for classification purposes.
You can do this in MATLAB using ‘dlnetwork’. You can pass the data as sequence to an LSTM architecture and as an image to CNN architecture which are two different networks then combine using ‘additionLayer.’ Make sure your dimension of the output are same for both the network. Then pass this combined output through fully connected layer and then a SoftMax function. This is how the architecture should like.
Please go through the following documentation to learn more,
Also please go through this example to learn more about using CNN and RNN together,
Also, for this type of combined networks you need to use Datastores to train the network in MATLAB, please go through the following documentation to learn more,
Hope this helps.
0 Comments
See Also
Categories
Find more on Deep Learning Toolbox in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!