Concatenation Error when using LSTM Layers in a Hybrid Actor (RL Toolbox)

Hey,

Unfortunately, I run into a concatenation error when I try to use LSTM layers combined with a hybrid actor in my RL framework. There would be an easy way to fix that in the sequenceExperienceArray.m file from the RL toolbox, however, I cannot save the changes because I get the error "Access denied".

Did anyone else experience this and found a workaround?

Thanks!

[Anay]

Hi Joschua,

It would be helpful if you can share more details about how this concatenation error is occuring. The "Access denied" error you are facing is likely arising because you are directly trying to edit files which are on MATLAB's installation directory and you dont't have the required permissions. I would suggest you to try and resolve the issue without editing such files to avoid unwanted complications in the future.

Answer 1

[Umeshraja]

Hi @Joschua,

I understand you are encountering a concatenation error with LSTM layers in a hybrid actor. The problem typically comes from dimension mismatches in network architectures.

To resolve this, ensure that the outputs from the LSTM and any other layers (such as fully connected layers) have compatible dimensions before concatenation. MATLAB functions like permute and reshape can help standardize the tensor formats.

% Example alignment for LSTM output (batch_size × sequence_length × features)
lstm_output_reshaped = permute(lstm_output, [3 1 2 4]); 

You may find this MATLAB Answers thread helpful for understanding dimension consistency during concatenation:

https://www.mathworks.com/matlabcentral/answers/2036941-why-is-the-error-thrown-inputs-to-concatenations-layer-must-have-consistent-formats

Also, when designing reinforcement learning network architectures, it's a good practice to use analyzeNetwork to validate layer connections before training

For further reference, here are the official MATLAB documentation pages:

https://www.mathworks.com/help/matlab/ref/permute.html

https://www.mathworks.com/help/matlab/ref/reshape.html

Hope this helps!