MATLAB Answers

DDPG Agent isn't learning (reward 0 for every episode)

14 views (last 30 days)
I suspect that the SampleTime Ts has something to do with it, I set Ts = 1e-6, but the trainnig is still going very fast.
I tried the same thing that was used for the Water Tank Model:
Just like in the example my system is very well controlled with a PI-Controller, but the DDPG Agent isn't learning anything.
What exactly is wrong with this system?

Accepted Answer

Emmanouil Tzorakoleftherakis
The reason why you see 0 rewards is because thw IsDone flag (which is used to terminate episodes early) is immediately set to true at the beginning of each episode. Either set it to false, or set the appropriate logic for you system.
Note that there are other things that need to be fixed in your setup, the most notable one being the agent sample time (which is currently very small) and the episode duration/max steps (which is currently very large). Unless you adjust these to more reasonable values for RL, your training will take days
Emmanouil Tzorakoleftherakis
Tha's what I am saying, you may need to reconsider your inputs/outputs. I am not sure what you mean by "made it even worse". If you are not seeing good training results, there could be a lot of other reasons including training options and hyperparameters, networks architectures etc. I would give the duty cycle idea another try and spend more time on the hyperparameters

Sign in to comment.

More Answers (0)

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!