some of the saved agents in DQN reinforcement learning algorithm do not reproduce the training rewards
6 views (last 30 days)
Show older comments
masoud k on 21 Sep 2020
Commented: Abdul Basith Ashraf on 3 Apr 2021
I am training a DQN agent in my reinforcement learning problem in Simulink Matlab 2019a. I have set a save criteria of agents with episode rewards equal to 6. My problem is that after a number of episodes, some agents will save based on my save criteria, but when I use them, some of the agents do not reproduce rewards equal to 6 or more but the amount of rewards is less than 6. If anyone could help me I will appriciate.
Emmanouil Tzorakoleftherakis on 24 Sep 2020
This is a common misconception. When you train an agent and you get a certain reward out of a specific episode, if you were to stop training and run the same episode again with the same agent you would most likely get a different response from the agent and thus a different episode reward.
There are a few reasons for that:
1) During training, the agent explores different options based on some probabilistic exploration strategy. For DQN this strategy is related to the epsilon parameter in DQN options. After training, the agent does not explore anymore and only relies on the underlying neural network for inference, so it makes sense that results are different.
2) It is best practice to randomize various elements of the environment during training to get a more robust policy. In that case there will also be differences in how the environment behaves during and after training, so agent will respond differently as well.
3) Some agents are stochastic. This by itself implies different decisions/behavior even if everything else in the environment remains deterministic.
Hope that helps
Abdul Basith Ashraf on 3 Apr 2021
Are you saying that we should use an agent from a higher episode number?
More Answers (0)
Find more on Reinforcement Learning in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!Start Hunting!