Clear Filters
Clear Filters

Computation Time Reinforcement Learning Toolbox

9 views (last 30 days)
Hello everyone
Following up on my question here, I have managed to get the example of the ball balancing system running. Now when I try to do the training on my machine, it doesn't take a few minutes as specified, but rather a few hours or days, since the simulation progresses slower than real time. I tried using the parallel computing toolbox, but didn't get significantly better results. I am using a 6 core Intel i5-8500 CPU at 3 GHz. Is this only an issue of my machine, or are there other ways to speed up the simulation?
Cheers

Accepted Answer

Ari Biswas
Ari Biswas on 24 Feb 2021
Training the SAC agent in the ball balance example could take as long as a day, generally speaking. We are working on performance improvements for future releases but as of now you may only get minor performance benefits.
1. Disable the Simscape Mechanics Explorer as mentioned here. You will not be able to see the animation in this case. You can always turn it back on after training is finished.
2. Disable the training visualization by setting the 'Plots' property in the training options to 'none'. You can also set the 'Verbose' property to view progress in the command window instead.
The other (more involving) approach is to experiment with different reward functions or tuning the agent hyperparameters.
  2 Comments
green_ananas
green_ananas on 25 Feb 2021
Thanks Ari, a day sounds about right to me. I was only wondering whether I did something wrong. After trying what you proposed, it seems that the limiting factor is indeed the simulation itself, especially the contact force and friction between the ball and the plate seems to be quite expensive.
Ari Biswas
Ari Biswas on 25 Feb 2021
Yes, the demo has high fidelity contact force modeling. You can experiment with different solver settings (solver type, absolute and relative tolerances) to check for any peformance improvements, but at the cost of simulation accuracy.

Sign in to comment.

More Answers (0)

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!