Clear Filters
Clear Filters

Is there a way to visualize simulation result (after RL agent training) at the command line?

5 views (last 30 days)
Hi!
Is there any way to visualize the result of a simulation at the command line, given a simulation result (as on the following picture for example)?
I mean, I can run the following lines to get the simulation result, but how could I then display it? How should I for example open the Reinforcement Learning Designer app at the command line and then 'plot' the simResults?
% Perform simulations
simResults = sim(env,agentPPO,simOptions);
Is there also a way of registering the "experience1" variable in the workspace?

Accepted Answer

Neha
Neha on 14 May 2024
Edited: Neha on 14 May 2024
Hi Nicolas,
I understand you're looking to export simulation results from the Reinforcement Learning Designer app to the workspace for visualization. Here's how you can do it:
  1. After obtaining the results in the app, click on "Save Session" within the "Reinforcement Learning" tab. This action saves a .mat file in your workspace.
  2. Access the results through the variable: "RLDesignerSession.Data.SimResults". This variable is a struct array containing all the experiments.
  3. For instance, to access the reward timeseries data for the first simulation of the first experiment, use: "RLDesignerSession.Data.SimResults(1).Data(1).Reward".
  4. To visualize the results, sum up the rewards for each simulation to obtain the cumulative reward. This data can then be used to plot a bar graph similar to the one in the Reinforcement Learning Designer App.
You can refer to the following documentation link for more information on the fields of the "RLDesignerSession.Data.SimResults" struct:
  1 Comment
Nicolas CRETIN
Nicolas CRETIN on 17 May 2024
Edited: Nicolas CRETIN on 17 May 2024
Thank you so much Neha!
Here is the code I finally used if someone is interrested (and the graphical result, in the case where the agent has been very poorly trained):
% -------- perform training --------
trainingStats = train(agentPPO, env, trainOpts);
% -------- perform the simulation to double check the training --------
% Create simulation options
simOptions = rlSimulationOptions();
simOptions.MaxSteps = 10;
simOptions.NumSimulations = 8;
% Perform simulations
simResults = sim(env,agentPPO,simOptions);
% plot reward for these simulations
sizeSimResults = size(simResults);
cumulativeReward = zeros(sizeSimResults);
for i = 1:sizeSimResults(1)
cumulativeReward(i) = sum(simResults(1).Reward.Data);
end
simIndex = 1:sizeSimResults(1);
meanReward = mean(cumulativeReward) * ones(sizeSimResults);
figure('Name', 'Simulation results');
bar(simIndex, cumulativeReward);
hold on
plot(simIndex, meanReward,'r--')
hold off
xlabel('Simulation index');
ylabel('Cumulative reward');
title('Cumulative rewards for each simulation');
legend('Cumulative Reward', 'Mean Reward');
% let's admit that simulation ii has a bad reward
% we want to display the output speed for this episode, to get to know what
% happened
ii = 1;
figure("Name", "Investigate bad simulation reward")
plot(simResults(ii).SimulationInfo.logsout{4}.Values.Time, ...
simResults(ii).SimulationInfo.logsout{4}.Values.Data)
hold on
plot(simResults(ii).SimulationInfo.logsout{5}.Values.Time, ...
simResults(ii).SimulationInfo.logsout{5}.Values.Data)
hold off
xlabel('time (seconds)');
ylabel('output speed (tr/min)');
title('Investigate bad simulation reward');
legend('reference', 'output speed');

Sign in to comment.

More Answers (0)

Products


Release

R2022a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!