How to Use the Reinforcement Learning Toolbox to Draw Observations While Training？

Hi!

How to Use the Reinforcement Learning Toolbox to Draw Observations While Training？Here is my code:

ObservationInfo = rlNumericSpec([12 1]);

% Initialize Action settings

ActionInfo = rlNumericSpec([6 1], ...

'LowerLimit', [-1; -1; -1; -1; -1; -1], ...

'UpperLimit', [1; 1; 1; 1; 1; 1]);

%Env

env = rlFunctionEnv(ObservationInfo,ActionInfo,'myStepFunction','myResetFunction');

% Simulation time and sample rate

Ts = 0.02;

% %% Deep Neural Network Options

% %Define the critic network

statePath = [

imageInputLayer([12 1 1],'Normalization','none','Name','observation')

fullyConnectedLayer(400,'Name','CriticStateFC1')

reluLayer('Name', 'Criticrelu1')

fullyConnectedLayer(300,'Name','CriticStateFC2')];

actionPath = [

imageInputLayer([6 1 1],'Normalization','none','Name','action')

fullyConnectedLayer(300,'Name','CriticActionFC1')];

commonPath = [

additionLayer(2,'Name','add')

reluLayer('Name','CriticCommonRelu')

fullyConnectedLayer(1,'Name','CriticOutput')];

criticNetwork = layerGraph();

criticNetwork = addLayers(criticNetwork,statePath);

criticNetwork = addLayers(criticNetwork,actionPath);

criticNetwork = addLayers(criticNetwork,commonPath);

criticNetwork = connectLayers(criticNetwork,'CriticStateFC2','add/in1');

criticNetwork = connectLayers(criticNetwork,'CriticActionFC1','add/in2');

criticOpts = rlRepresentationOptions('LearnRate',1e-03,'GradientThreshold',1);

critic = rlQValueRepresentation(criticNetwork,ObservationInfo,ActionInfo,...

'Observation',{'observation'},'Action',{'action'},criticOpts);

%Define the actor network

actorNetwork = [

imageInputLayer([12 1 1],'Normalization','none','Name','observation')

fullyConnectedLayer(400,'Name','ActorFC1')

reluLayer('Name','ActorRelu1')

fullyConnectedLayer(300,'Name','ActorFC2')

reluLayer('Name','ActorRelu2')

fullyConnectedLayer(6,'Name','ActorFC3')

tanhLayer('Name','ActorTanh')

scalingLayer('Name','ActorScaling','Scale',max(ActionInfo.UpperLimit))];

actorOpts = rlRepresentationOptions('LearnRate',1e-04,'GradientThreshold',1);

actor = rlDeterministicActorRepresentation(actorNetwork,ObservationInfo,ActionInfo,'Observation',{'observation'},'Action',{'ActorScaling'},actorOpts);

%% Set Agent and DDPG Options

agentOpts = rlDDPGAgentOptions(...

'SampleTime',Ts,...

'TargetSmoothFactor',1e-3,...

'ExperienceBufferLength',1e5,...

'DiscountFactor',0.99,...

'MiniBatchSize',128);

agentOpts.NoiseOptions.Variance = 0.6;

agentOpts.NoiseOptions.VarianceDecayRate = 1e-5;

agent = rlDDPGAgent(actor,critic,agentOpts);

%% Set Training Options

maxepisodes = 100;

trainOpts = rlTrainingOptions(...

'MaxEpisodes',maxepisodes,...

'MaxStepsPerEpisode',1000,...

'ScoreAveragingWindowLength',50,...

'Verbose',false,...

'Plots','training-progress',...

'StopTrainingCriteria','AverageReward',...

'StopTrainingValue',0,...

'SaveAgentCriteria','EpisodeReward',...

'SaveAgentValue',0);

%% Training

%Train the DDPG algorithm on the enviroment.

trainingStats = train(agent,env,trainOpts);

I would be grateful if you could help me!

Answer 1

[Emmanouil Tzorakoleftherakis]

You can use the information on plotting and visualization from this page to plot/visualize information during training

[Francisco Serra]

Hi @Emmanouil Tzorakoleftherakis Which page?

[Harold]

Hello @Emmalevel devil I'm sorry, but I don't see any information on this page about plotting and visualization techniques during training. Could you please provide the page again or perhaps share the specific section where this information is located? I'd be happy to help once I have the necessary context.

[Emmanouil Tzorakoleftherakis]

Updated the link above