How to set block parameters in a Matlab RL reset function

21 views (last 30 days)
I am using a reset function as a part of the rlSimulinkEnv and calling it in the main script. In the reset function, when I try to reset a constant block, this works as expected. However, when I try to reset variable found in a simulink block like a revolute joint, the value will be reset at the beginning of the first episode, but then never again.
agentBlk = [mdl '/RL Agent'];
observationInfo = rlNumericSpec([23 1]);
actionInfo = rlNumericSpec([6 1]);
observationInfo.Name = 'observation';
actionInfo.Name = 'action';
env = rlSimulinkEnv(mdl,agentBlk, observationInfo, actionInfo);
env.ResetFcn = @(in)handResetFcn(in,sep,yL_end,l);
Here is the reset function. In this case, the 'goal_x' and 'goal_y' variables correspond to constant blocks and reset every episode. The joint angles will not reset every time. The will be set to the first episode's random value for every episode.
function in = handResetFcn(in,sep,yL_end)
x = sep+.1*(rand-.5);
y = yL_end+.05*rand;
in = in.setVariable('goal_x',x);
in = in.setVariable('goal_y',y);
tl1_0 = pi/2 + .1*(rand-.5);
tl2_0 = pi/2 + .1*(rand-.5);
in = setBlockParameter(in,'FullHand_ChangeInitCondModel/Environment/Revolute Joint','PositionTargetValue',num2str(tl1_0));
in = setBlockParameter(in,'FullHand_ChangeInitCondModel/Environment/Revolute Joint1','PositionTargetValue',num2str(tl2_0));
end
  1 Comment
ajshank
ajshank on 2 Jun 2022
Edited: ajshank on 2 Jun 2022
I can confirm this is the case for Matlab R2021b. Other tricks such as assigning a workspace variable myVar for the block/mask parameter, and calling assignin("base", "myVar", val) in the reset function don't work either.

Sign in to comment.

Answers (1)

Steve Miller
Steve Miller on 28 Nov 2022
Hi Carson,
I suspect the reason this did not work is because you have not configured the target position to be a run-time parameter. If you do not configure this to be a run-time parameter, your sweep may not recognize that the value has changed from run to run. You can see in example Quadruped Robot Locomotion Using DDPG Agent that the joint targets are configured as run-time parameters.
Below is a screenshot showing the setting you need to change on the blocks.
--Steve

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!