How to set block parameters in a Matlab RL reset function
21 views (last 30 days)
Show older comments
I am using a reset function as a part of the rlSimulinkEnv and calling it in the main script. In the reset function, when I try to reset a constant block, this works as expected. However, when I try to reset variable found in a simulink block like a revolute joint, the value will be reset at the beginning of the first episode, but then never again.
agentBlk = [mdl '/RL Agent'];
observationInfo = rlNumericSpec([23 1]);
actionInfo = rlNumericSpec([6 1]);
observationInfo.Name = 'observation';
actionInfo.Name = 'action';
env = rlSimulinkEnv(mdl,agentBlk, observationInfo, actionInfo);
env.ResetFcn = @(in)handResetFcn(in,sep,yL_end,l);
Here is the reset function. In this case, the 'goal_x' and 'goal_y' variables correspond to constant blocks and reset every episode. The joint angles will not reset every time. The will be set to the first episode's random value for every episode.
function in = handResetFcn(in,sep,yL_end)
x = sep+.1*(rand-.5);
y = yL_end+.05*rand;
in = in.setVariable('goal_x',x);
in = in.setVariable('goal_y',y);
tl1_0 = pi/2 + .1*(rand-.5);
tl2_0 = pi/2 + .1*(rand-.5);
in = setBlockParameter(in,'FullHand_ChangeInitCondModel/Environment/Revolute Joint','PositionTargetValue',num2str(tl1_0));
in = setBlockParameter(in,'FullHand_ChangeInitCondModel/Environment/Revolute Joint1','PositionTargetValue',num2str(tl2_0));
end
1 Comment
Answers (1)
Steve Miller
on 28 Nov 2022
Hi Carson,
I suspect the reason this did not work is because you have not configured the target position to be a run-time parameter. If you do not configure this to be a run-time parameter, your sweep may not recognize that the value has changed from run to run. You can see in example Quadruped Robot Locomotion Using DDPG Agent that the joint targets are configured as run-time parameters.
Below is a screenshot showing the setting you need to change on the blocks.
--Steve
0 Comments
See Also
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!