how to change the reward parameter in RL toolbox every time step?

Hi,

I am using RL toolbox for energy mangement of a energy storage. my reward is like follow:

C*P where C is a parameter ant P the action. As the problem is energy mangemnt, the C which is the price of the power is difffrent every time step (for this exampl 1 hour)

currently in my simulink model i use a ps counter to count evert time sample of simulink and give a value to C for each time sample, but its causing problems. Is there any way to acess the time step number of rl toolbox?

Answer 1

[Emmanouil Tzorakoleftherakis]

One way is to implement your logic in your reward subsystem using a MATLAB Fcn block. Execute the MATLAB Fcn block every Ts time units (1hr in your problem) and add a persistent variable that keeps track of how many times the reward subsystem has been accessed.

One comment on your RL problem design - if C depends on time only, that should also be used as an observation (alternatively, you could use the time stamp itself).

[mary]

Thank you very much for your answer. My question is how can I execute my matlab FCN every Ts time unit?

Currently, I use a PS counter and my function is plented as an if function. I wanted to know if there is any better way?

Thank you very much on the hint.

I have another question can we assign a dynamic greedy epsilon with RL toolbox? my RL finds the optimal answer in the beginiging of the training but loses it and finds converges to another sub optimal reward.

[Emmanouil Tzorakoleftherakis]

If you right click the MATLAB Fcn block, select Block Parameters and change the Sample Time.

You cannot assign dynamic epsilon values I believe, but you can change the decay rate and the minimum epsilon values to allow the agent to explore more (see here for example).

[mary]

Thank you very much for your answer. However, I hope dynamic epsilon rate and also learn factor needs to be added in future versions.