Options for Q-learning agent
to specify options for creating Q-learning agents. To create a Q-learning agent, use
For more information on Q-learning agents, see Q-Learning Agents.
For more information on the different types of reinforcement learning agents, see Reinforcement Learning Agents.
opt = rlQAgentOptions
rlQAgentOptions object for use as an argument when creating a
Q-learning agent using all default settings. You can modify the object properties using
option properties using
name-value pairs. For example,
opt = rlQAgentOptions(
creates an option set with a discount factor of
0.95. You can specify
multiple name-value pairs. Enclose each property name in quotes.
EpsilonGreedyExploration — Options for epsilon-greedy exploration
Options for epsilon-greedy exploration, specified as an
EpsilonGreedyExploration object with the following
|Probability threshold to either randomly select an action or select the action that maximizes the state-action value function. A larger value of |
|Minimum value of |
At the end of each training time step, if
Epsilon is greater than
EpsilonMin, then it is updated using the following
Epsilon = Epsilon*(1-EpsilonDecay)
Epsilon is conserved between the end of an episode and
the start of the next one. Therefore, it keeps on uniformly decreasing over multiple
episodes until it reaches
If your agent converges on local optima too quickly, you can promote agent exploration by
To specify exploration options, use dot notation after creating the
opt. For example, set the epsilon value to
opt.EpsilonGreedyExploration.Epsilon = 0.9;
CriticOptimizerOptions — Critic optimizer options
Critic optimizer options, specified as an
rlOptimizerOptions object. It allows you to specify training parameters of
the critic approximator such as learning rate, gradient threshold, as well as the
optimizer algorithm and its parameters. For more information, see
SampleTime — Sample time of agent
1 (default) | positive scalar |
Sample time of agent, specified as a positive scalar or as
-1. Setting this
-1 allows for event-based simulations.
Within a Simulink® environment, the RL Agent block
in which the agent is specified to execute every
of simulation time. If
block inherits the sample time from its parent subsystem.
Within a MATLAB® environment, the agent is executed every time the environment advances. In
SampleTime is the time interval between consecutive
elements in the output experience returned by
-1, the time interval between
consecutive elements in the returned output experience reflects the timing of the event
that triggers the agent execution.
DiscountFactor — Discount factor
0.99 (default) | positive scalar less than or equal to 1
Discount factor applied to future rewards during training, specified as a positive scalar less than or equal to 1.
|Q-learning reinforcement learning agent|
Create Q-Learning Agent Options Object
rlQAgentOptions object that specifies the agent sample time.
opt = rlQAgentOptions(SampleTime=0.5)
opt = rlQAgentOptions with properties: EpsilonGreedyExploration: [1x1 rl.option.EpsilonGreedyExploration] CriticOptimizerOptions: [1x1 rl.option.rlOptimizerOptions] SampleTime: 0.5000 DiscountFactor: 0.9900 InfoToSave: [1x1 struct]
You can modify options using dot notation. For example, set the agent discount factor to
opt.DiscountFactor = 0.95;
Introduced in R2019a