rlOptimizer

Optimizer object for actors and critics

Syntax

algobj = rlOptimizer

algobj = rlOptimizer(algOptions)

Description

Create an optimizer object that updates the learnable parameters of an actor or critic in a custom training loop

algobj = rlOptimizer creates a default optimizer object. You can modify the object properties using dot notation.

algobj = rlOptimizer(algOptions) creates an optimizer object with properties specified by the optimizer options object algOptions.

Examples

Create Optimizer Object

Use rlOprimizer to create a default optimizer algorithm object to use for the training of an actor or critic in a custom training loop.

myAlg=rlOptimizer

myAlg = 
  rlADAMOptimizer with properties:

           GradientDecayFactor: 0.9000
    SquaredGradientDecayFactor: 0.9990
                       Epsilon: 1.0000e-08
                     LearnRate: 0.0100
        L2RegularizationFactor: 1.0000e-04
             GradientThreshold: Inf
       GradientThresholdMethod: "l2norm"

By default, the function returns an rlADAMOptimizer object with default options. You can use dot notation to change some parameters.

myAlg.LearnRate = 0.1;

You can now create a structure and set its CriticOptimizer or ActorOptimizer field to myAlg. When you call runEpisode, pass the structure as an input parameter. The runEpisode function can then use the update method of myAlg to update the learnable parameters of your actor or critic.

Create Optimizer Object Specifying an Option Set

Use rlOprimizerOptions to create an optimizer option object. Specify the algorithm as "rmsprop" and set the learning rate to 0.2.

myOptions=rlOptimizerOptions("Algorithm","rmsprop", ...
              LearnRate=0.2);

Use rlOprimizer to create an optimizer algorithm object to use for the training of an actor or critic in a custom training loop. Specify the optimizer option set myOptions as input parameter.

myAlg=rlOptimizer(myOptions)

myAlg = 
  rlRMSPropOptimizer with properties:

    SquaredGradientDecayFactor: 0.9990
                       Epsilon: 1.0000e-08
                     LearnRate: 0.2000
        L2RegularizationFactor: 1.0000e-04
             GradientThreshold: Inf
       GradientThresholdMethod: "l2norm"

The function returns an rlRMSPropOptimizer object with default options. You can use dot notation to change some parameters.

myAlg.GradientThreshold = 2;

You can now create a structure and set its CriticOptimizer or ActorOptimizer field to myAlg. When you call runEpisode, pass the structure as an input parameter. The runEpisode function can then use the update method of myAlg to update the learnable parameters of your actor or critic.

Input Arguments

algOptions — Algorithm options object

default Adam option set (default) | rlOptimizerOptions object

Algorithm options object, specified as an rlOptimizerOptions object.

Example: rlOptimizerOptions(Algorithm="sgdm",LearnRate=0.2)

Output Arguments

algobj — Algorithm optimizer object

rlADAMOptimizer, rlSGDMOptimizer, or rlRMSPropOptimizer object

Algorithm optimizer object, returned as an rlADAMOptimizer, rlSGDMOptimizer, or rlRMSPropOptimizer object. The runEpisode function uses the update method of the returned object to update the learnable parameter of an actor or critic.

Version History

Introduced in R2022a