Setting Options :: Setting Up an Optimization (Optimization Toolbox™)

Setting Options

On this page…
Default Options Changing the Default Settings Tolerances and Stopping Criteria Checking Validity of Gradients or Jacobians

On this page…

Checking Validity of Gradients or Jacobians

Default Options

The options structure contains options used in the optimization routines. If, on the first call to an optimization routine, the options structure is not provided, or is empty, a set of default options is generated. Some of the default options values are calculated using factors based on problem size, such as MaxFunEvals. Some options are dependent on the specific solver algorithms, and are documented on those solver function reference pages.

Optimization Options provides an overview of all the options in the options structure.

Changing the Default Settings

The optimset function creates or updates an options structure to pass to the various optimization functions. The arguments to the optimset function are option name and option value pairs, such as TolX and 1e-4. Any unspecified properties have default values. You need to type only enough leading characters to define the option name uniquely. Case is ignored for option names. For option values that are strings, however, case and the exact string are necessary.

help optimset provides information that defines the different options and describes how to use them.

Here are some examples of the use of optimset.

Returning All Options

optimset returns all the options that can be set with typical values and default values.

Determining Options Used by a Function

The options structure defines the options passed to solvers. Because functions do not use all the options, it can be useful to find which options are used by a particular function. There are several ways of determining the available options:

Consult the combined Optimization Options table.
To find detailed descriptions of the options and their default values, consult the Options table in the function reference pages. For example, the available options for fmincon appear in this table.
To find available options at the command line, pass the name of the function (in this example, fmincon) to optimset:
```
optimset('fmincon')
```
or
```
optimset fmincon
```
or
```
optimset(@fmincon)
```
This statement returns a structure. Generally, fields not used by the function have empty values ([]); fields used by the function are set to their default values for the given function. However, some solvers have different default values depending on the algorithm used. For example, fmincon has a default MaxIter value of 400 for the trust-region-reflective and active-set algorithms, but a default value of 1000 for the interior-point algorithm. optimset fmincon returns [] for the MaxIter field.
You can also check the available options and their defaults in the Optimization Tool. These can change depending on problem and algorithm settings. These three pictures show how the available options for derivatives change as the type of supplied derivatives change:

Setting More Than One Option

You can specify multiple options with one call to optimset. For example, to reset the Display option and the tolerance on x, enter

options = optimset('Display','iter','TolX',1e-6);

Updating an options Structure

To update an existing options structure, call optimset and pass the existing structure name as the first argument:

options = optimset(options,'Display','iter','TolX',1e-6);

Retrieving Option Values

Use the optimget function to get option values from an options structure. For example, to get the current display option, enter the following:

verbosity = optimget(options,'Display');

Displaying Output

To display output at each iteration, enter

options = optimset('Display','iter');

This command sets the value of the Display option to 'iter', which causes the solver to display output at each iteration. You can also turn off any output display ('off'), display output only at termination ('final'), or display output only if the problem fails to converge ('notify').

Choosing an Algorithm

Some solvers explicitly use an option called LargeScale for choosing which algorithm to use: fminunc, linprog, and others. Other solvers do not use the LargeScale attribute explicitly, but have an option called Algorithm instead: fmincon, fsolve, and others. All algorithms in Algorithm solvers are large scale, unless otherwise noted in their function reference pages. The term large-scale is explained in Large-Scale vs. Medium-Scale Algorithms. For all solvers that have a large-scale algorithm, the default is for the function to use a large-scale algorithm (e.g., LargeScale is set to 'on' by default).

To use a medium-scale algorithm in a solver that takes the LargeScale option, enter

options = optimset('LargeScale','off');

For solvers that use the Algorithm option, choose the algorithm by entering

options = optimset('Algorithm','algorithm-name');

algorithm-name is the name of the chosen algorithm. You can find the choices in the function reference pages for each solver.

Large-Scale vs. Medium-Scale Algorithms. An optimization algorithm is large scale when it uses linear algebra that does not need to store, nor operate on, full matrices. This may be done internally by storing sparse matrices, and by using sparse linear algebra for computations whenever possible. Furthermore, the internal algorithms either preserve sparsity, such as a sparse Cholesky decomposition, or do not generate matrices, such as a conjugate gradient method. Large-scale algorithms are accessed by setting the LargeScale option to on, or setting the Algorithm option appropriately (this is solver-dependent).

In contrast, medium-scale methods internally create full matrices and use dense linear algebra. If a problem is sufficiently large, full matrices take up a significant amount of memory, and the dense linear algebra may require a long time to execute. Medium-scale algorithms are accessed by setting the LargeScale option to off, or setting the Algorithm option appropriately (this is solver-dependent).

Don't let the name "large-scale" mislead you; you can use a large-scale algorithm on a small problem. Furthermore, you do not need to specify any sparse matrices to use a large-scale algorithm. Choose a medium-scale algorithm to access extra functionality, such as additional constraint types, or possibly for better performance.

Defining Output Functions and Plot Functions

Output functions and plot functions give information on the progress of a solver. They report results after each iteration of a solver. Additionally, they can halt a solver. For more information and examples of their use, see Output Functions and Plot Functions.

Call output or plot functions by passing a function handle, or a cell array of function handles. For output functions, the syntax is

options = optimset('OutputFcn',@outfun);

options = optimset('OutputFcn',...
    {@outfun1,@outfun2,...});

For plot functions, the syntax is

options = optimset('PlotFcns',@plotfun);

options = optimset('PlotFcns',...
    {@plotfun1,@plotfun2,...});

There are several predefined plot functions. See the solver function reference pages for a list, or view a list in the PlotFcns entry of the Options Structure table.

Tolerances and Stopping Criteria

The number of iterations in an optimization depends on a solver's stopping criteria. These criteria include:

First-order optimality measure
Tolerance TolX
Tolerance TolFun
Tolerance TolCon
Bound on number of iterations taken MaxIter
Bound on number of function evaluations MaxFunEvals

First-order optimality measure is defined in First-Order Optimality Measure. Iterations and function evaluations are discussed in Iterations and Function Counts. The remainder of this section describes how Optimization Toolbox solvers use stopping criteria to terminate optimizations.

Set tolerances and other criteria using optimset as explained in Setting Options.

TolX is a lower bound on the size of a step, meaning the norm of (x_i – x_i+1). If the solver attempts to take a step that is smaller than TolX, the iterations end. TolX is sometimes used as a relative bound, meaning iterations end when |(x_i – x_i+1)| < TolX*(1 + |x_i|), or a similar relative measure.
TolFun is a lower bound on the change in the value of the objective function during a step. If |f(x_i) – f(x_i+1)| < TolFun, the iterations end. TolFun is sometimes used as a relative bound, meaning iterations end when |f(x_i) – f(x_i+1)| < TolFun(1 + |f(x_i)|), or a similar relative measure.
TolFun is also a bound on the first-order optimality measure. If the optimality measure is less than TolFun, the iterations end. TolFun can also be a relative bound.
TolCon is an upper bound on the magnitude of any constraint functions. If a solver returns a point x with c(x) > TolCon or |ceq(x)| > TolCon, the solver reports that the constraints are violated at x. TolCon can also be a relative bound.

Note TolCon operates differently from other tolerances. If TolCon is not satisfied (i.e., if the magnitude of the constraint function exceeds TolCon), the solver attempts to continue, unless it is halted for another reason. A solver does not halt simply because TolCon is satisfied.

There are two other tolerances that apply to particular solvers: TolPCG and MaxPCGIter. These relate to preconditioned conjugate gradient steps. For more information, see Preconditioned Conjugate Gradient Method.

There are several tolerances that apply only to the interior-point algorithm in the solver fmincon. See Optimization Options for more information.

Checking Validity of Gradients or Jacobians

Many solvers allow you to supply a function that calculates first derivatives (gradients or Jacobians) of objective or constraint functions. You can check whether the derivatives calculated by your function match finite-difference approximations. This check can help you diagnose whether your derivative function is correct.

If a component of the gradient function is less than 1, "match" means the absolute difference of the gradient function and the finite-difference approximation of that component is less than 1e-6.
Otherwise, "match" means that the relative difference is less than 1e-6.

The DerivativeCheck option causes the solver to check the supplied derivative against a finite-difference approximation only at the initial point. If the finite-difference and supplied derivatives do not match, the solver reports the discrepancy, and asks whether you want to continue. If the derivatives match, the solver reports the differences, and continues without asking.

When DerivativeCheck is 'on' and GradObj or Jacobian is 'on', the solver compares your objective gradient function against a finite difference approximation. Similarly, when DerivativeCheck is 'on' and GradConstr is 'on', the solver compares your constraint gradient function against a finite-difference approximation.

Choose an initial point with nonzero, noninteger components when using DerivativeCheck. Often, gradients match at a simple initial point, but not at other points. Furthermore, as shown in Checking Derivatives with the Optimization Tool, the small inaccuracy inherent in finite difference approximations can be especially problematic at 0 or other integer components.

How to Check Derivatives

At the MATLAB command line:
1. Set the GradObj, GradConstr, or Jacobian options to 'on' with optimset. Make sure your objective or constraint functions supply the appropriate derivatives.
2. Set the DerivativeCheck option to 'on'.
Using the Optimization Tool:
1. In the Problem Setup and Results pane, choose Derivatives: Objective function: Gradient supplied or Nonlinear constraint function: Derivatives: Gradient supplied. Make sure your objective or constraint functions supply the appropriate derivatives.
2. In the Options pane, check User-supplied derivatives > Validate user-supplied derivatives

Best Practices for Checking Derivatives

Choose a random initial point, such as randn(size(x0)). If all components have lower and upper bounds, choose a random point satisfying the bounds: rand(size(x0)).*(ub-lb) + lb. Here x0 has same size as your initial vector, lb is the vector of lower bounds, and ub is the vector of upper bounds.
Use central finite differences (for applicable solvers). These differences are more accurate than the default forward differences. To do so:
- At the MATLAB command line, set FinDiffType option to 'central' with optimset.
- Using the Optimization Tool, in the Approximated derivatives pane, set Type to central differences.

Example: Checking Derivatives of Objective and Constraint Functions

Objective and Constraint Functions. Consider the problem of minimizing the Rosenbrock function within the unit disk as described in Example: Nonlinear Constrained Minimization. The rosenboth function calculates the objective function and its gradient:

function [f g H] = rosenboth(x)

f = 100*(x(2) - x(1)^2)^2 + (1-x(1))^2;

if nargout > 1
    g = [-400*(x(2)-x(1)^2)*x(1)-2*(1-x(1));
        200*(x(2)-x(1)^2)];
    
    if nargout > 2
        H = [1200*x(1)^2-400*x(2)+2, -400*x(1);
            -400*x(1), 200];  
    end
end

rosenboth calculates the Hessian, too, but this example does not use the Hessian.

The unitdisk2 function correctly calculates the constraint function and its gradient:

function [c,ceq,gc,gceq] = unitdisk2(x)
c = x(1)^2 + x(2)^2 - 1;
ceq = [ ];

if nargout > 2
    gc = [2*x(1);2*x(2)];
    gceq = [];
end

The unitdiskb function incorrectly calculates gradient of the constraint function:

function [c ceq gc gceq] = unitdiskb(x)
c = x(1)^2 + x(2)^2 - 1;
ceq = [ ];

if nargout > 2
    gc = [x(1);x(2)]; % Gradient incorrect: off by a factor of 2
    gceq = [];
end

Checking Derivatives at the Command Line.

Set the options to use the interior-point algorithm, gradient of objective and constraint functions, and the DerivativeCheck option:
```
options = optimset('Algorithm','interior-point',...
    'DerivativeCheck','on','GradObj','on','GradConstr','on');
```
Setting Algorithm to 'active-set' works, too. Best practice would set FinDiffType to 'central'.

Solve the minimization with fmincon using the erroneous unitdiskb constraint function:

[x fval exitflag output] = fmincon(@rosenboth,...
   [-1;2] + randn(2,1),[],[],[],[],[],[],@unitdiskb,options);
Objective function derivatives:
Maximum relative discrepancy between derivatives = 2.60089e-008
Nonlinear inequality constraint derivatives:
Maximum relative discrepancy between derivatives = 1
Caution: user-supplied and forward finite-difference derivatives 
 do not match within 1e-006 relative tolerance.
Maximum relative difference occurs in element (1,1):
  User-supplied constraint gradient:     -3.25885
  Finite-difference constraint gradient: -6.51769
Strike any key to continue or Ctrl-C to abort.

The constraint function does not match the calculated gradient, encouraging you to check the function for an error.

Replace the unitdiskb constraint function with unitdisk2 and run the minimization again:

[x fval exitflag output] = fmincon(@rosenboth,...
   [-1;2] + randn(2,1),[],[],[],[],[],[],@unitdisk2,options);
Objective function derivatives:
Maximum relative discrepancy between derivatives = 3.59821e-008
Nonlinear inequality constraint derivatives:
Maximum relative discrepancy between derivatives = 1.14322e-008

Local minimum found that satisfies the constraints...

Checking Derivatives with the Optimization Tool.

Note Take care when using DerivativeCheck with the Optimization Tool. If the derivative check fails, there is no indication in the Optimization Tool. Check the Command Window to see the message and act, if necessary.

To set up the example using correct derivative functions, but starting from [0 0], using the Optimization Tool:

Launch the Optimization Tool by entering optimtool at the command line.
Set the Problem Setup and Results pane to match the following figure:
Set the Options pane to match the following figure:

Press the Start button under Run solver and view results.

The output screen displays

However, the process does not continue. The MATLAB command window displays:

Objective function derivatives:
Maximum relative discrepancy between derivatives = 1.49012e-006
Caution: user-supplied and forward finite-difference derivatives
 do not match within 1e-006 relative tolerance.
Maximum relative difference occurs in element 2 of gradient:
  User-supplied gradient:      0
  Finite-difference gradient:  1.49012e-006
Strike any key to continue or Ctrl-C to abort.

The forward finite difference approximation is inaccurate enough at [0 0] that the derivative check fails.

Strike a key in the command window, and the optimization continues to completion.

The derivative check succeeds when you select central differences in the Approximated derivatives > Type pane. The derivative check also succeeds when you select the initial point [-1 2], or most random points.

Passing Extra Parameters

Examining Results

Recommended Products

Get MATLAB trial software

Includes the most popular MATLAB recorded presentations with Q&A sessions led by MATLAB experts.

Get the Interactive Kit