Setting Options :: Setting Up an Optimization (Optimization Toolbox™)

The options structure contains options used in the optimization routines. If, on the first call to an optimization routine, the options structure is not provided, or is empty, a set of default options is generated. Some of the default options values are calculated using factors based on problem size, such as MaxFunEvals. Some options are dependent on the specific solver algorithms, and are documented on those solver function reference pages.

Optimization Options Reference provides an overview of all the options in the options structure.

Changing the Default Settings

The optimset function creates or updates an options structure to pass to the various optimization functions. The arguments to the optimset function are option name and option value pairs, such as TolX and 1e-4. Any unspecified properties have default values. You need to type only enough leading characters to define the option name uniquely. Case is ignored for option names. For option values that are strings, however, case and the exact string are necessary.

help optimset provides information that defines the different options and describes how to use them.

Here are some examples of the use of optimset.

Returning All Options

optimset returns all the options that can be set with typical values and default values.

Determining Options Used by a Function

The options structure defines the options passed to solvers. Because functions do not use all the options, it can be useful to find which options are used by a particular function. There are several ways of determining the available options:

Consult the combined Optimization Options Reference table.
To find detailed descriptions of the options and their default values, consult the Options table in the function reference pages. For example, the available options for fmincon appear in this table.
To find available options at the command line, pass the name of the function (in this example, fmincon) to optimset:
```
optimset('fmincon')
```
or
```
optimset fmincon
```
or
```
optimset(@fmincon)
```
This statement returns a structure. Generally, fields not used by the function have empty values ([]); fields used by the function are set to their default values for the given function. However, some solvers have different default values depending on the algorithm used. For example, fmincon has a default MaxIter value of 400 for the trust-region-reflective, sqp, and active-set algorithms, but a default value of 1000 for the interior-point algorithm. optimset fmincon returns [] for the MaxIter field.
You can also check the available options and their defaults in the Optimization Tool. These can change depending on problem and algorithm settings. These three pictures show how the available options for derivatives change as the type of supplied derivatives change:

Setting More Than One Option

You can specify multiple options with one call to optimset. For example, to reset the Display option and the tolerance on x, enter

options = optimset('Display','iter','TolX',1e-6);

Updating an options Structure

To update an existing options structure, call optimset and pass the existing structure name as the first argument:

options = optimset(options,'Display','iter','TolX',1e-6);

Retrieving Option Values

Use the optimget function to get option values from an options structure. For example, to get the current display option, enter the following:

verbosity = optimget(options,'Display');

Displaying Output

To display output at each iteration, enter

options = optimset('Display','iter');

This command sets the value of the Display option to 'iter', which causes the solver to display output at each iteration. You can also turn off any output display ('off'), display output only at termination ('final'), or display output only if the problem fails to converge ('notify').

Choosing an Algorithm

Some solvers explicitly use an option called LargeScale for choosing which algorithm to use: fminunc, linprog, and others. Other solvers do not use the LargeScale attribute explicitly, but have an option called Algorithm instead: fmincon, fsolve, and others. All algorithms in Algorithm solvers are large scale, unless otherwise noted in their function reference pages. The term large-scale is explained in Large-Scale vs. Medium-Scale Algorithms. For all solvers that have a large-scale algorithm, the default is for the function to use a large-scale algorithm (e.g., LargeScale is set to 'on' by default).

For help deciding which algorithm to use, see Choosing the Algorithm.

To use a medium-scale algorithm in a solver that takes the LargeScale option, enter

options = optimset('LargeScale','off');

For solvers that use the Algorithm option, choose the algorithm by entering

options = optimset('Algorithm','algorithm-name');

algorithm-name is the name of the chosen algorithm. You can find the choices in the function reference pages for each solver.

Large-Scale vs. Medium-Scale Algorithms. An optimization algorithm is large scale when it uses linear algebra that does not need to store, nor operate on, full matrices. This may be done internally by storing sparse matrices, and by using sparse linear algebra for computations whenever possible. Furthermore, the internal algorithms either preserve sparsity, such as a sparse Cholesky decomposition, or do not generate matrices, such as a conjugate gradient method. Large-scale algorithms are accessed by setting the LargeScale option to on, or setting the Algorithm option appropriately (this is solver-dependent).

In contrast, medium-scale methods internally create full matrices and use dense linear algebra. If a problem is sufficiently large, full matrices take up a significant amount of memory, and the dense linear algebra may require a long time to execute. Medium-scale algorithms are accessed by setting the LargeScale option to off, or setting the Algorithm option appropriately (this is solver-dependent).

Don't let the name "large-scale" mislead you; you can use a large-scale algorithm on a small problem. Furthermore, you do not need to specify any sparse matrices to use a large-scale algorithm. Choose a medium-scale algorithm to access extra functionality, such as additional constraint types, or possibly for better performance.

Defining Output Functions and Plot Functions

Output functions and plot functions give information on the progress of a solver. They report results after each iteration of a solver. Additionally, they can halt a solver. For more information and examples of their use, see Output Functions and Plot Functions.

Call output or plot functions by passing a function handle, or a cell array of function handles. For output functions, the syntax is

options = optimset('OutputFcn',@outfun);

options = optimset('OutputFcn',...
    {@outfun1,@outfun2,...});

For plot functions, the syntax is

options = optimset('PlotFcns',@plotfun);

options = optimset('PlotFcns',...
    {@plotfun1,@plotfun2,...});

There are several predefined plot functions. See the solver function reference pages for a list, or view a list in the PlotFcns entry of the Options Structure table.

Tolerances and Stopping Criteria

The number of iterations in an optimization depends on a solver's stopping criteria. These criteria include several tolerances you can set. Generally, a tolerance is a threshold which, if crossed, stops the iterations of a solver.

Set tolerances and other criteria using optimset as explained in Setting Options.

TolX is a lower bound on the size of a step, meaning the norm of (x_i – x_i+1). If the solver attempts to take a step that is smaller than TolX, the iterations end. TolX is sometimes used as a relative bound, meaning iterations end when |(x_i – x_i+1)| < TolX*(1 + |x_i|), or a similar relative measure.
TolFun is a lower bound on the change in the value of the objective function during a step. If |f(x_i) – f(x_i+1)| < TolFun, the iterations end. TolFun is sometimes used as a relative bound, meaning iterations end when |f(x_i) – f(x_i+1)| < TolFun(1 + |f(x_i)|), or a similar relative measure.
TolFun is also a bound on the first-order optimality measure. If the optimality measure is less than TolFun, the iterations end. TolFun can also be a relative bound. First-order optimality measure is defined in First-Order Optimality Measure.

TolCon is an upper bound on the magnitude of any constraint functions. If a solver returns a point x with c(x) > TolCon or |ceq(x)| > TolCon, the solver reports that the constraints are violated at x. TolCon can also be a relative bound.

Note TolCon operates differently from other tolerances. If TolCon is not satisfied (i.e., if the magnitude of the constraint function exceeds TolCon), the solver attempts to continue, unless it is halted for another reason. A solver does not halt simply because TolCon is satisfied.

MaxIter is a bound on the number of solver iterations. MaxFunEvals is a bound on the number of function evaluations. Iterations and function evaluations are discussed in Iterations and Function Counts.

There are two other tolerances that apply to particular solvers: TolPCG and MaxPCGIter. These relate to preconditioned conjugate gradient steps. For more information, see Preconditioned Conjugate Gradient Method.

There are several tolerances that apply only to the fmincon interior-point algorithm. For more information, see Optimization Options Reference.

Checking Validity of Gradients or Jacobians

Many solvers allow you to supply a function that calculates first derivatives (gradients or Jacobians) of objective or constraint functions. You can check whether the derivatives calculated by your function match finite-difference approximations. This check can help you diagnose whether your derivative function is correct.

If a component of the gradient function is less than 1, "match" means the absolute difference of the gradient function and the finite-difference approximation of that component is less than 1e-6.
Otherwise, "match" means that the relative difference is less than 1e-6.

The DerivativeCheck option causes the solver to check the supplied derivative against a finite-difference approximation at just one point. If the finite-difference and supplied derivatives do not match, the solver errors. If the derivatives match to within 1e-6, the solver reports the calculated differences, and continues iterating without further derivative checks. Solvers check the match at a point that is a small random perturbation of the initial point x0, modified to be within any bounds. Solvers do not include the computations for DerivativeCheck in the function count; see Iterations and Function Counts.

How to Check Derivatives

At the MATLAB command line:
1. Set the GradObj, GradConstr, or Jacobian options to 'on' with optimset. Make sure your objective or constraint functions supply the appropriate derivatives.
2. Set the DerivativeCheck option to 'on'.
Using the Optimization Tool:
1. In the Problem Setup and Results pane, choose Derivatives: Objective function: Gradient supplied or Nonlinear constraint function: Derivatives: Gradient supplied. Make sure your objective or constraint functions supply the appropriate derivatives.
2. In the Options pane, check User-supplied derivatives > Validate user-supplied derivatives

Central finite differences are more accurate than the default forward finite differences. To use central finite differences:

At the MATLAB command line, set FinDiffType option to 'central' with optimset.
Using the Optimization Tool, in the Approximated derivatives pane, set Type to central differences.

Example: Checking Derivatives of Objective and Constraint Functions

Objective and Constraint Functions. Consider the problem of minimizing the Rosenbrock function within the unit disk as described in Example: Nonlinear Constrained Minimization. The rosenboth function calculates the objective function and its gradient:

function [f g H] = rosenboth(x)

f = 100*(x(2) - x(1)^2)^2 + (1-x(1))^2;

if nargout > 1
    g = [-400*(x(2)-x(1)^2)*x(1)-2*(1-x(1));
        200*(x(2)-x(1)^2)];
    
    if nargout > 2
        H = [1200*x(1)^2-400*x(2)+2, -400*x(1);
            -400*x(1), 200];  
    end
end

rosenboth calculates the Hessian, too, but this example does not use the Hessian.

The unitdisk2 function correctly calculates the constraint function and its gradient:

function [c,ceq,gc,gceq] = unitdisk2(x)
c = x(1)^2 + x(2)^2 - 1;
ceq = [ ];

if nargout > 2
    gc = [2*x(1);2*x(2)];
    gceq = [];
end

The unitdiskb function incorrectly calculates gradient of the constraint function:

function [c ceq gc gceq] = unitdiskb(x)
c = x(1)^2 + x(2)^2 - 1;
ceq = [ ];

if nargout > 2
    gc = [x(1);x(2)]; % Gradient incorrect: off by a factor of 2
    gceq = [];
end

Checking Derivatives at the Command Line.

Set the options to use the interior-point algorithm, gradient of objective and constraint functions, and the DerivativeCheck option:

rng(0,'twister'); % for reproducibility--DerivativeCheck
    % randomly perturbs the initial point
options = optimset('Algorithm','interior-point',...
    'DerivativeCheck','on','GradObj','on','GradConstr','on');

Solve the minimization with fmincon using the erroneous unitdiskb constraint function:

[x fval exitflag output] = fmincon(@rosenboth,...
   [-1;2],[],[],[],[],[],[],@unitdiskb,options);
____________________________________________________________
   Derivative Check Information  

Objective function derivatives:
Maximum relative difference between user-supplied 
and finite-difference derivatives = 1.84768e-008.

Nonlinear inequality constraint derivatives:
Maximum relative difference between user-supplied 
and finite-difference derivatives = 1.
 User-supplied constraint derivative element (2,1):     1.99838
 Finite-difference constraint derivative element (2,1): 3.99675
____________________________________________________________

Error using validateFirstDerivatives
Derivative Check failed:
User-supplied and forward finite-difference derivatives
do not match within 1e-006 relative tolerance.

Error in fmincon at 805
    validateFirstDerivatives(funfcn,confcn,X, ...

The constraint function does not match the calculated gradient, encouraging you to check the function for an error.

Replace the unitdiskb constraint function with unitdisk2 and run the minimization again:

[x fval exitflag output] = fmincon(@rosenboth,...
   [-1;2],[],[],[],[],[],[],@unitdisk2,options);

____________________________________________________________
   Derivative Check Information  

Objective function derivatives:
Maximum relative difference between user-supplied 
and finite-difference derivatives = 1.28553e-008.

Nonlinear inequality constraint derivatives:
Maximum relative difference between user-supplied 
and finite-difference derivatives = 1.46443e-008.

Derivative Check successfully passed.
____________________________________________________________


Local minimum found that satisfies the constraints...

Checking Derivatives with the Optimization Tool. To set up the example using correct derivative functions, but starting from [0 0], using the Optimization Tool:

Launch the Optimization Tool by entering optimtool at the command line.
Set the Problem Setup and Results pane to match the following figure:
Set the Options pane to match the following figure:
Press the Start button under Run solver and view results.
The output screen displays

The forward finite difference approximation is inaccurate enough near [0 0] that the derivative check fails.
To use the more accurate central differences, select central differences in the Approximated derivatives > Type pane:
Click Run solver and view results > Clear Results, then Start. This time the derivative check is successful:

The derivative check also succeeds when you select the initial point [-1 2], or most random points.

Passing Extra Parameters

Examining Results

Free Optimization Interactive Kit

Learn how to use optimization to solve systems of equations, fit models to data, or optimize system performance.

Get free kit

Trials Available

Try the latest version of optimization products.

Get trial software