fminunc (Optimization Toolbox)

fminunc finds a minimum of a scalar function of several variables, starting at an initial estimate. This is generally referred to as unconstrained nonlinear optimization.

x = fminunc(fun,x0)

starts at the point x0 and finds a local minimum x of the function described in fun. x0 can be a scalar, vector, or matrix.

x = fminunc(fun,x0,options)

minimizes with the optimization parameters specified in the structure options. Use optimset to set these parameters.

x = fminunc(fun,x0,options,P1,P2,...)

passes the problem-dependent parameters P1, P2, etc. directly to the function fun. Pass an empty matrix for options to use the default values for options.

[x,fval] = fminunc(...)

returns in fval the value of the objective function fun at the solution x.

[x,fval,exitflag] = fminunc(...)

returns a value exitflag that describes the exit condition.

[x,fval,exitflag,output] = fminunc(...)

returns a structure output that contains information about the optimization.

[x,fval,exitflag,output,grad] = fminunc(...)

returns in grad the value of the gradient of fun at the solution x.

[x,fval,exitflag,output,grad,hessian] = fminunc(...)

returns in hessian the value of the Hessian of the objective function fun at the solution x.

Function Arguments contains general descriptions of arguments passed in to fminunc. This section provides function-specific details for fun and options:


`fun`	The function to be minimized. `fun` is a function that accepts a vector `x` and returns a scalar `f`, the objective function evaluated at `x`. The function `fun` can be specified as a function handle. x = fminunc(@myfun,x0) where `myfun` is a MATLAB function such as function f = myfun(x) f = ... % Compute function value at x `fun` can also be an inline object. x = fminunc(inline('norm(x)^2'),x0); If the gradient of `fun` can also be computed and the `GradObj` parameter is `'on'`, as set by options = optimset('GradObj','on') then the function `fun` must return, in the second output argument, the gradient value `g`, a vector, at `x`. Note that by checking the value of `nargout` the function can avoid computing `g` when `fun` is called with only one output argument (in the case where the optimization algorithm only needs the value of `f` but not `g`). function [f,g] = myfun(x) f = ... % Compute the function value at x if nargout > 1 % fun called with 2 output arguments g = ... % Compute the gradient evaluated at x end
	The gradient is the partial derivatives of `f` at the point `x`. That is, the `i`th component of `g` is the partial derivative of `f` with respect to the `i`th component of `x`. If the Hessian matrix can also be computed and the `Hessian` parameter is `'on'`, i.e., `options = optimset('Hessian','on')`, then the function `fun` must return the Hessian value `H`, a symmetric matrix, at `x` in a third output argument. Note that by checking the value of `nargout` you can avoid computing `H` when `fun` is called with only one or two output arguments (in the case where the optimization algorithm only needs the values of `f` and `g` but not `H`). function [f,g,H] = myfun(x) f = ... % Compute the objective function value at x if nargout > 1 % fun called with two output arguments g = ... % Gradient of the function evaluated at x if nargout > 2 H = ... % Hessian evaluated at x end end The Hessian matrix is the second partial derivatives matrix of `f` at the point `x`. That is, the (`i`,`j`)th component of `H` is the second partial derivative of `f` with respect to `x`_i and `x`_j, . The Hessian is by definition a symmetric matrix.
`options`	Options provides the function-specific details for the `options` parameters.

Function Arguments contains general descriptions of arguments returned by fminunc. This section provides function-specific details for exitflag and output:


`exitflag`	Describes the exit condition:
	`> 0`	The function converged to a solution `x`.
	`0`	The maximum number of function evaluations or iterations was exceeded.
	`< 0`	The function did not converge to a solution.
`output`	Structure containing information about the optimization. The fields of the structure are
	`iterations`	Number of iterations taken
	`funcCount`	Number of function evaluations
	`algorithm`	Algorithm used
	`cgiterations`	Number of PCG iterations (large-scale algorithm only)
	`stepsize`	Final step size taken (medium-scale algorithm only)
	`firstorderopt`	Measure of first-order optimality: the norm of the gradient at the solution `x`

fminunc uses these optimization parameters. Some parameters apply to all algorithms, some are only relevant when you are using the large-scale algorithm, and others are only relevant when you are using the medium-scale algorithm.You can use optimset to set or change the values of these fields in the parameters structure options. See Optimization Parameters, for detailed information.

The LargeScale option specifies a preference for which algorithm to use. It is only a preference, because certain conditions must be met to use the large-scale algorithm. For fminunc, you must provide the gradient (see the preceing description of fun to see how) or else use the medium-scale algorithm.:


`LargeScale`	Use large-scale algorithm if possible when set to `'on'`. Use medium-scale algorithm when set to `'off'`.

Large-Scale and Medium-Scale Algorithms. These parameters are used by both the large-scale and medium-scale algorithms:


`DerivativeCheck`	Compare user-supplied derivatives (gradient) to finite-differencing derivatives.
`Diagnostics`	Display diagnostic information about the function to be minimized.
`Display`	Level of display. `'off'` displays no output; `'iter'` displays output at each iteration; `'final'` (default) displays just the final output.
`GradObj`	Gradient for the objective function that you define. See the preceding description of `fun` to see how to define the gradient in `fun`. You must provide the gradient to use the large-scale method. It is optional for the medium-scale method.
`MaxFunEvals`	Maximum number of function evaluations allowed.
`MaxIter`	Maximum number of iterations allowed.
`OutputFcn`	Specify a user-defined function that an opimization function calls at each iteration. See Output Function.
`TolFun`	Termination tolerance on the function value.
`TolX`	Termination tolerance on `x`.

Large-Scale Algorithm Only. These parameters are used only by the large-scale algorithm:

Hessian If 'on', fminunc uses a user-defined Hessian (defined in fun), or Hessian information (when using HessMult), for the objective function. If 'off', fminunc approximates the Hessian using finite differences.

HessMult

Function handle for Hessian multiply function. For large-scale structured problems, this function computes the Hessian matrix product H*Y without actually forming H. The function is of the form

```
W = hmfun(Hinfo,Y,p1,p2,...)
```

where Hinfo and the additional parameters p1,p2,... contain the matrices used to compute H*Y.
The first argument must be the same as the third argument returned by the objective function fun.

```
[f,g,Hinfo] = fun(x,p1,p2,...)
```

The parameters p1,p2,... are the same additional parameters that are passed to fminunc (and to fun).

```
fminunc(fun,...,options,p1,p2,...)
```

Y is a matrix that has the same number of rows as there are dimensions in the problem. W = H*Y although H is not formed explicitly. fminunc uses Hinfo to compute the preconditioner.

Note 'Hessian' must be set to 'on' for Hinfo to be passed from fun to hmfun.

See Nonlinear Minimization with a Dense but Structured Hessian and Equality Constraints for an example.

HessPattern Sparsity pattern of the Hessian for finite differencing. If it is not convenient to compute the sparse Hessian matrix H in fun, the large-scale method in fminunc can approximate H via sparse finite differences (of the gradient) provided the sparsity structure of H -- i.e., locations of the nonzeros -- is supplied as the value for HessPattern. In the worst case, if the structure is unknown, you can set HessPattern to be a dense matrix and a full finite-difference approximation is computed at each iteration (this is the default). This can be very expensive for large problems, so it is usually worth the effort to determine the sparsity structure.

MaxPCGIter Maximum number of PCG (preconditioned conjugate gradient) iterations (see Algorithms).

PrecondBandWidth Upper bandwidth of preconditioner for PCG. By default, diagonal preconditioning is used (upper bandwidth of 0). For some problems, increasing the bandwidth reduces the number of PCG iterations.

TolPCG Termination tolerance on the PCG iteration.

TypicalX Typical x values.

Medium-Scale Algorithm Only. These parameters are used only by the medium-scale algorithm:


`DiffMaxChange`	Maximum change in variables for finite-difference gradients.
`DiffMinChange`	Minimum change in variables for finite-difference gradients.
`LineSearchType`	Line search algorithm choice.

After a couple of iterations, the solution, x, and the value of the function at x, fval, are returned.

To minimize this function with the gradient provided, modify the M-file myfun.m so the gradient is the second output argument

and indicate that the gradient value is available by creating an optimization options structure with the GradObj parameter set to 'on' using optimset.

After several iterations the solution, x, and fval, the value of the function at x, are returned.

fminunc is not the preferred choice for solving problems that are sums of squares, that is, of the form

Instead use the lsqnonlin function, which has been optimized for problems of this form.

To use the large-scale method, you must provide the gradient in fun (and set the GradObj parameter to 'on' using optimset). A warning is given if no gradient is provided and the LargeScale parameter is not 'off'.

Large-Scale Optimization. By default fminunc chooses the large-scale algorithm if the user supplies the gradient in fun (and the GradObj parameter is set to 'on' using optimset). This algorithm is a subspace trust region method and is based on the interior-reflective Newton method described in [2],[3]. Each iteration involves the approximate solution of a large linear system using the method of preconditioned conjugate gradients (PCG). See Trust-Region Methods for Nonlinear Minimization and Preconditioned Conjugate Gradients.

Medium-Scale Optimization. fminunc, with the LargeScale parameter set to 'off' with optimset, uses the BFGS Quasi-Newton method with a mixed quadratic and cubic line search procedure. This quasi-Newton method uses the BFGS ([1],[5],[8],[9]) formula for updating the approximation of the Hessian matrix. You can select the DFP ([4],[6],[7]) formula, which approximates the inverse Hessian matrix, by setting the HessUpdate parameter to 'dfp' (and the LargeScale parameter to 'off'). You can select a steepest descent method by setting HessUpdate to 'steepdesc' (and LargeScale to 'off'), although this is not recommended.

The default line search algorithm, i.e., when the LineSearchType parameter is set to 'quadcubic', is a safeguarded mixed quadratic and cubic polynomial interpolation and extrapolation method. You can select a safeguarded cubic polynomial method by setting the LineSearchType parameter to 'cubicpoly'. This second method generally requires fewer function evaluations but more gradient evaluations. Thus, if gradients are being supplied and can be calculated inexpensively, the cubic polynomial line search method is preferable. A full description of the algorithms is given in the Standard Algorithms chapter.

The function to be minimized must be continuous. fminunc might only give local solutions.

fminunc only minimizes over the real numbers, that is, x must only consist of real numbers and f(x) must only return real numbers. When x has complex variables, they must be split into real and imaginary parts.

Large-Scale Optimization. To use the large-scale algorithm, the user must supply the gradient in fun (and GradObj must be set 'on' in options). See Table 2-4, Large-Scale Problem Coverage and Requirements, for more information on what problem formulations are covered and what information must be provided.

[1] Broyden, C.G., "The Convergence of a Class of Double-Rank Minimization Algorithms," Journal Inst. Math. Applic., Vol. 6, pp. 76-90, 1970.

[2] Coleman, T.F. and Y. Li, "An Interior, Trust Region Approach for Nonlinear Minimization Subject to Bounds," SIAM Journal on Optimization, Vol. 6, pp. 418-445, 1996.

[3] Coleman, T.F. and Y. Li, "On the Convergence of Reflective Newton Methods for Large-Scale Nonlinear Minimization Subject to Bounds," Mathematical Programming, Vol. 67, Number 2, pp. 189-224, 1994.

[4] Davidon, W.C., "Variable Metric Method for Minimization," A.E.C. Research and Development Report, ANL-5990, 1959.

[5] Fletcher, R.,"A New Approach to Variable Metric Algorithms," Computer Journal, Vol. 13, pp. 317-322, 1970.

[6] Fletcher, R., "Practical Methods of Optimization," Vol. 1, Unconstrained Optimization, John Wiley and Sons, 1980.

[7] Fletcher, R. and M.J.D. Powell, "A Rapidly Convergent Descent Method for Minimization," Computer Journal, Vol. 6, pp. 163-168, 1963.

[8] Goldfarb, D., "A Family of Variable Metric Updates Derived by Variational Means," Mathematics of Computing, Vol. 24, pp. 23-26, 1970.

[9] Shanno, D.F., "Conditioning of Quasi-Newton Methods for Function Minimization," Mathematics of Computing, Vol. 24, pp. 647-656, 1970.

Learn more about the latest releases of MathWorks products:

Select an option