Accelerating the pace of engineering and science

# fminunc

Find minimum of unconstrained multivariable function

## Equation

Finds the minimum of a problem specified by

$\underset{x}{\mathrm{min}}f\left(x\right)$

where f(x) is a function that returns a scalar.

x is a vector or a matrix; see Matrix Arguments.

## Syntax

x = fminunc(fun,x0)
x = fminunc(fun,x0,options)
x = fminunc(problem)
[x,fval] = fminunc(...)
[x,fval,exitflag] = fminunc(...)
[x,fval,exitflag,output] = fminunc(...)

## Description

fminunc attempts to find a minimum of a scalar function of several variables, starting at an initial estimate. This is generally referred to as unconstrained nonlinear optimization.

 Note:   Passing Extra Parameters explains how to pass extra parameters to the objective function, if necessary.

x = fminunc(fun,x0) starts at the point x0 and attempts to find a local minimum x of the function described in fun. x0 can be a scalar, vector, or matrix.

x = fminunc(fun,x0,options) minimizes with the optimization options specified in options. Use optimoptions to set these options.

x = fminunc(problem) finds the minimum for problem, where problem is a structure described in Input Arguments.

Create the problem structure by exporting a problem from Optimization app, as described in Exporting Your Work.

[x,fval] = fminunc(...) returns in fval the value of the objective function fun at the solution x.

[x,fval,exitflag] = fminunc(...) returns a value exitflag that describes the exit condition.

[x,fval,exitflag,output] = fminunc(...) returns a structure output that contains information about the optimization.

[x,fval,exitflag,output,grad] = fminunc(...) returns in grad the value of the gradient of fun at the solution x.

[x,fval,exitflag,output,grad,hessian] = fminunc(...) returns in hessian the value of the Hessian of the objective function fun at the solution x. See Hessian.

## Input Arguments

Function Arguments contains general descriptions of arguments passed into fminunc. This section provides function-specific details for fun, options, and problem:

 fun The function to be minimized. fun is a function that accepts a vector x and returns a scalar f, the objective function evaluated at x. The function fun can be specified as a function handle for a file`x = fminunc(@myfun,x0)`where myfun is a MATLAB® function such as```function f = myfun(x) f = ... % Compute function value at x```fun can also be a function handle for an anonymous function.`x = fminunc(@(x)norm(x)^2,x0);`If the gradient of fun can also be computed and the GradObj option is 'on', as set by`options = optimoptions(@fminunc,'GradObj','on')`then the function fun must return, in the second output argument, the gradient value g, a vector, at x. The gradient is the partial derivatives ∂f/∂xi of f at the point x. That is, the ith component of g is the partial derivative of f with respect to the ith component of x.If the Hessian matrix can also be computed and the Hessian option is 'on', i.e., options = optimoptions(@fminunc,'GradObj','on','Hessian','on'), then the function fun must return the Hessian value H, a symmetric matrix, at x in a third output argument. The Hessian matrix is the second partial derivatives matrix of f at the point x. That is, the (i,j)th component of H is the second partial derivative of f with respect to xi and xj, ∂2f/∂xi∂xj. The Hessian is by definition a symmetric matrix.Writing Scalar Objective Functions explains how to "conditionalize" the gradients and Hessians for use in solvers that do not accept them. Passing Extra Parameters explains how to parameterize fun, if necessary. options Options provides the function-specific details for the options values. problem objective Objective function x0 Initial point for x solver 'fminunc' options Options created with optimoptions

## Output Arguments

Function Arguments contains general descriptions of arguments returned by fminunc. This section provides function-specific details for exitflag and output:

 exitflag Integer identifying the reason the algorithm terminated. The following lists the values of exitflag and the corresponding reasons the algorithm terminated. 1 Magnitude of gradient smaller than the TolFun tolerance. 2 Change in x was smaller than the TolX tolerance. 3 Change in the objective function value was less than the TolFun tolerance. 5 Predicted decrease in the objective function was less than the TolFun tolerance. 0 Number of iterations exceeded MaxIter or number of function evaluations exceeded MaxFunEvals. -1 Algorithm was terminated by the output function. -3 Objective function at current iteration went below ObjectiveLimit. grad Gradient at x hessian Hessian at x output Structure containing information about the optimization. The fields of the structure are iterations Number of iterations taken funcCount Number of function evaluations firstorderopt Measure of first-order optimality algorithm Optimization algorithm used cgiterations Total number of PCG iterations ('trust-region' algorithm only) stepsize Final displacement in x ('quasi-newton' algorithm only) message Exit message

### Hessian

fminunc computes the output argument hessian as follows:

• When using the 'quasi-newton' algorithm, the function computes a finite-difference approximation to the Hessian at x using

• The objective function fun if you do not supply the gradient

• When using the 'trust-region' algorithm, the function uses

• options.Hessian, if you supply it, to compute the Hessian at x

• A finite-difference approximation to the Hessian at x, if you supply only the gradient

## Options

fminunc uses these optimization options. Some options apply to all algorithms, some are only relevant when you are using the trust-region algorithm, and others are only relevant when you are using the quasi-newton algorithm. Use optimoptions to set or change options. See Optimization Options Reference for detailed information.

### All Algorithms

All fminunc algorithms use the following options:

### trust-region Algorithm Only

The trust-region algorithm uses the following options:

Hessian

If 'on', fminunc uses a user-defined Hessian for the objective function. The Hessian is either defined in the objective function (see fun), or is indirectly defined when using HessMult.

If 'off' (default), fminunc approximates the Hessian using finite differences.

HessMult

Function handle for Hessian multiply function. For large-scale structured problems, this function computes the Hessian matrix product H*Y without actually forming H. The function is of the form

```W = hmfun(Hinfo,Y)
```

where Hinfo contains the matrix used to compute H*Y.

The first argument must be the same as the third argument returned by the objective function fun, for example by

```[f,g,Hinfo] = fun(x)
```

Y is a matrix that has the same number of rows as there are dimensions in the problem. W = H*Y although H is not formed explicitly. fminunc uses Hinfo to compute the preconditioner. See Passing Extra Parameters for information on how to supply values for any additional parameters hmfun needs.

 Note   'Hessian' must be set to 'on' for fminunc to pass Hinfo from fun to hmfun.

See Minimization with Dense Structured Hessian, Linear Equalities for an example.

HessPattern

Sparsity pattern of the Hessian for finite differencing. Set HessPattern(i,j) = 1 when you can have ∂2fun/∂x(i)x(j) ≠ 0. Otherwise, set HessPattern(i,j) = 0.

Use HessPattern when it is inconvenient to compute the Hessian matrix H in fun, but you can determine (say, by inspection) when the ith component of the gradient of fun depends on x(j). fminunc can approximate H via sparse finite differences (of the gradient) if you provide the sparsity structure of H — i.e., locations of the nonzeros — as the value for HessPattern.

In the worst case, when the structure is unknown, do not set HessPattern. The default behavior is as if HessPattern is a dense matrix of ones. Then fminunc computes a full finite-difference approximation in each iteration. This can be very expensive for large problems, so it is usually better to determine the sparsity structure.

MaxPCGIter

Maximum number of PCG (preconditioned conjugate gradient) iterations, a positive scalar. The default is max(1,floor(numberOfVariables/2)). For more information, see Algorithms.

PrecondBandWidth

Upper bandwidth of preconditioner for PCG, a nonnegative integer. By default, fminunc uses diagonal preconditioning (upper bandwidth of 0). For some problems, increasing the bandwidth reduces the number of PCG iterations. Setting PrecondBandWidth to Inf uses a direct factorization (Cholesky) rather than the conjugate gradients (CG). The direct factorization is computationally more expensive than CG, but produces a better quality step towards the solution.

TolPCG

Termination tolerance on the PCG iteration, a positive scalar. The default is 0.1.

### quasi-newton Algorithm Only

The quasi-newton algorithm uses the following options:

 HessUpdate Method for choosing the search direction in the Quasi-Newton algorithm. The choices are:'bfgs', the default'dfp''steepdesc'See Hessian Update for a description of these methods. InitialHessMatrixThis option will be removed in a future release. Initial quasi-Newton matrix. This option is only available if you set InitialHessType to 'user-supplied'. In that case, you can set InitialHessMatrix to one of the following:A positive scalar — The initial matrix is the scalar times the identity.A vector of positive values— The initial matrix is a diagonal matrix with the entries of the vector on the diagonal. This vector should be the same size as the x0 vector, the initial point. InitialHessTypeThis option will be removed in a future release. Initial quasi-Newton matrix type. The options are:'identity''scaled-identity', the default'user-supplied' — See InitialHessMatrix ObjectiveLimit A tolerance (stopping criterion) that is a scalar. If the objective function value at an iteration is less than or equal to ObjectiveLimit, the iterations halt, since the problem is presumably unbounded. The default value is -1e20.

## Examples

Minimize the function $f\left(x\right)=3{x}_{1}^{2}+2{x}_{1}{x}_{2}+{x}_{2}^{2}$.

Create a file myfun.m:

```function f = myfun(x)
f = 3*x(1)^2 + 2*x(1)*x(2) + x(2)^2;    % Cost function```

Then call fminunc to find a minimum of myfun near [1,1]:

```x0 = [1,1];
[x,fval] = fminunc(@myfun,x0);```

After a few iterations, fminunc returns the solution, x, and the value of the function at x, fval:

```x,fval

x =
1.0e-006 *
0.2541   -0.2029

fval =
1.3173e-013```

To minimize this function with the gradient provided, modify myfun.m so the gradient is the second output argument:

```function [f,g] = myfun(x)
f = 3*x(1)^2 + 2*x(1)*x(2) + x(2)^2;    % Cost function
if nargout > 1
g(1) = 6*x(1)+2*x(2);
g(2) = 2*x(1)+2*x(2);
end```

Indicate that the gradient value is available by creating optimization options with the GradObj option set to 'on' using optimoptions. Choose the 'trust-region' algorithm, which requires a gradient.

```options = optimoptions('fminunc','GradObj','on','Algorithm','trust-region');
x0 = [1,1];
[x,fval] = fminunc(@myfun,x0,options);```

After several iterations fminunc returns the solution, x, and the value of the function at x, fval:

```x,fval

x =
1.0e-015 *
0.1110   -0.8882

fval =
6.2862e-031```

To minimize the function f(x) = sin(x) + 3 using an anonymous function

```f = @(x)sin(x)+3;
x = fminunc(f,4);```

fminunc returns a solution

```x

x =
4.7124```

## Notes

fminunc is not the preferred choice for solving problems that are sums of squares, that is, of the form

$\underset{x}{\mathrm{min}}{‖f\left(x\right)‖}_{2}^{2}=\underset{x}{\mathrm{min}}\left({f}_{1}{\left(x\right)}^{2}+{f}_{2}{\left(x\right)}^{2}+...+{f}_{n}{\left(x\right)}^{2}\right)$

Instead use the lsqnonlin function, which has been optimized for problems of this form.

To use the trust-region method, you must provide the gradient in fun (and set the GradObj option to 'on' using optimoptions). A warning is given if no gradient is provided and the Algorithm option is 'trust-region'.

## Limitations

The function to be minimized must be continuous. fminunc might only give local solutions.

fminunc only minimizes over the real numbers, that is, x must only consist of real numbers and f(x) must only return real numbers. When x has complex variables, they must be split into real and imaginary parts.

### Trust Region Optimization

To use the trust-region algorithm, you must supply the gradient in fun (and GradObj must be set 'on' in options).

Trust Region Algorithm Coverage and Requirements

Must provide gradient for f(x) in fun.

• Provide sparsity structure of the Hessian, or compute the Hessian in fun.

• The Hessian should be sparse.

expand all

### Trust Region Algorithm

The trust-region algorithm requires that you supply the gradient in fun and set GradObj to 'on' using optimoptions. This algorithm is a subspace trust-region method and is based on the interior-reflective Newton method described in [2] and [3]. Each iteration involves the approximate solution of a large linear system using the method of preconditioned conjugate gradients (PCG). See fminunc trust-region Algorithm, Trust-Region Methods for Nonlinear Minimization and Preconditioned Conjugate Gradient Methodpt.

### Quasi-Newton Algorithm

The quasi-newton algorithm uses the BFGS Quasi-Newton method with a cubic line search procedure. This quasi-Newton method uses the BFGS ([1],[5],[8], and [9]) formula for updating the approximation of the Hessian matrix. You can select the DFP ([4],[6], and [7]) formula, which approximates the inverse Hessian matrix, by setting the HessUpdate option to 'dfp' (and the Algorithm option to 'quasi-newton'). You can select a steepest descent method by setting HessUpdate to 'steepdesc' (and Algorithm to 'quasi-newton'), although this is not recommended.

## References

[1] Broyden, C.G., "The Convergence of a Class of Double-Rank Minimization Algorithms," Journal Inst. Math. Applic., Vol. 6, pp. 76-90, 1970.

[2] Coleman, T.F. and Y. Li, "An Interior, Trust Region Approach for Nonlinear Minimization Subject to Bounds," SIAM Journal on Optimization, Vol. 6, pp. 418-445, 1996.

[3] Coleman, T.F. and Y. Li, "On the Convergence of Reflective Newton Methods for Large-Scale Nonlinear Minimization Subject to Bounds," Mathematical Programming, Vol. 67, Number 2, pp. 189-224, 1994.

[4] Davidon, W.C., "Variable Metric Method for Minimization," A.E.C. Research and Development Report, ANL-5990, 1959.

[5] Fletcher, R., "A New Approach to Variable Metric Algorithms," Computer Journal, Vol. 13, pp. 317-322, 1970.

[6] Fletcher, R., "Practical Methods of Optimization," Vol. 1, Unconstrained Optimization, John Wiley and Sons, 1980.

[7] Fletcher, R. and M.J.D. Powell, "A Rapidly Convergent Descent Method for Minimization," Computer Journal, Vol. 6, pp. 163-168, 1963.

[8] Goldfarb, D., "A Family of Variable Metric Updates Derived by Variational Means," Mathematics of Computing, Vol. 24, pp. 23-26, 1970.

[9] Shanno, D.F., "Conditioning of Quasi-Newton Methods for Function Minimization," Mathematics of Computing, Vol. 24, pp. 647-656, 1970.