Learn more about Optimization Toolbox

Writing Objective Functions

On this page…
Writing Objective Functions Jacobians of Vector and Matrix Objective Functions Anonymous Function Objectives Maximizing an Objective

Writing Objective Functions

This section relates to scalar-valued objective functions. For vector-valued or matrix-valued objective functions, see Jacobians of Vector and Matrix Objective Functions. For information on how to include extra parameters, see Passing Extra Parameters.

An objective function M-file can return one, two, or three outputs. It can return:

• A single double-precision number, representing the value of f(x)

• Both f(x) and its gradient ∇f(x)

• All three of f(x), ∇f(x), and the Hessian matrix H(x)=∂²f/∂x_i∂x_j

You are not required to provide a gradient for some solvers, and you are never required to provide a Hessian, but providing one or both can lead to faster execution and more reliable answers. If you do not provide a gradient or Hessian, solvers may attempt to estimate them using finite difference approximations or other numerical schemes.

Some solvers do not use gradient or Hessian information. You should "conditionalize" an M-file so that it returns just what is needed:

• f(x) alone

• Both f(x) and ∇f(x)

• All three of f(x), ∇f(x), and H(x)

For example, consider Rosenbrock's function

which is described and plotted in Example: Nonlinear Constrained Minimization. The gradient of f(x) is

and the Hessian H(x) is

Function rosenboth returns the value of Rosenbrock's function in f, the gradient in g, and the Hessian in H if required:

function [f g H] = rosenboth(x)
% Calculate objective f
f = 100*(x(2) - x(1)^2)^2 + (1-x(1))^2;

if nargout > 1 % gradient required
    g = [-400*(x(2)-x(1)^2)*x(1)-2*(1-x(1));
        200*(x(2)-x(1)^2)];
    
    if nargout > 2 % Hessian required
        H = [1200*x(1)^2-400*x(2)+2, -400*x(1);
            -400*x(1), 200];  
    end

end

nargout checks the number of arguments that a calling function specifies; see Checking the Number of Input Arguments in the MATLAB Programming Fundamentals documentation.

The solver fminunc, designed for unconstrained optimization, allows you to minimize Rosenbrock's function. Tell fminunc to use the gradient and Hessian by setting options:

options = optimset('GradObj','on','Hessian','on');

Run fminunc starting at [–1, 2]:

[x fval] = fminunc(@rosenboth,[-1; 2],options)
Local minimum found.

Optimization completed because the size of the gradient
is less than the default value of the function tolerance.

x =
    1.0000
    1.0000

fval =
  1.9310e-017

If you have a license for Symbolic Math Toolbox™ software, you can calculate gradients and Hessians automatically, as described in Example: Using Symbolic Math Toolbox Functions to Calculate Gradients and Hessians.

Back to Top

Jacobians of Vector and Matrix Objective Functions

Some solvers, such as fsolve and lsqcurvefit, can have objective functions that are vectors or matrices. The only difference in usage between these types of objective functions and scalar objective functions is the way to write their derivatives. The first-order partial derivatives of a vector-valued or matrix-valued function is called a Jacobian; the first-order partial derivatives of a scalar function is called a gradient.

Jacobians of Vector Functions

If x represents a vector of independent variables, and F(x) is the vector function, the Jacobian J(x) is defined as

If F has m components, and x has k components, J is a m-by-k matrix.

For example, if

then J(x) is

Jacobians of Matrix Functions

The Jacobian of a matrix F(x) is defined by changing the matrix to a vector, column by column. For example, the matrix

is rewritten as a vector f:

The Jacobian of F is defined as the Jacobian of f,

If F is an m-by-n matrix, and x is a k-vector, the Jacobian is a mn-by-k matrix.

For example, if

the Jacobian of F is

Jacobians with Matrix-Valued Independent Variables

If x is a matrix, the Jacobian of F(x) is defined by changing the matrix x to a vector, column by column. For example, if

then the gradient is defined in terms of the vector

With

and f the vector form of F as above, the Jacobian of F(X) is defined to be the Jacobian of f(x):

So, for example,

If F is an m-by-n matrix, and x is a j-by-k matrix, the Jacobian is a mn-by-jk matrix.

Back to Top

Anonymous Function Objectives

Anonymous functions are useful for writing simple objective functions. Rosenbrock's function is simple enough to write as an anonymous function:

anonrosen = @(x)(100*(x(2) - x(1)^2)^2 + (1-x(1))^2);

Check that this evaluates correctly at (–1,2):

>> anonrosen([-1 2])
ans =
   104

Using anonrosen in fminunc yields the following results:

options = optimset('LargeScale','off');
[x fval] = fminunc(anonrosen,[-1;2],options)

Local minimum found.

Optimization completed because the size of the gradient
is less than the default value of the function tolerance.

x =
    1.0000
    1.0000

fval =
  1.2262e-010

Back to Top

Maximizing an Objective

All solvers are designed to minimize an objective function. If you have a maximization problem, that is, a problem of the form

then define g(x) = –f(x), and minimize g.

For example, to find the maximum of tan(cos(x)) near x = 5, evaluate:

[x fval] = fminunc(@(x)-tan(cos(x)),5)
Warning: Gradient must be provided for trust-region method;
  using line-search method instead. 
> In fminunc at 356

Local minimum found.

Optimization completed because the size of the gradient is less than
the default value of the function tolerance.

x =
    6.2832

fval =
   -1.5574

The maximum is 1.5574 (the negative of the reported fval), and occurs at x = 6.2832. This is correct since, to 5 digits, the maximum is tan(1) = 1.5574, which occurs at x = 2π = 6.2832.

Back to Top

Choosing a Solver

Writing Constraints

© 1984-2009- The MathWorks, Inc.    -   Site Help   -   Patents   -   Trademarks   -   Privacy Policy   -   Preventing Piracy   -   RSS