Writing Objective Functions :: Optimization Overview (Optimization Toolbox™)

Optimization Toolbox™

Writing Objective Functions

On this page…
Writing Objective Functions Jacobians of Vector and Matrix Objective Functions Anonymous Function Objectives Maximizing an Objective Passing Extra Parameters

On this page…

Writing Objective Functions

Jacobians of Vector and Matrix Objective Functions

Anonymous Function Objectives

Maximizing an Objective

Passing Extra Parameters

Writing Objective Functions

This section relates to scalar-valued objective functions. For vector-valued or matrix-valued objective functions, see Jacobians of Vector and Matrix Objective Functions.

An objective function M-file can return one, two, or three outputs. It can return:

A single double-precision number, representing the value of f(x)
Both f(x) and its gradient ∇f(x)
All three of f(x), ∇f(x), and the Hessian matrix H(x)=∂²f/∂x_i∂x_j

You are not required to provide a gradient for some solvers, and you are never required to provide a Hessian, but providing one or both can lead to faster execution and more reliable answers. If you do not provide a gradient or Hessian, solvers may attempt to estimate them using finite difference approximations or other numerical schemes.

Some solvers do not use gradient or Hessian information. You should "conditionalize" an M-file so that it returns just what is needed:

f(x) alone
Both f(x) and ∇f(x)
All three of f(x), ∇f(x), and H(x)

For example, consider Rosenbrock's function

which is described and plotted in Example: Nonlinear Constrained Minimization. The gradient of f(x) is

and the Hessian H(x) is

Function rosenboth returns the value of Rosenbrock's function in f, the gradient in g, and the Hessian in H if required:

function [f g H] = rosenboth(x)
% Calculate objective f
f = 100*(x(2) - x(1)^2)^2 + (1-x(1))^2;

if nargout > 1 % gradient required
    g = [-400*(x(2)-x(1)^2)*x(1)-2*(1-x(1));
        200*(x(2)-x(1)^2)];
    
    if nargout > 2 % Hessian required
        H = [1200*x(1)^2-400*x(2)+2, -400*x(1);
            -400*x(1), 200];  
    end

end

nargout checks the number of arguments that a calling function specifies; see Checking the Number of Input Arguments in the MATLAB^® Programming Fundamentals documentation.

The solver fminunc, designed for unconstrained optimization, allows you to minimize Rosenbrock's function. Tell fminunc to use the gradient and Hessian by setting options:

options = optimset('GradObj','on','Hessian','on');

Run fminunc starting at [–1, 2]:

[x fval] = fminunc(@rosenboth, [-1; 2], options)
Optimization terminated: first-order optimality less than OPTIONS.TolFun,
 and no negative/zero curvature detected in trust region model.
x =
    1.0000
    1.0000
fval =
  1.9310e-017

Jacobians of Vector and Matrix Objective Functions

Some solvers, such as fsolve and lsqcurvefit, can have objective functions that are vectors or matrices. The only difference in usage between these types of objective functions and scalar objective functions is the way to write their derivatives. The first-order partial derivatives of a vector-valued or matrix-valued function is called a Jacobian; the first-order partial derivatives of a scalar function is called a gradient.

Jacobians of Vector Functions

If x represents a vector of independent variables, and F(x) is the vector function, the Jacobian J(x) is defined as

If F has m components, and x has k components, J is a m-by-k matrix.

For example, if

then J(x) is

Jacobians of Matrix Functions

The Jacobian of a matrix F(x) is defined by changing the matrix to a vector, column by column. For example, the matrix

is rewritten as a vector f:

The Jacobian of F is defined as the Jacobian of f,

If F is an m-by-n matrix, and x is a k-vector, the Jacobian is a mn-by-k matrix.

For example, if

the Jacobian of F is

Jacobians with Matrix-Valued Independent Variables

If x is a matrix, the Jacobian of F(x) is defined by changing the matrix x to a vector, column by column. For example, if

then the gradient is defined in terms of the vector

With

and f the vector form of F as above, the Jacobian of F(X) is defined to be the Jacobian of f(x):

So, for example,

If F is an m-by-n matrix, and x is a j-by-k matrix, the Jacobian is a mn-by-jk matrix.

Anonymous Function Objectives

Anonymous functions are useful for writing simple objective functions, without gradient or Hessian information. Rosenbrock's function is simple enough to write as an anonymous function:

anonrosen = @(x)(100*(x(2) - x(1)^2)^2 + (1 - x(1))^2);

Check that this evaluates correctly at (–1,2):

>> anonrosen([-1 2])
ans =
   104

Using anonrosen in fminunc yields the following results:

[x fval] = fminunc(anonrosen, [-1; 2])
Warning: Gradient must be provided for trust-region method;
  using line-search method instead.
> In fminunc at 265
Optimization terminated: relative infinity-norm of gradient less than options.TolFun.
x =
    1.0000
    1.0000
fval =
  1.2262e-010

Maximizing an Objective

All solvers are designed to minimize an objective function. If you have a maximization problem, that is, a problem of the form

then define g(x) = –f(x), and minimize g.

For example, to find the maximum of tan(cos(x)) near x = 5, evaluate:

[x fval] = fminunc(@(x)-tan(cos(x)),5)
Warning: Gradient must be provided for trust-region method;
  using line-search method instead.
> In fminunc at 265
Optimization terminated: relative infinity-norm of gradient less than options.TolFun.
x =
    6.2832
fval =
   -1.5574

The maximum is 1.5574 (the negative of the reported fval), and occurs at x = 6.2832. This is correct since, to 5 digits, the maximum is tan(1) = 1.5574, which occurs at x = 2π = 6.2832.

Passing Extra Parameters

Sometimes objective (or constraint) functions have parameters in addition to the independent variable. There are three methods of including these parameters:

Global variables are troublesome because they do not allow names to be reused among functions. It is better to use one of the other two methods.

For example, suppose you want to minimize the function

(2-1)

for different values of a, b, and c. Solvers accept objective functions that depend only on a single variable (x in this case). The following sections show how to provide the additional parameters a, b, and c. The solutions are for parameter values a = 4, b = 2.1, and c = 4 near x₀ = [0.5 0.5] using fminunc.

Anonymous Functions

To pass parameters using anonymous functions:

Write an M-file containing the following code:

function y = parameterfun(x,a,b,c)
y = (a - b*x(1)^2 + x(1)^4/3)*x(1)^2 + x(1)*x(2) + ...
    (-c + c*x(2)^2)*x(2)^2;

Assign values to the parameters and define a function handle f to an anonymous function by entering the following commands at the MATLAB prompt:
```
a = 4; b = 2.1; c = 4; % Assign parameter values
x0 = [0.5,0.5];
f = @(x)parameterfun(x,a,b,c)
```

Call the solver fminunc with the anonymous function:

[x,fval] = fminunc(f,x0)

The following output is displayed in the command window:

Warning: Gradient must be provided for trust-region method;
  using line-search method instead.
> In fminunc at 265
Optimization terminated: relative infinity-norm of gradient less than options.TolFun.
x =
   -0.0898    0.7127
fval =
   -1.0316

You can create anonymous functions of more than one argument. For example, to use lsqcurvefit, first create a function that takes two input arguments, x and xdata:

fh = @(x,xdata)(sin(x).*xdata +(x.^2).*cos(xdata));
x = pi; xdata = pi*[4;2;3];
fh(x, xdata)

ans =

    9.8696
    9.8696
   -9.8696

Now call lsqcurvefit:

% Assume ydata exists
x = lsqcurvefit(fh,x,xdata,ydata)

Nested Functions

To pass the parameters for Equation 2-1 via a nested function, write a single M-file that

Accepts a, b, c, and x0 as inputs
Contains the objective function as a nested function
Calls fminunc

Here is the code for the M-file for this example:

function [x,fval] =  runnested(a,b,c,x0) 
[x,fval] = fminunc(@nestedfun,x0);
% Nested function that computes the objective function     
    function y = nestedfun(x)
        y = (a - b*x(1)^2 + x(1)^4/3)*x(1)^2 + x(1)*x(2) +...
            (-c + c*x(2)^2)*x(2)^2;     
    end
end

Note that the objective function is computed in the nested function nestedfun, which has access to the variables a, b, and c.

To run the optimization, enter:

a = 4; b = 2.1; c = 4;% Assign parameter values
x0 = [0.5,0.5];
[x,fval] = runnested(a,b,c,x0)

The output is the same as in Anonymous Functions.

Global Variables

Global variables can be troublesome, it is better to avoid using them. To use global variables, declare the variables to be global in the workspace and in the functions that use the variables.

Write an M-file containing code for your function:

function y = globalfun(x)
global a  b  c
y = (a - b*x(1)^2 + x(1)^4/3)*x(1)^2 + x(1)*x(2) + ...
    (-c + c*x(2)^2)*x(2)^2;

In your MATLAB workspace, define the variables and run fminunc:

global a b c;
a = 4; b = 2.1; c = 4; % Assign parameter values
x0 = [0.5,0.5];
[x,fval] = fminunc(@globalfun,x0)

The output is the same as in Anonymous Functions.

Introduction to Optimization Toolbox™ Solvers

Writing Constraints