File Exchange

## Automatic Differentiation with Matlab Objects

version 1.4 (137 KB) by

Automatically compute derivatives of functions, without using finite-difference approximations.

4.16667
6 Ratings

Updated

Automatic differentiation is a technique for computing the derivatives of a function using the chain rule. Matlab objects make it easy to implement automatic differentiation. Note that this package is implemented in a rather old version of Matlab. You may need to edit it for newer versions.

An example of using automatic differentiation to compute the value and derivative of the Rosenbrock function at the point [1,2] is as follows:

x=adiff([1,2]); % create the automatic differentiation object at [1,2]
rosen = 100*(x(1)^2-x(2))^2+(x(1)-1)^2; % compute rosenbrock func.
[x,dx] = adiffget(x); % retrieve the value x and derivative dx

Then x = 100 and dx = [-400,200].

The adiff object includes a helper function to convert any optimization without derivatives into one with derivatives. For example, if you have a function f which you wish to optimize, but it doesn't compute derivatives, it is usually enough to call

fminunc('autodiff',x0,options,'f',...)

The zip file includes a pdf help file.

Ivar Eskerud Smith

Ryan Robinett

### Ryan Robinett (view profile)

William McIlhagga

### William McIlhagga (view profile)

An Zhou: I just noticed that in an earlier comment I said it only produces scalar output. That's wrong. (I can't edit that comment to correct it.)

William McIlhagga

### William McIlhagga (view profile)

To An Zhou:

thanks for using the toolbox. However, it *does* handle vector output; the example on page 3 of the documentation shows a case where a 10 element vector is transformed into a four element output, giving a 4 by 10 Jacobian.

Most of the examples are concentrated on scalar outputs, because those are the kinds of outputs you can optimize, which is the main reason you'd want derivatives (but of course not the only one).

As far as speed goes, it depends on what you do. In a recent project (logistic regression) I only noticed a 15% slowdown compared to hand-crafted code, but the end result was easier to understand and maintain.

An Zhou

### An Zhou (view profile)

It is pretty efficient in the sense that close to 4-5 times the runtime of the original function.

Yet I would really like to see it being able to handle vector output. Would it be very difficult?

cheng joylin

### cheng joylin (view profile)

William McIlhagga

### William McIlhagga (view profile)

To Jeffrey Steward:

The documentation says that adiff has restrictions on transpose. (See page 4 where it says that your example _won't_ work). However, you can get a quadratic form to work if you bracket things just right. Writing

f = x'*(A*x)

does work.

lgstarn

### lgstarn (view profile)

Doesn't seem to be able to handle a really simple function like this:

x'*A*x

where A is a matrix. Try this

A = eye(2);
f = x'*A*x;

and there is an error in mtimes.m

Error using *
Inner matrix dimensions must agree.

William McIlhagga

### William McIlhagga (view profile)

To Edward: For simplicity, the package was coded to work solely on functions that take a column vector and return a scalar argument. I can't see reshape helping much because of that restriction. After you've got the derivative though, you can extract it from the adiff object as an ordinary matlab vector (using adiffget) and reshape that.

Edward

### Edward (view profile)

How hard would it be to have the function reshape supported? Currently it throws errors. I think because numel(a) = 1, where a is an adiff object.

I am just beginning to learn about auto differentiation and am pretty excited about something that seems to be a very powerful tool. Thanks for helping me get started.

William McIlhagga

### William McIlhagga (view profile)

It's just plain forward auto.

Jichao Yin

Dan

### Dan (view profile)

The idea is great. It is impossible to compute jacobian with respect to numerous variables using standard Symbolic Toolbox.
However I found that the expression x'*A*x causes an error in Matlab 2010a. I was able to correct it by writing x'*(A*x) and correcting the 'adiffadiff' case in mtimes.m file (transposition should be added to first factors).
Is there a way to allow free usage of matrix expressions (i.e. free transposition usage and maybe allow to calculate derivatives with respect to a matrix)? I haven't found any matlab library that allows of doing it with respect to vector of variable length.