Code covered by the BSD License

Highlights from Automatic Differentiation with Matlab Objects

4.0
4.0 | 5 ratings Rate this file 20 Downloads (last 30 days) File Size: 137 KB File ID: #26807 Version: 1.4

Automatic Differentiation with Matlab Objects

William McIlhagga (view profile)

01 Mar 2010 (Updated )

Automatically compute derivatives of functions, without using finite-difference approximations.

File Information
Description

Automatic differentiation is a technique for computing the derivatives of a function using the chain rule. Matlab objects make it easy to implement automatic differentiation. Note that this package is implemented in a rather old version of Matlab. You may need to edit it for newer versions.

An example of using automatic differentiation to compute the value and derivative of the Rosenbrock function at the point [1,2] is as follows:

x=adiff([1,2]); % create the automatic differentiation object at [1,2]
rosen = 100*(x(1)^2-x(2))^2+(x(1)-1)^2; % compute rosenbrock func.
[x,dx] = adiffget(x); % retrieve the value x and derivative dx

Then x = 100 and dx = [-400,200].

The adiff object includes a helper function to convert any optimization without derivatives into one with derivatives. For example, if you have a function f which you wish to optimize, but it doesn't compute derivatives, it is usually enough to call

fminunc('autodiff',x0,options,'f',...)

The zip file includes a pdf help file.

MATLAB release MATLAB 5.3.1 (R11.1)
16 Jun 2016 Ryan Robinett

Ryan Robinett (view profile)

31 May 2016 William McIlhagga

William McIlhagga (view profile)

An Zhou: I just noticed that in an earlier comment I said it only produces scalar output. That's wrong. (I can't edit that comment to correct it.)

Comment only
31 May 2016 William McIlhagga

William McIlhagga (view profile)

To An Zhou:

thanks for using the toolbox. However, it *does* handle vector output; the example on page 3 of the documentation shows a case where a 10 element vector is transformed into a four element output, giving a 4 by 10 Jacobian.

Most of the examples are concentrated on scalar outputs, because those are the kinds of outputs you can optimize, which is the main reason you'd want derivatives (but of course not the only one).

As far as speed goes, it depends on what you do. In a recent project (logistic regression) I only noticed a 15% slowdown compared to hand-crafted code, but the end result was easier to understand and maintain.

Comment only
30 May 2016 An Zhou

An Zhou (view profile)

It is pretty efficient in the sense that close to 4-5 times the runtime of the original function.

Yet I would really like to see it being able to handle vector output. Would it be very difficult?

31 Aug 2015 cheng joylin

cheng joylin (view profile)

22 Aug 2014 William McIlhagga

William McIlhagga (view profile)

To Jeffrey Steward:

The documentation says that adiff has restrictions on transpose. (See page 4 where it says that your example _won't_ work). However, you can get a quadratic form to work if you bracket things just right. Writing

f = x'*(A*x)

does work.

Comment only
21 Aug 2014 lgstarn

lgstarn (view profile)

Doesn't seem to be able to handle a really simple function like this:

x'*A*x

where A is a matrix. Try this

A = eye(2);
f = x'*A*x;

and there is an error in mtimes.m

Error using *
Inner matrix dimensions must agree.

Comment only
24 May 2013 William McIlhagga

William McIlhagga (view profile)

To Edward: For simplicity, the package was coded to work solely on functions that take a column vector and return a scalar argument. I can't see reshape helping much because of that restriction. After you've got the derivative though, you can extract it from the adiff object as an ordinary matlab vector (using adiffget) and reshape that.

Comment only
22 May 2013 Edward

Edward (view profile)

How hard would it be to have the function reshape supported? Currently it throws errors. I think because numel(a) = 1, where a is an adiff object.

I am just beginning to learn about auto differentiation and am pretty excited about something that seems to be a very powerful tool. Thanks for helping me get started.

Comment only
29 Mar 2011 William McIlhagga

William McIlhagga (view profile)

It's just plain forward auto.

Comment only
29 Mar 2011 Jichao Yin

21 Nov 2010 Dan

Dan (view profile)

The idea is great. It is impossible to compute jacobian with respect to numerous variables using standard Symbolic Toolbox.
However I found that the expression x'*A*x causes an error in Matlab 2010a. I was able to correct it by writing x'*(A*x) and correcting the 'adiffadiff' case in mtimes.m file (transposition should be added to first factors).
Is there a way to allow free usage of matrix expressions (i.e. free transposition usage and maybe allow to calculate derivatives with respect to a matrix)? I haven't found any matlab library that allows of doing it with respect to vector of variable length.