Automatic differentiation is a technique for computing the derivatives of a function using the chain rule. Matlab objects make it easy to implement automatic differentiation. Note that this package is implemented in a rather old version of Matlab. You may need to edit it for newer versions.
An example of using automatic differentiation to compute the value and derivative of the Rosenbrock function at the point [1,2] is as follows:
x=adiff([1,2]); % create the automatic differentiation object at [1,2]
rosen = 100*(x(1)^2-x(2))^2+(x(1)-1)^2; % compute rosenbrock func.
[x,dx] = adiffget(x); % retrieve the value x and derivative dx
Then x = 100 and dx = [-400,200].
The adiff object includes a helper function to convert any optimization without derivatives into one with derivatives. For example, if you have a function f which you wish to optimize, but it doesn't compute derivatives, it is usually enough to call
The zip file includes a pdf help file.
An Zhou: I just noticed that in an earlier comment I said it only produces scalar output. That's wrong. (I can't edit that comment to correct it.)
To An Zhou:
thanks for using the toolbox. However, it *does* handle vector output; the example on page 3 of the documentation shows a case where a 10 element vector is transformed into a four element output, giving a 4 by 10 Jacobian.
Most of the examples are concentrated on scalar outputs, because those are the kinds of outputs you can optimize, which is the main reason you'd want derivatives (but of course not the only one).
As far as speed goes, it depends on what you do. In a recent project (logistic regression) I only noticed a 15% slowdown compared to hand-crafted code, but the end result was easier to understand and maintain.
It is pretty efficient in the sense that close to 4-5 times the runtime of the original function.
Yet I would really like to see it being able to handle vector output. Would it be very difficult?
To Jeffrey Steward:
The documentation says that adiff has restrictions on transpose. (See page 4 where it says that your example _won't_ work). However, you can get a quadratic form to work if you bracket things just right. Writing
f = x'*(A*x)
Doesn't seem to be able to handle a really simple function like this:
where A is a matrix. Try this
x = adiff([1,2]);
A = eye(2);
f = x'*A*x;
and there is an error in mtimes.m
Error using *
Inner matrix dimensions must agree.
To Edward: For simplicity, the package was coded to work solely on functions that take a column vector and return a scalar argument. I can't see reshape helping much because of that restriction. After you've got the derivative though, you can extract it from the adiff object as an ordinary matlab vector (using adiffget) and reshape that.
How hard would it be to have the function reshape supported? Currently it throws errors. I think because numel(a) = 1, where a is an adiff object.
I am just beginning to learn about auto differentiation and am pretty excited about something that seems to be a very powerful tool. Thanks for helping me get started.
It's just plain forward auto.
really good. I used this developed a simple single-phase nonlinear reservoir simulator. One question: is this adiff based on Reverse AD (Rad) or Forward AD (Fad)? Based on its usage, looks Rad to me compared to Sacado or original Fad
The idea is great. It is impossible to compute jacobian with respect to numerous variables using standard Symbolic Toolbox.
However I found that the expression x'*A*x causes an error in Matlab 2010a. I was able to correct it by writing x'*(A*x) and correcting the 'adiffadiff' case in mtimes.m file (transposition should be added to first factors).
Is there a way to allow free usage of matrix expressions (i.e. free transposition usage and maybe allow to calculate derivatives with respect to a matrix)? I haven't found any matlab library that allows of doing it with respect to vector of variable length.
Changed code & documentation to fix problem noted by Dan.
Minor bug fix to atan function
Create scripts with code, output, and formatted text in a single executable document.