Sparse Automatique Differentiation

Forward automatic differentiation using operator overloading and sparse jacobians

This project implements a Matlab/ forward automatic differentiation method, (wikipedia definition https://en.wikipedia.org/wiki/Automatic_differentiation#Forward_accumulation) based on operator overloading. This does not provide backward mode. It enables precise and efficient computation of the Jacobian of a function. This contrasts with numerical differentiation (a.k.a finite differences) that is unprecise due to roundoff errors and that cannot exploit the sparsity of the derivatives.
In contrast with most existing automatic differentiation Matlab toolboxes:
1) Derivatives are represented as sparse matrices, which yield to large speedups when the Jacobian of the function - we aim to differentiate is sparse or when intermediate accumulated Jacobian matrices are sparse (see the image denoising example) .
2) N dimensional arrays are supported while many Matlab automatic differentiation toolboxes only support scalars, vectors and 2D matrices

It is likely that the speed could be improved by representing Jacobian matrices by their transpose, due to the way Matlab represents internally sparse matrices

a simple example using a 3D array:

>> f=@(x) sum(x.^2,3);
>> full(AutoDiffJacobianAutoDiff(f,ones(2,2,2)))
ans =

2.0000 0 0 0 2.0000 0 0 0
0 2.0000 0 0 0 2.0000 0 0
0 0 2.0000 0 0 0 2.0000 0
0 0 0 2.0000 0 0 0 2.0000

see the github page https://github.com/martinResearch/MatlabAutoDiff for more examples and a more detailed explaination