# Open source auto-differentiation for MATLAB?

Are there any open-source auto-differentiation libraries for MATLAB?

I am aware of commercial packages such as Tomlab/MAD and plenty of C++ libraries, but I can't find many more for MATLAB other than Automatic Differentiation for Matlab package on Matlab File Exchange which isn't a major package.

As a bare minimum of functionality, I need to be able to handle vectors as variables. In that MathWorks package, I reference, for example, here is a test case:

val = [1 2 3]; %Some value
x = myAD(val); %Creates a vector with that value.

x .* [4 5 6]; %This should give the derivatives [4; 5; 6]
x_inner = sum(x.* x);% To get the inner product.  The derivative here is [2 4 6] here
%Ideally, the above would simply be x_inner = x * x'

• When I last used AdMat some years ago it was quite capable, but it doesn't satisfy your requirement of being open source. Jul 20, 2014 at 3:52

The SINTEF Matlab Reservoir Simulation Toolbox includes a GPL-licensed AD library. The usage is mostly geared towards numerical applications in subsurface flow, but the library itself is usable for more general purposes.

Here is a basic runthrough of your example as you would run it from the base directory of MRST:

startup;

val = [1; 2; 3];
y = x.*[4; 5; 6];
disp 'Jacobian:'
full(y.jac{1})

z = sum(x.*x);
disp 'Jacobian:'
full(z.jac{1})
Jacobian:

% MRST AD also supports multiple vector valued functions
a = ones(5, 1);
b = ones(5, 1);
[a, b] = initVariablesADI(a, b);
c = a.*b + 2.*b;
full(c.jac{1})
full(c.jac{2})


There exists a work-in-progress user guide at the website, where chapter 7 has some usage of the AD library. This library is primarily geared towards problems that require all Jacobian values and is vectorized using forward mode only. It does include some nice things though, like 2d table interpolation and so on. It has been used for moderate size non-linear problems (order of 500,000 unknowns).

The non-linear solvers and components are heavily focused on reservoir applications at the moment. This part of the library is presently being rewritten and may in the future have some use outside of this specific domain. However, if you just want the AD part, the ADI class and the initVariablesADI function should be sufficient.

Disclaimer: I am one of the developers attached to this research group.

• Thanks, sounds great. I will take a look. Are there features for sparsity patterns, etc.? Jul 21, 2014 at 17:45
• Presently all Jacobians are stored as cell arrays of sparse matrices (for groupings of variables). We did investigate having subclasses/treatment for some specific sparsity patterns, but as our linear systems are mostly from unstructured grids the overhead was too large to gain any benefit. Jul 21, 2014 at 17:48
• Thanks. This seems like a good library. It doesn't appear to have support for matrix-vector multiplications, which would be nice based on my current structure, but I may be able to do it with loops. Jul 22, 2014 at 22:38

This article in SIAM Review describes how to implement operator-overloading automatic differentiation in MATLAB, and gives a good introduction to the technology.

• Thank you, let me take a look. For completeness, it looks like the files are located in: academics.davidson.edu/math/neidinger/IntroAD&OOP.zip Jul 19, 2014 at 23:45
• This is a great explanation of AD and matlab, but doesn't support vector based variables as far as I can tell. I would have a (runtime determined) vector of variables, so individually defining each wouldn't work. See my adjusted test case in the description. Jul 20, 2014 at 0:15

I realize this is an old question, but when looking for this myself today I found ADiGator, which is open source, and seems to handle vectors. I haven't tested it yet myself, but it seems to be actively developed.

• Thanks @Petter T, I really appreciate it. After trying a bunch, I ended up deciding that operator overloading based ones were just too slow in matlab. Class access is just too slow compared to generic programming and no overhead abstractions in C++. Of the source code control ones, my solution was ADImat, but I am not sure how supported it is. Looking forward to trying out ADiGator. Aug 21, 2015 at 16:49

If you are really interested in open source and like to promote/support it, then you should probably take a look at the open source matlab clone octave. There is an extension to it, called ad that supports automatic forward differentiation. Unfortunately, it seems to be unmaintained currently.

The wikipedia page for automatic differentiation has links to many software packages or libraries, including several for MATLAB. http://en.wikipedia.org/wiki/Automatic_differentiation#Software

• Thanks. Yeah, I have looked at them and it is pretty out of date. most seem to be proprietary or just something someone threw together for an older version of matlab without core functionality. I was hoping there was a major implementation, but I am starting to doubt it exists. Jul 21, 2014 at 14:46

KPP translates a (chemical or other) ODE specification into Fortran77, Fortran90, C, or Matlab simulation code that implements time derivative function, its Jacobian, and its Hessian, together with a suitable numerical integration scheme. Sparsity in Jacobian/Hessian is carefully exploited to obtain computational efficiency.

CasADi's backbone is a symbolic framework implementing forward and reverse mode of AD on expression graphs to construct gradients, large-and-sparse Jacobians and Hessians. These expression graphs, encapsulated in Function objects, can be evaluated in a virtual machine or be exported to Matlab, Python, or C code.