# Questions tagged [automatic-differentiation]

Often referred to as Algorithmic-Differentiation or AD -- a technique to automatically generate code that evaluates the derivative of a function. AD repeatedly applies the chain rule and classical rules of calculating derivatives. AD usually takes a block of code representing a function and returns a block of code representing that function's derivative.

42 questions
Filter by
Sorted by
Tagged with
139 views

### Auto differentiation with JAX in python and ForwardDiff.jl in Julia give matrices with different values but same structure. Are the results right?

Using ForwardDiff in Julia gives me the following values for 2 matrices I have tried to use JAX to auto differentiate the same function in Python but I am getting the following results for A and B ...
110 views

### "Don't take the derivative of the approximation but approximate the derivative"..or something like this

Don't take the derivative of the approximation but approximate the derivative or something similar. I don't quite remember where I heard this but I am trying to find some work on the support or ...
• 21
117 views

### Automatic Differentiation In the Presence of Jump Points

I have a complex monte-carlo cashflow model that traditionally uses the finite difference (FD) method to calculate its derivative at any given point. To improve model performance, I coded forward-mode ...
1 vote
95 views

### Automatic differentiation (AD) of a loss function which maps unitary matrix onto number

Is it possible to estimate whether automatic differentiation (AD) techniques could enable a more efficient way to repeatedly compute the derivative $\delta L / \delta u^*_{ij}$ of a specific loss ...
• 111
342 views

### Computing the second derivative using Automatic Differentiation

Does anyone have any resources I could follow that explains how to compute the Nth derivative using both forward and backward autodiff I understand how to compute the first derivatives Any assistance ...
1 vote
97 views

### Automatic Differentiation using foward mode on matrices

Whilst googling I see reverse mode automatic differentiation (AD) tends to be used when optimising neural networks. Would it not be better to use forward mode and treat your input as a single variable,...
303 views

### state of automatic differentiation

I've been working with TensorFlow and I'm very impressed with its automatic differentiation capabilities. I'm wondering what the state of the art in automatic differentiation for finite element ...
• 760
791 views

### How to compute the Hessian using auto differentiation?

Assume that $f$ is defined as a composition of functions: $$f=f_2 \circ f_1$$ where $f_1:\mathbb{R}^n \rightarrow \mathbb{R}^{m_1}$ and $f_2:\mathbb{R}^{m_1} \rightarrow \mathbb{R}^n$. We can compute ...
• 169
457 views

### Automatic differentiation of a numerical solver

We often want to use numerical methods to evolve a system in time. That is, for a set of differential equations, we can specify some parameters $\bar{\theta}$ and pass these into our numerical solver ...
• 233
83 views

### Automatic differentiation necessary for large optimal control problems?

I am investigating ways to solve an optimal control problem in an embedded way, preferably in Java. The system is modeled with triple integrator dynamics $u=\dddot{x}$ and solved with multiple ...
844 views

### Why are dual numbers needed only in forward-mode autodiff?

I'm trying to understand autodiff better, and specifically the connection between autodiff and dual numbers, and why dual numbers are needed in the first place. The pytorch help pages about autodiff [...
75 views

• 468
2k views

### Fast Automatic Differentiation for numpy?

I would like to use automatic differentiation to calculate gradients to function written in numpy. I've come across a number of packages, including autograd tangent chainer But none of them seem ...
• 363
172 views

### Automatic differentiation of barycentric rational functions

By a barycentric rational interpolant we understand a function of the form \begin{align*} r(t) := \frac{\sum_{i=0}^{n-1} \frac{w_i y_i}{t-t_i} }{ \sum_{i=0}^{n-1} \frac{w_i}{t-t_i}} \end{align*} In ...
• 2,155
847 views

### Automatic Differentiation - reverse accumulation of linear system solve

I am studying the reverse mode of automatic differentiation. The reverse mode of automatic differentiation allows the efficient computation of a the derivative of a single dependent variable $y$ with ...
• 301
4k views

### Derivatives of a Chebychev polynomial

I am using Chebychev collocation nodes for approximation, and my problem requires me to calculate derivatives of the polynomial. I have been reading from a few sources, but I am not sure I understand ...
275 views

### Iterative linear solvers compatible with automatic differentiation?

I'm using automatic differentiation on a function that contains a sparse nonsymmetric linear system to be solved. I was using BiCGStab to solve this part of the function, but noticed the derivatives ...
• 173
1 vote
178 views

### Is casadi suitable for data fitting?

Quite often I do fit some ODE or DAE systems to my data (small to medium sized problems). Via the assimulo package, I found Casadi and read a bit about the language modellica. Casadi offers automatic ...
• 343
88 views

### $(1+x^M)^{1/M}$ need to be able to calculate any order derivatives vs. $x$ and $M$ for $x\ge 0$ and $M\gt 2$

cannot delete my own question, so I try to overwrite it instead...
785 views

### Constructing sparsity pattern of the Jacobian of a FORTRAN subroutine

I need to calculate the Jacobian matrix of a subroutine F(U). Both F and U are of size N(=O($10^5$)). Using Tapenade, I differentiated the routine in tangent mode. I cannot calculate the full Jacobian ...
• 531
1k views

### Open source auto-differentiation for MATLAB?

Are there any open-source auto-differentiation libraries for MATLAB? I am aware of commercial packages such as Tomlab/MAD and plenty of C++ libraries, but I can't find many more for MATLAB other than ...
• 376
616 views

### Regarding automatic differentiation, is source-code-transformation (STC) more efficient than operator-overloading (OO)?

We are working on a Bayesian model for a space-time process, and are using a No-U-Turn sampler (NUTS) that requires a model for the log-probability and it's gradient with respect to the model ...
• 2,086
1 vote
2k views

### Calculating Divergence in COMSOL

Is it computationally safe and accurate to use the following equation in COMSOL to compute the divergence of the vector quantity J (instead of using its general built-in equations that have $\nabla$ ...
3k views

### When should I use C++ expression templates in computational science, and when should I *not* use them?

Suppose that I'm working on a scientific code in C++. In a recent discussion with a colleague, it was argued that expression templates could be a really bad thing, potentially making software ...
• 30.3k
389 views

### Is there a tool out there that can generate interval extensions of Fortran (or C) functions by parsing Fortran (or C) code?

Case studies in my PhD thesis require that I have interval extensions of Fortran subroutines in CHEMKIN-II (apologies for the link; it's the best one I could find for a package no longer distributed ...
• 30.3k