Assume I have two functions $f$ and $g$, with derivatives of $g$ at point $x$ and derivatives of $f$ at point $g(x)$ available.

What is the fastest way of computing derivatives of $f \circ g (x)$ ?

  • 1
    $\begingroup$ (d(f(g(x))/dx)(x) = (df/dg)(g(x)) * (dg/dx)(x) $\endgroup$ – sssssssssssss Aug 10 '17 at 15:11
  • 3
    $\begingroup$ You don't provide a lot of context here, but using a forward-mode automatic differentiation approach could work well. If you're looking for numerical results and are working in a language that supports user-defined types, generic functions and operator overloading, then I suggest using a "Dual Number" approach. Strictly speaking, it may not be faster than hand-writing the derivatives of your functions (in terms of total operations), but (in my experience) it is less error-prone. en.wikipedia.org/wiki/Automatic_differentiation $\endgroup$ – Tyler Olsen Aug 10 '17 at 16:17
  • $\begingroup$ I've addressed this topic in more detail on a different StackExchange site before, so you may be interested in checking it out there. quant.stackexchange.com/questions/21885/… $\endgroup$ – Tyler Olsen Aug 10 '17 at 16:24

For real valued function g(x), you can do that as follows,

\begin{equation} \frac{\textrm{d} f}{\textrm{d} x}= \frac{\textrm{d} f}{\textrm{d} g}\frac{\textrm{d} g}{\textrm{d} x} \approx \frac{\textrm{d} f}{\textrm{d} g}\frac{\textrm{Im}\left\{g(x+i\epsilon)\right\}}{\epsilon} \end{equation} where $i$ is imaginary number. $\textrm{Im}\{z\}$ gives the imaginary part of the complex number z.

You can implement that in any language which can do operations on complex numbers. You can have very small $\epsilon$, to have small truncation error $O(\epsilon^2)$ and avoid subtractive cancellation error. I don't know how fast it is but is very fast to implement.

For more details see,

Squire, William; Trapp, George, Using complex variables to estimate derivatives of real functions, SIAM Rev. 40, No.1, 110-112 (1998). ZBL0913.65014.


Not the answer you're looking for? Browse other questions tagged or ask your own question.