Complex differentiation of linear solvers

Question

I have a linear system $$Ax=b$$ which I'm solving approximately, and I need to take the frechet derivative of x with respect to z. Were I solving the problem exactly (either analytically or to machine 0) I'd simply say $$\frac{dx}{dz}\vec{v} = \frac{d A^{-1}b}{dz}\vec{v} = \left(\frac{dA}{dz}A^{-1}\frac{dA}{dz}b + A^{-1}\frac{db}{dz}\right)\vec{v}$$ Since I am not solving the system exactly (2 orders of accuracy say), I thought the best way to get the sensitivity, would be a complex perturbation to the linear system. I.e, solve the linear system, but where it is: $$A(z+i\epsilon {v})x = b(z+i\epsilon\vec{v})$$ And I could then take the imaginary part of x. Will this work the way I want it to? Then if I wanted the adjoint differentiation, of the linear solve, would I give a complex perturbation to the matrix and transpose it?

This is related to my previous question posted here: Derivatives of Approximate Matrix inverses

Answer 1

It does indeed work to linearize the forward mode this way, and will correspond to machine precision. The reverse or adjoint mode doesn't work this way despite my wishes to the contrary. The derivation I posted doesn't correspond exactly to machine precision, but the numerical tests I did showed that the magnitude of the error depends on the linear tolerance of the approximate linear solve and the magnitude of the right hand side.