I am cross posting this question to the mathermatics stack exchange. please find it either at this link, https://math.stackexchange.com/q/2952989/430980, or below:

I have a question concerning the derivatives of approximate matrix inverses. I have a system, $$Ax = b$$ which I solve approximately (and with an iterative method) with: $$\Delta x = x - \tilde{A}^{-1}b$$ I would like to take the derivative of this process, i.e. find $$\frac{d\Delta x}{dF} = \frac{dx}{dF} - \frac{d\tilde{A}^{-1}}{dF}b - \tilde{A}^{-1}\frac{db}{dF}$$ I know that if I had an ideal inverse, my expression would be without tildes as I'd have the exact result: $$\frac{d\Delta x}{dF} = \frac{dx}{dF} - {A}^{-1}\frac{dA}{dF}{A}^{-1}b - \tilde{A}^{-1}\frac{db}{dF}$$ But for this, I cannot start with the typical assumption that $$AA^{-1} = I$$ and instead have $$A\tilde{A}^{-1} = C \neq I$$ Does anyone have any ideas for resources or techniques for such problems? Thank you.

Edit: To add some additional context, I'm using a Newton method in a CFD solver, so my linear system is $$\frac{\partial R}{\partial u}\Delta u= -R(u)$$ I'm attempting to differentiate the process. I solve the linear system using point Jacobi, and I break up the matrix into diagonal and off diagonal, so it looks like so: $$D \Delta u^{k+1}= -R(u) - \frac{\partial R(u) }{\partial u}\Delta u^{k}$$

  • $\begingroup$ It depends how you build $\tilde{A}^{-1}$. If $C$ is known, then you have $\tilde{A}^{-1} = A^{-1} C$ and you can differentiate the right hand side. $\endgroup$
    – user7440
    Oct 12, 2018 at 21:25
  • $\begingroup$ C is not known. I am using an iterative method. $\endgroup$
    – EMP
    Oct 13, 2018 at 0:21
  • $\begingroup$ It seems to be like you might not have enough information: you have a bunch of unknown matrices, with no explicit known relationships between them, and you want to differentiate one of the unknown matrices. Somewhere in there would have to be a way to relate the unknowns to the input data, but I don't see that in the question. Could you be more explicit about how exactly $\tilde A$ is computed, for example? $\endgroup$
    – Kirill
    Oct 13, 2018 at 10:36
  • $\begingroup$ I added an edit with some additional information. Hope that is helpful. Thank you $\endgroup$
    – EMP
    Oct 13, 2018 at 13:30
  • $\begingroup$ This problem seems similar to the works of Nemili et al., see papers here researchgate.net/profile/Anil_Nemili2 (Anil is a friend and collaborator of mine) $\endgroup$
    – cfdlab
    Oct 13, 2018 at 14:04

1 Answer 1


Call the approximate inverse $V$, then its products with $A$ are $$AV = I+E_{a},\quad VA = I+E_{v}$$ where $E_k$ and $E_a$ are "small" random error matrices which characterize your iterative method.

The error matrices are related since $$VAV=V+VE_a=V+E_vV\implies VE_a=E_vV$$ $$AVA=A+E_aA=A+AE_v\implies AE_v=E_aA$$

Pick one of the two products, differentiate, and solve for $dV$ $$\eqalign{ V\,dA + dV\,A &= dE_v \cr dV\,A &= dE_v - V\,dA \cr dV\,AV &= dE_vV - V\,dA\,V \cr dV &= (dE_vV - V\,dA\,V)\,(AV)^{-1} \cr }$$ When $\,(dE_vV)\,$ is negligible
$$\eqalign{ dV &= -(V\,dA\,V)\,(AV)^{-1} \cr }$$

  • $\begingroup$ Hey Greg, I'm using a point Jacobi method to solve the linear system. Does thay clarify it at all? $\endgroup$
    – EMP
    Oct 13, 2018 at 3:43
  • $\begingroup$ I added some additional information $\endgroup$
    – EMP
    Oct 13, 2018 at 13:29
  • $\begingroup$ @EMP Your original question was stated in terms of $(A,x,b)$, but your updated information is in terms of $(R,u)$ and various derivatives. I'm having trouble connecting the two. $\endgroup$
    – greg
    Oct 13, 2018 at 15:13
  • $\begingroup$ Hi Greg, The original statement was an attempt to be as broad as possible, but some people commented requesting additional information. So I tried to provide the information that indicates I am using newtons method to find the root of the nonlinear function R. the matrix A is Rs derivative, the vector x is the state field update $\Delta u$ and the right hand side vector, b is the negative of the R function evaluated at the current state. $\endgroup$
    – EMP
    Oct 13, 2018 at 15:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.