I'm using automatic differentiation on a function that contains a sparse nonsymmetric linear system to be solved. I was using BiCGStab to solve this part of the function, but noticed the derivatives produced by the automatic differentiation that were carried through BiCGStab were diverging in some cases. Is there an iterative linear solver with proven convergence using automatic differentiation?

  • 4
    $\begingroup$ Your question doesn't really make sense to me: if you're solving a linear problem, why do you need a derivative? I'm missing some information -- could you edit your question to add more details? $\endgroup$ – Christian Clason Jul 14 '17 at 7:35
  • $\begingroup$ Thanks for requesting more details. It was quite unclear before. Does it make more sense? The finding the derivatives w.r.t my parameters in the linear system is not so much the goal as a hurdle. It is one part of a large objective function. $\endgroup$ – bfletch Jul 14 '17 at 7:55
  • $\begingroup$ So you have to differentiate a (presumably implicit) function, which requires you to solve a linear system. If you do that iteratively, you find that the computed derivatives are not correct (in what sense precisely? are you checking via a finite difference test?). The first thing to do is to check whether the linear systems are solved accurately enough. If this is the case, it's the automatic differentiation that is unstable rather than the linear solver, so you'd need to find a more robust AD method or try to get more exact derivatives. $\endgroup$ – Christian Clason Jul 14 '17 at 8:26
  • $\begingroup$ (cont'd) Otherwise, you can try a different solver depending on the structure of the system (e.g., is it semidefinite or indefinite, do you have estimates of the spectrum). As a test, you can try to brute force it by using GMRES without restarts and maximal number of iterations equal to the size of the system. (It won't be competitive, but if any iterative method works at all, this will.) $\endgroup$ – Christian Clason Jul 14 '17 at 8:28
  • $\begingroup$ @bfletch were you able to find the source of your problem? I'd be interested to know what led to your directional derivatives diverging... Btw, may I ask what AD tool you are using? $\endgroup$ – GoHokies Jul 16 '17 at 9:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.