I tried some python optimization functions and some of them needed Jacobian matrix prior for faster convergence. I understand Jacobians are basically transformation matrices that data from one space to another or carrying gradients information. Can someone explain me with some literature references, how the speed up is achieved?

  • $\begingroup$ What you are asking is not very precise, it will be difficult for us to help you. Could you tell us more about what you try to optimize and which articles you read ? $\endgroup$
    – BrunoLevy
    May 4 '17 at 18:34
  • 3
    $\begingroup$ Read any book on numerical optimization (my favorite is the one by Nocedal and Wright) to learn about the difference between gradient optimization and Newton-type optimization. $\endgroup$ May 4 '17 at 19:28

You haven't told us exactly what optimization routine you're using, so it's difficult to provide a very specific answer to your question.

However, if you don't supply your own Jacobian function then the optimization routine that you're using is presumably approximating the derivatives using a finite difference approximation scheme. Computing finite difference approximations to the derivative requires many function evaluations and this slows down the optimization process. Furthermore, the inaccuracy of such approximate derivatives can cause the algorithm to require more steps to converge to an optimal solution and thus run more slowly.

The advantages of using exact analytical derivatives rather than finite difference approximations are discussed in most textbooks on nonlinear programming. See for example Practical Optimization by Gill, Murray, and Wright.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.