In terms of numerical optimization, the newton-rapson method requires a pos. definite Hessian $\nabla^2f$ respectively pos. curvature for computing the next step $p_k$ by solving $$\nabla^2 f p_k = -\nabla f$$

If I'm dealing with a vector valued function, e.g. a system of equations $F$, the newton step $p_k$ gets computed by $\nabla F p_k = - F$ without any calculation of the hessian. If in the latter case the newton-rapson diverges from my current point, would there be a possibility to check about the curvature of $F$?


The condition for finding a root that corresponds to positive definiteness of the Hessian in optimization, is that the function grows strictly monotonically.

But this is not a useful condition. That is because even in optimization, positive definiteness of the Hessian does not actually guarantee convergence of the unmodified Newton method. What is important is to focus on the remedy to the problem. The remedy is to use a line search procedure. If you understand how to write a line search procedure for optimization (e.g., based on the Wolfe and Goldstein conditions), then you will also know how to write a line search procedure for root finding.

| cite | improve this answer | |
  • $\begingroup$ I'm actually interested in studying Newton Method's undesirable behaviour. Sometimes, the algorithm seems to "climb up" and/or "jump around". While the latter case is often caused e.g. by inflection points with F(x) ~= 0, I was wondering if there was an explanation for choosing a wrong descent direction. If there was a possibility for checking curvature (or something like the pos. definiteness of the Hessian for F if F was not a system of equations), would that be a reasonable approach? $\endgroup$ – RockedSalad121 Jun 19 at 16:23
  • $\begingroup$ Maybe. But we already have very good methods to deal with the problem: Line search of trust region methods. You might want to read up on those in the context of optimization in the book by Nocedal and Wright. $\endgroup$ – Wolfgang Bangerth Jun 20 at 23:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.