I have a system of the form


where $A$ is basically a matrix originating from the Finite Element Method.

I try to solve it using the Newton method:

$$R = A(u_{i}) f(u_{i}) - b $$ $$u_{i+1}=u_{i} - J^{-1}R$$

where $J$ is the Jacobian of the residuum $R$ with respect to $u$.

At a first glance, it works fine. The residuum drops over several orders of magnitudes within just a few iterations. It seems that it converges to an error small enough for my application.

But still - after a couple of iterations, it stagnates/oscillates at a small error. I wonder whether this is normal or whether I should suspect a bug in my code.

What I found out so far:

  • The remaining error is around $10^{-7}$ with entries in $J$ around $10^{-8}$ and in $u$ around $10^{3}$. I suspect that this is not yet extreme enough to cause problems with limited floating point precision.
  • The remaining error becomes smaller when the derivatives $dA/du$ and $df/du$ become larger.

I'm of course happy with an answer to my problem, but I'm also happy with some further reading on the topic.

  • 1
    $\begingroup$ What is the PDE and the geometry you are looking at? $\endgroup$
    – user21
    Commented Jun 8, 2017 at 13:21
  • 2
    $\begingroup$ How do you compute $J^{-1}R$? And how do you define "error"? $\endgroup$ Commented Jun 8, 2017 at 19:25
  • $\begingroup$ What is the (estimated) condition number of your Jacobian? What is your machine precision? $\endgroup$ Commented Jun 9, 2017 at 16:34
  • $\begingroup$ Thanks for your answers. I will try to find out more as soon as possible. $\endgroup$
    – Michael
    Commented Jun 10, 2017 at 8:48

1 Answer 1


I'd suspect a bug in the code unless you understand exactly why the seen behavior is expected.

As for a particular references on how to write nonlinear solvers (globally convergent newton solvers), I was very happy working with:

  1. Newton Methods for Nonlinear Problems,
    Affine Invariance and Adaptive Algorithms;
    Deufelhard, Peter; Springer, 2006 (Second Printing)
    ISBN 978-3-642-23898-7 (Softcover)

  2. Reference implementations for (1),

  3. A Family of Newton Codes for Systems of Highly Nonlinear Equations (1991); Nowak, U., Weimann, L.; ?,

The NLEQ_ERR solver outlined in (1),(2) works very well for my applications.

Another nice book with simple non-linear examples (including the (tangent) stiffness matrices of each newton step is):

  • Advanced Topics in Finite Element Analysis of Structures: With Mathematica and MATLAB Computations; Bhatti M. Asghar; Wiley; ISBN: 978-0-471-64807-9

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.