Newton's method goes to zero determinant Jacobian - Computational Science Stack Exchange most recent 30 from scicomp.stackexchange.com 2019-11-17T14:55:54Z https://scicomp.stackexchange.com/feeds/question/21781 https://creativecommons.org/licenses/by-sa/4.0/rdf https://scicomp.stackexchange.com/q/21781 4 Newton's method goes to zero determinant Jacobian Sylvain B. https://scicomp.stackexchange.com/users/18836 2016-01-11T10:54:35Z 2016-01-18T14:08:06Z <p>I am using the Newton's method to solve $3\times3$ systems. </p> <p>For some particular cases, it turns out that at a given iteration, the Jacobian matrix cannot be inverted and that its determinant is very close to zero (looking at the matrix, there are terms that are around <code>1e+0</code> and others that are <code>1e-15</code>). </p> <p>After investigations, it is clear that one variable has no influence on the system when it is close to the solution. </p> <p>What is the most clever thing to do with such an issue ?</p> <p>I would like to have an algorithm that can adapt itself to such situations when they happen.</p> <hr> <p><em>EDIT</em> </p> <p>It is about an optical optimisation problem. The point is to add a surface to an optical system so that it fits some optical properties. The Newton's method finds the roots of a function that takes as input the parameters of the surface and outputs the differences between the optical properties computed and the targeted properties. </p> <p>I noticed that if the system is complex enough, then we have convergence. But if the system is too simple, the surface is more spherical and the Jacobian goes to very small values, because some parameters like astigmatism axis become influence-less. </p> https://scicomp.stackexchange.com/questions/21781/-/21851#21851 3 Answer by Christian Clason for Newton's method goes to zero determinant Jacobian Christian Clason https://scicomp.stackexchange.com/users/1804 2016-01-18T14:08:06Z 2016-01-18T14:08:06Z <p>From what you describe, you have an ill-posed problem: The solution is not unique (not even locally). A standard way of dealing with this is the following: Instead of trying to solve $F(x)=y$, where $x$ is your vector of geometrical parameters, $y$ the vector of optical parameters, and $F$ the mapping that computes the latter via the former, you minimize $$J(x) = \frac12\|F(x)-y\|^2 + \frac\alpha2\|x\|^2$$ for some (small) $\alpha&gt;0$. The last term ensures (local) uniqueness: Among all solutions of $F(x)=y$, it will pick the one with minimal norm (in your case, if any angle will give the same surface, the minimizer will have zero angle).</p> <p>You can then compute a minimizer using any optimization method (e.g., BFGS with line search globalization), as described in the book by Nocedal and Wright <em>Numerical Optimization</em>.</p> <p>For most methods, you need the gradient of $J(x)$, which is given by $$\nabla J(x) = \nabla F(x)^T (F(x)-y) + \alpha x,$$ where $\nabla F(x)^T$ is the transpose of the Jacobian. If it is at all possible, you should try to compute the Jacobian analytically and (approximately) evaluate that. (If you add the mathematical description of your $F$, we might be able to help with that.)</p>