Preconditionning for solving a non-linear system of equations with least squares

Question

I am trying to solve a large system of non-linear equations (about a few hundred equations and variable but with less variable than equations). Given that the system is really sparse and large I am using the optimize.least_square solver of SciPy with the Trust Regions method.

But for some examples, it takes quite long to find a solution and sometimes it lands on what I suppose are local minimums or saddle points.

I assume these problems may come from the fact that the problem is poorly conditioned. So I tried to add weights on the equations in order to give the approximatively the same scale to all the residuals, it improved the speed and convergence but not completely.

So I wanted to know: is there a tool in python either do detect if the function is badly conditioned or to precondition it before calling the solver?

Here is the parameters I use for the solver :

solution = optimize.least_squares(system, xinput, method='trf, loss='soft_l1', f_scale=10, x_scale='jac')

Answer 1

It is possible that the problem with the long execution times is not a conditioning issue, which usually require problem-specific knowledge to fix. You can leverage the sparsity structure of the jac_sparsity input argument. This takes in a matrix with zeros in the same places as your Jacobian as either an array or sparse matrix. This will significantly speed up the solution time for the trust region subproblems as it will use a method specifically for sparse Jacobians