1
$\begingroup$

I have been dealing with the standard libraries of scipy.optimize for rootfinding and optimization problems, but the problems i want to solve are very large, which makes the standard solvers run out of memory.

I've tried the large scale solvers (newton_krylov) but they are in my experience extremely unreliable no matter how good my initial guess or condition number is, they seem to only work in very coarse grids.

So there are any other libraries that can deal with large scale optimization and/or rootfinding problems? I am primarily using both types to solve systems of non-linear equations, so i require array optimization instead of scalar.

This is the kind of problem i tried to solve as a toy problem with the standard large scale solvers, but i run into multiple problems regarding convergence

Improve this question
$\endgroup$
4
  • 1
    $\begingroup$ Do you have a representative optimization problem to show? $\endgroup$
    – Richard
    Commented Apr 24, 2023 at 15:59
  • $\begingroup$ I edited one type of problem i am dealing with (an EDP) which using the standard large scale solvers fails very often even for simple cases, and even with the advice given regarding preconditioning they still fail too often. $\endgroup$
    – Klaus3
    Commented Apr 24, 2023 at 16:04
  • 1
    $\begingroup$ If Newton-Krylov methods are failing to converge (not just converging slowly) for a wide variety of initial guesses, good conditioning, and preconditioners, then I would suggest you describe your problem in more detail in this post as Newton-Krylov methods are used widely in large-scale applications so there must be something unique in your problem causing this issue. $\endgroup$
    – whpowell96
    Commented Apr 24, 2023 at 21:56
  • $\begingroup$ Have you tried any of the big stand-alone optimization packages? $\endgroup$
    – Wolfgang Bangerth
    Commented Apr 24, 2023 at 22:12

1 Answer 1

Reset to default
3
$\begingroup$

PETSc is widely-used and highly performant library that has a wrapper in Python (https://petsc.org/release/), but I suspect that with the problems you are encountering in SciPy, simply swapping the solver library won't have a large effect. Newton-Krylov methods at thier core are pretty similar across different libraries modulo efficiency and options. Convergence issues are resolved by better preconditioning and better globalization methods. Without more information on your particular problem, no specific advice can be given. The problem you linked had horrible conditioning with no preconditioner, so it is no surprise it failed to converge. Newton-Krylov methods are powerful tools, but they are tools you must understand and tailor to your specific problem to efficiently obtain accurate solutions.

Here are some related posts you may find helpful:

When is Newton-Krylov not an appropriate solver?

Why is Newton's method not converging?

Why is my iterative linear solver not converging?

Improve this answer
$\endgroup$
2
  • $\begingroup$ I've spent the week investigating your solution and also another packages, like the multiple solvers of SUNDIALS, Gekko, pyomo, pymoo... and i've got a pretty unanimous conclusion: There is something wrong with the solvers of scipy, because all of the solvers managed to solve, with no problems, the same poorly conditioned problem of the heat equation of the question you answered. No Preconditioning required and the convergence was super fast aswell. $\endgroup$
    – Klaus3
    Commented Apr 28, 2023 at 21:15
  • $\begingroup$ I've spent more time than i'd like to admit trying to "fix" my (very simple) toy problem so that the newton_krylov algorithm of scipy converges, with very limited success. Then i trusted my common sense and experience with other packages of NL solvers of matlab and other paid programs, that did not require this kind of work in this simple problem, and concluded that something fishy is going on in scipy. The success of the others solvers of different libraries pretty much confirms my suspicion. $\endgroup$
    – Klaus3
    Commented Apr 28, 2023 at 21:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.