The problem I am trying to solve involves minimising a function with respect to a large number (probably 10,000+) of parameters. I can cheaply compute both its Jacobian and its Hessian. The Hessian is very helpful; I am not entirely sure the function is convex, but using the
Newton-CG algorithm in
result = minimize(to_minimize, f, jac=jac, hess=hess, method='Newton-CG')
So far so good, but unfortunately,
scipy.optimize.minimize is unable to deal with sparse Hessians, so I have to convert the extremely sparse Hessian to a dense matrix. Internally, I believe the
Newton-CG method multiplies that Hessian with other things, which I assume will also be much slower than using its sparseness.
In short, my question is: does anyone know of a library that can exploit the sparse Hessian, preferably using the
Newton-CG algorithm, or do I have to write one myself? It would be nice if it were in python, but I'm happy to use C++ if that is more fruitful.