# Python-accessible industry-standard for unconstrained minimization that converges to machine precision?

I have an unconstrained minimization problem of many variables for which I know the gradient exactly. I turned to the conjugate gradient method contained in scipy.optimize.minimize (which uses the Polak-Ribiere algorithm), but it throws a LineSearchError when I try to converge the algorithm beyond the square root of machine precision. This seems to be a common occurence with a certain class of line-search algorithms.

The square root of machine precision is not enough for my purposes. Is there a robust algorithm available that uses an approximate line-search, or something similar, which does enable one to converge to machine precision?

• What exactly do you mean with machine precision? Do you mean single/double precision? Why are you looking to converge to the square root of that? – MPIchael Sep 25 '19 at 14:26
• How well conditioned is your problem? Do you know the condition number at the optimal solution? – Brian Borchers Sep 25 '19 at 16:50
• I am looking to converge beyond sqrt(eps) (double precision) because I am after a scalar which contains two contractions with the variable vector at a local minimum, meaning the precision of that scalar will be again the square root of the precision of the variables. That turns out to be ~1e-4, and that's insufficient for my problem. @BrianBorchers I don't know the condition number, except that it should be fairly well behaved. – Kappie001 Sep 25 '19 at 17:09
• If the condtion number is $10^{k}$, then the best you can hope for in double precision is $16-k$ digits of relative precision in the answer. – Brian Borchers Sep 25 '19 at 17:55
• I have no reason to believe that there's anything wrong with the code that you're using. A good working hypothesis is that your problem has a condition number of around $10^8$ and that the scipy function is doing as well as any routine could using double precision. – Brian Borchers Sep 25 '19 at 18:57

I totally agree with the discussion in the comments: it is quite likely that your problem has a large enough condition number that you are seeing the problems converging beyond $$\sqrt{\epsilon_\text{mach}}$$.
class mpmath.calculus.optimization.MDNewton(ctx, f, x0, **kwargs)
`