10
$\begingroup$

I am getting the warning in the post subject when attempting to optimize a function in Python with the scipy.optimize.fmin_bfgs function. The complete output:

Warning: Desired error not necessarily achieveddue to precision loss

     Current function value: nan
     Iterations: 1
     Function evaluations: 18
     Gradient evaluations: 3

It is not a fatal error, I am getting answers, but they are far from the optimum I search.

Are there any typical "noob" errors that might generate this error? I started working with this package only yesterday; as it is, I'm not even sure where to start looking. Any help is appreciated!

Improve this question
$\endgroup$
1
  • $\begingroup$ If the current function value of "nan" is being correctly reported, that would seem to indicate that your objective function is returning nan values- if so that would be something you need to fix. $\endgroup$
    – Brian Borchers
    Commented Dec 12, 2023 at 21:48

2 Answers 2

Reset to default
12
$\begingroup$

It is helpful in situations like this to take a look at the source code. This is easy because everything in scipy is open source!

As you can see from reading the source code, the warning message is printed when warnflag==2. This gets set elsewhere in the code when the linesearch function returns None (it fails).

So why does linesearch fail? The goal of an optimization algorithm is to find the minima of some objective function through a successive set of iterations. The line search, in this case, is trying to find a step size where the approximations in BFGS are still valid. When the Hessian of your function or its gradient are ill-behaved in some way, the bracketed step size could be computed as zero, even though the gradient is non-zero.

I guess my suggestion is to either go to the literature (Nocedal and Wright has a good discussion of line search and the BFGS method) or to your function and make sure that it is well-behaved in the region you are searching.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thanks, I had already checked out the source code. Alas, the problem is my lack of theoretical knowledge of BFGS :-) I managed to use BFGS successfully with a non-vectorized implementation, so I guess my matrices are not behaving as expected. That might be a direction for finding out the problem. $\endgroup$
    – ACEG
    Commented Apr 21, 2012 at 17:38
  • $\begingroup$ If you state the actual function you are trying to minimize and the region you're searching in, we may be able to give you some more clues about how to proceed. $\endgroup$
    – Aron Ahmadia
    Commented Apr 21, 2012 at 18:48
  • 2
    $\begingroup$ Thanks, I solved the problem in the meantime. The error had (as often happens) absolutely nothing to do with the problematic results. My hunch was right, I was passing matrices where the method expected transposed matrices. Now I still get the initial warning, but there are many more iterations and I get good values. $\endgroup$
    – ACEG
    Commented Apr 23, 2012 at 6:55
4
$\begingroup$

I encountered the same issue where, during the iteration, it stopped at 1 and the optimal value found stayed at the initial value.

I delved into the source code of SciPy and discovered that an exception is thrown from the line_search_wolfe2 process. The step size becomes very small during the line search iteration, preventing the function value from decreasing. However, I noticed that the function value does decrease when I combine the initial value with the gradient. Eventually, I realized that one term in the gradient was incorrectly signed. After the gradient was revised, the waring disappeared and the optimial value was found.

So, based on my experience, if you notice precision loss warning in the first iteration of the process, it's crucial to check that your objective function and gradient are correctly written. Paying close attention to the sign of the function is especially important.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.