8
$\begingroup$

I am getting the warning in the post subject when attempting to optimize a function in Python with the scipy.optimize.fmin_bfgs function. The complete output:

Warning: Desired error not necessarily achieveddue to precision loss

     Current function value: nan
     Iterations: 1
     Function evaluations: 18
     Gradient evaluations: 3

It is not a fatal error, I am getting answers, but they are far from the optimum I search.

Are there any typical "noob" errors that might generate this error? I started working with this package only yesterday; as it is, I'm not even sure where to start looking. Any help is appreciated!

$\endgroup$
9
$\begingroup$

It is helpful in situations like this to take a look at the source code. This is easy because everything in scipy is open source!

As you can see from reading the source code, the warning message is printed when warnflag==2. This gets set elsewhere in the code when the linesearch function returns None (it fails).

So why does linesearch fail? The goal of an optimization algorithm is to find the minima of some objective function through a successive set of iterations. The line search, in this case, is trying to find a step size where the approximations in BFGS are still valid. When the Hessian of your function or its gradient are ill-behaved in some way, the bracketed step size could be computed as zero, even though the gradient is non-zero.

I guess my suggestion is to either go to the literature (Nocedal and Wright has a good discussion of line search and the BFGS method) or to your function and make sure that it is well-behaved in the region you are searching.

$\endgroup$
  • $\begingroup$ Thanks, I had already checked out the source code. Alas, the problem is my lack of theoretical knowledge of BFGS :-) I managed to use BFGS successfully with a non-vectorized implementation, so I guess my matrices are not behaving as expected. That might be a direction for finding out the problem. $\endgroup$ – ACEG Apr 21 '12 at 17:38
  • $\begingroup$ If you state the actual function you are trying to minimize and the region you're searching in, we may be able to give you some more clues about how to proceed. $\endgroup$ – Aron Ahmadia Apr 21 '12 at 18:48
  • 1
    $\begingroup$ Thanks, I solved the problem in the meantime. The error had (as often happens) absolutely nothing to do with the problematic results. My hunch was right, I was passing matrices where the method expected transposed matrices. Now I still get the initial warning, but there are many more iterations and I get good values. $\endgroup$ – ACEG Apr 23 '12 at 6:55
  • $\begingroup$ Link is rotten... update link and include relevant bits in the answer itself $\endgroup$ – innisfree Jul 18 '17 at 5:47
  • $\begingroup$ rotted link fixed. $\endgroup$ – Aron Ahmadia Sep 4 '17 at 20:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.