0
$\begingroup$

Based on my basic understanding of the BFGS method, the algorithm will iterate until the gradient norm is less than or equal to a set value called "gtol" in the case of Python.

However, when using this method, and checking the output the following is showing:

Iterations: 2 Function evaluations: 76 Gradient evaluations: 13

"Desired error not necessarily achieved due to precision loss."

This got me confused. Shouldn't the algorithm iterate multiple times to reach convergence? How did it reach convergence by only iterating twice? In this case, why can we specify a maximum number of iterations if the algorithm converges this quickly?

Improve this question
$\endgroup$
9
  • $\begingroup$ If the algorithm was started very close to a local minimum, then it could very easily have converged in two iterations. Have you checked whether the norm of the gradient satisfies the tolerance at the point returned by the routine? $\endgroup$
    – Brian Borchers
    Commented Feb 4, 2021 at 15:51
  • $\begingroup$ I am not sure how to check using python but I am assuming that yes the tolerance criteria was satisfied. However, I cannot comprehend how the algorithm evaluated the objective function 76 times in two iterations. Shouldn't both occur almost equally? Since the main loop of the BFGS algorithm is to iterate from k=0 to k=n until the tolerance is satisfied. $\endgroup$
    – Habib
    Commented Feb 4, 2021 at 16:04
  • $\begingroup$ I would also like to add that I received this message when the optimization was terminated: "Desired error not necessarily achieved due to precision loss." $\endgroup$
    – Habib
    Commented Feb 4, 2021 at 16:07
  • 2
    $\begingroup$ Does this answer your question? scipy.optimize.fmin_bfgs: "Desired error not necessarily achieved due to precision loss" $\endgroup$
    – Brian Borchers
    Commented Feb 4, 2021 at 16:32
  • 1
    $\begingroup$ That's nonsmooth. I would suggest that you create a new questions including this information and ask for suggestions on how to solve the optimization problem. $\endgroup$
    – Brian Borchers
    Commented Feb 4, 2021 at 18:39

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.