In my class notes, the author says:

"If $f:\mathbb{R}^n \to \mathbb{R}$ is bounded below and $p_k$ is a descent direction and the $\alpha-\beta$ also known as Armijo-Goldstein condition is met then either $\nabla f(x_k) \to 0$ or the angle between $\nabla f(x_k)$ and $p_k$ approaches 90 degrees."

I think the author meant $\| \nabla f(x_k) \| \to 0$ as this would imply that we are approaching the stationary point of $f$??

or in the worst case scenario $\nabla f(x_k)$ and $p_k$ becomes orthogonal. It means that $p_k$ still decreases the $f$ but the rate of decrease decreases?

  • $\begingroup$ If $\nabla f(x_k)\rightarrow 0$, then clearly also $\|\nabla f(x_k)\|\rightarrow 0$. The opposite is in fact also true: The two conditions are equivalent. $\endgroup$ – Wolfgang Bangerth Jan 21 at 15:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.