New answers tagged iterative-method
4
These humps occur in the context of optimization when the solution is approached but we then go past it.
This frequently occurs in gradient descent with momentum, where they appear regularly on plots because the gradient and momentum both decay in magnitude together. Usually this suggests a little too much momentum, since we're retaining speed as we reach ...
Top 50 recent answers are included
