1

(Not exactly an answer to my question, but some empirical experiments which might give guidance to what a worst-case analysis of the L-BFGS iteration of a quadratic function should result in). I tested directly translated algorithm pseudocode implementations of GD and L-BFGS. See the code and all my experiments here. Even in the nicest setting you can think ...


1

From a pure convergence point of view, I believe the only thing that's necessary is to satisfy the convergence criteria from a globalization method such as a line-search or trust-region. This is generally required for convergence even if you used the exact gradient when determining your search direction. Meaning, BFGS will not generally converge by itself ...


Only top voted, non community-wiki answers of a minimum length are eligible