In practice, L-BFGS is frequently held comparably to other inexact QN methods, and it provides a middle ground of sorts between Hestenes–Stiefel CG and BFGS as memory goes from zero to infinity (Numerical Optimization Ch. 7). Lots of empirical results show L-BFGS outperforms GD, at least in certain large classes of settings, like very strongly convex and smooth functions.
Liu and Nocedal 1989 prove that the L-BFGS descent direction angle with the gradient at the $k$-th iterate, $\theta_k$, satisfies the lower bound $\cos^2\theta_k\ge \delta>0$ eventually, which under moderate smoothness ensures GD-like convergence properties: since the descent agrees with GD enough, under line search satisfying Wolfe conditions, we get global eventual convergence and for smooth and strongly convex functions we achieve linear convergence.
However, the $\cos^2\theta_k\ge \delta$ result, the local and global convergence properties come from general theorems about any descent method with Wolfe conditions met. There's nothing special about L-BFGS. In fact, the inequalities used for the $\cos^2\theta_k\ge \delta$ result seem to get weaker as memory increases.
The linear local rate with worse constants is counter-intuitive and feels like a gap between theory and practice.
Are any theoretical results known where L-BFGS improves on GD's linear rate (or even its smoothness constants) in sufficiently regular settings, ideally in a way monotonically improving with memory use?