I have read this and other threads on this site on BFGS, but I still don't have a clear understanding of what it's meant by low-rank updates.

For example, I read the following in this book:

The approach adopted by quasi-Newton methods (of which the BFGS algorithm is the most prominent) is to approximate the inverse of the Hessian $H$, with a matrix $M_t$ that is iteratively refined by low rank updates to become a better approximation of $H^{−1}$.

What's exactly low-rank update? How is it different from, say, a high-rank update?

A low rank update to an $n$ by $n$ matrix $A$ is an update of the form
$A=A+UU^{T}$
where $U$ is matrix with $n$ rows but very few columns (typically just one or two.) If the matrix $UU^{T}$ has $k$ columns, then its rank is at most $k$, so this is a low-rank update to the $A$ matrix.
• perhaps the second sentence should read "If the matrix $U$ has $k$ columns, then the rank of $UU^T$ is (at most) $k$"? – GoHokies Sep 25 '16 at 20:15