In an application I have to solve a series of positive definite linear systems of the form $A^TA x = A^Tb$ (i.e. normal equations). The next system is obtained from the previous one by adding and/or removing a small number of columns to $A$ (usually one or two). The vector $b$ stays the same throughout. It seems clear that some kind of up-/downdating strategy should be applied, however, my questions are:
What kind of up-/downdating shall I choose (QR, Cholesky, Matrix-Inversion-Lemma,others?,...)?
On what conditions and requirements shall I make my choice (high precision needed, ill-conditioned matrices, well-conditioned ones, speed,...)?
P.S.: You may assume that $A$ and $A^T$ can be applied fast.