The conjugate gradient method suffers from an accumulation of errors as it continues. For this reason it is unwise to use it as a direct solver. My question is, would it help to recalculate the residual every 5 or so steps in an attempt to mitigate these sorts of errors even when used as an iterative solver? I've not heard of people doing this but it seems kind of obvious. If it wouldn't help, can you explain why?
Nobody does this. It seems like everyone who has thought about the issue has come to the conclusion that the error one incurs by not computing the residual exactly is small and that computing it every few iterations would be a waste of time.
The reason, I believe, is that even for large systems with hundreds of thousands or hundreds of millions of unknowns, we typically only do a few hundred or thousand iterations at most. This is not enough for errors to significantly accrue due to round-off.
This tutorial talks about recalculating the residual every 50 iterations to mitigate round-off errors.