Lanczos/Arnoldi/Rietz/CG-like algorithm share the same core strategy... In each, a little miracle appears, most of the Gram-Schmidt inner products are zeroes! In others words, new direction need only get orthonormalized with the one previous updated search direction (or two in Lanczos case), no need to check other previous directions (because of zeros). Rather a big miracle when $A$ is sparse and $n = 10^6$.
In case of non-linear inverse/eigenproblem, this property is not true, but depending of the "nonlinearity rate", the values could be small and easily neglected or threshold since the iterative nature of the algorithm assumed that the residual must decrease in each step. So this property remains useful.
Any suggestions are welcome for understanding why the new direction is "already" orthogonal to the set of all (previous+1) directions. Math is ok, but there is a deep principle behind this thing. It looks like a hidden tridiagonalization process that happens at the same time. Is there an easy way to handle the trick?