In the original paper of Conjugate Gradients, the authors mention that if we pick the canonical basis $\{e_1,e_2,\ldots,e_n\}$, to obtain A-orthonormal vectors, we end up with the Gaussian elimination method (section $12$). My question is the following:

Apart from this canonical basis and of-course the Krylov subspace, are there other attractive subspaces to look at?

For instance, intuitively, it looks like if $A$ is well-conditioned, picking the subspace generated by $\{b,A^Tb,(A^T)^2b,(A^T)^3b,\ldots\}$ might be attractive option. If $A$ is the most well-conditioned matrix, the method is guaranteed to converge in $1$ iteration. Has there been some analysis done on other possible subspaces (apart from Krylov and the canonical ones)?

  • $\begingroup$ You typically need a justification for why you would look into a different subspace generation. One specific issue with your example is that in many large scale computational codes, the linear solver is used in a newton solve, in this case its easier to implement and cheaper to use the frechet derivatives of the residual to compute the jacobian vector product, rather than the jacobian transpose vector product. $\endgroup$
    – EMP
    Nov 27 '19 at 19:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.