I am reading the Chapra and Canale book on numerical methods, and was working through the chapters on solving linear systems. Now the book goes through direct methods including Gaussian Elimination, as well as LU factorization, versus iterative methods like Jacobi iteration, Gauss-Seidel, and conjugate gradient methods.
Now I was not clear what the current thinking was on which methods are best to use under what circumstances. So, if I am coming from the perspective of solving PDEs, then I will have to discretize the domain and repeatedly solve large systems of ODEs--leading to large systems of matrices. Most of the examples I have seen for solving large systems involves using iterative methods (Jacobi or Gauss-Seidel) for small the medium systems, and then moving to Krylov subspaces or GMRES for larger systems. So I never see any mention of LU factorization, etc., in any of these contexts.
Hence, I was not clear on when to use LU factorization, or QR factorization or Cholesky decomposition versus the seemingly more popular iterative methods. What is the current thinking on which methods work best under what circumstances.
