I'm doing a Lanczos diagonalization of a large sparse matrix (~2 million elements). Almost all of the steps in the Lanzcos algorithm are done in parallel on the GPU, except for diagonalizing the Lanczos matrix to check for convergence. For that, I've been using the TQLI algorithm from Numerical Recipes. Are there methods for finding the eigensystem of a tridiagonal matrix that are parallel or easily parallelizable? Does a parallel version of TQLI exist?
TQL cannot be parallelized.
The standard parallel algorithm is that of Cuppen:
A divide and conquer method for the symmetric tridiagonal eigenproblem, 1980.
A parallel divide and conquer algorithm for the symmetric eigenvalue problem on distributed memory architectures, 1999
I recommend using Parallel Multiple Relatively Robust Representations (PMRRR) for the parallel tridiagonal eigensolve. It can compute all of the eigenpairs of the tridiagonal matrix in $O(n^2)$ work in parallel. You can find an overview of the method here. There is also the implementation in ScaLAPACK, which is discussed here.