I want to know methods which are fully parallelizable on CUDA architecture. I have implemented the Jacobi and Conjugate Gradient methods and now Im thinking about the Bi-Conjugate gradient method. I was using the cublas library to simplify some basic algebra operations. Someone said me that Gauss-Seidel is semi-parallelizable. I am new on CUDA, so I cannot understand some issues in the Cholesky descomposition or Gaussian-Elimination papers like:
I can't understand the new algorithms, so I am saying that I have to use the classic numerical algorithms(I am not able to "create" that kind of algorithms), something like.
The point is, does there exist any "theory" about parallelizing of those algorithms? Because it was easy for me the Jacobi parallelization and conjugate gradient methods, there was some saxpy and gemv operations inside the algorithms.
For example in the Cholesky's case the inner loop depends of the first, how we can solve this?
for k = 1 to n ... for j = k + 1 to n
PS: I always developed the algorithms on C++ and from there I started to parallelizing process( it means the process of "discovering" how to parallelize )