I am looking for recommendations on matured C++ solver for Linear Sparse Algebra problems. The goal is to select between more or less GPU hardware agnostic libraries/frameworks that can be compiled on both Linux and Windows with quite high control on host-device memory. My current research narrows down to the following candidates:

  • CUDA - matured, more than BLAS, cross platform, run on NVIDIA only
  • OpenCL - matured, competitor to CUDA
  • HIP - less matured, but picks lot from CUDA, runs on both NVIDIA & AMD too, no cost on porting between OpenCL and CUDA
  • Intel One API (DPC++) - implements SYCL, seems to be only MKL/LAPACK wrapper now, not sure how it works with GPU system, since MKL is more CPU-centric - seems immature
  • Magma - matured, looks like big competitor to CUDA, smaller community

I am close to start with CUDA, since it has a lot of examples in the toolkit, and then potentially migrate to HIP (with hipify) or to Intel's DPC++ SYCL guide.

Do you have a link/article/opinion that will help me in challenging these considerations?

  • $\begingroup$ Maybe you do not have much choice about this but I was actually under the impression that sparse solvers on GPUs really have no advantage over CPUs. Has this changed? $\endgroup$ May 13 at 15:18
  • $\begingroup$ Ginkgo is another library that provides sparse, iterative solvers with backends for OpenMP, CUDA, HIP, and DPC++. $\endgroup$ May 13 at 15:23
  • 1
    $\begingroup$ @KyleMandli Current GPUs have high memory bandwidth compared to CPUs. So, they perform really well for sparse, iterative solvers. I don't know how they do for sparse-direct solvers though. $\endgroup$ May 13 at 15:25
  • $\begingroup$ @NeilLindquist Good point although I assume this gets difficult when you go to multi-node types of setups? $\endgroup$ May 13 at 18:30
  • $\begingroup$ @KyleMandli I think the difficulties for multi-node are similar between GPU and CPU systems for iterative solvers. Just call GPU kernels instead of CPU kernels and do host-device memcpy's before/after the MPI calls (or use CUDA-aware MPI). But, I haven't worked with any distributed, GPU-accelerated, sparse codes, so there might be details I'm unaware of. $\endgroup$ May 13 at 21:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.