What libraries can compute dense Cholesky factorizations in distributed-memory environments (preferably using MPI). I'm using C/C++, and have access to a cluster of between 1 and 40 nodes, where each node has 12 cores. I've heard of ScaLAPACK but am interested in alternatives.
1 Answer
In alphabetical order (disclaimer: I am the main author of Elemental):
DPLASMA
Distributed Parallel Linear Algebra Software for Multicore Architectures (DPLASMA) is a relatively recent and ongoing effort by Bosilca et al. to extend PLASMA to distributed-memory machines. Version 1.0.0 supports distributed Cholesky factorizations, among many other operations. DPLASMA is written in C and uses MPI. Their website links to the following handout from SC12, though there has been significant progress since then. This is the only software in the list which currently makes use of DAG-scheduling, and it appears to have begun development around 2010.
Elemental
Elemental is a modern retake and extension of many of the ideas behind PLAPACK and ScaLAPACK (described below). It is written in C++ and currently sits on top of BLAS, LAPACK, and MPI. Please see this recent presentation for a more detailed overview and some short example programs. Development began around 2009 and is still picking up steam. Contrary to PLAPACK and ScaLAPACK, its characteristic communication pattern is allgather.
PLAPACK
Parallel Linear Algebra PACKage (PLAPACK) was developed from around 1997-2000 and introduced the idea of providing an easy means (in the sense of just calling a routine named Copy) of redistributing (sub)matrices from one distribution to another. The main author of PLAPACK has actively contributed to Elemental's development. PLAPACK was written with C and MPI, it sits on top of BLAS and LAPACK, and its characteristic communication pattern is broadcast.
ScaLAPACK
Scalable Linear Algebra PACKage (ScaLAPACK) has been under development since about 1995 and is thus the earliest (and also most commonly used) of the bunch. ScaLAPACK is primarily written in Fortran, but several of its routines are written in C, and most of its communication is performed in terms of the Basic Linear Algebra Communication Subprograms, which is (now) typically implemented using MPI. It was purposely designed so that its high-level code resembles that of LAPACK, and the local computations that occur are typically calls to BLAS and LAPACK. The characteristic communication pattern in ScaLAPACK is broadcast.
openblas_set_num_threads(1)
when I'm using Elemental? $\endgroup$