5
$\begingroup$

If I use scalapack and pblas, and the code is run in serial (1x1 blacs process grid), how well does scalapack and pblas revert to the performance of lapack/blas?

I am particularly interested in the pzhegvx and pzheevx routines with $n \approx 1000$ and the pzgemm and pzherk routines with $m,n \approx 1000$ and $k \approx 100$. These are the lower bounds in the space of reasonable inputs. Otherwise, as Jeff points out, scalapack wouldn't make a whole lot of sense.

Improve this question
$\endgroup$

2 Answers 2

Reset to default
5
$\begingroup$

Not well. If serial is a common case, it is important to wrap and drop down to lapack for serial execution.

I implemented this in my code. For 2013 MKL pzhegvx with $n \approx 100 (1000)$ seems to incur 30% (100%) overhead compared to zhegvx when executed in serial.

This seems high to me, so I'm a little worried about my implementation. Note that I inline zhegvx to enable re-use of the factorization of $B$.

Improve this answer
$\endgroup$
4
  • $\begingroup$ Make sure that with LAPACK you're using single threaded BLAS. Serial LAPACK using multithreaded BLAS 3 will be faster than parallel ScaLAPACK running as a single processes (i.e., mpiexec -n 1) using single threaded BLAS 3. Bottomline is that set OMP_NUM_THREADS variable to 1. There might also be some MKL specific variables. $\endgroup$
    – stali
    Feb 21, 2014 at 16:17
  • $\begingroup$ @stali It is: -lmkl_sequential $\endgroup$
    – Max Hutchinson
    Feb 21, 2014 at 16:21
  • $\begingroup$ It's been a while since I used MKL. $\endgroup$
    – stali
    Feb 21, 2014 at 16:22
  • $\begingroup$ Newer Intel compilers support the flag -mkl={sequential,parallel(,cluster)}. This is particularly useful if one desires static linking. $\endgroup$
    – Jeff Hammond
    Feb 22, 2014 at 16:21
4
$\begingroup$

For matrices of that size, I'm not sure if you want to use ScaLAPACK at all.

If you've got the ScaLAPACK code already, it shouldn't be hard to implement your own logic to drop into LAPACK instead. At the very least, doing that will allow you to perform the experiments required to answer your own question.

Improve this answer
$\endgroup$
3
  • $\begingroup$ In many cases, n,m,k are much larger. I am trying to avoid putting in that logic if it won't help. Although not 'hard,' it would be quite involved. $\endgroup$
    – Max Hutchinson
    Jan 24, 2014 at 14:20
  • $\begingroup$ I would recommend wrapping your linear algebra in an object that makes this transparent. ScaLAPACK has a horrible interface anyways, so I hope you're not using it directly anyways. $\endgroup$
    – Jeff Hammond
    Jan 25, 2014 at 21:24
  • $\begingroup$ I bit the bullet; my results are in a separate answer. I actually like the ScaLAPACK interface, as far as < F90 interfaces go, but that is another matter... $\endgroup$
    – Max Hutchinson
    Feb 21, 2014 at 14:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.