0
$\begingroup$

I have a distributed matrix in block cyclic layout. Is there an efficient way to out/in place transpose a distributed matrix with scalapack?

Context: I am trying to diagonalize the transpose of a distributed matrix with pzheevd

EDIT: Thanks to Ian Bush, It fixed the problem. Bit curious if there is a nice way to do it

$\endgroup$
4
  • 1
    $\begingroup$ Why do you need to form the transpose? It seems to me that as the matrix is hermitian, you can just diag "as is", the evals stay the same and you just complex conjugate the evecs. $\endgroup$
    – Ian Bush
    Oct 12, 2022 at 14:11
  • $\begingroup$ @IanBush, Thanks for the comment, Yes that fixes the issues, but I am bit curious, If there is any way to do it efficiently. I will remove the context so that I would be a more general question. $\endgroup$ Oct 12, 2022 at 18:28
  • 3
    $\begingroup$ There is no efficient way to do it - a matrix transpose for a distributed object is just not a good thing to do on a distributed memory machine. If you have to do it in scalapack pzgeadd is a simple way to do it, just set beta to zero - intel.com/content/www/us/en/develop/documentation/… On any decent number of nodes the overhead due to the addition will be utterly insignificant compared to the communications cost. Write your algorithm not to need the explicit transpose. $\endgroup$
    – Ian Bush
    Oct 12, 2022 at 19:28
  • 1
    $\begingroup$ @IanBush Thanks again, That answers the question perfectly. If possible, please post this as an answer so that I can accept. $\endgroup$ Oct 12, 2022 at 20:07

1 Answer 1

2
$\begingroup$

Unfortunately there really is no efficient way to do it; transposing a matrix stored as a distributed object on a distributed memory machine is just inherently a bad thing to be doing as it potentially requires a many to many communication which can never scale. This is the reason behind the poor scalability of the standard implementation of FFTs on distributed memory architectures.

The best way to do it is therefore to write the implementation of your algorithm in such a way that it never needs to explicitly form the transpose. Clever use of the transpose options in (P)BLAS and (Sca)LAPACK can often achieve this. However if you must do it for a ScaLAPACK matrix an easy option is to use the p*geadd family of functions, setting the beta argument to zero.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.