3
$\begingroup$

If you call a library like LAPACK or BLAS (which are written in FORTRAN and use column major order) from a C-like language that uses row major order, won't you lose performance and use a lot of memory due to the creation of transposed matrices? Tests are best, of course, but can anyone with experience tell me whether optimized libraries still improve performance, even after that overhead?

$\endgroup$
2
  • $\begingroup$ I haven't tested the performance penalty, but you might use a library or a toolkit that hides away memory layout. E.g. in NumPy (relevant parts written in C), you can choose the memory layout on array object creation. $\endgroup$
    – AlexE
    Commented Aug 12, 2014 at 7:00
  • $\begingroup$ If I am not mistaken, BLAS and LAPACK have a "transpose" argument that tells if you want the operation done with $A$ or with $A^T$. So you can effectively pass matrices to BLAS either in row-major or column-major order, you just have to specify it. No explicit transpositions are performed. Of course, some algorithms might perform better with row-major and some with column-major storage, but this seems a different issue. $\endgroup$ Commented Aug 12, 2014 at 12:38

3 Answers 3

3
$\begingroup$

Many of these libraries have C interfaces that swap the meaning of the ordering internally without swapping the data. Also, using C-style double pointers for 2-D matrices is probably the wrong choice in the first place (due to the double lookup), so you can make your data appear column-major by indexing into a linear C array or use libraries that work around these issues.

$\endgroup$
1
  • $\begingroup$ It turned out that the LAPACK port I was using, CLAPACK, didn't have that option but a more recent one called LAPACKE did. $\endgroup$
    – Emre
    Commented Aug 14, 2014 at 22:25
2
$\begingroup$

You can use the identity C^T = (AB)^T = B^T A^T to use BLAS to compute on row-major arrays from C.

In any case, the overhead of transposition is usually not a bottleneck compared to O(N^3) BLAS3 or LAPACK routines, but I should note that I am oft to point out counterexamples from my own research.

$\endgroup$
0
$\begingroup$

Straight from the horse's mouth (https://www.netlib.org/lapack/lapacke.html#_array_arguments),

Note that using row-major ordering may require more memory and time than column-major ordering, because the routine must transpose the row-major order to the column-major order required by the underlying LAPACK routine.

So yes, at least for LAPACKE, there is a performance penalty if you use a layout that requires transposition internally. I've confirmed by testing dgels with some old version of OpenBLAS and indeed, two extra allocations occur when using row-major layout. I couldn't get a good measurement of how it impacts performance.

On the other hand, for OpenBLAS' CBLAS interface, no extra allocations occur for any of the configurations.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.