31
$\begingroup$

I have been looking into C++ linear algebra libraries for a project I've been working on. Something that I still don't have any grasp on is the connection of BLAS and LAPACK to other linear algebra libraries.

Looking through this article on linear algebra libraries I found it interesting that:

  • some libraries are independent from BLAS and LAPACK
  • some require BLAS and LAPACK
  • some have optional interfaces to BLAS and LAPACK
  • and, as I understand it, you can use BLAS and LAPACK to solve linear algebra problems directly

I can imagine that some libraries are simply C++ interfaces to BLAS and LAPACK libraries written in C and Fortran and others have implemented their own substitute routines, but

  1. What are the implications of the optional interfaces to BLAS and LAPACK? What are you loosing by opting out, and what are the libraries doing instead?

  2. Do any of the libraries provide more than just an interface? For example, UMFPACK is written in C and has optional interfaces to BLAS and LAPACK. What can UMFPACK (or other libraries) do that BLAS and LAPACK can't on their own?

Improve this question
$\endgroup$

3 Answers 3

Reset to default
17
$\begingroup$

As far as I know, Lapack is the only publicly available implementation of a number of algorithms (nonsymmetric dense eigensolver, pseudo-quadratic time symmetric eigensolver, fast Jacobi SVD). Most libraries that don't rely on BLAS+Lapack tend to support very primitive operations like matrix multiplication, LU factorization, and QR decomposition. Lapack contains some of the most sophisticated algorithms for dense matrix computations that I don't believe are implemented anywhere else.

So to answer your questions (at least partially),

  1. By opting out of BLAS/Lapack, you are typically not missing functionality (unless the optional interface was designed so that there is no substitute implementation, which is rare). If you wanted to do very sophisticated operations, those other libraries probably don't implement it themselves anyways. Since BLAS can be highly tuned to your architecture, you could be missing out on huge speedups (an order of magnitude speed difference is not unheard of).

  2. You mention UMFPACK, which is for sparse matrix factorization. BLAS/Lapack is only concerned about dense matrices. UMFPACK at some level needs to work on medium size dense problems, which it can do using custom implementations or by calling BLAS/Lapack. Here the difference is only in speed.

If speed is of great concern, try to use a library that supports optional BLAS/Lapack bindings, and use them in the end when you want things faster.

Improve this answer
$\endgroup$
5
$\begingroup$

  1. Good implementations of the BLAS and LAPACK routines (most importantly the BLAS routines) can be much faster than naive straight forward implementations of the same functions. However, efficient implementations typically include optimizations that are very specific to the particular computer that you're running on. Even different models of processors from the same manufacturer (e.g. Intel x86-64 processors) often require very different code to get good performance. By supplying optimized BLAS/LAPACK libraries to a software package, you can typically speed up the code compared to using unoptimized routines. However, since many casual users may not have the expertise to install optimized routines, it's common to also provide an option to use generic unoptimized linear algebra routines.

  2. UMFPACK is a library of routines for linear algebra on sparse matrices (matrices with a high proportion of 0 entries.) It can use BLAS/LAPACK to handle dense matrices (or dense blocks within matrices) that it encounters.

Improve this answer
$\endgroup$
1
$\begingroup$

Short version: they are libraries written primarily in Fortran that are used for Numerical operations in many languages - even some C programs due to their sheer speed and optimizations; They are also some of the only open source implementations of many algorithms :)

You don't have to use libraries unless they have dependencies; iirc most of those are pretty independent, and you can always write your own math functions, such as better vectorized ones for your architecture

Improve this answer
$\endgroup$
6
  • 1
    $\begingroup$ Most optimized BLAS and LAPACK routines have long since ceased being written Fortran. The fastest BLAS and LAPACK routines are not generally the ones downloaded from netlib. The vendors of most processors sell or distribute optimized versions of the BLAS and LAPACK designed specifically for their chips. $\endgroup$
    – Bill Barth
    Jul 25, 2013 at 11:12
  • $\begingroup$ Sorry I mean the ones distributed on the net - the general ones; the chip specific ones of course aren't as portable: i.e. some of Intel's vector tricks don't work that well on AMD chips, which is why they have a disclaimer about the library. And I'm pretty sure those are rebranded under another name no? $\endgroup$
    – Eiyrioü von Kauyf
    Jul 25, 2013 at 13:58
  • $\begingroup$ @BillBarth BLAS definitely but are you sure that LAPACK routines are also rewritten? Afaik as long BLAS 3 is highly performing (or multithreaded) then its all good. $\endgroup$
    – stali
    Jul 25, 2013 at 19:04
  • 1
    $\begingroup$ I think of the BLAS and LAPACK as the names of the functions and the interface. There are lots different implementations (ACML, ESSL, MKL, ATLAS, etc.). $\endgroup$
    – Bill Barth
    Jul 26, 2013 at 2:19
  • 1
    $\begingroup$ those are just interfaces .. .like ATLAS they're not the actual library. $\endgroup$
    – Eiyrioü von Kauyf
    Jul 26, 2013 at 3:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.