General understanding of Intel MKL, threads and MPI

Question

Preface

I seem to lack a fundamental understanding of best practise recommendations given by Intels MKL user guides for using MKL in threaded applications. So let's clarify it together.

Wording and the question

There are especially two different ways to optimize numerical code. Either OpenMP or the Message Passing Interface (MPI) - and combinations of both. It seems to me MKL settles with OpenMP internally out of the box. Intel recommends to not combine OpenMP parallelization with a manual parallelization (e.g., a domain decomposition at higher level) with the aid of MPI (see here).

I am confused now about what to do in multi-processor environments (read clusters) with many physical computation units. My first approach towards the topic was:

• Use MPI, do a domain decomposition of the problem, distribute the small chunks to all CPUs in the cluster and use sequential MKL (=MKL without OpenMP) there.

I especially ask myself how it could be done differently when relying only on multi-threaded MKL (=MKL with OpenMP) in the cluster: What if one matrix in a matrix vector product is to large to fit into memory of one compute node? Will it be automatically spread over all compute nodes by MKL?

Answer 1

Nothing stops you from decomposing the problem up yourself and feeding the relevant partitioned data into MKL sequentially, or even in parallel. It will work as long as you avoid data races, but you may experience performance penalties unless you are very careful about how you do it.

The reason it's discouraged to combine OpenMP code with MKL is that OpenMP is generally not equipped to handle the parallel scheduling challenge of nested parallelism (this is slowly starting to change). Thus the overhead of this limitation results in decreased performance.

There is no issue in doing this with MPI however, but you will have to be very careful with how you set process affinities in the MPI ranks and thread affinities in OpenMP, or else you will have again very degraded performance. You should look at the MPI documentation as well as the OpenMP documentation for how to coordinate the two correctly. It's not possible to give general advice here because there is no standardized way yet to handle this - you have to read the documentation of your OpenMP and MPI.

Alternatively, Intel also releases a version of MKL that is threaded with the Threading Building Blocks (TBB) library. This library can very efficiently handle nested parallelism, but then you will have to use TBB to achieve this (while it is possible to mix OpenMP and TBB, it is not recommended). This means you can call MKL within TBB parallelized code and experience less performance penalty than if you had done the same within OpenMP.

Edit: I should add that MKL is strictly focused on single node performance, thus any extra parallelism you add to it in the form of OpenMP or MPI must be managed yourself. It will not distribute matrices for you nor perform block linear algebra. You will have to block the matrices yourself and perform the correct sequence of BLAS or LAPACK calls to achieve the result you want.