I seem to lack a fundamental understanding of best practise recommendations given by Intels MKL user guides for using MKL in threaded applications. So let's clarify it together.

Wording and the question

There are especially two different ways to optimize numerical code. Either OpenMP or the Message Passing Interface (MPI) - and combinations of both. It seems to me MKL settles with OpenMP internally out of the box. Intel recommends to not combine OpenMP parallelization with a manual parallelization (e.g., a domain decomposition at higher level) with the aid of MPI (see here).

I am confused now about what to do in multi-processor environments (read clusters) with many physical computation units. My first approach towards the topic was:

  • Use MPI, do a domain decomposition of the problem, distribute the small chunks to all CPUs in the cluster and use sequential MKL (=MKL without OpenMP) there.

I especially ask myself how it could be done differently when relying only on multi-threaded MKL (=MKL with OpenMP) in the cluster: What if one matrix in a matrix vector product is to large to fit into memory of one compute node? Will it be automatically spread over all compute nodes by MKL?

  • 1
    $\begingroup$ I fail to see in your link to the Intel website where it says anything that implies that you should not combine OpenMP with MPI. Could you specify what passage on that page you're referring to? $\endgroup$ – LedHead Feb 18 '18 at 17:55
  • $\begingroup$ @led23head Maybe I got Intel wrong but the sentence Your application is intended to be run on a single thread, like a message-passing Interface (MPI) application seems to suggest that nested parallelism by hybrid MPI+OpenMP is not recommended. Might be wrong, though. $\endgroup$ – pbx Feb 18 '18 at 22:04
  • $\begingroup$ It's saying that you should almost always use the threaded library. There are some special cases in which you might want to use the sequential library, and one of them might be if your code is MPI parallelized. I think it's talking about the case in which you plan to have multiple MPI processes running on each compute node. It's not saying you shouldn't do OpenMP inside MPI. $\endgroup$ – LedHead Feb 19 '18 at 0:30

Nothing stops you from decomposing the problem up yourself and feeding the relevant partitioned data into MKL sequentially, or even in parallel. It will work as long as you avoid data races, but you may experience performance penalties unless you are very careful about how you do it.

The reason it's discouraged to combine OpenMP code with MKL is that OpenMP is generally not equipped to handle the parallel scheduling challenge of nested parallelism (this is slowly starting to change). Thus the overhead of this limitation results in decreased performance.

There is no issue in doing this with MPI however, but you will have to be very careful with how you set process affinities in the MPI ranks and thread affinities in OpenMP, or else you will have again very degraded performance. You should look at the MPI documentation as well as the OpenMP documentation for how to coordinate the two correctly. It's not possible to give general advice here because there is no standardized way yet to handle this - you have to read the documentation of your OpenMP and MPI.

Alternatively, Intel also releases a version of MKL that is threaded with the Threading Building Blocks (TBB) library. This library can very efficiently handle nested parallelism, but then you will have to use TBB to achieve this (while it is possible to mix OpenMP and TBB, it is not recommended). This means you can call MKL within TBB parallelized code and experience less performance penalty than if you had done the same within OpenMP.

Edit: I should add that MKL is strictly focused on single node performance, thus any extra parallelism you add to it in the form of OpenMP or MPI must be managed yourself. It will not distribute matrices for you nor perform block linear algebra. You will have to block the matrices yourself and perform the correct sequence of BLAS or LAPACK calls to achieve the result you want.

  • $\begingroup$ Thanks for the very detailed analysis of my rather general and less specific question. Especially your ´Edit´ section helps clarifying. So the best way (or rather: a pretty senseful approach) for maximum performance on clusters seems to be sequential MKL and manual domain decompositions by means of MPI? Provided that data races in MPI are omitted. $\endgroup$ – pbx Feb 18 '18 at 22:18
  • 1
    $\begingroup$ If you're on a cluster, then the simplest approach to get very good performance out of MKL would be to use one MPI rank per node ad use the OpenMP version of MKL within that node. The reason is that MKL is highly tuned for using the full node. There are other strategies that are equally valid, but more complex to execute - such as using one MPI rank per socket or one MPI rank per NUMA zone. Simply setting the number of ranks equal to the number of nodes and the number of OpenMP threads equal to CPU cores per node though should result in decent performance of the MKL library. $\endgroup$ – Reid.Atcheson Feb 18 '18 at 22:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.