1
$\begingroup$

As of 2015, what shared memory parallel computing solutions are available?

What are the advantages and disadvantages of each for various use cases arising in high performance scientific computing?

I am primarily interested in things that work with C or C++, similar to OpenMP or Threading Building Blocks; or perhaps other low-level languages that can be a reasonable alternative to C++ for scientific computing (e.g. Fortran or some of the new languages designed for parallel scientific computing, like Chapel, though I'm not sure if any of these have reached a usable quality already).

I am looking for a brief high level overview that could be useful for a beginner to decide which direction to follow for his/her particular use case.

Improve this question
$\endgroup$
5
  • $\begingroup$ What's your use case? Focusing on shared-memory models may limit your problem size unnecessarily. $\endgroup$
    – Bill Barth
    Jan 28, 2015 at 19:27
  • $\begingroup$ @BillBarth This is a question specifically about shared memory solutions, not about one particular use case. $\endgroup$
    – Szabolcs
    Jan 28, 2015 at 19:46
  • $\begingroup$ I think the question is too vague as it stands and likely to draw more opinions that solid answers. Having the ability to understand your needs will help people guide you in the right direction. Some programming models may be better suited to certain use cases than others. Also, if you ever need to work with others in the future, using something esoteric may limit that ability considerably. $\endgroup$
    – Bill Barth
    Jan 28, 2015 at 20:11
  • $\begingroup$ @Bill No, I believe you misunderstood the question. Imagine you don't know at all what options you have for parallelization. You heard about OpenMP. Is there anything else people use at all? If yes, what? Are they direct alternatives or are they completely interchangeable? See e.g. here. The question is not vague at all. Also I'm only interested in high-profile solutions only not every little obscure library with only 1-2 users. E.g. if someone asked what are the main options for GPU computing, CUDA and OpenCL would come up. $\endgroup$
    – Szabolcs
    Jan 28, 2015 at 20:16
  • $\begingroup$ @Bill I tried to phrase the question so that it will be useful not just for me but others too. But what I personally want to find out: many people use OpenMP and I understand when it's an appropriate choice. Are there any alternatives at all? TBB seems to be one (is it?). Are there others? The second part of the question: how do the alternatives compare? A possible answer might be: "if you do scientific computing, you never need to look at anything else than OpenMP". But I don't think that's the case. Maybe I'm wrong though. So I asked. $\endgroup$
    – Szabolcs
    Jan 28, 2015 at 23:26

1 Answer 1

Reset to default
3
$\begingroup$

You have two basic choices with a distant third. Everything else is not really advisable in my opinion right now. You can use Pthreads (and Boost::Thread) if you want to do 100% of the work yourself. You will have to write your own barriers and reductions, generate thread-local storage, and generally do all the heavy lifting. With all that work, comes all of the control you could want.

OpenMP gets rid of most of these problems by hiding most of the details. If you need to do both task parallelism and worksharing, you'll need a more modern implementation. GCC is coming along, but I haven't compared its completeness or performance to the Intel compiler in a long time. The Intel compiler's OpenMP implementation is traditionally pretty good. With OpenMP you get parallel loops, tasks, atomic/master/single regions for free which seem to cover 99% of scientific computing needs (where that means PDE and ODE simulation and linear algebra). There are lots of libraries (like the Intel MKL) that use/cooperate with OpenMP, so that's an advantage, too. With OpenMP 4.0, you get offload to accelerators, though I think that Intel is the only one supporting that and only for Xeon Phi.

TBB, while now open sourced, is a much less used and not standardized. That being said, it is very powerful, has many OpenMP-like constructs, and is very C++y. However, if you try to share your TBB code with anyone, they may not be able to help you since it's much less familiar.

All of the other things I can think of have less adoption, are not robust, have poor programmability, or have poorer performance. My list for that includes OpenCL (on the CPU not GPU), Haskell, Julia, Chapel, X10, UPC, Co-Array Fortran, Python Multiprocessing, etc., etc.

Unless you're a language nerd who likes explore new languages for fun, I wouldn't recommend writing a scientific app that needs to see the light of day, have performance, and use a library in anything but C/C++/Fortran with OpenMP for threading. When it comes to traditional scientific computing, there are too many libraries out there that expect a C/threads interface to waste your time reinventing the wheel.

Improve this answer
$\endgroup$
5
  • $\begingroup$ When/why would I choose "raw threads" (like pthreads) instead of something like OpenMP? I didn't consider this choice at all. I'm assuming typical scientific computing applications where usually all threads do the same thing (on different data). $\endgroup$
    – Szabolcs
    Jan 29, 2015 at 2:02
  • $\begingroup$ When you want some element of control that's too hard to get with OpenMP, usually. $\endgroup$
    – Bill Barth
    Jan 29, 2015 at 2:06
  • $\begingroup$ Can you give an example of something like this? $\endgroup$
    – Szabolcs
    Jan 29, 2015 at 2:14
  • $\begingroup$ Single program with long-running service processing incoming connections in one thread and dealing with I/O or the database in another. I can't think of a more science-y app off the top my head. $\endgroup$
    – Bill Barth
    Jan 29, 2015 at 3:07
  • $\begingroup$ I list a few scienc-y areas where using threads (and in fact the TBB) make sense in my lectures here (39,30): math.tamu.edu/~bangerth/videos.html $\endgroup$
    – Wolfgang Bangerth
    Jan 29, 2015 at 5:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.