0
$\begingroup$

I have a numerical code which is MPI+OpenMP (hybrid) parallelized and an available computational resource of 32 nodes with 16 cores on each node. The code has been tested for MPI scalability up to 16 cores on different nodes and OpenMP scalability on 16 cores on a single node.

Can I assume the hybrid parallelization to be scalable on $32\times 16$ cores?

Edit: MPI+OpenMP is done in such a way that MPI processes are launched at the beginning itself and shared memory parallelization is done within each process using a few PARALLEL DO loops.

Improve this question
$\endgroup$
1
  • $\begingroup$ Do any of the threaded sections of the code also include MPI calls, or are the two logically separated? $\endgroup$
    – origimbo
    Apr 28, 2016 at 14:23

2 Answers 2

Reset to default
6
$\begingroup$

No. You need to test it to that scale, especially if you have MPI calls within OpenMP regions.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Since there is a restriction on total CPU time used by different users in the facility I am availing, it will be difficult to perform a test to the scale. $\endgroup$
    – boltz
    Apr 30, 2016 at 6:11
  • $\begingroup$ a) They don't all have to fit in one job. b) If you do a strong scaling test so that the problem sized is fixed globally, then the run time should get shorter as you scale up. $\endgroup$
    – Bill Barth
    Apr 30, 2016 at 13:51
2
$\begingroup$

As a general rule, I would test out the combination of cores and nodes, since scalability depends on so many factors: having enough to do on each node and then core, task / data parallelism in your problem, how well the code is architected to perform MPI communications without interrupting the openMP threads, etc. Most of the times one can avoid MPI calls from within openMP loops, and I have.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.