2
$\begingroup$

Short version of question: We have a Opteron 6272 cluster, an we have recently started bilding FX 9370 based small cluster. Problem is that we are not able to get desired speed, i.e scaling with number or processes is not optimal.

Full story: When we ran CPMD on Opteron 6272 as a single process, on a single node (hence without any parallel component) we got execution time of say 44 sec. when same thing was repeated on FX 9370 time to finish up job was ~27 sec. but as number of processes are increased (max=8) opteron 6272 scales up very nicely as compared to 9370 and at towards the end manages to just better the 9370 performance.

As processor is almost twice as fast (as given by single process performance), shouldn't i expect similar trend in parallel performance?

MPICH2 was compiled with ACML, and x86_open64 compilers from amd to avoid any bottleneck. preformance with intel compilers was also same.

Here is the link to Intel MPI benchmarking results. http://pastebin.com/keACK8Qz

Improve this question
$\endgroup$
2
  • $\begingroup$ MPICH does not make any BLAS/LAPACK calls so the use of ACML is irrelevant here. It is certainly relevant to CPMD though. $\endgroup$
    – Jeff Hammond
    Commented Feb 22, 2014 at 16:38
  • $\begingroup$ We tried with Intel mkl also. But results were same $\endgroup$
    – ipcamit
    Commented Feb 22, 2014 at 17:09

1 Answer 1

Reset to default
2
$\begingroup$

The single-node performance of all of the MPI stacks I know of has always been very good. They all used shared memory behind the scenes to move data around between tasks. Your IMB results look fine.

These are two very different processors, and you may be getting killed by the difference in memory bandwidth available to you. The 9370 is a desktop processor, and the 6272 is designed for servers. I haven't done the complete research, but it's very possible that while a single core is better on the 9370, you've run out of memory bandwidth by the time you get to 8 cores.

Less likely but also possible:

  • There's some problem with task pinning that preventing optimal layout of the tasks on the node
  • Frequency scaling under Linux is turning down the clock frequency more on the 9370 as you scale up to conserve power or keep the processor cool

Can you post your CPMD results as an update?

Improve this answer
$\endgroup$
6
  • $\begingroup$ Thank you for replying Bill. Scaling by linux kernel should not be an issue as earlier while trying to install ATLAS math library I disabled it. Now it runs on single frequency, thought i will check once more. Can you be bit more specific about CPMD results (which)? Here is list of time taken by some simulation with respect to number of processes. (On same as well as different nodes, connected by Gbit ethernet). pastebin.com/8m2JXFMR $\endgroup$
    – ipcamit
    Commented Feb 14, 2014 at 4:14
  • $\begingroup$ Thanks, but I wanted to compare the 9370 to the 6272 since you said the 6272 was faster. Maybe I'll see something in the data. With the data we have here, I'm having a hard time understanding why the times are getting longer as you add cores. They should either get shorter (with a fixed size problem) or stay nearly flat (with a fixed size problem per core). Are you adding together the times of various tasks within each run? $\endgroup$
    – Bill Barth
    Commented Feb 14, 2014 at 4:35
  • $\begingroup$ Sorry for the delay. I deleted my earlier comment to avoid any confusion. Anyway... yes the time give above are time take in completing each calculation as provided by "time mpirun..." command. Please find below the detailed set of timing on Opteron 6272. I have provided all the data regarding communication from the obtained output file. Please let me know if u want anything more pastebin.com/jg2i09cv $\endgroup$
    – ipcamit
    Commented Feb 14, 2014 at 15:35
  • $\begingroup$ It would be very helpful if you put the timings for the data in your original question in a format where the results for both cards was comparable side-by-side. These runs you have just posted contemplate thousands of seconds, and the times in your original question is tens of seconds. I can't understand how to make a comparison with what you've given. $\endgroup$
    – Bill Barth
    Commented Feb 14, 2014 at 19:18
  • $\begingroup$ I apologize profusely for the delay, but recently there were certain serious problems with opteron cluster so i will upload the required data as soon as possible. Thank you for your assistance so far. sorry again $\endgroup$
    – ipcamit
    Commented Feb 17, 2014 at 4:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.