Not really much more to say;

Of several computers available for use, these are two of the larger ones; one has 2x E5-2670, and the other has 4x E5-4640. The problems we're looking at essentially boil down to large matrix diagonalisation problems (i.e. i have been dealing with 125K rank matrices, and in the past have looked at larger ones. Both sparse and non sparse, using PARPACK).

All intuition would tell me that the Quad CPU 4640 should be much faster, but it's not, and I can't work out why at all. Both have hyperthreading enabled, both have 250GB ram.

Could it be that the gains from the parallelisation of PARPACK beyond 16 cores just start to diminish, and are not worth adding the additional processors due to the increase in inter cpu communcation required?

The codes are written in fortran if that makes much difference.

  • $\begingroup$ Hyperthreading usually kills performance for HPC apps, so it's entirely possible that the performance difference between two separate nodes and one shared-memory node is entirely masked by the drop in performance due to the use of hyperthreads. Also, it might matter how you are doing your parallelism. Is it entirely using MPI, or are you doing MPI+OpenMP? $\endgroup$ – Bill Barth May 28 '14 at 18:23
  • $\begingroup$ It's likely that this computation is limited by memory bandwidth effects. What are the memory configurations on the two systems? $\endgroup$ – Brian Borchers May 28 '14 at 18:39
  • $\begingroup$ According to the first reference I checked, the E5-4640 is clocked at 2.4 GHz, vs. 2.6 GHz for the E5-2670. Both have 4 DDR3 memory channels. I'm not too surprised that the extra cores with no extra memory bandwidth don't help with something like this. $\endgroup$ – Brian Borchers May 28 '14 at 20:23
  • 1
    $\begingroup$ @BrianBorchers, there should be additional memory bandwidth added with every socket added since the memory controllers are integrated with the chips. $\endgroup$ – Bill Barth May 28 '14 at 21:31
  • 2
    $\begingroup$ I'm surprised nobody has asked for any quantitative performance numbers. How big is the difference? $\endgroup$ – Victor Liu May 28 '14 at 22:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.