CPU benchmarks available online mostly focus on desktop apps/games and rarely on serial/parallel numerical kernels, specially sparse ones (e.g., MatMult). Some benchmarks like NAS/SciMark exists but are rarely used by popular benchmarking sites on lastest/greatest CPUs.

When it comes to codes that use sparse/dense numerical libraries such as PETSc/ScaLAPACK how does one decide which machine to buy/use specially w.r.t. to multicore performance? E.g. would I be better off with an AMD FX8350 or a Intel Core i7-3770K as my Desktop or should I request allocation on a Xeon or Opteron based cluster, specially for small/medium (i.e., less than 4000 cores) type jobs where one has more options.

I know from personal experience that Intel chips have performed significantly better in the last few years but what about the latest AMD stuff?

EDIT: I am specifically interested in PETSc's MatMult kernel (for unstructured FE matrices) which I have been told is memory bound. Performance in terms of percentage of peak FLOPS using all cores I think would be useful.

  • $\begingroup$ please expand a little more about exactly what codes you intend to be running and some more of their performance characteristics. For example, there's a huge difference between the type of machine I'd recommend for somebody with a non-scalable, memory-bound sparse PDE code that hasn't been ported beyond x86 vs somebody with a scalable dense problem that has been ported to CUDA+MPI and is floating-point bound. $\endgroup$ Nov 1, 2012 at 16:26
  • $\begingroup$ Yes it is a sparse PDE based code that can use both iterative and sparse direct solvers (e.g., MUMPS) via PETSc and I am interested in AMD's latest offerings (Opteron 62XX and FX8350 on the Desktop). $\endgroup$
    – stali
    Nov 1, 2012 at 16:38
  • $\begingroup$ please update your question, title and tags. I'd do it for you but I know you know how to edit questions :) $\endgroup$ Nov 1, 2012 at 17:39
  • $\begingroup$ Specifically, I know the PETSc developers follow the petsc tag, so it's helpful to put it on questions that relate to PETSc (and they know quite a bit about the performance of various architectures). The final thing to note is what the fastest known solvers are for your problem (direct/multigrid etc.) and if you are targeting desktop or cluster performance (and if a cluster/supercomputer, at what size problem). $\endgroup$ Nov 1, 2012 at 17:41
  • $\begingroup$ Have you looked at spec.org/cpu2006 ? They have floating point and integer benchmarks for simulation codes on tons of different hardware. I don't know if any of the benchmarks use PETSc, but you may find one that does similar work to what you are considering. $\endgroup$
    – tpg2114
    Nov 1, 2012 at 18:19

3 Answers 3


First, sparse direct is completely different from sparse iterative. You cannot reliably predict performance if you don't have a good understanding of what your code is doing. For sparse MatMult, MatSolve, MatSOR, and similar kernels, you have an arithmetic intensity of no more than 1 flop/4 bytes of memory bandwidth. Meanwhile, most recent multicore chips can perform about 5 flops/byte. This means that even if cache is reused perfectly, you cannot get better than $1/20 = 5\%$ of floating point peak. This is a hard barrier that no amount of "tuning" can circumvent. In practice, it is very important to order your matrices so that cache is reused well. If you choose an ordering that produces poor reuse, the realized peak will be several times lower. In practice, with decent orderings and sufficiently large subdomains, PETSc sparse matrix kernels frequently get $80\%$ of memory bandwidth peak or better. Look at the STREAM benchmarks to determine the realizable bandwidth. Some machines, such as Ranger at TACC, cannot realize better than $50\%$ of peak bandwidth.

If you resort to assembled sparse matrices to be solved using iterative methods, there is absolutely no point buying cores. Buy memory bandwidth. I recommend testing algorithms with assembled matrices, then switching to mostly-unassembled methods (perhaps matrix-free multigrid), when appropriate, if you want to get high machine utilization. A better choice of algorithm can give an order of magnitude speedup in so many cases that there's no point micro-tuning a suboptimal method.

  • $\begingroup$ This is very helpful because the STREAM benchmarks are fairly common. $\endgroup$
    – stali
    Nov 2, 2012 at 1:58

There are benchmarks that are derived from typical scientific computing codes doing sparse matrix vector stuff in SPEC CPU 2006, see http://www.spec.org/cpu2006/ . In particular, the 447.dealII benchmark in this suite derives from the deal.II library and spends most of its time in exactly the kind of operations that you describe for your workload. The advantage of this is that you can query the SPEC database for the peak rates on this benchmark to see how various processors (variants) perform on this one benchmark. Pretty much every CPU built over the past 10 years is in that database.

(Disclaimer: I'm the author of 447.dealII.)


Raw benchmarks are correlated with performance for certain, but to have a real answer about what is optimal for your particular problem, you need to profile your code on the system in question. IMHO it is impossible to say authoritatively without some actual performance testing on your code.

That said, as mentioned in the question thread, most matrix/vector operations are memory bound, and become fairly dependent on so many factors, like structure of the matrix, how your compiler optimizes memory access, cache-hits, ram latency, OS foolery, the list goes on and on. I think it is very hard to make any substantive claims without actual testing on the system in question.

I have had similar experience with Intel's hardware as you mention. Especially when using the icc tool-chain, as it seems to really know how to write machine code that uses the full capabilities of the underlying metal.

  • $\begingroup$ I would love to SSH into a machine but the question is where to find one with the latest/greatest CPU. I think most users dont have access to the newest stuff. Lets say you're requesting CPU time on a cluster and you need to choose one that uses Xeon vs one that uses Opteron or maybe you're buying a Desktop. Now what will you do. Though I do understand that benchmarking is not an easy task given all the factors that you mention. $\endgroup$
    – stali
    Nov 1, 2012 at 20:41
  • $\begingroup$ Also as far as sparse MatMult goes I havent seen a difference of more than 35% (on matrices arising from unstructured low order FE code) across GNU, Intel, Open64, Cray, PGI and Solaris Studio compilers (even with different opt switches). However I have seen a Xeon that is about twice as fast over older Opterons. Though I am not sure about the new ones which really led to the question. $\endgroup$
    – stali
    Nov 1, 2012 at 20:52
  • $\begingroup$ Most computing centers that allow you to apply for time allow you to apply for a "startup" or "benchmarking" allocation in order to see how your code performs. We do this so that folks can write the best-informed proposals possible. $\endgroup$
    – Bill Barth
    Nov 1, 2012 at 22:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.