3
$\begingroup$

I apologise if this is somewhat of a rookie question. So, from my understanding, on a GPU board, far more of the space is allocated to ALUs compared to CPUs which have far more cache available. This should mean that a typical GPU is capable of higher FLOP counts. This I understood to be the advantage of a GPU (in a nutshell). However, looking at the numbers for some of the state of the art processors on the market, Nvidia's 3090 card outputs around 0.5 TFLOPS (double precision) at \$1499, while Intel's Xeon 3175-X processor outputs 1.5 TFLOPS (also double precision I presume) at \$2999. So the CPU is providing higher double precision FLOP count per dollar.

I find these figures to be a bit confusing. What exactly is the advantage of the GPU if not in overall (or effective) FLOP counts? Are GPUs only advantageous when performing single or half precision operations?

source for 3090 FLOP counts: https://en.wikipedia.org/wiki/GeForce_30_series

source for Xeon FLOP counts: https://www.intel.com/content/dam/support/us/en/documents/processors/APP-for-Intel-Xeon-Processors.pdf

Improve this question
$\endgroup$
4
  • $\begingroup$ Please clarify your specific problem or provide additional details to highlight exactly what you need. As it's currently written, it's hard to tell exactly what you're asking. $\endgroup$
    – Community Bot
    Feb 14, 2022 at 14:41
  • 2
    $\begingroup$ I think my question is pretty clear. Could down voters please comment what part of it they find objectionable? $\endgroup$
    – Enforce
    Feb 14, 2022 at 14:42
  • 2
    $\begingroup$ Compare in single precision. Nvidia cripples the double precision performance on all but the most expensive Tesla GPUs. $\endgroup$
    – Brian Borchers
    Feb 14, 2022 at 18:00
  • 2
    $\begingroup$ @Enforce RTX 3090 is a consumer-grade GPU that is not designed for high double-precision performance. NVIDIA makes a different line of professional-grade GPUs (formerly the Tesla brand) for HPC applications requiring high double-precision performance. Current models would be A100 PCIe or A100 SXM4 (also available with 80 GB of memory). $\endgroup$
    – njuffa
    Feb 14, 2022 at 21:23

2 Answers 2

Reset to default
3
$\begingroup$

As others said in the comments, the RTX 3090 was designed for single-precision performance primarily for gaming or other single-precision uses. In which case, its TFLOP count is around 35 TFLOPS. Compared to Xeon's 1.5 TFLOPs, it's blown out of the water.

Let's give Xeon a little headroom and say you could get 3 TFLOP performance for a single-precision algorithm. This matches the performance of my laptop's GTX 1650, which boasts around 3.5 TFLops of single-precision computing power. Yet this GPU was $300 or so, compared to Xeon's huge costs. I've personally benchmarked my laptop GPU (the 1650) vs some Xeon I had access to on batched small-matrix diagonalization, and found about a 2.5x speed increase between a single-precision GPU algorithm and its double-precision equivalent run on the Xeon.

Make sure to compare apples to apples. Intel Xeons are professional/cluster-grade CPUs, and you wouldn't find those in a typical laptop/desktop, whereas 3090s are for casual use. As many have said, you should instead compare to the professional-grade GPUs NVIDIA charges a boatload more money for, which indeed have 20 TFLOPs+ performance for double precision.

Personally, I am happy with single-precision for many of my physics simulations, so an RTX 3090 (or even 3080, with 29 TFLOPs of single-precision compute) is like a supercomputer in its own right, with the compute power of like 16 Xeons. Except communication is faster on a single GPU than 16 nodes!

What I haven't seen are the TFLOPs for using Tensor Cores. In many computational science cases, matrix multiplication is a key element, and you can then use tensor cores. That will mean the RTX 3090 elevates to over 100 TFLOPs (if all your time is spent on the matrix multiplications).

Improve this answer
$\endgroup$
3
$\begingroup$

Dollars per FLOP isn’t really the question, it’s dollars per solution. Estimate or measure both if you can, and buy the best you can afford that gets you the best solutions in good time. Either use your estimates or find somewhere to run benchmarks. You may find a computing center which has all the kinds of architectures you want that you could use for free or cheaper than buying a couple of nodes for yourself. Depending on eligibility requirements, you might be able to use what you need at our site for free.

Improve this answer
$\endgroup$
5
  • $\begingroup$ Could you expand to explain how that changes the picture? One could expect that the FLOPs-per-solution ratio is constant, as a first approximation. $\endgroup$
    – Federico Poloni
    Feb 15, 2022 at 8:06
  • 1
    $\begingroup$ When I said “solution” above, I meant whole answer for whatever problem you’ve set yourself. That might be one matrix inversion of a certain size or next week’s weather. You probably can’t afford the latter computer, so spend your money with a cloud provider or get your cycles for free from a computing center like mine. If your estimate says you can do a whole-virus simulation on something you can buy and run, buy it! I ended up spending an extra 1.5 years in grad school because we were running our own small cluster. Then I ended up working for the computing center for way more money. $\endgroup$
    – Bill Barth
    Feb 15, 2022 at 14:20
  • 1
    $\begingroup$ Flops/solution is not constant. Different algorithms may do very different numbers of floating point operations. Furthermore, parallel algorithms on different hardware can vary dramatically in their ability to make use of the peak performance of the hardware. $\endgroup$
    – Brian Borchers
    Feb 16, 2022 at 18:29
  • $\begingroup$ @BrianBorchers, of course, different initial guesses could change how many FLOPS are required to get a solution, too. My biased opinion is to spend your money on staff and get compute for free from a center who’s getting paid to give away cycles for science. Most regions have such centers. $\endgroup$
    – Bill Barth
    Feb 16, 2022 at 19:04
  • $\begingroup$ @BillBarth I'm entirely in agreement with you- my comment was a response to Frederico's comment that "One could expect that the FLOPs-per-solution ratio is constant". $\endgroup$
    – Brian Borchers
    Feb 16, 2022 at 19:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.