# How to properly calculate CPU and GPU FLOPS performance?

## Problem

I'm trying to calculate CPU / GPU FLOPS performance but I'm not sure if I'm doing it correctly.

Let's say we have:

• A Kaby Lake CPU (clock: 2.8 GHz, cores: 4, threads: 8)
• A Pascal GPU (clock: 1.3 GHz, cores: 768).

This Wiki page says that Kaby Lake CPUs compute 32 FLOPS (single precision FP32) and Pascal cards compute 2 FLOPS (single precision FP32), which means we can compute their total FLOPS performance using the following formulas:

CPU:

TOTAL_FLOPS = 2.8 GHz * 4 cores * 32 FLOPS = 358 GFLOPS


GPU:

TOTAL_FLOPS = 1.3 GHz * 768 cores * 2 FLOPS = 1996 GFLOPS


## Questions

1. [SOLVED] Most of the guides I've seen (like this one) are using physical cores in the formula. What I don't understand is why not use threads (logical cores) instead? Weren't threads created specifically to double the floating point calculations performance? Why are we ignoring them then?

2. [SOLVED] Am I doing it correctly at all? I couldn't find a single reliable source for calculating FLOPS, all the information on the internet is contradicting. For the i7 7700HQ Kaby Lake CPU I found FLOPS values as low as 29 GFLOPS even though the formula above gives us 358 GFLOPS. I don't know what to believe.

3. Is there a cross-platform (Win, Mac, Linux) library in Node.js / Python / C++ that just returns all the GPU stats like shading cores, clock, available instruction sets (or FP32, FP64 FLOPS values) so I could calculate the max theoretical performance myself? It's quite ridiculous that we cannot get the FLOPS stats from the CPU / GPU directly, instead we have to download and parse a wiki page to get the value. Even when using C++, it seems (I don't actually know) we have to download the 2 GB CUDA toolkit just to get access to the basic Nvidia GPU information like the amount of cores - which would make it practically impossible to make the app available for others, since no one would download a 2 GB app.

• As a partial answer I believe what you are calling "threads" is a trick that allows for a core to host what looks like two threads at a time (hyper-threading) while only real having one actual physical core to compute with. I am not certain entirely about the details of how Intel did this but I think it has to do with filling in holes in pipelines and such. This will not in principle happen if you are computing something heavy but for a lot of more common use cases for a desktop OS this does make sense. If you are interested in actual compute throughput though this is usually not counted. – Kyle Mandli Nov 17 '20 at 16:15
• @KyleMandli thanks for the clarification, I suppose that makes sense – AlekseyHoffman Nov 17 '20 at 16:27
• One part of the proposed computation is frequency. I assume you are aware that with modern hardware, there is not the frequency. Operating frequency will differ based on temperature and power draw (e.g. most GPUs), or instruction set usage and utilization (e.g. most x86 CPUs), and possibly all of the mentioned factors. – njuffa Nov 17 '20 at 20:42
• You'll have to replace MHz everywhere by GHz. – Wolfgang Bangerth Nov 17 '20 at 21:10
• There's no single "actual" performance. For instance, when multiplying large matrices on Volta GPUs, my "actual" performance is close to theoretical, 90 Tops/second. Meanwhile training resnet-50, it's more like 20 Tops/second -- medium.com/@yaroslavvb/… – Yaroslav Bulatov Nov 17 '20 at 21:43

You can calculate GFLOP rates this way, but the numbers are pretty meaningless on today's hardware:

• Floating point operations require a variable number of clock cycles. An addition is generally cheaper than a multiplication, but each generally takes more than one clock cycle of the 2.8 billion cycles you quite.

• When you have hyperthreading, you have two threads running on one core, but the core will still have only one floating point addition unit and so the two threads can't execute floating point additions at the same time.

• Floating point operations are energy hungry, and energy is converted into heat. When you do a lot of FLOPs, processors overheat and step down their clock frequencies.

• If you use the right instructions, you can do floating point multiply-add (FMA) operations that make a multiplication-and-addition faster than doing these operations separately.

• Similarly, with SIMD instructions, a core can do the same operation on multiple pieces of data at the same time -- say, add four pairs of floating point numbers together, yielding 4 FLOPs at the same time. But this requires having a problem where an algorithm actually requires this to happen, rather than using the results of the first addition in the second one. As a consequence, SIMD instructions only contribute to the speed with which some algorithms can be executed, but not others.

• Most importantly, you will generally want to do operations on data from memory, but moving data from main memory onto the processor takes far far longer than actually doing any operations on the data -- like a factor of 100 longer (order of magnitude). So you generally don't see even a small fraction of the theoretical floating point performance of processors in real applications: generally substantially less than 10% of the theoretical peak performance.

In other words, calculating peak performance has become sort of a meaningless business: It has nothing very much to do with the actual performance of a processor.

• You might also discuss how SIMD floating-point units can increase the theoretical peak performance. – Brian Borchers Nov 17 '20 at 21:59
• Thanks for your input, guys, I understand those points and understand how advanced instructions sets affect floating point performance. I guess I'll just stick with the theoretical max for now. I wish there was at least a formula that would approximate the actual FLOPS performance just from the time it takes for CPU to compute a specific function. – AlekseyHoffman Nov 17 '20 at 22:10
• @AlekseyHoffman There is no formula, just measurements. That's why the TOP 500 list is based on actual measurements of performance, not theoretical top performance. – Wolfgang Bangerth Nov 18 '20 at 18:03
• @BrianBorchers Yes, good idea. – Wolfgang Bangerth Nov 18 '20 at 18:10

Yoy can read in Russian - how to calculate FLOPS.

GHz doesn't show FLOPS. One processor with the same GHz can be much faster than the other with the same GHz.

P.S. gpu-s "rx 590" and very old "r7 250x" have almost the same GHz. But ... this is even not correct to compare their performance)

• Hi welcome to scicomp! In stackexchange is better to have post self-contained (see here ). Please, for improve the post, try to edit the answer with the core information of the article. – Mauro Vanzetto Jan 11 at 9:19