How to properly calculate CPU and GPU FLOPS performance?

Question

Problem

I'm trying to calculate CPU / GPU FLOPS performance but I'm not sure if I'm doing it correctly.

Let's say we have:

• A Kaby Lake CPU (clock: 2.8 GHz, cores: 4, threads: 8)
• A Pascal GPU (clock: 1.3 GHz, cores: 768).

This Wiki page says that Kaby Lake CPUs compute 32 FLOPS (single precision FP32) and Pascal cards compute 2 FLOPS (single precision FP32), which means we can compute their total FLOPS performance using the following formulas:

CPU:

TOTAL_FLOPS = 2.8 GHz * 4 cores * 32 FLOPS = 358 GFLOPS

GPU:

TOTAL_FLOPS = 1.3 GHz * 768 cores * 2 FLOPS = 1996 GFLOPS

Questions

1. [SOLVED] Most of the guides I've seen (like this one) are using physical cores in the formula. What I don't understand is why not use threads (logical cores) instead? Weren't threads created specifically to double the floating point calculations performance? Why are we ignoring them then?

2. [SOLVED] Am I doing it correctly at all? I couldn't find a single reliable source for calculating FLOPS, all the information on the internet is contradicting. For the i7 7700HQ Kaby Lake CPU I found FLOPS values as low as 29 GFLOPS even though the formula above gives us 358 GFLOPS. I don't know what to believe.

3. Is there a cross-platform (Win, Mac, Linux) library in Node.js / Python / C++ that just returns all the GPU stats like shading cores, clock, available instruction sets (or FP32, FP64 FLOPS values) so I could calculate the max theoretical performance myself? It's quite ridiculous that we cannot get the FLOPS stats from the CPU / GPU directly, instead we have to download and parse a wiki page to get the value. Even when using C++, it seems (I don't actually know) we have to download the 2 GB CUDA toolkit just to get access to the basic Nvidia GPU information like the amount of cores - which would make it practically impossible to make the app available for others, since no one would download a 2 GB app.

Answer 1

You can calculate GFLOP rates this way, but the numbers are pretty meaningless on today's hardware:

• Floating point operations require a variable number of clock cycles. An addition is generally cheaper than a multiplication, but each generally takes more than one clock cycle of the 2.8 billion cycles you quite.

• When you have hyperthreading, you have two threads running on one core, but the core will still have only one floating point addition unit and so the two threads can't execute floating point additions at the same time.

• Floating point operations are energy hungry, and energy is converted into heat. When you do a lot of FLOPs, processors overheat and step down their clock frequencies.

• If you use the right instructions, you can do floating point multiply-add (FMA) operations that make a multiplication-and-addition faster than doing these operations separately.

• Similarly, with SIMD instructions, a core can do the same operation on multiple pieces of data at the same time -- say, add four pairs of floating point numbers together, yielding 4 FLOPs at the same time. But this requires having a problem where an algorithm actually requires this to happen, rather than using the results of the first addition in the second one. As a consequence, SIMD instructions only contribute to the speed with which some algorithms can be executed, but not others.

• Most importantly, you will generally want to do operations on data from memory, but moving data from main memory onto the processor takes far far longer than actually doing any operations on the data -- like a factor of 100 longer (order of magnitude). So you generally don't see even a small fraction of the theoretical floating point performance of processors in real applications: generally substantially less than 10% of the theoretical peak performance.

In other words, calculating peak performance has become sort of a meaningless business: It has nothing very much to do with the actual performance of a processor.

Answer 2

The FLOP measure for GPU's is supposed to represent the peak theoretical 32b float processing speed by any means necessary. In every modern instance, that means every single shading unit doing as many FMA instructions in parallel as possible.

Now I should mention that the latest GPUs also have dedicated matrix calculation units and even though their theoretical peak far outperforms the FMA theoretical peak, they are typically measured separately since they have a bit of a niche use case and their performance alone doesn't translate well to anything that isn't mostly pipelined matrix operations which is most things, even games.

For CPU's you can employ the same logic - just forcing as many 32b float operations through as possible. In case of modern CPU's that means 512b SIMD FMA. There is a catch however, no consumer processor to date has implemented AVX -512 in a way that the CPU frequency remains at max speed. In reality, using 512b SIMD instructions will make the CPU slow down to around 65% of its max turbo. This still provides some advantage over 256b SIMD instructions (which retain max CPU frequency) but not as much as you'd think.

Another thing to note is that FMA instructions have the pipeline spacing of 1 clock on both modern GPUs and CPUs. The instruction itself may take several cycles to finish but as far as throughput is concerned, you basically get one FMA done per one clock cycle.

Now as for your specific example

1. A 1.3 GHz 768 core GPU will have the max FMA performance of 1.3(GHz) x 768(cores) x 2(because FMA is technically two calculations in one instruction) = 1996.8GFLOPS
2. A 2.8 GHz 4 core CPU with AVX256 will have the max FMA performance of 2.8 x 4 x 2 x 8(because you can do 8 32b instructions with a 256b SIMD instruction) = 179.2GFLOPS

Neat. GPUs typically have 10x more peak theoretical performance so this is rather normal. Now a few words on this "theoretical":

Even though FMA doubles the number for both GPUs and CPUs, neither of them end up doing that very often at all in any kind of consumer application. When doing some cool 3d math, you'll likely be having 9/10 instructions not being FMA.

CPU cores are generally far easier to keep fed. This means that GPU shaders have to be polished to extreme levels to reach anywhere near half of the theoretical peak whereas with CPU's you just need to make sure that you are using SIMD instructions where appropriate (something that modern compilers are pretty good at doing automatically).

CPU's excel at doing things in serial because they have far higher clock speeds and don't get penalized for branching like GPU's do. As any tutorial on writing GPU shaders will tell you about branching - just don't. Avoid whenever possible because when one shading unit in a compute unit branches, all of them do. A basic if/else branch will result in nearly halving the performance within that block whereas 3+ branching possibilities or.. oh my.. nested branching will grind things to an icy halt.

Because of this, programmers can use more analytical algorithms for CPU's which are far more energy efficient and often even better performing. Of course, this really depends on the task. There is only so much benefit one can reap from an analytical approach to localized pixel effects for example whereas no amount of brute force will make a conditional stack go faster.

Answer 3

Yoy can read in Russian - how to calculate FLOPS.

GHz doesn't show FLOPS. One processor with the same GHz can be much faster than the other with the same GHz.

P.S. gpu-s "rx 590" and very old "r7 250x" have almost the same GHz. But ... this is even not correct to compare their performance)