1
$\begingroup$

There have been many papers quoting FLOP to quote the performance of a specific approach in machine learning. For example,

We trained two models with different capacities: BlazePose Full (6.9 MFlop, 3.5M Params) and BlazePose Lite (2.7 MFlop, 1.3M Params).

I assume they measured the number of Mega FLOP needed to run the model on input. But they did not explain how.

This may be specific to neural networks, but probably isn't. This is not the same question as how to calculate the floating point operations per second a machine is capable of (which there are plenty of answers to).

Improve this question
$\endgroup$
13
  • 1
    $\begingroup$ If you know the details of your calculations you should be able to count the number of additions (subtractions) and multiplications (divisions) involved. For example, for calculation of the factorial n! you would need (n-1) multiplications. So if n=1e6 that would be 1 MFlop. $\endgroup$
    – Maxim Umansky
    Aug 19 '20 at 18:26
  • $\begingroup$ They might have also taken the known values of FLOPs/cycle and cycles/second for a given processor and multiply it with number of cores and time spent. That would give an approximation, but seeing that they 6.9MFLOPs -which is probably also an approximation- this calculation wouldn't be too far away from the truth. $\endgroup$
    – Abdullah Ali Sivas
    Aug 19 '20 at 20:57
  • 1
    $\begingroup$ @AbdullahAliSivas It is extremely optimistic to assume that this approximation would work. Various operations differ a lot by memory and cache usage, and are going to execute at wildly different FLOP/s speeds. $\endgroup$
    – Federico Poloni
    Aug 20 '20 at 7:03
  • 1
    $\begingroup$ perf stat -e <counter> foo.x $\endgroup$
    – user14717
    Aug 20 '20 at 13:11
  • $\begingroup$ Not all processes use CUDA, but for CUDA applications, we can use stackoverflow.com/questions/14812446/… $\endgroup$
    – Ben Butterworth
    Aug 20 '20 at 14:20
1
$\begingroup$

All processors have counters that can be used to count all sorts of things between a point A in your program, and a point B. Examples are the number of floating point operations performed, the number of branches encountered, the number of cache misses, etc.

I don't know, of course, what the authors of the paper you quote did, but it's not very difficult to actually count how many FP operations were performed -- in fact, it's much easier to do that than to estimate how many operations a specific algorithm would have to perform.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Note that not all counters work correctly on all processors. $\endgroup$
    – Richard
    Sep 19 '20 at 2:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.