5
$\begingroup$

I am still rather new on here and I hope question is suitable for this forum otherwise please help me migrate it to greener pastures.

I am an electrical engineer specializing in applying mathematics for building cool algorithms. Sometimes in my work I try to write code that helps compiler generate code that would run as fast as hardware would allow it to. The other day, I realized that when I changed type from 32 bit float to 64 bit doubles and compiled my code.. it ran much faster.

Is it true that modern CPU architectures SIMD instructions and other quirks in CPUs are more optimized for double than for single precision floating point arithmetics?

To me it seems to be true, at least my local machine. I was surprised by it as I figured you would be able to fit twice the amount of floats on same logics. Maybe it is because of demand in industry, that people in general use double precision so much more than float, which makes them focus on good support for double precision arithmetics?

Or if I am wrong and it is not the prioritization of the CPU design maybe it is the prioritization of the compiler?

Here is some minimal godbolt example with matrix-vector multiplication of Nx8 matrices https://godbolt.org/z/gPR9wj

It is interesting for me to see for example how clang 9.1.0 and gcc 8.1 produce very different output w.r.t. code size.

I am not able to conclude very much regarding float vs double though. (We can change scal typedef to float or double as we desire.)

Improve this question
$\endgroup$
8
  • 2
    $\begingroup$ Can you isolate the phenomenon in a minimum working example? If it work true that SIMD was optimized for double on your architecture it should be simple to demonstrate this. $\endgroup$
    – Richard
    Commented May 31, 2019 at 20:49
  • $\begingroup$ Random thought: if you were using a convergent algorithm with a fine tolerance it might be possible that with floating point your algorithm oscillates outside of that tolerance for a long while before finding two values that subtract to fall within it. Using double could fix this. I mention this mostly to illustrate that the specifics of your code may be important. $\endgroup$
    – Richard
    Commented May 31, 2019 at 20:52
  • $\begingroup$ @Richard It is faster also without taking fewer iterations required into consideration, which is what I really doubted if it would be reasonable. The problem right now with minimal examples is that the code is part of a huge computational library and also closed source. I am starting to think maybe when compiling with float arrays that bad memory alignment might have been the saboteur to be able to load SIMD registries. I can see if I maybe can produce some small minimal example of same phenomenon. But if so then probably on another platform with another compiler. Agh what a mess. $\endgroup$
    – mathreadler
    Commented May 31, 2019 at 22:41
  • 3
    $\begingroup$ It doesn't seem as though there's really an answerable question here. Per my previous suggestion, you can build simple test cases to see if various hypotheses are correct. You could also try profiling before and after the change to try to isolate where the speed-up is coming from. But I don't think there's much we can tell you here on SE. $\endgroup$
    – Richard
    Commented May 31, 2019 at 23:19
  • $\begingroup$ It really is impossible to say anything concrete without seeing some code that can demonstrate this issue. It doesn't even need to be specifically minimal, so long as there is at least something to go on. $\endgroup$
    – Kirill
    Commented Jun 1, 2019 at 14:34

2 Answers 2

Reset to default
8
$\begingroup$

I have looked at your simple code example, and my suspicion is that what you observe in loss of speed is due to the C-heritage requirement that right-hand side expressions be evaluated using intermediate double precision operations, even when the source variables are single precision and the output location (left-hand side destination) is single precision.

The purpose of this was to avoid a loss of accuracy in intermediate calculations, but if a programmer is unaware of this, simply changing types as you did will induce type conversions from float to double on each source variable and from double to float on storing the left-eval result.

In your matrix multiplication code the floating point computation appears as one line in a nested loop, so that the four generated type conversions might easily compare to the time cost of the one multiply and one addition of double precision "intermediate" values.

Since C++ 11 there has been a standard "define" FLT_EVAL_METHOD in header file <cfloat> which exposes an implementation dependent parameter to reflect what (if any) extra precision is necessitated for intermediate expression evaluation. At first glance one might hope that FLT_EVAL_METHOD of 0 means no unnecessary conversions or promotions would be done, by a more careful reading reveals that evaluations are always allowed to be "calculated as if all intermediate results have infinite range and precision." For this reason I would not treat that parameter as a guarantee that lowered precision will be honored.

Ones best bet is to look at the generated assembly language files to see how often floating point conversions are inserted by the compiler. I'll try to use your example code to illustrate this a bit later.

Improve this answer
$\endgroup$
4
  • $\begingroup$ Wow I would not have been able to figure this out on my own. Thank you. Is there some way to tell a compiler to relax this requirement? $\endgroup$
    – mathreadler
    Commented Jun 2, 2019 at 17:00
  • 2
    $\begingroup$ I don't know of a compiler flag to ignore or relax that part of the C specification. Many C/C++ family compilers allow "inline assembly" code to be written into surrounding high-level language so that instructions can be given that circumvent specifications like I described. $\endgroup$
    – hardmath ♦
    Commented Jun 2, 2019 at 18:08
  • 1
    $\begingroup$ A related (but different) discussion took place for this older SO Question, "Need to use floats for performance yet want double-precision calculations". $\endgroup$
    – hardmath ♦
    Commented Jun 3, 2019 at 5:23
  • $\begingroup$ There is no such C-heritage requirement in modern clang/gcc. The C++ standard leaves open how floating-point arithmetic is done and the compiler may decide how to do it. Clang and gcc decide to follow the IEEE-754 standard, which doesn't allow for intermediate higher-precision rounding. The assembly code produced by the example on godbolt looks like it's doing correct IEEE-754 rounding. I believe that this bit of your answer contradicts what is implemented by mainstream compilers. $\endgroup$
    – Kirill
    Commented Jun 4, 2019 at 11:41
4
$\begingroup$

There are multiple possible explanations:

  • with floats and doubles the different number of computations happen due to, say, number of iterations/function evaluations (or something else, as pointed out by @Richard in the comments)
  • there is some type conversions/implementations (say templated code) that are non-optimal with float types as opposed to doubles. I once had this issue while overspecializing template implementations.
  • C++ standard library can choose a different path for standard algorithms depending on the type traits. A recent example that I encountered: the sorting algorithm for double and my custom implementation of complex<double>, where due to the fact I had a non-trivial constructor, the quicksort algorithm switched to merge sort way later than it would have for "simple datatype". However, such things would usually benefit from a smaller 32-bit datatype.

It is very unlikely that the SIMD to be more efficient for double than for float, unless you are running on some exotic architecture with and exotic compiler. On the contrary, right now the trend is to optimize for single- and even half-precision (influence of deep learning), as opposed to double and quad.

Things to try:

  • check obvious items I touched in this answer above
  • minimal reproducible example (as @Richard pointed out)
  • Godbolt on the minimum reproducible example to see what is going on
Improve this answer
$\endgroup$
1
  • $\begingroup$ You make very good points. Thanks. I do agree that single and half precision is getting more attention lately, but as far as I know this attention is often on other pieces of hardware than general purpose CPUs. Like GPUs or even dedicated ML chips. $\endgroup$
    – mathreadler
    Commented Jun 1, 2019 at 11:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.