I have looked at your simple code example, and my suspicion is that what you observe in loss of speed is due to the C-heritage requirement that right-hand side expressions be evaluated using intermediate double precision operations, even when the source variables are single precision and the output location (left-hand side destination) is single precision.

The purpose of this was to avoid a loss of accuracy in intermediate calculations, but if a programmer is unaware of this, simply changing types as you did will induce type conversions from float to double on each source variable and from double to float on storing the left-eval result.

In your matrix multiplication code the floating point computation appears as one line in a nested loop, so that the four generated type conversions might easily compare to the time cost of the one multiply and one addition of double precision "intermediate" values.

I'll add some documentation links for you when I'm not posting from my phone. You could add some timing data to quantify how much slower the single precision code runs (and maybe a snippet of the generated asm files). But if my suspicion is correct the way you wrote the code is guaranteed to do all the work for float-types that would be needed for double-type variables and more!

I have looked at your simple code example, and my suspicion is that what you observe in loss of speed is due to the C-heritage requirement that right-hand side expressions be evaluated using intermediate double precision operations, even when the source variables are single precision and the output location (left-hand side destination) is single precision.

The purpose of this was to avoid a loss of accuracy in intermediate calculations, but if a programmer is unaware of this, simply changing types as you did will induce type conversions from float to double on each source variable and from double to float on storing the left-eval result.

In your matrix multiplication code the floating point computation appears as one line in a nested loop, so that the four generated type conversions might easily compare to the time cost of the one multiply and one addition of double precision "intermediate" values.

Since C++ 11 there has been a standard "define" FLT_EVAL_METHOD in header file <cfloat> which exposes an implementation dependent parameter to reflect what (if any) extra precision is necessitated for intermediate expression evaluation. At first glance one might hope that FLT_EVAL_METHOD of 0 means no unnecessary conversions or promotions would be done, by a more careful reading reveals that evaluations are always allowed to be "calculated as if all intermediate results have infinite range and precision." For this reason I would not treat that parameter as a guarantee that lowered precision will be honored.

Ones best bet is to look at the generated assembly language files to see how often floating point conversions are inserted by the compiler. I'll try to use your example code to illustrate this a bit later.