7
$\begingroup$

I have been working with an algorithm, which uses

  1. additions of floating point vectors,
  2. (sparse matrix of floats)x(dense vector of floats) dot products

I recently found out that I can get the same functionality form the algorithm if I set everything (vectors and matrices) to integers. The details are nontrivial and boring, but I now know it works.

I still remember how floating point operations work (mantissa, exponents etc) and to me this seems much more complicated integer operations. I thus expected that modifying the algorithm to use pure integer operations would accelerate things. I am trying to ballpark by how much.

Here are now my questions:

  1. Are integer operations generally faster than float?
  2. Has hardware been specialised to floats so much that floats are actually preferable?
  3. FLOPS can be used to specifically quantify the how good a piece of hardware can handle floats. What metric should/could we look at to find out the same thing about integers?
$\endgroup$
6
$\begingroup$

There is a nice discussion on StackOverflow regarding floating point vs integer operations. In short, the performance of the operations depends a lot on

  • processor architecture
  • how the data is stored in memory and in which order it is accessed
  • if (and which) SSE/AVX/AVX2/etc instructions are used (and how efficiently)

This probably provides some insight into the questing #1 regarding the generality of integer-types being faster than floats.

However, FLOPS are well known for a good reason as it is a standard measure in the performance of the numerical software. In numerical computing, it is generally not easy to go from floating types to integer without

  • changing the nature of mathematical operations (division, trigonometric functions, square roots, special functions, most of the linear algebra, etc)
  • having to implement special (long) integer arithmetic to "emulate" those operations – which will definitely negate all the possible benefits of using integer types instead of floating types.

Of course, there are probably counter-examples, where the usage of integer types is natural (say, incidence matrices for graphs).

Now, regarding the performance metrics. A slightly simplified formula from Wiki:

$$ \text{FLOPS}=\text{cores}\times\frac{\text{cycles}}{\text{second}}\times\frac{\text{FLOPS}}{\text{cycle}} $$ Also, see the discussion in this question on CompSci and this question on SO.

Now, for judging integer operations, we can have IPS

$$ \text{IPS}=\text{cores}\times\frac{\text{cycles}}{\text{second}}\times\frac{\text{Instructions}}{\text{cycle}} $$

Notice the difference: For FLOPS, we use FLOPS/cycle, while for IPS, we use general instructions per cycle. Again, it will depend on the processor, if it can actually do more integer-arithmetic operations per cycle (compared to FLOPS). Notice, that there is some averaging done on many levels:

  • average # instructions per cycle
  • hidden dependence on the order of instructions
  • ... and mainly on the data access pattern - which is the common bottleneck for the modern codes.

So, in short, I would answer one question #2 that floating-point calculations have been optimized a lot, so the unnatural use of integer types is very unlikely to bring you any dividends.

Small extra note:

I would also look into half-precision arithmetic. It may be helpful in your tasks provided that the architecture you are using can exploit it efficiently.

$\endgroup$
  • 3
    $\begingroup$ Along the same vein as half precision, if the OP's calculation can be expressed using integer arithmetic, and the range of them is limited (eg, to 16 bits), there might be speedup opportunities using AVX instructions (eg, AVX256 instructions with the _epi16 suffix will handle 16x 16-bit integers at a time). $\endgroup$ – rchilton1980 Oct 16 '18 at 13:24
  • 1
    $\begingroup$ I would posit that FLOPS is not "well known for a good reason as it is a standard measure in the performance of the numerical software" but that they are well known because they used to be a measure of performance. That is definitely no longer true today. Indeed, performance of many floating point codes is much better described by the speed with which data can be transferred from memory to the processor. What happens on the processor does not matter all that much any more if the data exceeds the size of the memory caches. $\endgroup$ – Wolfgang Bangerth Oct 16 '18 at 22:46
  • $\begingroup$ I see, from what I gather then: if I was to magically transform all the operations in an algo to integer ones - without altering anything else - i would stand to gain a FLOPS/IPS ratio in performance. That is assuming that I make proper use of SSE/AVX, and use my memory as I should. I have so many follow up question I feel like I should buy and book. $\endgroup$ – Dionysios Georgiadis Oct 17 '18 at 2:15
  • $\begingroup$ Regarding rchilton1980's comment though, yes: my algo is using very limited ints (up to say 20). I will check the AVX256, as you said. $\endgroup$ – Dionysios Georgiadis Oct 17 '18 at 2:15
4
$\begingroup$

Integer operations are generally faster than floating point operations, but the gap is far less than it was, say, 30 years ago when everyone was still counting FLOPS. The difference may be a factor of 3 or 5 now between a fp-fp operation and an integer-integer operation.

But in both cases, at least if your problems become large enough that they don't fit into the cache of the processor, is that you are limited by how fast data can be loaded from memory onto the processor. On today's processors, floating point computations are so fast that a processor sits idle for more than 90% of the time if you are doing, for example, a matrix-vector product with data that needs to come from main memory. So the bigger question in your application is how much memory you need to dedicate to store your matrix and vectors. If you are thinking of replacing double precision floating point numbers (64-bit) by regular integers (32-bit) and if you only need to store one integer per matrix or vector entry, then you have just saved 50% of your memory needs and your algorithm will run approximately twice as fast because only half the memory needs to be loaded. But if you were planning on only storing single precision floating point numbers (32-bit) then there is likely not going to be a large difference between the two implementations.

$\endgroup$
  • $\begingroup$ Your comment is fantastic, you addressed a concern I had not idea I might have. I can even use 16bit. How do I actually do what you said? I mean, would, say, numpy automatically handle all the smart memory management stuff you mentioned if I set the data type to int16? Or do I have to use some lower level language? $\endgroup$ – Dionysios Georgiadis Oct 17 '18 at 2:19
  • 1
    $\begingroup$ I don't know what numpy does. I've never used it, and so I wouldn't even know how to tell numpy what data type to use. $\endgroup$ – Wolfgang Bangerth Oct 17 '18 at 13:14

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.