1
$\begingroup$


Yesterday I was wondering how floats are handled in a computer and what they look like in binary... I learnt about the single-precision floating-point and I tried to see the limit of that format...

I first wrote this little Python (3.4) script

a,b = 1,1

while (a+b)-a-b == 0:
    a*=2.0

while (a+b)-a-b != 0:
    b+=1.0

print(a,b)

I wanted to see what specific exponent value of a could be create a lack of precision in the result (a+b)-a-b, and if increasing the value of b could correct it.

The output was :

9007199254740992.0 2.0

I noticed that 2^53 = 9007199254740992.0 and I though it was a bit disturbing : the binary representation of 53 is 110101, and this really doesn't look like any specific value...

The "2.0" let me a little less sceptical, as you just have to add 1 to increase the exponent value of b. It made more sense, but I realised I couldn't explain it properly too.

I modified my script to see the actual binary value of everything I was doing :

import struct

def binary(f):
    return str(bin(struct.unpack('!i',struct.pack('!f',f))[0])).replace('0b', '')

def status(a,b):
    return binary(a),binary(b),binary(a+b),binary(-a),binary(-b),binary(-a-b),binary((a+b)-a-b)

a,b = 1,1

while (a+b)-a-b == 0:
    a*=2.0
    print(status(a,b))


print("\n")

while (a+b)-a-b != 0:
    b+=1.0
    print(status(a,b))

print(a,b)

But what I saw didn't help me. I can't figure out why the lack of precision happens at that specific value of a, and why increasing b just once would correct it.

Could anyone help me understanding this ?

(I apologize in advance for my really poor English)

$\endgroup$
  • $\begingroup$ It might be insightful to play with floating point numbers that use less bits such as binary16 (half-precision) where you could see what each bit in a number does and enumerate all possible numbers easily. $\endgroup$ – jfs Feb 27 '16 at 11:49
4
$\begingroup$

You've run into the limits inherent in double precision floating point numbers, which python uses as its default float type (this is the same as a C double).

These model real numbers as

$(-1)^s \left(1+\sum_{i=1}^{52}\frac{b_{52-i}}{2^i}\right)\times 2^{e-1023}$

where $s$ is 0 or 1 (1 bit), $e$ is an 11 bit number and the $b_i$s fill the remaining bits. This indeed has a maximum value with integer stride of $2^{53}$ = 9007199254740992.0. After this value, the numbers which can be modelled start to increase by 2 each time, and the addition of 1.0 (i.e. b) underflows. On the other hand, you can still add 2.0 just fine.

The model effectively attempts to balance range (the size of exponents it accepts) with accuracy (the size of the quantised step it accepts). If you'd still like to play with single precision numbers and your python installation includes numpy, they can be accessed using the numpy.float32 function. Note that you will have to set your constants to be float32 as well, otherwise python automatic type conversion may prevent you seeing anything.

$\endgroup$
  • 1
    $\begingroup$ 1- Python supports IEEE-754 platforms and some functions may even rely on IEEE-754 e.g., math.fsum() but Python doesn't guarantee IEEE-754 semantics on some platforms. 2- it might be worth mentioning subnormal values and nan, inf (a single formula in the answer doesn't apply to some f.p. numbers). $\endgroup$ – jfs Feb 27 '16 at 11:26
  • $\begingroup$ @ J.F. Sebastian, Assuming you have the reputation to do it, feel free to edit this to a fuller answer if you want to, it sounds like you have a clearer vision of other important information than I do. $\endgroup$ – origimbo Feb 27 '16 at 21:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.