I have to work with values such as 1e-15 in my code but I can't. Indeed, because of low precision these values are equivalent to 0. Any ideas?
6
-
1$\begingroup$ I think you are using double. For your propose try to use type with more precision. See wiki double, quadruple. $\endgroup$– Mauro VanzettoApr 12, 2017 at 15:45
-
3$\begingroup$ Either I don't understand the question or you have a misunderstanding about floating point arithmetic on modern computers. I suggest taking a look at en.wikipedia.org/wiki/IEEE_floating_point and, in particular, the paper by Goldberg it references. By the way, IEEE 64-bit floating point arithmetic (aka double precision) is the highest precision implemented on most modern CPUs, so if you really need higher precision (and that is far from clear from your post), you will likely have to resort to a software library that implements this. $\endgroup$– Bill GreeneApr 12, 2017 at 17:07
-
1$\begingroup$ Here's my favorite question about IEEE-754: What fraction of 32-bit IEEE-754 numbers are smaller than the float epsilon? (Answer: ~40%.) Everyone gets this wrong, including myself at times. $\endgroup$– user14717Apr 12, 2017 at 17:41
-
1$\begingroup$ Double precision floating point numbers (64 bit wide) can definitely represent 1e-15 and distinguish it from zero. $\endgroup$– cfhApr 12, 2017 at 22:09
-
2$\begingroup$ It is important to know whether all your numbers are $O(10^{-15})$, in which case what you describe shouldn't happen and suggests something else went wrong, or whether some of your numbers are $O(10^{-15})$, but others are $O(1)$, in which case you are seeing limits of double-precision floating-point numbers, and might actually benefit from extra precision. $\endgroup$– KirillApr 13, 2017 at 2:25
|
Show 1 more comment
1 Answer
$\begingroup$
$\endgroup$
3
The best thing to do would be to re-scale your problem so that way you don't have such small values. If you can, do this and call it a day.
If every number is small, then you can still be fine because of the subnormal floating point numbers. People mentioned this in the comments, and its addressed here:
https://stackoverflow.com/questions/8341395/what-is-a-subnormal-floating-point-number
However, subnormal numbers have slower calculations, so it's normal to try and avoid them. Also, because they are less than machine epsilon, they do not behave well (the easiest example is that if you add 1 to any of them, you get 1 back). The will multiply and add in ways that round the gaps, and you pay and extra cost for calculating with them.
So rescale your problem.
Or use quads, BigFloats, Arbs, 128-bit decimal floating point numbers. If you're using Julia, BigFloats are in Base, Arbs are in ArbFloats.jl, and decimals are in DecFP.jl. Due to type-genericity, you can generally replace any algorithm with these number types and it should just work (even in code from packages, if the author didn't overtype their functions).
But first, try rescaling your problem.
answered Apr 13, 2017 at 1:06
-
3$\begingroup$ Subnormal values start at around $10^{-300}$, which is many many orders of magnitude away from $10^{-15}$. $\endgroup$– KirillApr 13, 2017 at 2:23
-
1$\begingroup$ "because they are less than machine epsilon, they do not behave well" I believe this is technically wrong: there are plenty of normal numbers between $0$ and $\epsilon$, and the property $1+x=1$ only ever implies $|x|<\epsilon$, not that $x$ is subnormal: in particular, arithmetic between $x$ and values similar in magnitude to $x$ (but not to $1$) will behave in just the normal way—that's just the point of "floating-point". $\endgroup$– KirillApr 13, 2017 at 2:42
-
$\begingroup$ Thanks for the correction. I guess I learned something too! $\endgroup$ Apr 13, 2017 at 4:20