2
$\begingroup$

From Computer Systems: a Programmer's Perspective:

With single-precision floating point

  • the expression (3.14+1e10)-1e10 evaluates to 0.0: the value 3.14 is lost due to rounding.

  • the expression (1e20*1e20)*1e-20 evaluates to +∞ , while 1e20*(1e20*1e-20) evaluates to 1e20.

  • How can I detect lost of precision due to rounding in both floating point addition and multiplication? (in C or Python)

  • What is the relation and difference between underflow and the problem that I described? Is underflow only a special case of lost of precision due to rounding, where a result is rounded to zero?

Thanks.

$\endgroup$
1
3
$\begingroup$

Floating-point exceptions may help you here.

C support varies by implementation (compiler) but see GCC here: https://www.gnu.org/software/libc/manual/html_node/FP-Exceptions.html

Python support is documented here: https://docs.python.org/2/library/fpectl.html

I’ve only used these features a few times, and then only with the Intel compiler (https://software.intel.com/content/www/us/en/develop/documentation/cpp-compiler-developer-guide-and-reference/top/compiler-reference/compiler-options/compiler-option-details/floating-point-options/fp-trap-qfp-trap.html ), but in that case, I was able to trap truncation and other non-fatal errors (fatal would be dividing by zero, for example).

$\endgroup$
3
  • 1
    $\begingroup$ Thanks. What is the relation and difference between underflow and the problem that I described? Is underflow only a special case of lost of precision due to rounding, where a result is rounded to zero? $\endgroup$ – Tim Oct 10 '20 at 20:12
  • $\begingroup$ I will have to write a test code to know for sure. I think the first addition throws an inexact exception (the most common one) but maybe it’s an overflow instead. $\endgroup$ – Jeff Hammond Oct 10 '20 at 20:21
  • $\begingroup$ There's a standard, IEEE, that defines all the floating point exceptions. Then there are implementations on the compiler side of things, gcc, intel, etc. If that was not enough most architectures also have a hand in it. As @Jeff suggested, you really unfortunately need to write some test codes to see what is going on for your platform/tool-chain of choice. My own experience would suggest it's not quite as diverse as I just mentioned but it sure is not uniform. $\endgroup$ – Kyle Mandli Oct 11 '20 at 0:32
2
$\begingroup$

Normally one does not try to detect loss of precision algorithmically, but rather analyzes and modifies algorithms to assess how they are affected by it.

For instance, in your first example you would run a (forward) error analysis and figure out that the summation error is bounded by $3 \cdot 10^{10} \mathsf{u}$, where $\mathsf{u}$ is machine precision, or you would show that the summation is backward stable so the summation has not done significantly more damage than storing that $10^{10}$ in a Float32 did in the first place.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.