How can I detect lost of precision due to rounding in both floating point addition and multiplication?

Question

From Computer Systems: a Programmer's Perspective:

With single-precision floating point

• the expression (3.14+1e10)-1e10 evaluates to 0.0: the value 3.14 is lost due to rounding.

• the expression (1e20*1e20)*1e-20 evaluates to +∞ , while 1e20*(1e20*1e-20) evaluates to 1e20.

• How can I detect lost of precision due to rounding in both floating point addition and multiplication? (in C or Python)

• What is the relation and difference between underflow and the problem that I described? Is underflow only a special case of lost of precision due to rounding, where a result is rounded to zero?

Thanks.

Answer 1

Floating-point exceptions may help you here.

C support varies by implementation (compiler) but see GCC here: https://www.gnu.org/software/libc/manual/html_node/FP-Exceptions.html

Python support is documented here: https://docs.python.org/2/library/fpectl.html

I’ve only used these features a few times, and then only with the Intel compiler (https://software.intel.com/content/www/us/en/develop/documentation/cpp-compiler-developer-guide-and-reference/top/compiler-reference/compiler-options/compiler-option-details/floating-point-options/fp-trap-qfp-trap.html ), but in that case, I was able to trap truncation and other non-fatal errors (fatal would be dividing by zero, for example).

Answer 2

Normally one does not try to detect loss of precision algorithmically, but rather analyzes and modifies algorithms to assess how they are affected by it.

For instance, in your first example you would run a (forward) error analysis and figure out that the summation error is bounded by $3 \cdot 10^{10} \mathsf{u}$, where $\mathsf{u}$ is machine precision, or you would show that the summation is backward stable so the summation has not done significantly more damage than storing that $10^{10}$ in a Float32 did in the first place.