15
$\begingroup$

I'm currently engaged in a large-scale C++ HPC project focused on numerical simulation, particularly Finite Element Method (FEM) simulations. Our project spans various Linux-based platforms and involves compilation with different architectures and compilers, including GCC, Clang, and ICC.

In our pursuit of optimal performance without compromising numerical precision and stability, we've encountered a challenge with certain optimization flags, such as -O3, which implicitly enable flags like -ffast-math, potentially sacrificing precision for marginal speed gains. Specifically, flags like -funsafe-math-optimizations, -ffinite-math-only, and -fno-rounding-math are activated alongside -O3, raising concerns.

To address this, we're exploring the implementation of tests that can alert us to deviations from floating-point standards compliance during compilation. Ideally, these tests would operate statically and provide warnings about problematic optimization configurations prior to runtime.

While we've come across references discussing IEEE 754 compliance and standard handling of edge cases (e.g., signaling NaNs, FTZ/DAZ), we haven't found guidance on implementing such tests.

My question is: Is there a standard approach or existing toolset for conducting tests to ensure adherence to IEEE 754 standards and appropriate handling of floating-point calculations in C++ projects? While we're aware of methods like checking defined macros (__FAST_MATH__, __FINITE_MATH_ONLY__) or querying std::numeric_limits<float>::is_iec559, we're seeking comprehensive solutions that cover all aspects of compliance and precision.

Any insights or recommendations would be greatly appreciated. Thank you!

These are the references I have consulted:

$\endgroup$
9
  • 7
    $\begingroup$ At least for GCC and Clang, -O3 does not (and has never) enable -ffast-math, both compilers only do that if you use -Ofast (which does a bunch of other risky things beyond just changing FP behavior). The very documentation that you linked in your question very clearly states this. $\endgroup$ Apr 5 at 11:58
  • $\begingroup$ There are existing IEEE 754 conformance test suites, including some written in C. I don't see any reason to hope for reliable tests that operate at compile time, but you could test prospective sets of compilation options before compile time by building such a test suite with them and then running it. And of course, your project should have its own test suite, including tests that assess whether the desired level of numeric stability and precision is attained. $\endgroup$ Apr 5 at 19:58
  • 3
    $\begingroup$ What do you mean exactly with "we've encountered a challenge with certain optimization flags"? I see that some of the answers are challenging the premise that you need to worry about -ffast-math; but from what you write it seems that you have actually encountered a case when enabling -ffast-math gives you trouble in practice. So it could help here to be more specific. $\endgroup$ Apr 6 at 8:58
  • 1
    $\begingroup$ Hi René, I am going to suggest some resources. IEEE754 compliance is not very big concern anymore; however there are many famous processor bugs (FDIV comes to mind immediately) that can affect computations (sometimes in a significant way). If you are developing software with security/safety requirements, -O3 and unsafe optimizations shouldn't be used. You can use Prof. Beebe's testing suite math.utah.edu/~beebe/software/ieee or Kahan's Paranoia people.math.sc.edu/Burkardt/c_src/paranoia/paranoia.html , if they fit your purpose. Also herbgrind can detect numerical issues. $\endgroup$ Apr 9 at 4:01
  • 1
    $\begingroup$ @FedericoPoloni You are asking the right question! We are doing many unit tests and are testing various compilation environments, adding new platforms or modifying existing platforms once in a while. The issue was raised while testing our code compiled with ICPX-2022.2 which implicitly activated fast-math with -O3. Our unit tests revealed a couple of discrepancies. What we are looking to achieve is to identify the underlying specific causes in order to render our code more robust to them and eventually to ban the problematic compilation flags. $\endgroup$ Apr 10 at 14:00

5 Answers 5

8
$\begingroup$

IEEE 754 conforming floating point arithmetic has rounding errors. The rounding errors are well defined and reproducible, so two calculations will give you the exact same results - with the same rounding errors.

When you say "potentially sacrificing precision for marginal speed gains" - that is not what is happening. For example using a fused multiply-add gains precision by avoiding one of two rounding errors, and speed gains can be far from marginal. What you lose is bit for bit identical results. There are other cases where your results are as precise as before, just slightly different.

Now if your calculation gives vastly different results (and it is not due to a bug in your software), then your nice bit-by-bit reproducible results will most likely have the same problem. You can just have multiple implementations giving you bit-by-bit identical nonsense answers, and one different, equally nonsense answer. If that happens you really want to check all your results.

$\endgroup$
4
  • 5
    $\begingroup$ This is untrue for the vast majority of IEEE-754 operations. In fact, for quite a few (e.g. Bessel functions AFAIK) rounding errors are essentially an unsolved math problem. I know +-/* and sqrt have defined rounding errors <1ULP, but those are the exception to the rule. $\endgroup$
    – MSalters
    Apr 5 at 9:47
  • $\begingroup$ @MSalters +, -, *, / and sqrt have defined errors. At most 1/2 ulp, and round to even. Anything else is up to your floating-point library. You may have a library that makes heroic efforts to have sin/cos with at most 0.5 or a good library that has at most 0.55 ulp for a very large range of numbers. $\endgroup$
    – gnasher729
    Apr 6 at 16:41
  • $\begingroup$ "Annex B.1: The format of an anonymous destination is defined by language expression evaluation rules." + "5.10 The values of the floating operands and the results of floating expressions may be represented in greater precision and range than that required by the type; the types are not changed thereby.)" appears to contradict this answer. $\endgroup$
    – TLW
    Apr 6 at 21:38
  • 1
    $\begingroup$ I understand that it is normal to expect slightly different results arising from discrepencies in the architecture, compilation environment and compiler configurations... We are not trying to get the same results with or without -Ofast. What we would like to obtain is a way to detect and identify implicitly activated risky optimizations. So a test (or a battery of tests) that warns us that something is not within standards would satisfy us. We are also trying to render our code more robust to these optimization flags. $\endgroup$ Apr 10 at 14:24
7
$\begingroup$

My suggestion is to just not worry about it. We run the ~13,000 tests of the deal.II project in both debug and release mode (the latter involving high optimization flags) and do not observe differences in the test results that would trigger the test to fail -- tests fail if numerical outputs differ by more than a relative tolerance of 1e-8 or by an absolute difference of 1e-6. In practice, I believe that the differences are much smaller than these tolerances.

In other words, the differences in output that result from using flags such as -ffast-math are very very small. Certainly much smaller than the finite element error or even just the error due to iterative solvers for linear systems.

$\endgroup$
14
  • 9
    $\begingroup$ only places I've ever run into issues with -ffast-math and related options are in special summation functions like Kahan summation, but in the few places I have this implemented I have stuff in the code to explicitly disable optimizations for just that function. $\endgroup$ Apr 5 at 2:38
  • $\begingroup$ Do you use 1e-8 tolerance for tests with doubles or with floats? What tolerance do you use for the other case? $\endgroup$ Apr 5 at 6:23
  • 4
    $\begingroup$ The big thing about -ffast-math is that it disables special-casing of NaN and infinity. For example isnan always returns false. $\endgroup$
    – jpa
    Apr 6 at 8:31
  • 1
    $\begingroup$ @WolfgangBangerth - the issue with that is that often the fastest approach for algorithms that "rarely" hit edge cases is to always execute with a NaN check somewhere. Unfortunately, performance is also the main reason to use -ffast-math... $\endgroup$
    – TLW
    Apr 6 at 21:45
  • 1
    $\begingroup$ The issue was raised because these implicitly activated optimization flags caused some unit tests to fail. For example, mesh partitionning can give vastly different results when -Ofast is activated. We didn't explicitly activate this flag. ICPX-2022.2 seem to activate -Ofastwhen -O3 is activated. We don't expect to use -Ofast, but would like to minimize the risks of having wrong results outside of what our unit tests covers. A good way to do this is to raise a red flag when something is not compliant to the standards. $\endgroup$ Apr 10 at 14:32
3
$\begingroup$

In complementation of others answers that say to write runnable tests with tolerances, I further add to not write static tests at all.

Optimizing compilers are in the business of generating results, instead of running code. That is, they may rework, reflow and transform the source code to something that has no resemblance to the original, to the point of no running no code at all, if it is possible to constant fold / constexpr all the code.

This occurs inside the compiler, within the compiler internal code. In other words, you may end up testing the internal compiler code for conformance, rather than the project code conformance.

$\endgroup$
1
  • 1
    $\begingroup$ We would like to test eveything that could go wrong. We have a lot of unit tests, which helped us identify the root cause of some of them failing on specific platforms: the implicit activation of -Ofast when using -O3 with ICPX-2022.2. So what we would like to obtain is a simple/light solution to raise a red flag when critial standards are not respected. $\endgroup$ Apr 10 at 14:37
2
$\begingroup$

I agree with the answer of @Wolfgang Bangerth.

If you insist on testing the effects yourself you could set up explicit assertions or even unit tests for the 5 rounding types and the recommended operations as stated in IEEE 754. That would give you a feel for where the erros actually occurr on each target platform. If you do that, please share your tests somewhere so other implementors may benefit too. There is published research on verifying compliance Testing IEEE-1788 compliance

also: collection: utah.edu

$\endgroup$
1
  • 1
    $\begingroup$ If I find a solution, I will gladly share it here, yes! We don't necessarly need a deep and comprehensive battery of tests... What we would like to obtain is a red flag when the compiler does something that breaks compliance to the critical standards. Since numerical stability and precision are both important in numerical simulation, we would like to, at the very least, be aware when something risky is being done implicitly by the compiler. $\endgroup$ Apr 10 at 14:46
0
$\begingroup$

I work in a related field and I'd say these are issues you should worry about. The problem is that deviations can be caused by benign compiler optimizations or by real software bugs. To address it, I suggest you use check-summing; after some or all of your system's calculations are complete, bitwise convert every floating point value to an unsigned integer and sum them into a checksum. I.e. -2.3421 becomes 0xc015e4f7 in hexadecimal. Record the checksum and on subsequent runs of the system check that the calculated checksum matches the stored one. If they don't you need to investigate. Perhaps it is a bug, or the new version of clang reorders some floating point operations, or the new CPU's simd unit is slightly different, etc.

In the end, you might end up with quite a lot of checksums depending on how many compilers and devices you support. That's fine because the alternative is subtle algorithmic bugs causing huge amount of trouble.

What you write about IEEE 754 compliance is confusing since it is the IEEE 754 implementation (hardware and/or software) that you use that may or may not be compliant - not your code. Some devices claiming compliance still have bugs affecting calculations. It's not something you can control.

$\endgroup$
4
  • 2
    $\begingroup$ I don't think this is practical. In my experience, you will get bitwise different results when using different compilers, compiler versions, processors, platforms. These results are functionally identical, but not bitwise. It is impractical to base testing on your approach; you'd be spending your whole day tracking down where all of the differences in your tests come from if you have a substantial number of tests. $\endgroup$ Apr 6 at 21:03
  • 1
    $\begingroup$ Welcome to Scicomp! There isn't really much to gain by going in that deep. let's say you have two different checksums for GCC and Clang because of slighly different operator ordering and inlining etc. Then what? Which is 'correct'? At some point we have to live with those errors as long as they don't change the results significantly. You will improve your results more if you simply add more finite elements to your domain. $\endgroup$
    – MPIchael
    Apr 9 at 6:30
  • $\begingroup$ As I wrote, you'll end up with a lot of checksums, each of which is correct for a specific compiler/version/hardware. The point is to distinguish between hardware differences and subtle coding bugs. It's a "you'll thank me later" kind of thing; more initial setup that saves days not bug hunting. $\endgroup$ Apr 9 at 17:11
  • 1
    $\begingroup$ Our unit tests where enough to identify the presence of a significant discrepency in the result. We don't really need something else to check that. What we would like to test is if something has broken compliance to the critical standards. In such case, we would like a red flag to be raised. Ideally, a test informing us of which standards are not respected would help us mitigate the side effects of this kind of situation, rendering our code more robust to non-compliance. $\endgroup$ Apr 10 at 14:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.