IEEE-754 NaNs and missing data

Question

I would like -if possible at all- to represent and handle missing data (in the statistical sense) within the standard IEEE-754 format. Seeing that for both SNaNs and QNaNs various bit representations are possible, I wonder if they all can arise from standard computations or some can be made inaccessible to paths other than the corresponding explicit bit storage, and thus safe and reserved to represent "empty slots" for missing data.

So first of all, does the standard dictate a specific behaviour in representing NaNs or can each processor/system treat them differently? And in the former case, can I do what's hinted at above? (assuming it's safe to detect such values e.g. in C++ via static_cast or a union)

The IEEE standard generically dictates NaN for certain undefined operations, however not the specific payloads. I wonder if there is a standard de facto for this. I would not care if external libraries each have their own payloads, but it's a more serious matter if standard C++ libs are not consistent.

Answer 1

I wonder if they all can arise from standard computations…

This is implementation-dependent, if a library assigns its own payload to its NaNs, its payload may well clash with your own payload. I'd say this is unlikely to be a huge problem, but check the documentation.

I also believe this approach is what R uses for missing data (https://cran.r-project.org/doc/manuals/R-lang.pdf Section 3.3.4). There are relevant existing answers to related questions on StackOverflow here and here.

So first of all, does the standard dictate a specific behaviour in representing NaNs or can each processor treat them differently?

An operation that receives one or more NaNs as inputs should usually return one of those NaNs as output, so the payload is preserved.

And in the former case, can I do what's hinted at above?

Yes, I believe this was the original idea for why NaNs should have a payload at all. It helps with debugging and is used for representing missing data.

assuming it's safe to detect such values e.g. in C++ via static_cast or a union

Indeed, since the binary representation of IEEE754 floats is known, it is safe to just extract the payload with reinterpret_cast and a little bit manipulation (e.g., this existing question on SO). static_cast doesn't do the right thing here, and doing this with union is (technically) undefined behaviour but would work correctly anyway.