[cfp-interest 3422] Re: sign bit of a NaN

[Joshua Cranmer]

On 3/28/2025 22:18, Damian McGuckin wrote:
>
> In looking at somebodies example, it was noticed that
>
>     const double nan = 0.0/0.0
>
> delivers
>
>     -nan ... with gcc 11
>     +nan ,,, with clang 17
>
> I know that the sign is irrelevant on a NaN but with something like 
> cproj() whose sign of its imaginary component does depend on this,
> one might get the wrong answer (alough not one which will compare as
> different).
>
> What should the above yield or is there not an answer to that question?

The short answer, as Vincent points out, is that the sign of a NaN 
result is unspecified.

The long answer (since I've had to some diving into this recently): IEEE 
754 doesn't prescribe any particular handling for NaN payloads and signs 
for most operations. Most hardware ISAs are, understandably, a lot less 
loose in their definitions. Instead, they usually prescribe specific 
payload handling, and the most common variant is to have some sort of 
"preferred NaN" (my terminology for it) that is returned when none of 
the input values are NaNs.

This "preferred NaN" varies from architecture to architecture. In x86, 
the preferred NaN is sign bit set and all bits (save the quiet NaN bit) 
in the payload set to 0. In ARM, the preferred NaN is sign bit not set 
and all bits (save the quiet NaN bit) in the payload set to 0. In SPARC, 
the preferred NaN is sign bit not set, all bits in the payload set to 1.

Compiler optimizations don't necessarily preserve these rules. 
LLVM/Clang, in its optimizations, will prefer to use the ARM-style 
preferred NaN as its set of payload bits when doing constant evaluation 
or constant folding of NaN values, even if the hardware (e.g., x86 or 
SPARC) chooses a different preferred NaN.