[Cfp-interest 1724] Re: short hex strings

[Jim Thomas]

Here is the IEEE 754-2019 definition of roundTiesToEven:

― roundTiesToEven, the floating-point number nearest to the infinitely precise result shall be delivered; if the two nearest floating-point numbers bracketing an unrepresentable infinitely precise result are equally near, the one with an even least significant digit shall be delivered; if that is not possible, the one larger in magnitude shall be delivered (this can happen for one-digit precision, possible with convertToDecimalCharacter for example, as when rounding 9.5 to one digit in which case both 9 and 1×10^1 have odd significands)

The resolution for the case where neither of the nearest representable numbers has an even least significant digit is new in IEEE 754-2019.

Given this specification, an implementation where the most significant digit is 1, would return 0x1p+2, not 0x1p+1, for the case in question.

- Jim Thomas

> On Jul 24, 2020, at 8:23 AM, Fred J. Tydeman <tydeman at tybor.com> wrote:
> 
> On Fri, 24 Jul 2020 17:10:01 +0200 Paul Zimmermann wrote:
>> 
>> for this kind of thing, I find giving a concrete example is very helpful for
>> the reader to understand the issue. The initial example where 3 can print as
>> 0x2p+1 or 0x3p0 or 0x6p-1 or 0xcp-2 would be nice to add (interestingly, we
>> can't have 1 for the first hexadecimal digit, assuming rounding is to nearest).
> 
> Yes we can have 0x1p+1 or  0x1p+2 (which I contend are the correct output
> if an implementation is only using 1 bit to the left of the radix point).
> 
> 
> ---
> Fred J. Tydeman        Tydeman Consulting
> tydeman at tybor.com      Testing, numerics, programming
> +1 (702) 608-6093      Vice-chair of PL22.11 (ANSI "C")
> Sample C99+FPCE tests: http://www.tybor.com
> Savers sleep well, investors eat well, spenders work forever.
> 
> _______________________________________________
> Cfp-interest mailing list
> Cfp-interest at oakapple.net
> http://mailman.oakapple.net/mailman/listinfo/cfp-interest