[Cfp-interest 2913] Re: incorrect definitions of *_DECIMAL_DIG and DECIMAL_DIG
Mike Cowlishaw
mfc at speleotrove.com
Thu Oct 12 11:42:08 PDT 2023
As discussed in the last CFP telecon, I asked Michel Hack for his
perspective on this, because he has a wealth of experience of decimal <->
binary conversions. Here are his comments:
* No rounding direction is mentioned in the comments in float.h, so we have
to assume that default rounding is implied, i.e. to-nearest. Btw, I don't
think it matters how ties are resolved (to even, the default, or away).
* 754 (2008 as well as 2019) only requires value-preserving round-trip
conversions for default rounding, defining Pmin(bf) which corresponds
to bf_DECIMAL_DIG -- and it also mentions M = max(Pmin (bf )) for all
supported binary formats bf, which corresponds to DECIMAL_DIG.
* Directed rounding in the same direction (up+up or down+down) for the
two conversions in a round-trip can change the value by one ulp for
a number of digits equal to or exceeding DECIMAL_DIG. My suspicion,
based on what I worked out for minimum precision needed for correct
radix-change rounding, is that 2*DECIMAL_DIG+K, where K is perhaps 7,
should be enough to preserve the value. (K depends slightly on the
exponent range, based on exploration; theoretical justification is
something I'm still looking for.)
* Directed rounding in opposite directions should be ok, but might need
one additional digit, i.e. DECIMAL_DIG+1, though I have not yet found
an example where this is needed.
Mike
More information about the Cfp-interest
mailing list