[Cfp-interest 2913] Re: incorrect definitions of *_DECIMAL_DIG and DECIMAL_DIG

Mike Cowlishaw mfc at speleotrove.com
Thu Oct 12 11:42:08 PDT 2023


 
As discussed in the last CFP telecon, I asked Michel Hack for his
perspective on this, because he has a wealth of experience of decimal <->
binary conversions.  Here are his comments:

 * No rounding direction is mentioned in the comments in float.h, so we have
   to assume that default rounding is implied, i.e. to-nearest. Btw, I don't
   think it matters how ties are resolved (to even, the default, or away).  

 * 754 (2008 as well as 2019) only requires value-preserving round-trip
   conversions for default rounding, defining Pmin(bf) which corresponds
   to bf_DECIMAL_DIG -- and it also mentions M = max(Pmin (bf )) for all
   supported binary formats bf, which corresponds to DECIMAL_DIG.

 * Directed rounding in the same direction (up+up or down+down) for the
   two conversions in a round-trip can change the value by one ulp for
   a number of digits equal to or exceeding DECIMAL_DIG.  My suspicion,
   based on what I worked out for minimum precision needed for correct
   radix-change rounding, is that 2*DECIMAL_DIG+K, where K is perhaps 7,
   should be enough to preserve the value. (K depends slightly on the
   exponent range, based on exploration; theoretical justification is
   something I'm still looking for.)

 * Directed rounding in opposite directions should be ok, but might need
   one additional digit, i.e. DECIMAL_DIG+1, though I have not yet found
   an example where this is needed.

Mike



More information about the Cfp-interest mailing list