[Cfp-interest 2262] Re: printf and rounding recommendation

[Vincent Lefevre]

[About N2731, 7.21.6.1p13]

On 2021-11-09 14:30:43 -0800, Jim Thomas wrote:
> The first part of the Recommended Practice (not included in [Cfp-interest 2256]) says:
> 
> For e, E, f, F, g, and G conversions, if the number of significant decimal digits is at most the maximum value M of the T_DECIMAL_DIG macros (defined in <float.h>), then the result should be correctly rounded. 
> 
> If the implementation does this for a directed rounding mode, then it could satisfy the RP for numbers of significant decimal digits greater than M by rounding (using the directed rounding mode) to M digits and padding with zeros.

Yes, but the issue is more with rounding to nearest.

Let's take an example: M = 6, ~= 1.2345678, and rounding to nearest.

If the number of significant decimal digits is 6, then the RP says
that the correctly rounded value 1.23457 should be output.

If the number of significant decimal digits is 7, then one considers
L = 1.23456 and U = 1.23457. According to the RP, 1.234560 is one of
the possible recommended outputs, since 1.23456 ≤ 1.234560 ≤ 1.23457.

Conclusion: By increasing the number of output digits, one has
decreased the accuracy!

And this can be the case for any number of decimal digits greater
than M.

IMHO, for rounding to nearest on more than M digits, there should
be an additional requirement: the error should not be larger than
the one for M digits. With this rule, it is still possible to use
the correctly rounded value on M digits and pad with zeros.

Note that this is a recommendation: if the error is slightly larger
than recommended because the algorithm attempts to round correctly
for almost all cases, this is not an issue.

-- 
Vincent Lefèvre <vincent at vinc17.net> - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)