[Cfp-interest 3020] Re: printf and rounding recommendation

[Jim Thomas]

This is about action item

>     Jim: C26: Issue 4: Draft a paper as per the resolution in the issues list.


Issue 4 refers to the recommended practice for the fprintf function.

CFP Messages 2256, 2262	
Problem: printf rounding issue. Error (for more than M digits) should not be any larger than for M digits.
Proposal: Add: For rounding to nearest on more than M digits, there should be an additional requirement: the error should not be larger than the one for M digits

Note that a proposal would be for an additional recommendation (not requirement), as this is recommended practice.

The relevant part of Cfp-interest 2262 (from Vincent Lefevre) is
> Let's take an example: M = 6, ~= 1.2345678, and rounding to nearest.
> 
> If the number of significant decimal digits is 6, then the RP says
> that the correctly rounded value 1.23457 should be output.
> 
> If the number of significant decimal digits is 7, then one considers
> L = 1.23456 and U = 1.23457. According to the RP, 1.234560 is one of
> the possible recommended outputs, since 1.23456 ≤ 1.234560 ≤ 1.23457.
> 
> Conclusion: By increasing the number of output digits, one has
> decreased the accuracy!
> 
> And this can be the case for any number of decimal digits greater
> than M.
> IMHO, for rounding to nearest on more than M digits, there should
> be an additional requirement: the error should not be larger than
> the one for M digits. With this rule, it is still possible to use
> the correctly rounded value on M digits and pad with zeros.
> 
> Note that this is a recommendation: if the error is slightly larger
> than recommended because the algorithm attempts to round correctly
> for almost all cases, this is not an issue.
I don’t recall CFP discussing the final paragraph (“Note that …”), and suggest we do so before preceding with a WG14 proposal.


- Jim Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.oakapple.net/pipermail/cfp-interest/attachments/20240304/729f9742/attachment.htm>