[Cfp-interest 3033] Re: printf and rounding recommendation

[Jim Thomas]

I believe CFP, after discussing the note at the end of Vincent’s message below, will want to proceed with the proposal. Here is a draft:

https://wiki.edg.com/pub/CFP/WebHome/C2Y proposal - Recommendation for printf rounding-20240309.pdf <https://wiki.edg.com/pub/CFP/WebHome/C2Y%20proposal%20-%20Recommendation%20for%20printf%20rounding-20240309.pdf>

- Jim Thomas

> On Mar 4, 2024, at 8:11 AM, Jim Thomas <jaswthomas at sbcglobal.net> wrote:
> 
> This is about action item
> 
>>     Jim: C26: Issue 4: Draft a paper as per the resolution in the issues list.
> 
> 
> Issue 4 refers to the recommended practice for the fprintf function.
> 
> CFP Messages 2256, 2262	
> Problem: printf rounding issue. Error (for more than M digits) should not be any larger than for M digits.
> Proposal: Add: For rounding to nearest on more than M digits, there should be an additional requirement: the error should not be larger than the one for M digits
> 
> Note that a proposal would be for an additional recommendation (not requirement), as this is recommended practice.
> 
> The relevant part of Cfp-interest 2262 (from Vincent Lefevre) is
>> Let's take an example: M = 6, ~= 1.2345678, and rounding to nearest.
>> 
>> If the number of significant decimal digits is 6, then the RP says
>> that the correctly rounded value 1.23457 should be output.
>> 
>> If the number of significant decimal digits is 7, then one considers
>> L = 1.23456 and U = 1.23457. According to the RP, 1.234560 is one of
>> the possible recommended outputs, since 1.23456 ≤ 1.234560 ≤ 1.23457.
>> 
>> Conclusion: By increasing the number of output digits, one has
>> decreased the accuracy!
>> 
>> And this can be the case for any number of decimal digits greater
>> than M.
>> IMHO, for rounding to nearest on more than M digits, there should
>> be an additional requirement: the error should not be larger than
>> the one for M digits. With this rule, it is still possible to use
>> the correctly rounded value on M digits and pad with zeros.
>> 
>> Note that this is a recommendation: if the error is slightly larger
>> than recommended because the algorithm attempts to round correctly
>> for almost all cases, this is not an issue.
> I don’t recall CFP discussing the final paragraph (“Note that …”), and suggest we do so before preceding with a WG14 proposal.
> 
> 
> - Jim Thomas

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.oakapple.net/pipermail/cfp-interest/attachments/20240309/53f86e93/attachment.htm>