[Cfp-interest] aA formatting with small precision modifier

[Jim Thomas]

>> 
>> What is the result of:
>>  printf("%.1Ha\n", DEC32_MAX);
>> Is it INFINITY or 1e97?  I believe it is INFINITY. 
>> As if round source value.
> 
> Yes, (0, 9999999, 90) -> inf
> 
> What if rounding down? what does "rounds the input, in the type, according to the current rounding direction for decimal floating-point operations, to the number of digits specified by the precision modifier" really mean?
> 
> (0, 1234567, 0) -> (0, 1000000, 0) or (0, 1, 6)? Check implementations. Former has same quantum exponent as input.
> 
> For the max case the latter isn't an option because it's at the limit of the exponent range:
>  (0, 9999999, 90) -> (0, 9000000, 90) -> 9.000000e96
> 
> Designed to reflect the quantum exponent?

In the Aug teleconference we agreed to check the 754-2008 standard for guidance on this issue and if none is given to use the first (quantum preserving) approach, where the intermediate rounding in the example above gives

(0, 1234567, 0) -> (0, 1000000, 0)

I didn't find any help in 754-2008 on this issue. Did anyone else?

-Jim

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.oakapple.net/pipermail/cfp-interest/attachments/20120821/8369f872/attachment.html