[Cfp-interest 1735] Re: footnote about sufficient %a formatting precision

[Paul Zimmermann]

       Hi Jim,

> Here’s a suggested replacement for footnote 296 to address the issues
> mentioned in the earlier message (below). 
> 
> https://wiki.edg.com/pub/CFP/WebHome/footnote_296_rev.pdf

shouldn't it be 16^P >= b^p instead? If we have a number with p digits in
radix b, and want to output it in radix B with P digits, we need
P >= 1 + p*log(b)/log(B) so that we can recover the input number exactly
(cf page 177 of Modern Computer Arithmetic). This means B^(P-1) >= b^p.
Here we have B=2 with P replaced by 4P+1 (4P because in hexadecimal, and
+1 because of the first hexadecimal digit which has at least one bit), thus
we get 2^4P >= b^p, i.e., 16^P >= b^p.

For example, for b=2 and p=4, this yields P=1 instead of P=2 for 16^P > b^p.

Note that equality can occur only when b is a power of two, say b=2^k.
If we want to distinguish such a case, then it should be 4P+1 >= pk
thus 16^P >= b^p/2, which would give P=1 for b=2 and p=5 instead of P=2.

Also should "the decimal-point character" be "the hexadecimal-point character"?

Paul