FYI: Pentium FPATAN can lose 36 bits (out of 64)

David Hough sun!Eng!David.Hough
Tue Dec 6 17:07:24 PST 1994


> But I think the atan problem reinforces the gravity of the FDIV bug in an
unexpected way... most general
scientific calculations that run all night on a Pentium WILL encounter the
FDIV bug, although the odds are small but non-zero that wrong conclusions
will result.

I spoke truer than I thought.   It turns out that last night Alex found 
two atan failures in his atan test program, but he couldn't confirm one
in a simple short program, so he only published the one he could confirm.

Today he determined that the second failure was actually a FDIV bug failure
caused by a division in his test routine that computes the high-precision
atan that is to be compared to that produced by the hardware under test.
So this is the second known example, after Prof. Nicely at Lynchburg,
of the FDIV bug leading to wrong conclusions in a program that was not
searching for it.    And it took less than overnight to find it.



More information about the Numeric-interest mailing list