FYI: another independent ATAN discovery (ATAN for real)

Zhishun Alex Liu uunet!Eng.Sun.COM!Alex.Liu
Wed Dec 7 10:58:18 PST 1994


I received e-mail from broadleyaturing.ucdavis.edu late last night
indicating that he is hitting a real FPATAN failure.  FPATAN of this
data point (with denominator 1 of course) sustains a loss of slightly
more than 36 bits out of a total of 64 - i.e. the error sustained is
slightly worse than the data point I hit the night before.  The new
data point is:

x                 0x4000 A7FFFF03 EAAEF800 (+2.624999765229162118629e+00)
Correct  atanl(x) 0x3FFF 9A78FD98 EA625820 (+1.206817340532114316703e+00)
Computed atanl(x) 0x3FFF 9A78FDA3 94FA1A5C (+1.206817345499033699002e+00)

Once again, the shortest bit pattern of the x cited above whose FPATAN
maintains an accuracy loss of more than 36 bits is:

x                 0x4000 A7FFFF03 EAAC0000 (+2.624999765229119930154e+00)
Correct  atanl(x) 0x3FFF 9A78FD98 EA61977E (+1.206817340532108970069e+00)
Computed atanl(x) 0x3FFF 9A78FDA3 94F959B9 (+1.206817345499028352259e+00)

    From broadleyaturing.ucdavis.edu Wed Dec  7 00:52 PST 1994
    From: broadleyaturing.ucdavis.edu (Bill Broadley)
    To: Alex.LiuaEng (Zhishun Alex Liu)
    Date: Wed, 7 Dec 1994 00:51:50 -0800 (PST)

    I found this worse case:
    Pentium:
    in=    3FF34F1F B31D4C4B (+1.206817340532114313234e+00)
    out=   3FF34F1F B4729F43 (+1.206817345499033633516e+00)
    diff=  BE35552F 80000000 (-4.966919320281704131048e-09)

    I386+387:
    in=    3FF34F1F B31D4C4B (+1.206817340532114313234e+00)
    out=   3FF34F1F B31D4C4B (+1.206817340532114313234e+00)
    diff=  00000000 00000000 (+0.000000000000000000000e+00)
     
    --
    Bill Broadley	Broadleyamath.ucdavis.edu	    UCD Math Sys-Admin
    Linux is great.	http://ucdmath.ucdavis.edu/~broadley            PGP-ok



More information about the Numeric-interest mailing list