Satan loves extended precision

Debjit Das Sarma uunet!seas.smu.edu!dassarma
Wed Jan 25 07:13:00 PST 1995


   As a side note, rounding the result of a single operation (+,-,*,/,sq root)
to 80 (=p) bits, then to 23 (=q) bits will produce the same answer as rounding
directly to 23 bits.

In general, double rounding for a single operation is not a problem if q <= (p-1)/2.

Debjit Das Sarma



More information about the Numeric-interest mailing list