What is hypot(NaN,Inf)?

David G. Hough on validgh validgh!dghaSun.COM
Sun Jan 13 14:06:56 PST 1991


The argument for 

     hypot(+-oo, y) is +oo for any y, even  a  NaN,
     and is exceptional only for a signaling NaN.

originates with W. Kahan and reflects a larger controversy
exemplified by nan**0: should it be nan or 1?

One point of view:

	Any function returning a floating-point result should
	return a NaN if any operand is a NaN.

This point of view is consistent and easy to remember.
The other point of view:

	When the value of a function is independent of any finite
or infinite value of a particular operand, it should return the
same result if that operand is NaN.

Contrived example:  The constant function three(x) that maps all
finite and infinite x to 3.0.  Surely three(NaN) mapped to 3.0
is more useful than three(NaN) mapped to NaN, even if not
consistent with what more interesting functions do with NaN.

------------------------------------------------------------------

Yet another point of view:  

	Functions should be defined in the way most useful to
	mathematical software users, regardless of consistency.

nan**0 can be examined from this point of view, but consider also
atan2(0,0).  Sometimes atan2 has been defined misleadingly in terms
of atan of a quotient, with a funny proviso to insure that the
full range of complex arguments (used here in the sense of the
angular portion of a polar coordinate) is achieved.  The more direct 
definition that corresponds to what atan2 is actually used for,
is the following (taken from Sun C 1.1 and Fortran 1.4 beta man
pages):

     atan2(y,x) and hypot(x,y) (see hypot(3M)) convert  rectangu-
     lar  coordinates  (x,y) to polar (r,th); atan2(y,x) computes
     th, the argument or phase, by computing an  arc  tangent  of
     y/x  in  the range -p to p.  atan2(0.0,0.0) is +-0.0 or +-p,
     in conformance with 4.3BSD.

 From this point of view the key is that converting
(0,0) to polar and back again will get you back to the origin rather
than gratuitously generating (nan,nan) as a simple-minded
application of a formula might do.  A further fine point of dispute
revolves around exactly what finite result should be returned for
the polar argument of (0,0), taking signed zeros into effect. 



More information about the Numeric-interest mailing list