UCBTEST early testing extended to comp.arch.arithmetic

[David G. Hough at validgh]

Happy new year... I just posted the following to comp.arch.arithmetic:



The following software is available for early testing to persons willing to
abide by the restrictions listed in the first paragraph:





This version of ucbtest is provided under non-disclosure, for testing
purposes, to individual subscribers to comp.arch.arithmetic.  Please indicate
your willingness to respect the non-disclosure by quoting this paragraph when
requesting ucbtest from dghavalidgh.com, and do not distribute the programs
further or report results except to numeric-interestavalidgh.com or
dghavalidgh.com.  But please report any porting problems or logic errors you
confirm; of particular interest are reports of successful ports to to Alpha
and PA-RISC workstations.  After external testing has turned up any small
avoidable problems, these programs will be made publicly available with this
copyright notice:
/*
 * ====================================================
 * Copyright (C) 1988-94 by Sun Microsystems, Inc. All rights reserved.
 *
 * Developed at SunPro, a Sun Microsystems, Inc. business.
 * Permission to use, copy, modify, and distribute this
 * software is freely granted, provided that this notice
 * is preserved.
 * ====================================================
 */

     UCBTEST is a suite of programs for testing certain difficult test cases
of IEEE 754 floating-point arithmetic.  The difficult test cases are obtained
from number-theoretic algorithms developed by Turing Award winner Prof. W.
Kahan, Department of Electrical Engineering and Computer Science, University
of California, Berkeley, as part of ongoing research into test methods for
computer arithmetic.  After Prof. Kahan presented a Floating-Point Indoctrina-
tion lecture series at Sun Microsystems in 1988, some lecture attendees and
former graduate students of Prof. Kahan wrote the ucbmultest, ucbdivtest, and
ucbsqrtest programs.

     ucbtest also includes versions of the paranoia and pirats programs,
ucbpartest and ucbpitest, as reworked by various collaborators.  ucbpartest's
current pass/fail criterion reflects one particular set of prejudices about
x**0 and double rounding during underflow; others may prefer different
choices.   ucbpitest is designed to detect how accurate an approximation to pi
is employed in the standard trigonometric functions, and whether it is used
consistently.

Porting

     All the ucbtest programs are intended for users with a good understanding
of IEEE 754 arithmetic and C.   If misused - especially after porting to a new
environment for the first time - they may report arithmetic "errors" when
nothing is amiss.   Nor do "error-free" results prove correct arithmetic,
although they may "lend artistic verisimilitude to an otherwise bald and
unconvincing" set of proofs.

     Some sample sh scripts, sunos4.sh and sunos5.sh for SPARC, and x86.sh for
PC's running SunOS 5, compile and run all the programs, for all three
floating-point types float, double, and long double; first with 10**3 test
points, then with 10**4, ... 10**9, unless a failure occurs or the user, pati-
ence exhausted, terminates the script with ^C.  The test programs are designed
to terminate with non-zero exit status in case they find an error.  Approxi-
mately 20 minutes are required on a 60MHz Pentium for the x86.sh script to
locate an instance of the infamous FDIV bug, although it can be seen in less
than a minute by compiling ucbdivtest with -DNTESTS=1000000 -DFLOAT etc.  This
however is unexpected since the program was not designed to seek out such
gross errors inherent in a particular division method, but rather looks for
least-significant-bit errors independent of division method.

     ucbmultest, ucbdivtest, and ucbsqrtest have been designed to run on
SPARCstations running SunOS 4 or SunOS 5, and on x86 PC's running on SunOS 5.
ucbdivtest and ucbsqrtest may be ported to other RISC/Unix systems with little
difficulty.   Systems with extended precision, non-Unix systems, and ucbmul-
test require more effort to provide the equivalents of the Sun functions
ieee flags etc.

     David Gay has provided a simple sample makefile to compile on SGI or
RS/6000 workstations, and a collection of .bat files for various PC compilers,
and suggested a number of other improvements.

     Some compile-time flags used in the porting process to Sun include:

-DNTESTS        number of test cases to try
-DDEBUG         turn out debugging output
-DFLOAT         test C float type
-DDOUBLE        test C double type
-DLONGDOUBLE    test C long double type
-DPROTOTYPED    ANSI-C compiler, so use function prototypes - important
                for float and long double
-DSUN IEEE      ieee flags, ieee retrospective available
-DSUN MATH      sunmath.h and libsunmath.a are available


Misapplication

     Often these programs appear to fail due to misapplication rather than
incorrect hardware or libraries.  Common causes include:

precision
     On extended-precision systems such as x86 and 68k, the extended-precision
     precision control must be set to the precision of the type of floating-
     point variable being tested.  On Sun systems,
     ieee flags("set","precision",...) is useful.

prototypes
     Traditional C and ANSI-C differ in their treatment of float function
     arguments and function results.   It may be necessary to declare addi-
     tional function prototypes to avoid problems.

traditional C expression evaluation
     Traditional C may evaluate expressions involving floats by promoting them
     first to double, causing double rounding errors. Compiling with "-
     fsingle" is helpful with /bin/cc on SunOS 4.

optimizers
     Optimizing compilers that allocate variables to extended-precision regis-
     ters and fail to round them to storage precision even in response to
     source-code assignments may cause problems.   Certain variables have been
     declared "volatile" in hope of avoiding those problems.

sqrt functions
     C compilers usually don't convert calls to sqrtf, sqrt, or sqrtl to
     inline machine sqrt instructions by default. Thus testing those instruc-
     tions, rather than the subroutine library, may require extra steps such
     as coding in assembly language.    Sun compilers invoked with "-xlibmil"
     provide the necessary inlining to test SPARC or x86 sqrt instructions.

Futures

     Minor work yet to be done:

*    Consolidation of the failure printing functions from ucb???test.c to
     ucbtest.c.

*    General cleanup and reconciliation of comments with source code.

     Major enhancements less likely to be incorporated soon:

*    Coonen's IEEE Test Vectors program and data.

*    Liu's Berkeley Elementary Functions tests.

*    Hough's tests for correctly-rounded base conversion.

*    Kahan's tests for SRT-PLA-style dividers.

     Please report bugs and portability enhancements for other systems to
David Hough, dghavalidgh.com.