first release of UCBTEST just about ready

[David G. Hough at validgh]

I hope to submit UCBTEST to netlib at the end of this week.

But I'll send a final early version to anybody who can commit to trying it
out on a non-Solaris system, revising the appropriate sample shell
script, and letting me know the results by Friday morning.
If you can't act that quickly, just wait for the netlib announcement.
Here's the README.




A B S T R A C T

UCBTEST is a suite of programs for testing certain difficult test cases of
IEEE 754 floating-point arithmetic.  This early access version of ucbtest is
provided to individual readers of numeric-interestavalidgh.com and
comp.arch.arithmetic.  Please report any porting problems or logic errors you
confirm to fdlibm-commentsasunpro.sun.com; of particular interest are versions
of ucbREADME/*.sh for non-Solaris Unix-like systems.  Some source code modules
have various copyright restrictions; please read and observe them.




     UCBTEST is a suite of programs for testing certain difficult test cases
of IEEE 754 floating-point arithmetic.  Some of the difficult test cases are
obtained from number-theoretic algorithms developed by Turing Award winner
Prof. W. Kahan, Department of Electrical Engineering and Computer Science,
University of California, Berkeley, as part of ongoing research into test
methods for computer arithmetic.

     After Prof. Kahan presented a Floating-Point Indoctrination lecture
series at Sun Microsystems in 1988, some lecture attendees and former graduate
students of Prof. Kahan wrote the ucbtest/mul.c, ucbtest/div.c, and
ucbtest/sqrt.c programs.

     ucbtest also includes versions of Kahan's paranoia and pirats programs,
ucbctest/par.c and ucbctest/pi.c, as reworked by various collaborators.
ucbctest/par.c's current pass/fail criterion reflects one particular set of
prejudices about x**0 and double rounding during underflow; others may prefer
different choices.   ucbctest/pi.c is designed to detect how accurate an
approximation to pi is employed in the standard trigonometric functions, and
whether it is used consistently.

     ucbeeftest determines the worst errors in common elementary transcenden-
tal functions. Note its additional usage restrictions set forth in the README
and source code.  It also comes with

phase1.c
     a setup program phase1.c to help create local.h in case the one supplied
     is inadequate.

findpi.c
     a setup program to help determine how accurate an approximation to pi is
     used in trigonometric argument reduction, to confirm the user's specifi-
     cation of that pi approximation to phase1.c.

beefscan.c
     a program to condense the output of ucbeeftest to a manageable level,
     quite brief when -DQUIET is specified during compilation.

     ucblibtest, for exercising test vectors of K-C Ng, is still under
development to fit the ucbtest paradigm.   Even so the current version may be
useful for some purposes.

Porting

     All the ucbtest programs are intended for users with a good understanding
of IEEE 754 arithmetic and C.   If misused - especially after porting to a new
environment for the first time - they may report arithmetic "errors" when
nothing is amiss.   Nor do "error-free" results prove correct arithmetic,
although they may "lend artistic verisimilitude to an otherwise bald and
unconvincing" proof.

     Some sample sh scripts in the ucbREADME directory, sc301.sh, pc201.sh,
sunos4.sh, and gcc.sh, for various compilers available on SunOS, compile and
run all the programs, for all three floating-point types float, double, and
long double.  The test programs are designed to terminate with non-zero exit
status in case they find an error, unless compiled with -DNOFAIL.  An instance
of the infamous Pentium FDIV flaw can be found in less than a minute by com-
piling ucbdivtest with -DNTESTS=1000000 -DSP etc.  This however is unexpected
since the program was not designed to seek out such gross errors inherent in a
particular division method, but rather looks for least-significant-bit errors
independent of division method.  The Pentium bug is not detected with -DDP or
-DQP in a reasonable amount of time.

     ucbmultest, ucbdivtest, and ucbsqrtest were developed on SPARCstations
running SunOS 4 or SunOS 5, and on x86 PC's running SunOS 5.   ucbdivtest and
ucbsqrtest may be ported to other RISC/Unix systems with little difficulty.
Systems with extended precision, non-Unix systems, and ucbmultest require more
effort to provide the equivalents of the Sun functions ieee flags etc.

     David Gay improved several aspects of the programs.

     Warren Ferguson suggested many improvements for better portability to
MSDOS, and has provided executables for MSDOS compiled with Borland C++ for
Windows; see ucbDOS/readme.

     Some compile-time flags used in the porting process to Sun include:

        -DNTESTS       number of test cases to try
        -DDEBUG        turn out debugging output
        -DSP           test IEEE single precision
        -DDP           test IEEE double precision
        -DQP           test IEEE extended precision
        -DFLOAT        test C float type
        -DDOUBLE       test C double type
        -DLONGDOUBLE   test C long double type
        -DSUN IEEE     ieee flags, ieee retrospective available
        -DSUN MATH     sunmath.h and libsunmath.a are available
        -DNO FUNCF     no ANSI-C reserved functions sqrtf, expf, etc.
        -DNOFAIL       do not abort on certain errors


Misapplication

     Misapplication often causes these programs to fail, reporting hardware or
software bugs where none exist.  Common causes include:

precision
     On extended-precision systems such as x86 and 68k, the extended-precision
     precision control must be set to the precision of the type of floating-
     point variable being tested.  On Sun systems,
     ieee flags("set","precision",...) is useful.

prototypes
     Traditional C and ANSI-C differ in their treatment of float function
     arguments and function results.   It may be necessary to declare addi-
     tional function prototypes to avoid problems.

traditional C expression evaluation
     Traditional C may evaluate expressions involving floats by promoting them
     first to double, causing double rounding errors. Compiling with "-
     fsingle" is helpful with /bin/cc on SunOS 4.

optimizers
     Optimizing compilers that allocate variables to extended-precision regis-
     ters and fail to round them to storage precision, even in response to
     source-code assignments, may cause problems.   Certain variables have
     been declared "volatile" in hope of avoiding those problems.  Particu-
     larly on PC's, compilers seem to have trouble correctly optimizing
     single-precision code so that it works with rounding precision set to
     single instead of extended; both Sun's compilers and GCC show problems of
     this sort.   Perhaps more "volatile" declarations are required.   On GCC,
     -ffloat-store sometimes helps.

sqrt functions
     C compilers usually don't convert calls to sqrtf, sqrt, or sqrtl to
     inline machine sqrt instructions by default. Thus testing those instruc-
     tions, rather than the subroutine library, may require extra steps such
     as coding in assembly language.    Sun compilers invoked with "-xlibmil"
     provide the necessary inlining to test SPARC or x86 sqrt instructions.

Source code

     Several of the programs are set up so that the C infrastructure may be
used to test the arithmetic and libraries of any language.  Sample wrappers
are provided for C and Fortran.  Outer main programs are in ucbctest and
ucbftest; they use inner main programs and utility functions from ucbtest to
invoke the test functions in libctest or libftest.

     Makefiles are provided in the ucbctest, ucbftest, libctest, and libftest
directories.  Sample *.sh scripts for invoking the Makefiles are in the
ucbREADME directory.   The scripts define -NTESTS=1000000; reduce that to sim-
plify debugging; increase it for a more comprehensive test.
ucb{mul,div,sqr,eef}test have been adjusted to that their run times are
approximately comparable for comparable NTESTS.  ucb{pi,par,findpi,lib}test
run times do not depend on NTESTS.

     The scripts define which precisions and which languages are to be tested,
and how each language is to be compiled.

     Some scripts use make -k to enable testing to continue past failures of
various sorts; for instance, the GCC scripts for SunOS 5.3 can't run ucbmul-
test because they don't have access to ieee flags in Sun's unbundled
libsunmath.a.

     Generally speaking, the source code uses -DSP, -DDP, and -DQP to distin-
guish single, double, and extended precision.

Directory information

     ucbctest and ucbftest contain the main programs for each test, in C and
Fortran respectively.

     libctest and libftest contain the test modules which invoke each function
to be tested, in C and Fortran respectively.   Each function is in a separate
module so that compilation failure on one will not affect other tests.   On
systems which lack some of the test functions, it is necessary to improvise;
see for instance libctest/testexpm1*.c.

     ucbtest contains the C logic for the programs that can be invoked from
main programs in C or Fortran.

     ucbeef contains the various source files invoked by ucbeeftest and
ucbfindpitest.

     ucblib contains the input files for ucblibtest.

     ucbREADME contains the readme and *.sh scripts.

     ucbDOS contains Warren Ferguson's adaptations of certain test programs to
work under DOS by compiling with Borland C++ under Windows.

Test information

     ucbpitest - attempts to determine a rational approximation A/B close to
the value of PI used in a system's trigonometric functions.   ucbctest/pi.c is
a complete translation into C of ucbftest/pi.F, derived from the pirats pro-
gram.

     ucbpartest - attempts to determine general properties of computer arith-
metic by ostensibly portable means.   ucbftest/par.F and ucbctest/par.c are
complete translations from the original Basic program of Kahan.

     ucbdivtest - by number-theoretic means, generates division test cases
nearly halfway between representable numbers in order to test the correctness
of alleged IEEE 754 implementations.  The logic is in ucbtest/div.c.
Exactly halfway cases don't arise in division.

     ucbsqrtest - by number-theoretic means, generates sqrt test cases nearly
halfway between representable numbers.    The logic is in ucbtest/sqr.c.
Exactly halfway cases don't arise in sqrt.

     ucbmultest - by number-theoretic means, generates multiplication test
cases that are exactly halfway and nearly halfway between representable
numbers.   The logic is in ucbtest/mul.c.

     ucbfindpitest - determines an approximation for pi suitable for use in
ucbeef/local.h.

     ucbeefscan - a postprocessor for ucbeeftest that summarizes its detailed
output.

     ucbeeftest - programs for testing sin, cos, atan, exp, expm1, log, and
log1p.  Random arguments are used to evaluate the test functions for com-
parison to independently computed values accurate to better than test function
precision.  In ucbtest, the program is compiled and run separately for each
function.  The interface is somewhat different from the original form of the
program, available from zaliuaeng.sun.com.

     ucblibtest -  hex test vectors for library functions in single, double,
and quad precision.  x86 double extended is not currently supported.   Consid-
erable additional work remains to be done on this program.  The source codes
available now may be helpful, but are still changing.

Futures

     Enhancements planned to be incorporated some day:

*    Ng's libm test vectors - ucblibtest - are still in the process of conver-
     sion to the ucbtest format.

*    Coonen's IEEE Test Vectors program and data.

*    Hough's tests for correctly-rounded base conversion.

*    Kahan's tests for SRT-PLA-style dividers.

     This readme may be formatted with commands like

        tbl readme | troff -ms


     Please report bugs and portability enhancements for other systems to
fdlibm-commentsasunpro.sun.com and to dghavalidgh.com.