LCAS, LAPACK and LINPACK

[James Demmel]

I would like to add the following computational experience regarding the
development of portable software that can accommodate VAX D-format. In the
LAPACK project we have spent much of the last two weeks trying to get our
rather comprehensive test code to pass all its tests on a VAX in D-format. It
passes everywhere else we have tried so far, including IEEE arithmetic,
Crays, IBMs, and VAX single precision (F-format). 

We have tried to write software that can robustly handle problems with
badly scaled data, including very small or very large values. The limits
on how small the data can be, vary for different classes of algorithm, but
may be SAFMIN/EPS, SQRT(SAFMIN) or SQRT(SAFMIN/EPS), where SAFMIN is a
"safe minimum" close to the underflow threshold, and EPS is the machine
precision. IN VAX D-format, these three expressions have the approximate
values 1.E-22, 1.E-20, and 1.E-11, which are hardly very tiny, especially
when compared with the machine precision of 1.E-17.

Our test software is designed to test the software close to these limits. The
narrow exponent range on the VAX (combined with the lack of gradual underflow)
has required us to do extra work, refining details of the routines and test
software, in order to get the routines to pass the tests. In the end we are
still left with a few large "test ratios" (basically scaled residual norms),
that we have not yet been able to diagnose or cure. Since these large residuals
only occur for badly scaled problems and only in VAX D-format, we have decided,
with some regret, to go ahead and release the software without having
eliminated them, since we do not want any more delay in releasing the software.
I mentioned this result to Pete Stewart, one of the original LINPACK 
developers, and he said that they had never gotten LINPACK to work in all 
cases in VAX D-format either, and had given up too. (In fact, Velvel Kahan 
and I began our work on the bidiagonal SVD just by trying to fix the underflow
bug that caused the LINPACK SVD to occasionally fail to converge in Vax D.)

This confirms my opinion that it is not our software that
is poorly designed, but the VAX D format. Of course, it would be probably be
possible in the end to fix the problems with either LINPACK or LAPACK, but the
point is that a lot of extra work is required, and extra complexity must be
added to the code, which would not be needed for most other floating-point
formats in current use. 

This (among other reasons) leads me to oppose a standard like LCAS that would
permit VAX D-format in the name of aiding portable software development. 

   Jim Demmel
   Math Dept. and CS Division
   U. C. Berkeley
   demmelacs.berkeley.edu