SPECmarks for 1991 - Sun and IBM

[David G. Hough on validgh]

Chris Aoki asks:

	What do you have against nasa7 and matrix300, other than
        the fact that they don't put in enough time calling the
        elementary functions  :) ?

SPEC applications should each be one realistic application,
and they should all be "orthogonal" in the sense of spanning as large
a space of realistic applications as possible.
nasa7 consists of seven kernels, allegedly from realistic applications,
which I believe; if so then ideally the seven applications should have individually
been SPEC candidates.
matrix300 consists of one kernel, allegedly from a realistic application,
but which really doesn't reflect the best way to do large matrix operations
these days.  The problem with the "kernels" approach is that like
dhrystone, linpack, or the livermore loops, small kernels tend to make
small machines look better than they really are for the kinds of
scientific computations that people actually buy computers for.
 From the results I posted yesterday you might think that there was
little advantage of a Sun-4/490 over a 4/70 other than running GCC;
if you had looked at Perfect Club results you might have concluded
otherwise.

Small kernels tend to make simple compilers look better than they really are,
as well, especially if they're tuned for them.  
We recently had a director-level brushfire because one
customer couldn't reproduce some dhrystone numbers that he'd seen
somewhere.  

You can tune a compiler for saxpy or dhrystone but you can't tune a compiler 
for overall SPEC performance except by laboriously improving every aspect of it.

Small kernels may make sense for I/O benchmarks (I don't know but I doubt it).