compiler comparison report available for preview

[David G. Hough at validgh]

An abstract follows.   I will send "tbl | troff -ms" source for
the complete report (50 pages) to anybody interested, provided they are
willing to treat it as confidential until published on USENET.   I'm looking
primarily for comments on the way the results are summarized - it's not easy
to fairly account for reams of data, some of which admits to multiple
interpretations.








Comparison of SPARCcompilers 3.0, Apogee 2.302 including KAP, GCC 2.5.8, and
F2C on a variety of realistic applications indicates that:

1)   SPARCcompilers 3.0 is clearly superior in -g compile time, a critical
     factor during program development.

2)   Apogee 2.302 compilers including KAP can provide superior execution-time
     performance for some programs, at the cost of considerable experimenta-
     tion with compile-time options and the risk of incorrect results.

3)   GCC 2.5.8 and F2C are robust but don't provide the same level of SPARC-
     specific optimization as SPARCcompilers or Apogee.

4)   The variation in relative execution-time performance of various com-
     pilers, among different test programs, is enormous, especially at higher
     levels of optimization.    One compiler's code will be twice as fast as a
     competitor's on one program, and twice as slow on another program.

     Since this study ended, new releases of all these products have become
available, but it appears that the major conclusions are unchanged:

1)   SPARCcompilers provide "industrial-strength" robustness and good SPARC
     code generation, and are available with the SPARCworks development
     environment.

2)   The GCC+F2C combination is worth investigating when programs must be com-
     pilable on many different platforms, compilers must be robust, and max-
     imum optimization, especially floating-point optimization, is not
     required.

3)   The Apogee compilers with KAP are worth investigating when considerable
     skilled manpower is available to tune the compiler options for each pro-
     gram.  Even so it is still advisable to measure SPARCcompilers as well
     since they sometimes provide superior available performance.

     A smaller study of Sun's PROcompilers 2.0.1, GCC 2.5.8. and F2C indicates
that:

1)   GCC appears slightly more robust than PROcompilers but neither optimizer
     is as fully debugged as SPARC.

2)   The combination of 2.0.1 cc + F2C provides best execution times at max-
     imum optimizations.

3)   PROcompilers 2.0.1 provides best compile times at -g.