a question and an answer about optimizing compilers; your chance to rebut

[Dr. Bernd Hartke]

In a recent posting, I complained about problems with installing LAPACK on
an HP735 workstation under HP-UX9.01, and David Hough answered:

> As for LAPACK, I have read that HP uses simple but fast algorithms for
> complex division that cause it to fail frequently with gratuitous
> intermediate overflow/underflow problems.
Without me having specified any of the errors I encountered, you are right:
I have decidedly more problems with complex/complex*16 than with single or
double precision. 

> Particularly at high levels of optimization using preprocessors (e.g. KAP
> which is built into HP, IBM, and SGI compilers, I think) you will run into
> many difficulties, some of which are due to the compilers and some to the
> source code to be compiled (although probably not the latter with LAPACK).
> Basically you are doing quality assurance for the compiler vendors and the
> source coders. 
True again - with LAPACK1.0a/1.0b under HP-UX9.01 on an HP720 I went to great 
lengths (for a humble end user) to get things working, e.g., manual insertion
of HP vector routines into the code (the preprocessor would not do that for
me in most cases), actual bug hunt in some of the source code, fine tuning
of machine constants away from what a canned routine claimed that they should
be, etc. And I ended up with still not all things working properly.
  The LAPACK staff was very helpful, but they did not have an HP expert at
hand. And an HP expert (actually, an HP employee) I contacted proved to be
not sufficiently interested in this issue to look for a working solution.

> ... There are many issues here, of language definitions,
> existing bodies of source code, kluges for SPEC results, and end user
> expectations, all of which are dear to my heart.
My original expectation as an end user was simply: I want to base the
application codes I am writing myself on a library that (i) is readily
available everywhere (ensuring portability, and cutting down the length of
my own programs), and that (ii) ensures some high level of base performance,
without too much manual intervention by me. The latter point looked like
being fulfilled by LAPACK, since it is based on the BLAS, which is so
commonplace that many computer vendors supply BLAS-versions tuned to their
own machines - with the exception of HP (they are promising it for HP-UX10.0,
as far as I have heard). Reliability and ease of installation were two issues
I was basically taking for granted. It looks like I was being quite naive...

> What do you think?  My belief is that most end users, most of the time, 
> just want to make their existing programs run faster UNCHANGED.  
> That includes makefiles and shell scripts, which are programs too.  
Let me say one or two more words as to user expectations and user effort: 
As I hinted at above, if I can expect some guarantee that the software will
turn out to fulfill points (i) and (ii) above (plus reliability), *and* if I
have to do it only once, I am perfectly willing to even dive into the source
code jungle and do some hand tuning in order to get good performance.  I am
much less willing to do this just for making the programs work at all. 
With respect to makefiles/shell scripts: I have recently installed quite a bit
of software here, and I have not yet seen a makefile or shell script (apart
from the totally trivial ones) that did not need some minor or major
modifications, particularly if it comes to X11. But I do not expect things
to work right away - this would mean that either the makefile is infinitely
clever about looking for libraries etc. (I have had a very elaborate one here
that differentiated between several dozen machines - but it still failed on
ours...), or all these issues are so much standardized that it becomes
akward to work with them at all.

But coming back to the LAPACK issue: I am still looking for someone who has
experience with the issue at hand (LAPACK1.1 on HP735 under HP-UX9.01) and
is willing to give me one or two concrete hints or some help.

    Bernd

--
Dr. Bernd Hartke                     Email: berndatc1.chemie.uni-bielefeld.de 
University of Bielefeld              Email: bxhawiffin.chem.ucla.edu           
Department of Chemistry              Tel. : +49-521-1066172                   
Theoretical Chemistry                FAX  : +49-521-1066146                   
33615 Bielefeld                      Raum : E 4 - 148                         
GERMANY