Vendor Claims - To Believe or Not to Believe ...

[Judy.Bamberger@SEI.CMU.EDU]

[CAVEAT:  I have some very dear friends who work for TeleSoft ...]

BUT - When was the last time one accepted any tool vendor's word 
about their own product without serious questioning and justifying
thereof?  And reproducing such a statement out of such context - 
to my naive view, anyway - is a bit irresponsible.

A statement like, "X's compiler generates as good a code as any C 
compiler" is a totally vacuous statement!  Querying about what 
benchmark was used to derive this wonderous conclusion is only the 
*BEGINNING* of the comparison!  USER BEWARE!  What optimisations 
were enabled in both compilation systems?  How much error checking 
was done in both code?  (Note that pragma SUPPRESS /= C-code with 
no checks!)  What kinds of language features were used?  What
compiler-specific coding paradigms were used to exploit the most
efficient code?  What host/target systems, compiler vendors,
ETC, ETC!

Having done some compiler benchmarking myself (in a former life -
JOVIAL (J73) versus assembly language), I have dealt with compiler
vendors and users both, as well as the overseeing DoD programme 
office, and I have seen just how statistics can be abused to an
amazing extent.  Let's be cognisant of what we say and imply, and
be careful about representations in context.

Respectfully,
Judy Bamberger

> 1. Precisely which Ada compilers produce better code than 
>    which C compilers? Not all compilers are created equal...

   The Telesoft TeleGen2 compiler, vs. all available C compilers,
   as of circa October 1988; contact Telesoft for details, since 
   the statement was made at the Telesoft User's Group meeting at
   the Tri-Ada '88 conference.  

> 2. What are the benchmarks and where can they be obtained?  If, in
>    order to generate good code, Ada programmers must limit themselves to
>    a `Pascal subset' of the language exactly what advantages have
>    accrued?