Vendor Claims - To Believe or Not to Believe ...

Vendor Claims - To Believe or Not to Believe ...

[Judy.Bamberger]

[CAVEAT:  I have some very dear friends who work for TeleSoft ...]

BUT - When was the last time one accepted any tool vendor's word 
about their own product without serious questioning and justifying
thereof?  And reproducing such a statement out of such context - 
to my naive view, anyway - is a bit irresponsible.

A statement like, "X's compiler generates as good a code as any C 
compiler" is a totally vacuous statement!  Querying about what 
benchmark was used to derive this wonderous conclusion is only the 
*BEGINNING* of the comparison!  USER BEWARE!  What optimisations 
were enabled in both compilation systems?  How much error checking 
was done in both code?  (Note that pragma SUPPRESS /= C-code with 
no checks!)  What kinds of language features were used?  What
compiler-specific coding paradigms were used to exploit the most
efficient code?  What host/target systems, compiler vendors,
ETC, ETC!

Having done some compiler benchmarking myself (in a former life -
JOVIAL (J73) versus assembly language), I have dealt with compiler
vendors and users both, as well as the overseeing DoD programme 
office, and I have seen just how statistics can be abused to an
amazing extent.  Let's be cognisant of what we say and imply, and
be careful about representations in context.

Respectfully,
Judy Bamberger

> 1. Precisely which Ada compilers produce better code than 
>    which C compilers? Not all compilers are created equal...

   The Telesoft TeleGen2 compiler, vs. all available C compilers,
   as of circa October 1988; contact Telesoft for details, since 
   the statement was made at the Telesoft User's Group meeting at
   the Tri-Ada '88 conference.  

> 2. What are the benchmarks and where can they be obtained?  If, in
>    order to generate good code, Ada programmers must limit themselves to
>    a `Pascal subset' of the language exactly what advantages have
>    accrued?

Re: Vendor Claims - To Believe or Not to Believe ...

[William Thomas Wolfe, 2847]

From article <8911171513.AA16049@fa.sei.cmu.edu>, by Judy.Bamberger@SEI.CMU.EDU:
> BUT - When was the last time one accepted any tool vendor's word 
> about their own product without serious questioning and justifying
> thereof?  And reproducing such a statement out of such context - 
> to my naive view, anyway - is a bit irresponsible.

   Forgive me, Judy... obviously you are aware of comparisons which
   were not supplied by vendors, which scientifically control every
   conceivable variable, and which are completely unassailable.  

   Since I am clearly citing comparisons which are less valuable, 
   please contribute the results of these more valuable comparisons 
   to the discussion so that all of us can increase our understanding. 


   Bill Wolfe, wtwolfe@hubcap.clemson.edu