Ada speed vs. C...

Ada speed vs. C...

[Bill Wolfe]

   In response to recent requests for comparisons of the speed of
   the code generated by Ada compilers vs. C compilers, the following
   results were originally described at the Tri-Ada '88 conference
   in Charleston, WV on October 26, 1988:
 
--------------------------------------------------------------------------
Among the C compilers chosen were those that produced the best times on 
16.77 Mhz 68020 based machines out of a list of published dhrystone results 
for 300 compilers.  The TeleSoft TeleGen2 optimizing compiler was compared 
against four other C compilers:

          Sun/Unix 3.2 C on a Sun 3/160
          GreenHills C on ISI optimum V machine
          Gnu C on a Sun 3/160
          MASSCOMP C on RTU 3.1, NC-5700 machine

MASSCOMP had the best C compiler, our Ada compiler only beat it by a very
small margin.

The whole point of the comparison was not to prove that Ada is better than
C but that it's possible for Ada compilers to generate code that is just
as good or better than C compilers.  It takes a lot of time and effort to
go from a compiler that meets the minimum language requirements to a
highly optimizing compiler.  Ada compilers will continue to improve and
you can expect to see more optimizing compilers from a variety of sources.

Gary Morris      {uunet,ucsd}!telesoft!garym       TeleSoft
N6FRT            garym@telesoft.com                San Diego, CA, USA
ASEL             garym@nynashamn.telelogic.se      +1 619 457 2700
"If something is worth doing, it's worth doing right."
--------------------------------------------------------------------------

   Bill Wolfe, wtwolfe@hubcap.clemson.edu

Re: Ada speed vs. C...

[Richard A Hammond]

In article <8347@hubcap.clemson.edu> wtwolfe@hubcap.clemson.edu (Bill Wolfe) writes:

Summary: Bill just reposted an article from  Gary Morris of Telesoft
which reported that the Dhrystone results for the Telesoft compiler on
a 16.67 Mhz 68020 based system beat out 4 C compilers on similar systems.
The results were reported in 1988 at the Tri-Ada conference.

As even Reinhold Weicker admits (posting to Usenet, also in Sigplan Notices):

1) Dhrystone uses only the "Pascal Subset" of Ada.

2) Ada runtime checks should be disabled for comparison to other languages!

3) There is some concern that string operations are over-represented,
   as HrDr Weicker notes: [ quoted from his posting of Feb 11, 1989]

It should be noted that this is a language-dependent issue:  Dhrystone was
first published in Ada, and with Ada or Pascal semantics, the time spent in
the string operations is, at least in all implementations known to me,
considerably smaller than in C.  In Ada and Pascal, assignment and comparison
of strings are operators defined in the language, and the upper bounds of the
strings occuring in Dhrystone are part of the type information known at
compilation time.  The compilers can therefore generate efficient inline code
whereas in C, the string operations must be expressed in terms of the C
library functions "strcpy" and "strcmp".  (This is probably the main reason
why on most systems known to me, the Ada and Pascal version are faster than
the C version.)

I admit that the string comparison in Dhrystone terminates later (after
scanning 20 characters) than most string comparisons in real programs.  For
consistency with the original benchmark, I didn't change the program despite
this weakness.

-- END QUOTE --

So, while it is nice to see that Ada compilers can do as well as or better than
C, be aware of the conditions of the test and be sure that you understand
what the benchmarks measure.  In particular, I'd suggest getting
the number for the runtime checks enabled run, since that
should be the way you actually run Ada programs.

Rich Hammond

Re: Ada speed vs. C...

[William Thomas Wolfe, 2847]

From hammondr@sunroof.crd.ge.com (Richard A Hammond):
> [some comments regarding the Dhrystone benchmark for Ada vs. C]

  The comments on that issue were:

     it is widely used in spite of its minor flaws.  One
     of the reasons for using it was that numerous C compilers
     have been benchmarked with it and the results have been 
     published and are readily available for comparison. 

  (Note: Followups and continued discussion to comp.lang.misc...)


  Bill Wolfe, wtwolfe@hubcap.clemson.edu
 

Re: Ada speed vs. C...

[Tim Olson]

In article <8347@hubcap.clemson.edu> wtwolfe@hubcap.clemson.edu (Bill Wolfe) writes:
| 
|    In response to recent requests for comparisons of the speed of
|    the code generated by Ada compilers vs. C compilers, the following
|    results were originally described at the Tri-Ada '88 conference
|    in Charleston, WV on October 26, 1988:
|  
| --------------------------------------------------------------------------
| Among the C compilers chosen were those that produced the best times on 
| 16.77 Mhz 68020 based machines out of a list of published dhrystone results 
| for 300 compilers.  The TeleSoft TeleGen2 optimizing compiler was compared 
| against four other C compilers:
| 
|           Sun/Unix 3.2 C on a Sun 3/160
|           GreenHills C on ISI optimum V machine
|           Gnu C on a Sun 3/160
|           MASSCOMP C on RTU 3.1, NC-5700 machine
| 
| MASSCOMP had the best C compiler, our Ada compiler only beat it by a very
| small margin.

Dhrystone should not be used to compare the performance of Ada
compilers vs. C compilers.  It over-emphasizes string operations,
which ends up penalizing C with its implementation of null-terminated
strings. Many compiler/machine combinations spend ~30% of their time
in strcpy() when running this benchmark!

Most real programs don't use the string routines nearly as much as
Dhrystone, so the performance difference between Ada and C on this
benchmark doesn't reflect real-world results.  I'm not saying that Ada
is necessarily slower than C on non-string-intensive code, but the
performance of a number of different benchmarks should be compared to
ensure that you aren't seeing an anomaly or artifact of a particular
benchmark.


	-- Tim Olson
	Advanced Micro Devices
	(tim@amd.com)

Re: Ada speed vs. C...

[David Kassover]

In article <29483@amdcad.AMD.COM> tim@amd.com (Tim Olson) writes:
>In article <8347@hubcap.clemson.edu> wtwolfe@hubcap.clemson.edu (Bill Wolfe) writes:
>|           Sun/Unix 3.2 C on a Sun 3/160
>|           GreenHills C on ISI optimum V machine
>|           Gnu C on a Sun 3/160
>|           MASSCOMP C on RTU 3.1, NC-5700 machine

Benchmark programs ennable you to tell how fast the BENCHMARK
PROGRAM (not *your* application) will run on a particular
machine.

As far as I am concerned, the only valid comparison above is
between the first and third, since they were compiled and run on
the same machine.

And should be given strong weight iff your application is the
Dhrystone benchmark.

And furthermore, I suspect that there is more than one way to
code the function of the Dhrystone benchmark programs in Ada,
thus giving different instruction sequences for a given compiler.

Is an orange inherently better than an apple?  Perhaps, if all
you care about is Vitamin C.  On the other hand, if you need to
alleviate constipation...  :-)

Enough, Already!

Re: Ada speed vs. C...

[Mike Percy]

From article <8347@hubcap.clemson.edu>, by wtwolfe@hubcap.clemson.edu (Bill Wolfe):
> 
[talks about an Ada compiler whose output code is faster than four C compilers]

Yawn...and maybe when I get back from lunch my compile will be done.  "All
I did was correct a misspelling in a comment in the linked-list package
specification..."  Ooops. 

Re: Ada speed vs. C...

[William Thomas Wolfe, 2847]

From grimlok@hubcap.clemson.edu (Mike Percy):
> Yawn...and maybe when I get back from lunch my [Ada] compile will be done.  
> "All I did was correct a misspelling in a comment in the linked-list 
> package specification..."  Ooops. 

   When you submit a program unit for compilation, the compiler has to
   assume that if all you wanted was to change a comment and get a compiler
   listing, you would have had the sense to do the compilation in a temporary
   program library rather than recompile a fundamental specification and
   thereby invalidate everything (probably the entire software system) that
   was compiled against that particular specification.  

   If you had changed something substantive, then the recompilation would 
   be necessary in order to propagate the effects of the change throughout 
   the system.  It would have to be a fundamental semantic or syntactic 
   change, since if it were merely an implementation detail the effects 
   would have been confined to the package body.   

   (Followups to comp.lang.misc...)


   Bill Wolfe, wtwolfe@hubcap.clemson.edu

Re: Ada speed vs. C...

[Wayne Wood]

In article <8347@hubcap.clemson.edu> wtwolfe@hubcap.clemson.edu (Bill Wolfe) writes:
>
>          Sun/Unix 3.2 C on a Sun 3/160
>          GreenHills C on ISI optimum V machine
>          Gnu C on a Sun 3/160
>          MASSCOMP C on RTU 3.1, NC-5700 machine
>
>MASSCOMP had the best C compiler, our Ada compiler only beat it by a very
>small margin.
>

somebody said there are lies, damn lies, and statistics.  i don't see any
stats here. 

why don't you post the complete table and let us make our own decisions?

as it is, we have to take your word for it... not very good salemanship.

/***   woody   ****************************************************************
*** ...tongue tied and twisted, just an earth bound misfit, I...            ***
*** -- David Gilmour, Pink Floyd                                            ***
****** woody@eos.arc.nasa.gov *** my opinions, like my mind, are my own ******/

Re: Ada speed vs. C...

[Vinod Grover]

In article <8347@hubcap.clemson.edu> wtwolfe@hubcap.clemson.edu (Bill Wolfe) writes:
>
>   In response to recent requests for comparisons of the speed of
>   the code generated by Ada compilers vs. C compilers, the following
>   results were originally described at the Tri-Ada '88 conference
>   in Charleston, WV on October 26, 1988:
> 
>--------------------------------------------------------------------------
>Among the C compilers chosen were those that produced the best times on 
>16.77 Mhz 68020 based machines out of a list of published dhrystone results 
>for 300 compilers.  The TeleSoft TeleGen2 optimizing compiler was compared 
>against four other C compilers:
>
>          Sun/Unix 3.2 C on a Sun 3/160
>          GreenHills C on ISI optimum V machine
>          Gnu C on a Sun 3/160
>          MASSCOMP C on RTU 3.1, NC-5700 machine
>
>MASSCOMP had the best C compiler, our Ada compiler only beat it by a very
>small margin.
>
>The whole point of the comparison was not to prove that Ada is better than
>C but that it's possible for Ada compilers to generate code that is just
>as good or better than C compilers.  It takes a lot of time and effort to
>go from a compiler that meets the minimum language requirements to a
>highly optimizing compiler.  Ada compilers will continue to improve and
>you can expect to see more optimizing compilers from a variety of sources.


I took the trouble to call TeleSoft and obtained some information about
these results. They sent me a brochure describing the Ada vs C results.
They report highest performance on a Sun-3 relative to 3 C compilers. 
The test is the PIWG Dhrystone. 

They also mention in that brochure that for the Ada version checks were
turned off. 

Though, of course Mr Wolfe conveniently omits this piece of information.  If
you turn off constraint checks, you are sacrificing the much-touted safety
of Ada. In *this respect*, I do not see how you are different from
programming in C or any other unsafe language.


---------
Vinod Grover

Disclaimer: I am not speaking for my employer.

Re: Ada speed vs. C...

[Steve Tynor]

In article <132920@sun.Eng.Sun.COM> grover@sun.UUCP (Vinod Grover) writes:
...
>They also mention in that brochure that for the Ada version checks were
>turned off. 
>
>Though, of course Mr Wolfe conveniently omits this piece of information.  If
>you turn off constraint checks, you are sacrificing the much-touted safety
>of Ada. In *this respect*, I do not see how you are different from
>programming in C or any other unsafe language.

Then you're ignoring all the compile-time benefits of Ada (a real type system,
separate specification/implemetation, etc.).  Just because certain run-time
checks were disabled does _not_ make Ada C.

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Eiffel: Accept no substitutes
                     
    Steve Tynor
    Georgia Tech Research Institute
    Artificial Intelligence Branch
    tynor@prism.gatech.edu