From mboxrd@z Thu Jan  1 00:00:00 1970
X-Spam-Checker-Version: SpamAssassin 3.4.5-pre1 (2020-06-20) on
	ip-172-31-74-118.ec2.internal
X-Spam-Level: 
X-Spam-Status: No, score=-1.9 required=3.0 tests=BAYES_00 autolearn=ham
	autolearn_force=no version=3.4.5-pre1
Date: 4 May 93 19:31:47 GMT
From: eachus@mitre-bedford.arpa  (Robert I. Eachus)
Subject: Re: ADA Performance
Message-ID: <EACHUS.93May4143147@spectre.mitre.org>
List-Id: <comp.lang.ada>

   I had said:

   >>    ...The first and most important is that you must compare two
   >> versions of the SAME program with the same data.

   And later I said...

   >>    Third use each the features of each language as they are expected
   >> to be used...

In article <1993May4.080922.22901@lth.se> dag@control.lth.se (Dag Bruck) writes
:

   > You either have to compare compilers with programs that are as similar
   > as possible, or compare languages by writing "idiomatic" code in each
   > language.  I think both kinds of comparisons are useful and complement
   > each other.

   Not a contradiction, just a fundamental problem with cross-language
benchmarking.  First you need two programs which accomplish the same
functional operations in the same order but are written in different
languages.  Second (or third :-) the should be written to take
advantage of the features of the language, not to avoid otherwise
normal operations.  For a good example assume I decide to use a 1024
point Fourier Tranform as a benchmark.  Okay, I should use the same
algorithm for both languages (assume FFT).  If I am going to use a
complex transform in FORTRAN I shouldn't do the transform modulo a
large integer in Ada.  I should use the same input data, read from the
same file, etc., etc.  But the Ada code might be much more elegently
written as a set of recursive calls using vector operations...as long
as I use the same algorithm to get the same answers.  (With reals they
may not be exact, but they should be very close.  In this case I
actually perfer to do the transform and the inverse and measure the
error...)

   On theother hand, if I want to the sum of the first 1,234,567
integers to benchmark Ada vs APL, I have an insoluble problem.  In APL
I will write:
       
       N <-- +/iota 1234567  

    and be very surprised to get anything other than an instant
response.  (All APL systems I know of use a compact representation for
such vectors, and compute the sum from the three numbers actually
generated.)  What is the equivalent code in Ada or FORTRAN?  Writing
an explict loop is probably wrong, and writing (X * (X+1)) / 2 is
worse.  So this particular benchmark is inappropriate for comparing
these languages.

    This was the problem with the benchmark proposed in the article
that started all this.  A truly agressive FORTRAN compiler can get rid
of almost everything, while no sensible Ada compiler is going to try
to determine, at at compile time, whether or not an exception is going
to be raised.  Put in vectors which contain many different values, and
rotate the vectors so that you get 100 different answers, print all of
the answers out, and you might get a valid benchmark.
--

					Robert I. Eachus

with Standard_Disclaimer;
use  Standard_Disclaimer;
function Message (Text: in Clever_Ideas) return Better_Ideas is...