Re: 10 rules for benchmarking (was Re: Teaching new tricks to an old dog (C++ -->Ada))

[Georg Bauhaus <sb463ba@user1.uni-duisburg.de>]

CTips wrote:
> Georg Bauhaus wrote:

> 
>> Anyway, here goes.
> 
> 
> This is pretty good work.

Thank you!


>>                      C             |             Ada
>> N        R       *T*         TC      *T*      TC      TU      TUC
 >> ...
 >> ...
>> 100000   2000    7.4         7.25    7.2      7.0     7.0     7.2
>> 200000   1000    7.38        7.26    7.22     6.9     6.8     7.1
> 
> 
> For some reason your numbers at the higher end are anamolous. The 
> checked code should not take less time than the unchecked code. There is 
> something else going on.

I found this puzzling, too. I'll try some more combinations next week.


> You're running on a 2-processor system, so you could be ping-ponging 
> between processors. Does the version of linux you are running have 
> processor affinity? Also, if you're running on an loaded system, this 
> would explain things.

The system showed a load of 0.0 between tests, runlevel == 1, no jobs.
I'll try again setting the affinity.


> Ada is not faster- the results where Ada is faster are not trustworthy. 
> Replace the C checks with the alternate form, and run on an *unloaded* 
> machine.

Well... To me,
                     T           TC      T        TC      TU      TUC
 >> 50000    4000    6.1  (6.4+) 6.7     6.1      6.4+-   5.9+-   7.1-
 >> 100000   2000    7.4         7.25    7.2      7.0     7.0     7.2
shows that apart from the 7.4/7.25 (C) and 7.2/7.0 (Ada) surprises
in the TC columns (where both programs run faster with checks _on_)
the unconstrained array version (Ada) appears indeed faster
than anything else in this row, interpreting the stable
7.0 (TU) as less than all other stable timings in the 100000/2000
row.

I agree that every number marked +- appears less trustworthy than
the others, even though the reason is not clear. The two 200k
arrays both almost fill the cache (256k PIII Coppermine), so maybe
there is some "fuzzy" loading going on ... Perhaps between the
procesors? I'll see next week.


>> It takes a lot of time to really find out what
>> and why. Still, you get fast code from either language front
>> end, so if an Ada compiler will check more things at
>> compile time, support more programming language, and you want
>> your compiler to do this, it can be a plus.
> 
> 
> And if you want performance, use C instead.

In the current context I prefer to let the timings in both T
columns favour neither language based on performance.
Neither is really performing better than the other when no
runtime checks have been added.
So I get the same performance out of C or Ada, with an additional
yield of either fewer or more compile time checks, respectively.


Georg