From mboxrd@z Thu Jan  1 00:00:00 1970
X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on polar.synack.me
X-Spam-Level: 
X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham
	autolearn_force=no version=3.4.4
X-Google-Thread: 103376,703c4f68db81387d
X-Google-Thread: 115aec,703c4f68db81387d
X-Google-Thread: f43e6,703c4f68db81387d
X-Google-Thread: 108717,a7c8692cac750b5e
X-Google-Attributes: gid103376,gid115aec,gidf43e6,gid108717,public
X-Google-Language: ENGLISH,ASCII-7-bit
Path: 
 g2news1.google.com!news3.google.com!newsfeed.stanford.edu!sn-xit-02!sn-xit-01!sn-post-01!supernews.com!corp.supernews.com!not-for-mail
From: CTips <ctips@bestweb.net>
Newsgroups: comp.lang.ada,comp.realtime,comp.software-eng,comp.programming
Subject: Re: 10 rules for benchmarking (was Re: Teaching new tricks to an
 old dog (C++ -->Ada))
Date: Fri, 11 Mar 2005 14:24:17 -0500
Organization: Posted via Supernews, http://www.supernews.com
Message-ID: <1133s3qnmqmbjfb@corp.supernews.com>
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US;
 rv:1.7.5) Gecko/20041217
X-Accept-Language: en-us, en
MIME-Version: 1.0
References: <4229bad9$0$1019$afc38c87@news.optusnet.com.au>
 <1110032222.447846.167060@g14g2000cwa.googlegroups.com>
 <871xau9nlh.fsf@insalien.org>
 <3SjWd.103128$Vf.3969241@news000.worldonline.dk>
 <87r7iu85lf.fsf@insalien.org> <1110052142.832650@athnrd02>
 <d0dkq6$781$1@titan.btinternet.com>
 <1110284070.410136.205090@o13g2000cwo.googlegroups.com>
 <395uqaF5rhu2mU1@individual.net> <112rs0bdr2aftdf@corp.supernews.com>
 <1inxxr988rxgg$.1w9dedak41k89.dlg@40tude.net>
 <112s1r0rf0o8nca@corp.supernews.com> <d0l2gq$egt$1@titan.btinternet.com>
 <112sonip5v4dca6@corp.supernews.com>
 <E1vXd.352556$w62.20389@bgtnsc05-news.ops.worldnet.att.net>
 <112t3de6fu04f38@corp.supernews.com>
 <1110396477.596174.285520@o13g2000cwo.googlegroups.com>
 <112vb2t8eonuhed@corp.supernews.com>
 <1110422108.925127.54110@o13g2000cwo.googlegroups.com>
 <11329cb96h2p19f@corp.supernews.com> <c92s0d.vgk.ln@hunter.axlog.fr>
 <113394jjvppao64@corp.supernews.com> <uk6oeaua9.fsf@obry.org>
In-Reply-To: <uk6oeaua9.fsf@obry.org>
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit
X-Complaints-To: abuse@supernews.com
Xref: g2news1.google.com comp.lang.ada:9159 comp.realtime:1298
 comp.software-eng:4866 comp.programming:17827
Date: 2005-03-11T14:24:17-05:00
List-Id: <comp.lang.ada>

Pascal Obry wrote:

> CTips <ctips@bestweb.net> writes:
> 
> 
>>BTW - all the psuedo-benchmarking people have done on this thread so far has
>>provided meaningless numbers - too small run-times, performance dominated by
>>cache misses etc.
> 
> 
> And the pseudo results from http://shootout.alioth.debian.org/great/ were fine
> to you some days ago :)

He's done it right:
  - he's running them on a Linux machine, which is more accurate than 
cygwin for measuring performance (and where sys/user really mean something).
   - he's using the improved times() which gives better resolution
   - he's controlling for the startup overhead.
   - he's using best of 4 runs.
   - he's picked data sizes that will not blow the cache
   - he's picked data sizes that will give decent run-times.

Like I said, if one knows what one is doing (which he does) then one can 
disregard the rules. If one does not know what one is doing (which you 
don't) then one should follow the rules.