Re: Computer Language Shootout

[tmoran@acm.org]

Just looking at some of the benchmark examples gives me a headache.  The
author himself gives some of the reasons they are "lies, damned lies, and
benchmarks".  Most are small, so comprehensibility of code is untested.
Speed is the most important thing, and you are apparently supposed to do
things the same way as the other programming language versions, so you are
pretty much left testing a particular compiler's code generator or library
routines (eg reading lines of text).  He allows tricks like reading the
entire source file in one operation, then scanning it from memory, which
is not testing the same thing as multiple single-line reads.  His
lines-of-code metric, which he admits is terrible, counts non-comment
source lines, so there's a strong pressure to avoid white space.  In the
example with threads he mentions that a solution using Python's
synchronized Queue class is not legitimate - I presume that means one
couldn't use the obvious Ada solution with protected types, but would have
to ignore that Ada advantage.  etc etc etc