Re: No call for Ada (was Re: Announcing new scripting/prototyping language)

[Marin David Condic <nobody@noplace.com>]

Well, from my experience with benchmarking for realtime systems, we 
generally drew on sample code that was typical of our control systems. 
Compiler A might do a real good job of optimizing one algorithm while 
Compiler B was better at another. This was done for purposes of 
selecting which Ada compiler we wanted to use for the given target - not 
for selecting a language.

We never attempted language-to-language benchmarking because we pretty 
much figured it was pointless. Too many variables to really get a 
meaningful result and we knew we could get realtime quality out of most 
languages usually used for the purpose. So we picked the language based 
on other factors (error reduction, improved productivity, etc) and then 
benchmarked the competitors in that category. Key to trying to do any 
evaluation of anything in a scientific manner is to hold "all other 
things being equal" and when it comes to code in different languages 
with different compilers, you have a tough time doing this.

A key result of our tests is that there are seldom any clear winners. It 
depends a lot on what your real-world code is going to look like. Some 
languages may be ruled out for lack of a competing implementation (if 
nobody makes an embedded compiler for your target, the game is over) but 
usually the "conventional" players are around. Then - depending on the 
specific compiler - there are various ways of getting optimal code for 
the algorithms you're interested in and usually you have to learn that 
along the way. Its seldom an exact science. Sooner or later you pick 
something and then get on with getting the job done. So long as you 
didn't pick something hopelessly inefficient, you usually find a way to 
get reasonable results with what you picked.

I'd offer again my observation that for probably 90% of the software 
development that goes on in the world, relative compiler/language 
inefficiency is a total non-issue and people ought not to sweat over it. 
Even if there was any truth to the "Ada is slow..." rumor (and a given 
compiler is twice as slow as some highly optimized C example?) you'll 
never even see it in most applications. One ought to then focus in on 
other important factors that go along with language selection such as 
available compilers, reliable implementations, improvements in error 
rates and/or productivity, available tools, available libraries, 
time-to-market issues, etc.

MDC


Jerry Coffin wrote:
> 
> IMO, this produces a benchmark that is so far departed from the real 
> world that while it may produce results that are accurate (for some 
> definition of the word) they're utterly devoid of relationship with 
> reality, and therefore of any real meaning.
> 
> If you want to do a comparison, you need to compare things how they're 
> really used.  There are certainly variations among programmers, but to 
> be meaningful the test code should fall well within the range of normal 
> variations.  We all know that "real programmers can write Fortran in any 
> language", but writing Fortran in Ada, C++, Java, etc., doesn't really 
> accomplish much, and the performance of such code is meaningless at 
> best, and more likely to be downright misleading.
> 


-- 
======================================================================
Marin David Condic
I work for: http://www.belcan.com/
My project is: http://www.jsf.mil/NSFrames.htm

Send Replies To: m   o   d   c @ a   m   o   g
                    c   n   i       c   .   r

     "Face it ladies, its not the dress that makes you look fat.
     Its the FAT that makes you look fat."

         --  Al Bundy

======================================================================