Re: GNAT versions ( was :Ada compiler for PC?)

[Russ Allbery <rra@stanford.edu>]

In gnu.misc.discuss, Robert Dewar <robert_dewar@my-dejanews.com> writes:

> One problem of course is that only ACT has access to the ACT regression
> suite. This is fundamental, it is mostly proprietary code, that we
> promise to guard very closely.  Similarly, the DEC test suite is
> proprietary code to which we have access, but most certainly not
> distribution rights.

This is actually a really interesting issue that faces any developer of a
free software compiler.

Fundamental to compiler development of any sort is extensive regression
testing, since a bug in a compiler will not only cause the compiler to not
function correctly, but may also propagate breakage to everything compiled
with the compiler, sometimes in extremely subtle ways that are damnably
hard to detect.  Extensive testing is considerably more important even
than for other critical system utilities, and the need is probably
equalled only by system kernels and libc.

One of the best sorts of tests are things that used to be broken and are
now fixed, that can be tested with every change to make sure they don't
break again.  (That's the definition of regression testing, in fact.)  But
what a compiler breaks on is certain code constructs, and other people's
code is quite frequently not under the same license as the compiler
itself.  Therefore, if one actually uses one of the best sources of a test
suite for a compiler, one ends up with a test suite that's covered under a
bunch of conflicting licenses, including, quite frequently, proprietary
ones (since people still do write a lot of proprietary software), and
which almost certainly can't be easily distributed with the compiler.

Companies are usually willing to let their internal code be used for
testing the compiler that they actually use, but they're much less likely
to be willing to have that code redistributed.  But not adding a good test
to the test suite due to licensing issues isn't very palatable either, nor
is trying to rewrite the test.  (In fact, the latter is often not even
possible, since the obscure interactions that caused the bug in the first
place usually disappear in a rewrite.)

It makes a great deal of sense for there to be a "traditional" software
support company associated with a free compiler effort that can take bug
reports and test code from companies and other sources and perform
regression tests against the free source base using that information, even
if the test code can't be redistributed.

-- 
Russ Allbery (rra@stanford.edu)         <URL:http://www.eyrie.org/~eagle/>