Re: GNAT versions ( was :Ada compiler for PC?)

[Robert Dewar <dewar@gnat.com>]

In article <yl676lnovr.fsf@windlord.stanford.edu>,
  Russ Allbery <rra@stanford.edu> wrote:

> In gnu.misc.discuss, Robert Dewar
> <robert_dewar@my-dejanews.com> writes:

<<many other good points snipped>>

> It makes a great deal of sense for there to be a
> "traditional" software support company associated with a
> free compiler effort that can take bug reports and test
> code from companies and other sources and perform
> regression tests against the free source base using that
> information, even if the test code can't be
> redistributed.

We certainly regard our test suite (now many millions of
lines long spreading out over nearly 6000 directories, and
close to a hundred thousand files) as a critical aid in
our quality control procedures. Other companies undoubtedly
have their own suites. Often this code is very fiercely
protected, and we routinely sign non-disclosure agreements
to get it.

A useful, but massive task, would be to sanitize this test
suite, by extracting the tests that are not subject to
non-disclosure (since this is the default for us, it would
take a large effort of contacting users etc), and creating
sanitized versions of bugs in proprietary code. That could
result in a useful freely available test. But I would
estimate that several person years of work is required
for this, and we just don't have the resources.

One very important point for us is that we also have
access to the Digital test suite from their extensively
used Ada 83 product (this includes basically all the bugs
that DEC ever ran into plus a number of really diabolical
complex purpose written tests [the ACT test suite also
includes hundreds of our own purpose-written tests,
generally when we add a new feature, we add a test to our
test suite].

In practice we find that the trio working together (the
ACVC tests, the ACT tests, and the DEC test suite) are
complementary, very often a new version of the compiler
passes 2 out of 3 of these tests!

We actually use the ACT regression suite very aggressively.
Our internal procedures allow anyone to checkin
modifications to one of the standard GNAT files, but *only*
after running the regression suite. This suite has been
run nearly ten thousand times in the last couple of years,
and we have found this a very effective way of controlling
development and making sure that regressions are not
introduced. Then each night we build on all targets, and
run the ACVC and DEC test suites.

In any case we are ahead in the Ada community in having
at least one freely available comprehensive test suite,
the ACVC tests, and anyone for example doing their own
fiddling with GNAT should most CERTAINLY be running the
ACVC tests as one good step to making sure they are not
introducing regressions.

Robert Dewar
Ada Core Technologies

-----------== Posted via Deja News, The Discussion Network ==----------
http://www.dejanews.com/       Search, Read, Discuss, or Start Your Own