Re: Five Years After

["Randy Brukardt" <randy@rrsoftware.com>]

"Maciej Sobczak" <see.my.homepage@gmail.com> wrote in message 
news:292710c7-3520-406f-9e6b-1fb030ada211@googlegroups.com...
>> Also having a validated compiler doesn't seem to matter as much as it did 
>> in the last century. ;-)
>
>Interestingly, within the safety-critical domain, "validation" (or 
>"qualification", which
> is a more specialized term) means more than ACATS tests can offer, and 
> outside
> of this domain nobody cares about it anyway. In some sense, this effort, 
> even
> though very well-intended and representing an objectively good engineering
> practice, was simply misplaced.

Everyone agrees that the ACATS test suite offers a lot of value to both 
implementers and users (implementers because an independently sourced and 
trustworthy set of tests is very helpful to creating and maintaining an Ada 
compiler -- it is especially good IMHO in preventing errors of omission, 
which are often the hardest to find; users because the test suite provides a 
modicum of portability; Ada compilers accept essentially the same language 
since they've all been tested with the same set of tests). This certainly is 
true since I'm paid money every year to maintain and enhance the test 
suite -- that wouldn't be the case if people didn't see a value.

The value to formal testing ("conformity assessment") is substantially less, 
especially nowdays when machines are fast enough that anyone can run the 
ACATS in a few hours. That is, you can do your own testing in a day or two 
to prove that your vendor is actually telling the truth about conformity. 
Why would you want to pay some formal testing $$$ to do what you can do 
yourself?

Thus, I've been spending some of the ACATS effort on making it easier to 
self-test. Using the new(ish) grading tools and some other tools that I'll 
be posting shortly as "submitted tools", one can essentially test a GNAT 
implementation against the ACATS without a lot of hand work. (You do have to 
understand enough about your implementation and your target to configure the 
ACATS itself.) The hope is to get the value of independent testing without 
the costs.

Of course, formal third-party testing is still available if someone wants 
it.

                         Randy.