Re: Ada Core Technologies and Ada95 Standards

[Ken Garlington <garlingtonke@lmtas.lmco.com>]

Robert Dewar wrote:

> The concern with the ACVC has always been that it both contributes to
> quality and detracts from it. It contributes by doing a lot of
> thorough testing of a wide spread set of features.

And the reason why we believe this is thorough testing? For the
same reason we thought ACVC 1.x was thorough - because the
people involved worked as hard as they could.

There are other ways to determine if testing is thorough. Why are none of
them used?

> It detracts by forcing resources to be spent on validation that miht
> better be spent on other activities that would be more cost effective
> in improving compiler quality.

Name one activity that all vendors should be doing, exclusive of ACVC testing.
Just one. If you can, I will accept this statement. If you can't, why should
I believe that vendors will spend money on activities to improve quality,
as opposed to more glossy brochures?

> In developing the 2.1 suite, the answer Ken's two questions have bot
> both been yes. Yes, it should grow, by adding more tests that are more
> user-oriented, i.e. more realistic with respect to typical use of the
> language, and of course it should grow by testing the Ada 95 features.

This will, of course, have the opposite effect: Adding these tests will,
in fact, detract from quality.

(I have no proof of this, but since you can offer no evidence as to how
the current ACVC contributes to quality - other than "it's better than
Pascal!" - I think my statement is just as valid.)

> Yes, it should shrink, by removing minimal value tests, or tests of
> pathological features not worth testing.

Nope. By deleting those tests, the quality of many compilers will actually
diminish. (Why not?)

> Ken, have you examined the tests in the new test suite. I think they are
> a significant step forward from the 1.11 tests, and it would be interesting
> to know what you think.

I guess I would be more impressed with objective evidence of the change in
test quality, rather than just an opinion (even my own). I guess that
objective evidence will just have to wait until the compilers are delivered,
and us users will be the guinea pigs once again.

It doesn't have to be that way. However, compiler vendors have their own
domain, I guess.

> The test suite development is an open process, and prereleaseed versions
> of the tests are availabel. Comments from everyone, including certainly
> users of Ada 95, are welcome.

I don't think you've quite grasped the concept. It's a radical one, to be
sure, requiring out of the box thinking. In summary, it is: "Rather than
change the number of tests, perhaps we should be thinking of changing the
_types_ of tests, using 'test' in the broad sense of 'a measure of quality'".

In the meantime, those regression test suites continue to grow....

-- 
LMTAS - "Our Brand Means Quality"