Re: Ada Core Technologies and Ada95 Standards

[dewar@cs.nyu.edu (Robert Dewar)]

Ken said

"Believe it or not, Robert Dewar wrote these two sentences in the
same post:

> ...the point is that Brian's test generator does not test any
> features not tested pretty thoroughly in the ACVC suite.

> We certainly plan to use this tool in testing GNAT.

Sounds like either (1) a waste of money, given that the ACVC tests
these features thoroughly, or (2) a good idea for all compiler vendors
to do."

You should definitely believe it because I certainly wrote those two
sentences, and they are not at all inconsistent. The fact that you
think they are shows you are still missing some important points.

There is a big difference between feature coverage and stress testing.
The ACVC has pretty complete feature coverage, for example, you will
find cases of all possible operators applied to all classes of types.

But that is very differnt from seeing if a compiler will fall apart
if you give it a complex expression containing thousands of tokens
(which is waht the NPL stress testing tool can do).

I use feature in the sense of "feature in the language", not a general
use applying to any possible characteristic of the language.

The ACVC test suite measures conformance, not capacity or performance.
For capacity and performance tests, you should look at the ACES test
suite which concentrates on these areas. Note that the development
of the ACES has been funded by the DoD, and use of this test suite
is considered to be of critical importance by many projects.