More bits on the Ada validation suite

More bits on the Ada validation suite

[Dale Worley]

Well, I agree that the Ada suite is very large compared to the suites
available for other languages, but I have my doubts that it is any
more thorough (relative to the number of language features to be
tested).  We have written on the order of 10,000 lines of Ada code
here and have run into 5 or so bugs in validated compilers.  Most of
these seem not to be peculiar special cases of the algorithms that the
compiler writers used, but rather certain requirements from the
Reference Manual that were just coded up wrong.  That is, if that
particular sentence of the RM had been tested by the suite, the bug
would have been caught.  This leads me to believe that the ACVC
doesn't cover the language very thoroughly.

Further evidence of this is that compliance with Chapter 13 is just
being added, and the statement in GCN (29 Aug 88):

	With the new set of tests, ACVC covers about three-quarters of
	the test objectives outlined in DOD's Ada implementation
	guidelines, Myers said.  The objectives will be covered
	completely when the tests are revised again next year, he
	said.

I guess what I really want to do is praise DOD for trying to develop
the first genuine validation suite for testing compilers, and damn
them (faintly) for not doing a comprehensive enough job of it.

Dale