ACVC test coverage

ACVC test coverage

[Vladimir G. Ivanovic]

Does anyone know the coverage of the ACVC suite?  I would expect (and
hope) that it would test 100% of the syntactic rules (i.e. the
grammar) and a high percentage of the semantic rules.

Re: ACVC test coverage

[Jon Jacky]

> (Vladimir G. Ivanovic asks....)
>
> Does anyone know the coverage of the ACVC suite?  I would expect (and
> hope) that it would test 100% of the syntactic rules (i.e. the
> grammar) and a high percentage of the semantic rules.

It may be less than that.   The limitations in an early version of the
validation suite are candidly discussed in a very informative paper by one of
the people who built it:

John B. Goodenough. Ada compiler validation: an example of software testing
theory and practice.  In: A.N. Haberman and U. Montinari (Eds.), System
Development and Ada, Proceedings of the CRAI Workshop on Software Factories
and Ada, Capri Italy, May 26 - 30, 1986.  Lecture Notes in Computer Science
No. 275.  New York, Springer-Verlag 1986, pps. 195 - 232

Goodenough wrote:

"The test suite today (Version 1.8) represents 2400 programs. ... The test
suite is not yet complete.  We have identified approximately 1400 additional
tests that need to be written just to cover all aspects of the language
adequately."

Things may have improved since Goodenough's paper was written, but I've seen
nothing more recent to compare it with. 

Jonathan Jacky				jon@gaffer.rad.washington.edu
Radiation Oncology RC-08		voice:	(206)-548-4117
University of Washington		FAX:	(206)-548-6218	
Seattle, Washington  98195
USA