Re: ACATS on Wikipedia.

["Derek M. Jones" <derek@_NOSPAM_knosof.co.uk>]

Randy,

> I'm not sure what question you are referring to (there doesn't seem to be
> one there):

The reference was to the talk page of the following Wikipedia entry:

http://en.wikipedia.org/wiki/ACATS

which included the sentence (now changed):

"The ACATS test suite is a very important aspect of the Ada programming 
language as it ensures that all major Ada compilers conform to the 
language standard."

to include:

> Conformity assessment does not ensure that a processor has no
> nonconformities to the Ada standard other than those, if any, documented in
> this report.  The compiler vendor declares that the tested processor
> contains no deliberate deviation from the Ada standard; a copy of this
> Declaration of Conformity is presented immediately after the certificate.

and:

> Also relevant is the second paragraph of the background of the current ACAA
> procedures (http://www.adaic.com/compilers/procs/3.0/ACAP30.html):
> 
> It is important to note the scope and intent of conformity assessment. The
> purpose of conformity assessment is to ensure that Ada processors achieve a
> high degree of conformity with the Ada standard ([Ada95] as corrected by
> [TC1]). Characteristics such as performance and suitability for a particular
> application are not specified by the standard, and thus are outside the
> scope of Ada conformity assessment. Moreover, the ACATS is a set of test
> programs intended to check broadly for correct implementation; it is not
> possible to exhaustively test for conformity. Thus, conformity is checked
> only to the extent of these tests; processors that are certified as
> conforming may fail to conform to the standard in ways peculiar to each,
> under particular circumstances.

I did not want to include the following because I was not sure
you would feel it appropriate in a Wikipedia entry.

>> How do we know that a validation suite correctly implements the
> requirements contained in a language >standard? One answer is here
> 
> The short answer is that we don't. Indeed, there have been a handful of
> cases where we've changed the Standard to match the ACATS tests, because
> implementations have been passing the tests for years, and strictly
> following the wording of the standard would have been incompatible with
> actual practice.
> 
> The longer answer is that we do via a number of ways:
> 
> 1) Vetting of test objectives (the test objective must be reasonably clear,
> and clearly relate to one or more rules in the Standard) [the ARG now has
> this responsibility];
> 
> 2) Dispute procedures (where vendors and users can object to a test that
> they think is wrong - this prevents incorrect tests from lasting a long
> time);
> 
> 3) Tracking of coverage (so that we can tell which rules in the standard
> have been tested, and which ones have not been - which guide which tests are
> the highest priority for construction).
> 
> There are a number of articles about this on adaic.org (look under Compilers
> and Conformity).

Wikipedia uses a creative commons license.  So I'm pleased you said:

> Feel free to use whatever part of this you want.