From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on polar.synack.me X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham autolearn_force=no version=3.4.4 X-Google-Thread: 103376,f5c1b0f8af65535 X-Google-Attributes: gid103376,public X-Google-Language: ENGLISH,ASCII-7-bit Path: g2news1.google.com!news3.google.com!news4.google.com!news.glorb.com!news-xfer.cox.net!peer01.cox.net!cox.net!news-out.ntli.net!newsrout1-gui.ntli.net!ntli.net!news.highwinds-media.com!newspeer1-win.ntli.net!newsfe2-gui.ntli.net.POSTED!53ab2750!not-for-mail From: "Derek M. Jones" User-Agent: Thunderbird 1.5 (Windows/20051201) MIME-Version: 1.0 Newsgroups: comp.lang.ada Subject: Re: ACATS on Wikipedia. References: <1204160.MAhx02haZG@linux1.krischik.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Message-ID: Date: Tue, 21 Feb 2006 00:21:29 GMT NNTP-Posting-Host: 86.15.173.248 X-Complaints-To: http://www.ntlworld.com/netreport X-Trace: newsfe2-gui.ntli.net 1140481289 86.15.173.248 (Tue, 21 Feb 2006 00:21:29 GMT) NNTP-Posting-Date: Tue, 21 Feb 2006 00:21:29 GMT Organization: ntl Cablemodem News Service Xref: g2news1.google.com comp.lang.ada:3028 Date: 2006-02-21T00:21:29+00:00 List-Id: Randy, > I'm not sure what question you are referring to (there doesn't seem to be > one there): The reference was to the talk page of the following Wikipedia entry: http://en.wikipedia.org/wiki/ACATS which included the sentence (now changed): "The ACATS test suite is a very important aspect of the Ada programming language as it ensures that all major Ada compilers conform to the language standard." to include: > Conformity assessment does not ensure that a processor has no > nonconformities to the Ada standard other than those, if any, documented in > this report. The compiler vendor declares that the tested processor > contains no deliberate deviation from the Ada standard; a copy of this > Declaration of Conformity is presented immediately after the certificate. and: > Also relevant is the second paragraph of the background of the current ACAA > procedures (http://www.adaic.com/compilers/procs/3.0/ACAP30.html): > > It is important to note the scope and intent of conformity assessment. The > purpose of conformity assessment is to ensure that Ada processors achieve a > high degree of conformity with the Ada standard ([Ada95] as corrected by > [TC1]). Characteristics such as performance and suitability for a particular > application are not specified by the standard, and thus are outside the > scope of Ada conformity assessment. Moreover, the ACATS is a set of test > programs intended to check broadly for correct implementation; it is not > possible to exhaustively test for conformity. Thus, conformity is checked > only to the extent of these tests; processors that are certified as > conforming may fail to conform to the standard in ways peculiar to each, > under particular circumstances. I did not want to include the following because I was not sure you would feel it appropriate in a Wikipedia entry. >> How do we know that a validation suite correctly implements the > requirements contained in a language >standard? One answer is here > > The short answer is that we don't. Indeed, there have been a handful of > cases where we've changed the Standard to match the ACATS tests, because > implementations have been passing the tests for years, and strictly > following the wording of the standard would have been incompatible with > actual practice. > > The longer answer is that we do via a number of ways: > > 1) Vetting of test objectives (the test objective must be reasonably clear, > and clearly relate to one or more rules in the Standard) [the ARG now has > this responsibility]; > > 2) Dispute procedures (where vendors and users can object to a test that > they think is wrong - this prevents incorrect tests from lasting a long > time); > > 3) Tracking of coverage (so that we can tell which rules in the standard > have been tested, and which ones have not been - which guide which tests are > the highest priority for construction). > > There are a number of articles about this on adaic.org (look under Compilers > and Conformity). Wikipedia uses a creative commons license. So I'm pleased you said: > Feel free to use whatever part of this you want.