From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on polar.synack.me X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham autolearn_force=no version=3.4.4 X-Google-Thread: 103376,f5c1b0f8af65535 X-Google-Attributes: gid103376,public X-Google-Language: ENGLISH,ASCII-7-bit Path: g2news1.google.com!news3.google.com!news4.google.com!border1.nntp.dca.giganews.com!nntp.giganews.com!local01.nntp.dca.giganews.com!nntp.megapath.net!news.megapath.net.POSTED!not-for-mail NNTP-Posting-Date: Mon, 20 Feb 2006 16:45:58 -0600 From: "Randy Brukardt" Newsgroups: comp.lang.ada References: <1204160.MAhx02haZG@linux1.krischik.com> Subject: Re: ACATS on Wikipedia. Date: Mon, 20 Feb 2006 16:50:48 -0600 X-Priority: 3 X-MSMail-Priority: Normal X-Newsreader: Microsoft Outlook Express 6.00.2800.1106 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1106 Message-ID: NNTP-Posting-Host: 64.32.209.38 X-Trace: sv3-PRw141O+wSauZA+M5Rz681DGPox4+HVSDpxcwW0ZfBXYNFR0wCdq1glNu9vtiqBzmv97saSmzDVr4rb!53kkCQni+EPCd0t+FaDznD5fD5UwD/nFgD8rCvBz/KrhC62141MIQTZBCK+lOnC9vsd5oCzP6199 X-Complaints-To: abuse@megapath.net X-DMCA-Complaints-To: abuse@megapath.net X-Abuse-and-DMCA-Info: Please be sure to forward a copy of ALL headers X-Abuse-and-DMCA-Info: Otherwise we will be unable to process your complaint properly X-Postfilter: 1.3.32 Xref: g2news1.google.com comp.lang.ada:3027 Date: 2006-02-20T16:50:48-06:00 List-Id: "Martin Krischik" wrote in message news:1204160.MAhx02haZG@linux1.krischik.com... > Hello again > > Anybody got an intelligent answer to this question: > > http://en.wikipedia.org/wiki/Talk:Ada_Conformity_Assessment_Test_Suite I'm not sure what question you are referring to (there doesn't seem to be one there): >It is incorrect to say that ACATS ensures a compiler conforms to a language standard. >When BSI were in the compiler validation business the validations certificates they issued >said "This is to certify that the language processor identified below was found to contain >no errors when tested with the identified validation suite." No mention of conforming to a >language standard there. I suspect the Ada validation certificates say something similar >(I don't have one to hand). Recent ACATS validation reports are available on line. (None are that recent, unfortunately.) I forget the exact contents of the certificate, but the preface to the test report includes the following: Conformity assessment does not ensure that a processor has no nonconformities to the Ada standard other than those, if any, documented in this report. The compiler vendor declares that the tested processor contains no deliberate deviation from the Ada standard; a copy of this Declaration of Conformity is presented immediately after the certificate. Also relevant is the second paragraph of the background of the current ACAA procedures (http://www.adaic.com/compilers/procs/3.0/ACAP30.html): It is important to note the scope and intent of conformity assessment. The purpose of conformity assessment is to ensure that Ada processors achieve a high degree of conformity with the Ada standard ([Ada95] as corrected by [TC1]). Characteristics such as performance and suitability for a particular application are not specified by the standard, and thus are outside the scope of Ada conformity assessment. Moreover, the ACATS is a set of test programs intended to check broadly for correct implementation; it is not possible to exhaustively test for conformity. Thus, conformity is checked only to the extent of these tests; processors that are certified as conforming may fail to conform to the standard in ways peculiar to each, under particular circumstances. ---- The other statement in the original is: >How do we know that a validation suite correctly implements the requirements contained in a language >standard? One answer is here The short answer is that we don't. Indeed, there have been a handful of cases where we've changed the Standard to match the ACATS tests, because implementations have been passing the tests for years, and strictly following the wording of the standard would have been incompatible with actual practice. The longer answer is that we do via a number of ways: 1) Vetting of test objectives (the test objective must be reasonably clear, and clearly relate to one or more rules in the Standard) [the ARG now has this responsibility]; 2) Dispute procedures (where vendors and users can object to a test that they think is wrong - this prevents incorrect tests from lasting a long time); 3) Tracking of coverage (so that we can tell which rules in the standard have been tested, and which ones have not been - which guide which tests are the highest priority for construction). There are a number of articles about this on adaic.org (look under Compilers and Conformity). Feel free to use whatever part of this you want. Randy Brukardt, ACAA Technical Agent.