From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on polar.synack.me X-Spam-Level: X-Spam-Status: No, score=-1.3 required=5.0 tests=BAYES_00,INVALID_MSGID autolearn=no autolearn_force=no version=3.4.4 X-Google-Language: ENGLISH,ASCII-7-bit X-Google-Thread: 103376,42427d0d1bf647b1 X-Google-Attributes: gid103376,public From: dewar@cs.nyu.edu (Robert Dewar) Subject: Re: Ada Core Technologies and Ada95 Standards Date: 1996/04/10 Message-ID: #1/1 X-Deja-AN: 146888586 references: <00001a73+00002c20@msn.com> <828038680.5631@assen.demon.co.uk> <828127251.85@assen.demon.co.uk> <315FD5C9.342F@lfwc.lockheed.com> organization: Courant Institute of Mathematical Sciences newsgroups: comp.lang.ada Date: 1996-04-10T00:00:00+00:00 List-Id: Ken said "I know that NPL has a tool that they sell that tests Ada compilers for bugs, tha t apparently provides much more coverage than the ACVC. Why should such a tool exist outside of the validation/certification process?" This is not true at all, and I guess Ken is only aware of this tool by rumour, since if he had used it he would now that it is not ni the business AT ALL of providing coverage testing. Instead this is a stress testing tool, it generates random very complex (and generally very unrealistic) examples of expressions and other constructs to see if the compiler can be broken by such stress testing. The tool is incidentally available from NPL to users, so Ken it is certainly something you could use to test a compiler yourself. In fact, I talked to Brian Wichman (the author of this tool and a similar one for Pascal), and the results they have obtained with these tools are quite surprising at least to me, in the extent to which they show quality differences between Ada and Pascal compilers. Most (all?) of the Pascal compilers they have tested have exhibited safety defects (defined as the generation of incorrect code). None of the Ada compilers have shown safety defects -- they have managed to break them but not persauded then to generate wrong code. Now in practice, I would expect that big projects such as Ken's can point to safety defects (defined this way) in the compilers they have used, and just as the ACVC cannot 100% guarantee conformance, the NPL tests cannot 100% guarantee safety, but they are a measure. I find it interesting that the Ada compilers fair so much better than the Pascal compilers. Brian at least ascribes this at least in part to the ACVC process.