From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on polar.synack.me X-Spam-Level: X-Spam-Status: No, score=-1.3 required=5.0 tests=BAYES_00,INVALID_MSGID autolearn=no autolearn_force=no version=3.4.4 X-Google-Language: ENGLISH,ASCII-7-bit X-Google-Thread: 103376,42427d0d1bf647b1 X-Google-Attributes: gid103376,public From: dewar@cs.nyu.edu (Robert Dewar) Subject: Re: Ada Core Technologies and Ada95 Standards Date: 1996/04/19 Message-ID: #1/1 X-Deja-AN: 148369845 references: <00001a73+00002c20@msn.com> <828038680.5631@assen.demon.co.uk> <828127251.85@assen.demon.co.uk> <315FD5C9.342F@lfwc.lockheed.com> <3160EFBF.BF9@lfwc.lockheed.com> <829851188.11037@assen.demon.co.uk> organization: Courant Institute of Mathematical Sciences newsgroups: comp.lang.ada Date: 1996-04-19T00:00:00+00:00 List-Id: John McCabe says "This paragraph confuses me. I believe that if the compiler vendors were producing high quality products that conformed to the language, then passing the ACVC suite should be a cinch. What you seem to be saying is that compiler vendors are assigning resources to ensure that they pass the ACVC suite, and in doing so are compromising the quality of their product." This is a very common misconception among people who don't know language design or compiler implementation very well. Indeed it is the same misconception that has lead people to misinterprete what conformance testing is all about. Perhaps I can put it this way. Suppose a vendor has resources to do exacty one of the following two tasks: 1. Rewrite the loop optimizer so that all loops run faster 2. Rewrite the handling of static expressions so that one very obscure test in the ACVC suite which has never shown up in a custoer programer and is very unlikely *ever* to show up in customer programs, under the condition that this rewriting is extensive and will likely cause regressions (in programs other than ACVC tests). Which do YOU think would contribute more to quality for most users? That's the potential trouble with mandated testing, it elevates very unimportant problems to maximum priority without any regard to the importance or impact of these problems. If we follow Ken's repeated request, and extend the scope of mandatory testing, then we distort things still more. That's the risk. The DoD policy in this area is that requiring the ACVC conformance testing is as far as it is desirable to go for general requirements. This policy of course recognizes that for certain purposes, other kinds of testing may be useful, which is why DoD has suppoted the development of the ACES suite. It is up to a project officer to specify this as a requirement if this is needed (such requirements are not uncommon). Ken, if your procurement did not specify this requirement, all one can ask is why not? Do you really need the DoD to tell you what testing needs to be done? In the commercial marketplace, the market determines what testing is desirable (for instance a lot of the C++ commercial market is comfortable with no testing whatsoever), but in other contexts he commercial marketplace requires testing, e.g. many commercial COBOL customers wlil only used NIST certified compilers. Why is it that DoD customers can't work the same way. By the way Ken, you ask how the DoD has determined that it is reasonable to generally require the ACVC testing and nothing more? I find it a bit odd that a DoD contractor should be asking this question to someone outside -- why not ask within the DoD, it's their policy! P.S. when I used critical in talking about non-critical banking applications, I was abbreviating not for mission-critical, but for safety-critical. Sorry for not being clear. But to clarify my point here. A banking applicatoin may well not care about ACES testing because they don't care about performance, and their own domain specific testing (e.g. actual testing of the application in question) shows that a given compiler works adeqately for their purposes.