From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on polar.synack.me X-Spam-Level: X-Spam-Status: No, score=-1.3 required=5.0 tests=BAYES_00,INVALID_MSGID autolearn=no autolearn_force=no version=3.4.4 X-Google-Language: ENGLISH,ASCII-7-bit X-Google-Thread: 103376,42427d0d1bf647b1 X-Google-Attributes: gid103376,public From: dewar@cs.nyu.edu (Robert Dewar) Subject: Re: Ada Core Technologies and Ada95 Standards Date: 1996/04/01 Message-ID: #1/1 X-Deja-AN: 145355448 references: <00001a73+00002c20@msn.com> <828038680.5631@assen.demon.co.uk> <828127251.85@assen.demon.co.uk> <315FD5C9.342F@lfwc.lockheed.com> organization: Courant Institute of Mathematical Sciences newsgroups: comp.lang.ada Date: 1996-04-01T00:00:00+00:00 List-Id: iKen Garlington says " It's probably just ignorance on my part about the ACVC process, but I don't get that same sense of rigor in the ACVC design. A lot of what's known about good processes for software testing (as documented by Beizer and others) isn't apparent from what little I've heard about the ACVC, and from reading the old ACVC 1.0 tests." Yup, you are right on target (it is ignorance on your part!). "1. We have a requirements specification that uniquely identifies each requirement. " Yes, of course this was what was done for ACVC version 1 (ever read the implementors guide? I guess not! This was the requirements spec for the testing) "2. We have a test or set of tests which can be traced back to each requirement." Yes, of course this was done (don't you see the objectives in the test traced back to the requirements, you said you read the tests). "3. We have consultations with the end user of the system to see if the tests are adequate, and reflect the usage of the system. " This is *especially* being done for the new ACVC 2 (I guess you are unfamiliar with the process here). Your comments on white box testing are not relevant for a general validatoin facility, though of course for a given compiler, these kind of procedures are followed. It would not be practical to incorporate all test progrms for all bugs found in all compilers into the ACVC (it would rapidly have tens of thousands of tests, and become completely unmanagable). For example, the GNAT regression tests now are larger than the whole ACVC test suite by a considerable factor. Also, the effort of taking every bug and putting it into ACVC form is out of the question. The Ada ACVC suite is by fac the most comprehensive test suite ever generated for a programming language. THe fact that it is stlil not truly comprehensive just serves to emphasize how complex compilers for modern large languages are. Ken, a minimal effort on your part invested in learning about the ACVC process would seem a worthwhile effort, and would certainly make your comments about the ACVC more informed. Have you even read John Goodenough's papers on the subject?