From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on polar.synack.me X-Spam-Level: X-Spam-Status: No, score=-1.3 required=5.0 tests=BAYES_00,INVALID_MSGID autolearn=no autolearn_force=no version=3.4.4 X-Google-Language: ENGLISH,ASCII-7-bit X-Google-Thread: 103376,42427d0d1bf647b1 X-Google-Attributes: gid103376,public From: dewar@cs.nyu.edu (Robert Dewar) Subject: Re: Ada Core Technologies and Ada95 Standards Date: 1996/04/24 Message-ID: #1/1 X-Deja-AN: 151307886 references: <00001a73+00002c20@msn.com> <828038680.5631@assen.demon.co.uk> <828127251.85@assen.demon.co.uk> <315FD5C9.342F@lfwc.lockheed.com> <3160EFBF.BF9@lfwc.lockheed.com> <829851188.11037@assen.demon.co.uk> <830205883.24190@assen.demon.co.uk> <317CB1C1.431F@lmtas.lmco.com> organization: Courant Institute of Mathematical Sciences newsgroups: comp.lang.ada Date: 1996-04-24T00:00:00+00:00 List-Id: Ken asked: "This is certainly true, and one of the questions I asked. If the ACVC is changing to be more user-oriented, how does the ACVC writer have this understanding of how the compiler will be used? If his understanding is in error, then the tests will still be focusing on the wrong things." Why not study the tests? Why not study the process being used to construct ACVC 2.1? This development is not being done in a vacuum. The basic answer is that each feature, instead of being tested in a formalistic way is instead resulting in the question: "How would this feature be used in a real program?", a question that was never even asked in the context of ACVC version 1. The resulting test is then reviewed by the ACVC review team, which represents implementors and users, to see how well it seems to match potential use. It is hard to establish objective criterion for how well one is doing in this process. The review group certainly has not found anyway of pre-establishing what will turn out to be typical usage. What *is* encouraging is the following, which actually could possibly be quantied from our data, with a lot of work. When we make proposed changes to GNAT, we run three kinds of tests: Our main test suite, which is surely user-oriented, since it is mostly user code. the old ACVC tests the new ACVC tests in pratice we find the general rests on the new ACVC tests closely mirror the general results on the main test suite. On the other hand, the old ACVC tests seem to pick up a somewhat separate set of errors, and often yield conflicting results (lots of problems when the other two sets show none, or no problems when the other two sets show lots).