From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on polar.synack.me X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,LOTS_OF_MONEY autolearn=ham autolearn_force=no version=3.4.4 X-Google-Language: ENGLISH,ASCII-7-bit X-Google-Thread: 103376,42427d0d1bf647b1 X-Google-Attributes: gid103376,public From: john@assen.demon.co.uk (John McCabe) Subject: Re: Ada Core Technologies and Ada95 Standards Date: 1996/03/29 Message-ID: <828127251.85@assen.demon.co.uk> X-Deja-AN: 144890499 x-nntp-posting-host: assen.demon.co.uk references: <00001a73+00002c20@msn.com> <828038680.5631@assen.demon.co.uk> newsgroups: comp.lang.ada Date: 1996-03-29T00:00:00+00:00 List-Id: dewar@cs.nyu.edu (Robert Dewar) wrote: <..snip...> >Wrong! The validation suite does contain tests for all parts of Ada 95 >including all the special needs annexes, and this is true "at the moment" >(where DO these rumours come from? :-) A particular employee of a particular Ada compiler vendor (who you probably know) in a presentation in Waterlooville England on 14th March 1995 stated that ACVC 2.0 consisted of only the parts of Ada that were common between Ada 83 and 95. I interpreted this to mean just the core language but looking back on it I can understand that this would also mean _parts_ of the specialised needs annexes. >It is certainly true that the >initial release of ACVC 2.0 and now 2.0.1 does not thoroughly cover >all new parts of the language, but as any Ada 95 compiler implementor >can tell you, they are definitely non-trivial, and any compiler passing >all or nearly all of these tests is a pretty complete Ada 95 compiler. Mmmm. Seems a bit contradictory ("all parts of Ada95" and "not all new parts") >As for Ada 83 validation not proving much, if you feel this way, probably <..snip..> I was obviously thinking of validation of Ada compilers in the same way that _my_ software is validated - i.e a full set of test cases proving that _all_ requirements have been met. If I cannot prove this, my software is not accepted by my customer. In the tools and utilities market this level of proof does not seem to be required. I design and implement systems for satellite instrument control. The software will have 1 user. I cannot just put a 1st version of the software onto a satellite and wait for the user to send me bug reports because by that time the chances are that the whole system could have failed and $250M worth of satellite is lying at the bottom of the Pacific Ocean. >What does validation do? It makes sure that the vendor has implemented >the entire language without significant gaps, and that the vendor has >implemented large parts of the language (those parts tested) accurately. >As a result, it is a good guarantee that the vendor undrstands the >language completely and thoroughly. There is a large difference between implementing the entire language and implementing it accurately. I find the number of faults with basic language handling in my present compiler rather disturbing. >Can a test suite do more than this? No! Can it do a better or worse job >of this? Sure. We think the ACVC 2.1 suite will turn out to be more >effective, because we have learned something in 12 years! In particular >we (the ACVC team and the reviewers) believe that the orientation to >more user-oriented testing will be helpful in this regard (compare some >typical 2.0 test with 1.11 tests, and you will see that the 2.0 tests >are much more like real programs -- the test writer testing a particular >feature thinks "how would this feature be used in a real program", and >constructs a real program to answer that question. That's good and basically the way it should be. >HOWEVER, although the suite will, we believe, be even more effective >than the 1.11 suite, no one would claim that it guarantees 100% >conformance or usability. If you hear anyone saying this, beware! >they do not know what they are talking about. I can accept that it is very difficult to prove a tool such as this completely but when I buy a _validated_ Ada compiler it is because I want to compile _valid_ _Ada_ code, not a subset of it! >There are many ways to evaluate a compiler. GNAT is validated, but it has <..snip..> I agree entirely with what you say here. It is obvious that the more users of a compiler, the more likely the bugs are found and sorted early on. That has been a problem with our compiler (MIL-STD-1750A version) because the user base is tiny. What is more disturbing however is that for every bug that seems to get fixed, the new release seems to contain even more! With GNAT you've probably got one of the largest user bases of any single compiler which can only help. I know GNAT is a very good "product" (I noticed in a posting some time ago you said GNAT is not a product but...) but the fact that it is available free of charge would lead me to be more understanding about its faults. When I pay $40000 for a piece of software development kit, I expect it to work. At the end of the day, I want validation to mean that the compiler can produce working object code from Ada source - and by that I mean the whole language - a subset is of no use to me. If that is not true of the compiler then I think that the term used to describe this examination should not be validation. Best Regards John McCabe