From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on polar.synack.me X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham autolearn_force=no version=3.4.4 X-Google-Language: ENGLISH,ASCII-7-bit X-Google-Thread: 103376,446231e9f9fb9a1c X-Google-Attributes: gid103376,public From: heatwole@erols.com (Kevin D. Heatwole) Subject: Re: ACVC tests Date: 1996/05/07 Message-ID: X-Deja-AN: 153512909 references: <831410279.2370.2@assen.demon.co.uk> organization: Erols Internet Services newsgroups: comp.lang.ada Date: 1996-05-07T00:00:00+00:00 List-Id: In article <831410279.2370.2@assen.demon.co.uk>, john@assen.demon.co.uk (John McCabe) wrote: >dewar@cs.nyu.edu (Robert Dewar) wrote: > >>Of course the team has not "contacted most users", who would number in >>the tens of thousands. In fact the experience in the past has been that, >>although the ACVC tests were generally available for review, it has been >>extremely difficult to get ANY tecnical input from anyone. Even vendors >>do not in general look at the tests in advance of the formal release >>of the suite, and it is extremely rare to get any technical input from >>users on the tests (I can't remember any example of such). Thus the >>phiolosophy behind the committee was precisely to get at least *some* >>users, implementors and testers looking at the test carefully in advance. > >I am surprised that the vendors attitude here since one would have >thought they would be keen to be the first on the market with a >validated product. > >Perhaps this attitude should be considered further. It may be, for >example, that they consider validation as a hinderance that does not >add any real value to their product. It could then be suggested that >this attitude is promoted by the users (like myself, or the mandators >like Dornier) not being strict enough with the vendors and asking for >VSRs and ACES results etc. > >Any comments? Well, I work for OC Systems and we would be considered an Ada vendor. I will not pretend to speak for OC Systems, but I can offer my attitude on this subject. The ACVC tests are a great resource for the compiler vendors and provide a measure of reassurance to our customers that we do indeed implement the Ada language. The test suite has literally cost millions of dollars to develop and is probably the best such suite of tests that I have ever encountered. If this test suite did not exist, a individual compiler vendor would be forced to develop such a test suite internally. This burden would increase the cost of Ada compilers to our end users (Ada compilers are already an incredibly expensive piece of software to develop and we need all the help we can get to cut these costs). Also as a vendor, we do not consider validation "as a hinderance". Rather, it is an essential part of developing a commercially acceptable Ada compiler. Indeed, even if there was no requirement to "validate" using this test suite, we would still want to incorporate this set of tests into our test suite. Vendors simply do not want to delivery buggy product (it will hurt our bottom line eventually). I wish that OC Systems had the resources to participate in the development of the new Ada95 ACVCs more. Unfortunately, developing an Ada95 compiler is a huge undertaking and resources must be allocated to this effort in view of market realities and current cash flow. We have chosen to try to implement the new features in Ada95 first (trying to implement them as best we can independent of running the new ACVCs - which are still in development and still have some bugs in them). When the compiler reaches a level of maturity that we feel it is beneficial to start running/debugging the ACVC tests, we focus our efforts in this direction. In this regards, the ACVCs provide valuable feedback to vendors developing Ada95 compilers as to how close the compiler is to being completed/robust. If you fail most of the ACVCs the first time around, you probably didn't do a good job implementing the features. Anyway, this is valuable feedback. As to the current ACVC 2.1 team (led by SAIC), they are doing an exellent job. We monitor all the email that they exchange and have some incite into their process. They have some very good people on the project. They are tasked to develop a test suite before there are compilers that implement the rules they are testing. This is a very difficult thing to do, since some of the rules in Ada are very subtle and without a reliable compiler to tell you that you messed up, it is very big undertaking (by the way, the ACVC team does have access to all the various vendors' Ada95 compilers which are at different levels of completeness. The ACVC team does a great job of sending the vendors bug reports when they encounter problems.). > >In general, however, I applaud your attempts here to extract more >detailed comments from users on the new test suite and, when I have >time, I'll see if I can be of any help! I think you had best hurry here if you expect to be of any help for ACVC 2.1. This test suite is nearing completion and you better get your licks in now. They have been developing the new Ada 95 tests for over two years now (I am not really sure how long they have been at it, but there are new tests from 1994 - the Ada83 tests in the suite were developed over the a number of years in the 1980s). Kevin Heatwole OC Systems, Inc.