From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on polar.synack.me X-Spam-Level: X-Spam-Status: No, score=-1.3 required=5.0 tests=BAYES_00,INVALID_MSGID autolearn=no autolearn_force=no version=3.4.4 X-Google-Language: ENGLISH,ASCII-7-bit X-Google-Thread: 103376,c0f035b936128b6c X-Google-Attributes: gid103376,public X-Google-Thread: 1014db,c0f035b936128b6c X-Google-Attributes: gid1014db,public From: Arthur Schwarz Subject: Re: Ada95 to ANSI_C converter Date: 1997/04/07 Message-ID: <334929E8.4E68@gdls.com>#1/1 X-Deja-AN: 231386728 References: <5htg0a$v8v$1@news.nyu.edu> To: Robert Dewar Organization: General Dynamics Land Systems Newsgroups: comp.lang.ada,comp.lang.c Date: 1997-04-07T00:00:00+00:00 List-Id: Robert Dewar wrote: > > Keith says > > <<(I *like* nasty test cases)>> > > The trouble with nasty test cases is that they can sometimes result in > implementors wasting huge amounts of time getting things right that are > of no possible concern to users. The old ACVC suite was full of such > things, and it is still quite often the case that we make a change > and it affects none of our user programs in our regression suite, but > it causes an obscure bug in some ACVC test. Equally, it is often the > case that a change blows away huge numbers of user tests, but the > ACVC hums merrily on with no errors. > > Anecdotally (we really should collect data on this, we have the raw data), > we have the impression that the Ada 95 tests tend to track user programs > more closely. They are certainly much less full of "nasty test cases". (Anecdotally): One program that I wrote (Fortran IV/66) broke the compiler. I was told by my manager that my program 'stressed' it (the compiler). And at that time and since I have wondered why! If the language permits a construct and the compiler doesn't accept it, or what's worse, the generated code doesn't perform the operation correctly, then the compiler is "broke", not the construct and not the language. I would then say that a "nasty test case" is just another language construct which must be accepted by the compiler. It's only nasty when the compiler can not process it correctly. The same goes for 'obscure', 'seldom used', and of course, 'of no possible use to users'. To my jaundiced eye this all seems to be an excuse to direct effort at 'big payoff' items, the ones that seem to be most likely of use. The problem is that the devil is in the details, and it is just those other, legal, small payoff items which cause great labor for the implementor to debug; transfering the cost of the compiler from developer to user. Your other comment that the ACVC test does not catch all errors is valid but I think not very meaningful. The complexity of language and compiler and computer and operating system and ... conspire to make exhaustive testing too exhaustive. If this is accepted as an accurate statement, it would seem unlikely that any test suite, except for very small 'languages', could ever be exhaustive enough to catch all possible compiler errors. And this is not to attempt to derate ACT or your own efforts. Only to say that IMO the issue is not meaningful in that failing to pass a "nasty test case" means that the language feature tested is not useable and the compiler erroneous. art My own opinions are often my own, seldom my companies.