From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on polar.synack.me X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham autolearn_force=no version=3.4.4 X-Google-Thread: 103376,43127f177a55dc41 X-Google-NewGroupId: yes X-Google-Attributes: gida07f3367d7,domainid0,public,usenet X-Google-Language: ENGLISH,ASCII-7-bit Received: by 10.68.0.170 with SMTP id 10mr170861pbf.2.1320754215749; Tue, 08 Nov 2011 04:10:15 -0800 (PST) Path: h5ni12440pba.0!nntp.google.com!news1.google.com!goblin2!goblin.stu.neva.ru!aioe.org!eternal-september.org!feeder.eternal-september.org!.POSTED!not-for-mail From: Brian Drummond Newsgroups: comp.lang.ada Subject: Re: limited allocated classwide types Date: Tue, 8 Nov 2011 12:10:14 +0000 (UTC) Organization: A noiseless patient Spider Message-ID: References: <0ed43f83-40e7-46d3-8cc4-e1c41f500d28@c1g2000vbw.googlegroups.com> <14ff4109-2f02-4dee-9638-68c1befc07c2@hc5g2000vbb.googlegroups.com> Mime-Version: 1.0 Injection-Date: Tue, 8 Nov 2011 12:10:14 +0000 (UTC) Injection-Info: mx04.eternal-september.org; posting-host="DkTdSjxOCm6DqG+Uf7eArg"; logging-data="7985"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX1/yd/ztQlptq6NOGnAAt6yda4gC1w0WvlY=" User-Agent: Pan/0.134 (Wait for Me; GIT cb32159 master) Cancel-Lock: sha1:Sqf4EtRyS6KYM8MGfP97ypu7tuk= Xref: news1.google.com comp.lang.ada:18855 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Date: 2011-11-08T12:10:14+00:00 List-Id: On Mon, 07 Nov 2011 22:25:38 -0600, Randy Brukardt wrote: > "Adam Beneschan" wrote in message > news:14ff4109-2f02-4dee-9638-68c1befc07c2@hc5g2000vbb.googlegroups.com... > On Nov 4, 2:53 am, Brian Drummond wrote: >>> has ACATS (test suite) not kept up with the language additions in >>> Ada-2005? And is it being maintained and updated alongside Ada-2012? >> >>I should probably let Randy answer this, but I think the answer to your >>questions is that it hasn't kept up, due to insufficient funding. > > Right, both funding and insufficient contributions of tests. (The latter > may be in part caused by the former.) I can understand that! And I appreciate that much high quality work goes into the Standard; perhaps expecting ACATS to keep up is being greedy! >> and the fact that one simple test >>breaks a compiler that passes ACATS isn't itself evidence that there's a >>flaw in ACATS. I believe ACATS is set up to check each feature and rule >>defined in the language (or something like that), but it can't test >>every combination of every feature. And this case seems to involve a >>combination of limited controlled types and interfaces (and perhaps >>extended return). I agree that it's impractical to test every combination of features. But perhaps it is possible to test, in addition to every feature individually, the sum of all of them. (Of course I admit even then, it may not have caught this particular corner case) > It is important to recognize that the ACATS tests do not guarantee > compiler correctness. A compilation system that correctly processes the > ACATS tests is not thereby deemed error-free, nor is it thereby deemed > capable of correctly processing all software that is submitted to it. Which does not negate the value of ACATS in any way. >>> Is there a mechanism for adding ad-hoc test cases like this one to >>> ACATS? One would like to think there is a more systematic effort to >>> add tests alongside features as they are developed. I should have been clearer that there were two independent questions here ... (1) can I or Simon B or Joe Bloggs submit tests ? This has been pretty well answered : Simon W pointed to submitted not-yet- ACATS tests. And... >>I'm wondering if there might be a need for some other public test suite >>that users can contribute to. I don't think ACATS would be an >>appropriate suite to for just dumping any test cases users find--that >>just doesn't seem like the right place--but perhaps someone could start >>a web site. > > I think that would be a great idea. It also would provide a place to > mine for possible future ACATS tests (it's hard work coming up with test > scenarios for C-Tests [that is, executable tests]). I absolutely agree that ACATS shouldn't be a dumping ground for random testcases ... but, if there were such an extensible test suite, it would inevitably contain the occasional nugget worth polishing up and adding to the canon. Perhaps a starting point is to GPL any testcases we submit against (FSF or GPL) Gnat, so that they can become part of the public record. (I understand that commercial bug reports against commercial compilers will be treated differently!) The second question : (2) when adding a new feature or language revision, is there a systematic effort to add tests for it? I think has been answered by the remarks on funding... Thanks, - Brian