From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on polar.synack.me X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham autolearn_force=no version=3.4.4 X-Google-Thread: 103376,7a3b1c6915ea1273 X-Google-NewGroupId: yes X-Google-Attributes: gida07f3367d7,domainid0,public,usenet X-Google-Language: ENGLISH,ASCII-7-bit Received: by 10.68.75.170 with SMTP id d10mr3832033pbw.6.1327017872938; Thu, 19 Jan 2012 16:04:32 -0800 (PST) MIME-Version: 1.0 Path: lh20ni201578pbb.0!nntp.google.com!news1.google.com!npeer02.iad.highwinds-media.com!news.highwinds-media.com!feed-me.highwinds-media.com!nx01.iad01.newshosting.com!newshosting.com!news2.euro.net!amsnews11.chello.com!nuzba.szn.dk!news.jacob-sparre.dk!pnx.dk!jacob-sparre.dk!ada-dk.org!.POSTED!not-for-mail From: "Randy Brukardt" Newsgroups: comp.lang.ada Subject: Re: How many Ada compiler bugs are here? Date: Thu, 19 Jan 2012 18:04:27 -0600 Organization: Jacob Sparre Andersen Research & Innovation Message-ID: References: <01dd6341-9c3c-4dcb-90f8-6ac66c65eb66@z1g2000vbx.googlegroups.com><0980607c-df6d-4769-bf6c-288c5666b40f@a11g2000vbz.googlegroups.com><7a8f802e-8d70-4d97-8b85-3f87dafa0899@u20g2000yqb.googlegroups.com> <75df28b7-e784-41a1-9c40-9520fb6f1c0d@v14g2000yqh.googlegroups.com> NNTP-Posting-Host: static-69-95-181-76.mad.choiceone.net X-Trace: munin.nbi.dk 1327017870 25931 69.95.181.76 (20 Jan 2012 00:04:30 GMT) X-Complaints-To: news@jacob-sparre.dk NNTP-Posting-Date: Fri, 20 Jan 2012 00:04:30 +0000 (UTC) X-Priority: 3 X-MSMail-Priority: Normal X-Newsreader: Microsoft Outlook Express 6.00.2900.5931 X-RFC2646: Format=Flowed; Original X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 Date: 2012-01-19T18:04:27-06:00 List-Id: "Adam Beneschan" wrote in message news:75df28b7-e784-41a1-9c40-9520fb6f1c0d@v14g2000yqh.googlegroups.com... >On Jan 18, 5:47 pm, "Randy Brukardt" wrote: >> "Brian Drummond" wrote in message >> ... >> > Recalling an earlier conversation : this (visibility of private parts) >> > strikes me as a candidate for submission to the ACATS suite; though it >> > would surprise me if ACATS doesn't cover this in some depth already. >> >> Keep in mind that the ACATS doesn't try to guess the errors that a >> compiler >> implementer might make, as that is nearly an infinite set. > >And some days it seems like I've made all of them. Been there, done that. >Of course, I could really quibble about what "nearly infinite" means. >However big the set of possible errors is, I'm sure it's still >billions, maybe even trillions, short of infinity. :) True enough. For the purposes of a test suite, it might as well be infinite, even if the actual number is "finite but very large". An ACATS with 10 billion tests would be hard to process... Aside: I note that security testers are putting a lot of work into "fuzzing" tools. One could imagine building such a tool for Ada syntax; it could create thousands of programs each containing a single error in many different ways. But I'd be somewhat dubious about the value of such a tool, as it would a take a lot of effort to create and probably wouldn't find a lot of errors. (Mainly because many compilers use a table-driven parser [Janus/Ada does], so the only way to get parsing errors is to have mistakes in the original grammar definition. Those happen, but they're relatively obvious and easy to fix when found. Even in compilers with hand-written parsers, [like GNAT], errors are fairly unlikely and obvious.) A fuzzing tool would be a lot more useful if it could generate Legality errors automatically, but I don't think that is possible beyond a very limited subset. Randy. -- Adam