From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on polar.synack.me X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham autolearn_force=no version=3.4.4 X-Google-Thread: 103376,9b7d3a51d0d8b6ee X-Google-Attributes: gid103376,domainid0,public,usenet X-Google-Language: ENGLISH,ASCII-7-bit Path: g2news1.google.com!news4.google.com!feeder1-2.proxad.net!proxad.net!feeder1-1.proxad.net!club-internet.fr!feedme-small.clubint.net!nuzba.szn.dk!news.jacob-sparre.dk!pnx.dk!not-for-mail From: "Randy Brukardt" Newsgroups: comp.lang.ada Subject: Re: Compiler quality Date: Mon, 1 Dec 2008 21:51:26 -0600 Organization: Jacob Sparre Andersen Message-ID: References: <20081115101632.5f98c596@cube.tz.axivion.com> <20081122011825.5354d1c1@cube.tz.axivion.com> <9cb27caa-8e9f-4123-ad36-4980c3032722@20g2000yqt.googlegroups.com> <492e7554$0$30237$9b4e6d93@newsspool1.arcor-online.net> <8fbe700c-9b80-46b5-8ebe-8d3ef8fe41ea@u14g2000yqg.googlegroups.com> <135789fa-1261-4c2f-9fa7-3225d19ecc00@x14g2000yqk.googlegroups.com> <492fc7bc$0$31869$9b4e6d93@newsspool3.arcor-online.net> NNTP-Posting-Host: static-69-95-181-76.mad.choiceone.net X-Trace: munin.nbi.dk 1228189957 13872 69.95.181.76 (2 Dec 2008 03:52:37 GMT) X-Complaints-To: news@jacob-sparre.dk NNTP-Posting-Date: Tue, 2 Dec 2008 03:52:37 +0000 (UTC) X-Priority: 3 X-MSMail-Priority: Normal X-Newsreader: Microsoft Outlook Express 6.00.2900.5512 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.5579 X-RFC2646: Format=Flowed; Original Xref: g2news1.google.com comp.lang.ada:2848 Date: 2008-12-01T21:51:26-06:00 List-Id: "Georg Bauhaus" wrote in message news:492fc7bc$0$31869$9b4e6d93@newsspool3.arcor-online.net... > Martin wrote: > >> To be fair to anon...that's not a SYNTAX bug. His claim was that >> compilers should be able to be 100% tested for syntax bugs by limiting >> the input so much that's it becomes practical to do on current >> machines. > > Considering the Ada 83 rule that 'Base must be prefixed to > 'Another_Attribute (i.e., have context), do Ada compilers > typically check this rule in the parts handling syntax? > (In the case of GNAT, the sem*.ad? files refer to the 'Base > attribute, AFAICS.) I don't think so; the typical syntax for attributes is something like: prefix APOSTROPHE IDENTIFIER where the capitalized items are terminals ' and an identifier. The only "identifier"s that have special handling in the syntax are the reserved words, and "base" is not reserved. A data point about anon's more general point. We discovered a syntax processing error (in our case, an error in the grammar input to the table generator for the syntax part of Janus/Ada 95) more than 6 1/2 years after the Ada 95 grammar was completed (and the associated compiler was in use by many people during the time period of February 1996 to November 2002). The error was allowing the declaration of abstract subprograms in places that they are not allowed, such as generic formal parameters. The point is that no amount of (sane) testing can find unintended additional capabilities -- because the only way to do that is to guess the possible errors, and that is not a practical thing to do as there is a very low probability of actually finding any errors. Thus such testing is not a good use of limited testing effort; there are many more things that are better uses of effort. For this reason, the ACATS does not test separately for syntax errors unless there is a significant probability that the error will be detected outside of the syntax portion of the compiler. After all, most compiler parsers are generated by tools these days, so the errors will almost certainly be limited to simple errors (omission, transposition, or substitution) in the input to those tools and can be detected by pretty much any use of a particular feature of the language. Thus the tests for other language rules provide enough testing for the syntax proper. (I believe that GNAT uses a hand-coded parser, and thus is at somewhat more risk for syntax processing errors than most other compilers, but that still seems to be a low priority error to detect compared to violations of other language rules.) Testing in general is a poor way to eliminate errors in a compiler. Back in 1997, I ran some coverage tests to determine what code in Janus/Ada has been executed in tests. (That doesn't prove that such code will work in all circumstances, but code that is executed is more likely to be correct than code that has never been known to be executed.) I ran pretty much every test and program that we had access to at the time (more than a thousand in-house tests, plus the ACATS, plus Claw and its tests, plus the compiler source code), and still was able only to show coverage on about 70% of the code in the part of the compiler I was testing. While some of the unreached paths detected internal compiler errors (that is, represented redundant checks), many of them represented combinations of language features that had simply not been tested. (Looking at a randomly selected piece of the compiler, some such cases are a timed internal protected entry call; selection of a component from a formal parameter of an unconstrained record type; and selection of a component using the "current instance" of a protected object.) My original idea was to write some test cases to try to reach the untouched code, but the magnitude of the task made it impractical. (It would make more sense to use "white box" unit tests to force coverage, but that brings up the possibility of spending a lot of time testing things that can't happen in real input programs. Not a great choice either way.) Randy.