From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on polar.synack.me X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham autolearn_force=no version=3.4.4 X-Google-Thread: 103376,60cf103f8ae4940d X-Google-Attributes: gid103376,domainid0,public,usenet X-Google-Language: ENGLISH,ASCII-7-bit Path: g2news1.google.com!news2.google.com!border1.nntp.dca.giganews.com!nntp.giganews.com!newsfeed00.sul.t-online.de!t-online.de!news-out2.kabelfoon.nl!newsfeed.kabelfoon.nl!bandi.nntp.kabelfoon.nl!news.banetele.no!uio.no!fi.sn.net!newsfeed1.fi.sn.net!news.song.fi!not-for-mail Date: Fri, 22 Aug 2008 20:17:46 +0300 From: Niklas Holsti User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.8) Gecko/20060628 Debian/1.7.8-1sarge7.1 X-Accept-Language: en MIME-Version: 1.0 Newsgroups: comp.lang.ada Subject: Re: Status of ayacc and aflex? References: <48acd484$0$24596$4d3efbfe@news.sover.net> <9ceb2207-6a3d-407c-84dc-885bfaa07ec1@j22g2000hsf.googlegroups.com> <48aebeb3$0$23599$4f793bc4@news.tdc.fi> In-Reply-To: Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Message-ID: <48aef41e$0$23587$4f793bc4@news.tdc.fi> Organization: TDC Internet Services NNTP-Posting-Host: laku61.adsl.netsonic.fi X-Trace: 1219425310 news.tdc.fi 23587 81.17.205.61:32957 X-Complaints-To: abuse@tdcnet.fi Xref: g2news1.google.com comp.lang.ada:1766 Date: 2008-08-22T20:17:46+03:00 List-Id: Ludovic Brenta wrote: > Niklas Holsti wrote: >>... >>I have not used the parsing functions in OpenToken... > > I too like the absence of a generator, and I'm used to writing my own > parsers by hand. Did you find that OpenToken helped a lot in doing > that? It does its job: lexical analysis. The parser gets to look at the tokens one by one; OpenToken provides a function to return the identity of the current token (an enumeration), another function to return the text string of the current token, and a procedure to advance to the next token. That is what I expect of a lexical analyser, and OpenToken gives me that (at least; I haven't really studied it thoroughly to see what else there may be). As far as I recall, the only wart I have found has to do with the error reporting when the input text has a sequence of characters that does not match any token -- I had to add a special "invalid token" definition (Opentoken.Recognizer.Nothing.Get) to find the line-number and column-number of the erroneous text. A minor detail. If I try to think of what might be missing, perhaps the main thing is context-dependent lexical analysis: the ability to say, for example, that I expect the next token to be an identifier, so please ignore the definitions of reserved keywords and just consider them identifiers, too. Of course I have designed my own languages not to need this (keywords are really reserved). As far as I know OpenToken has no general look-ahead facility, but it should not be too hard to build one yourself, on top of OpenToken, if you need it. Another good point about OpenToken and the absence of a generator phase: one can have several instances of OpenToken in the same application, for different languages, without any clashes of names or data. My application needs that. A caveat: The amounts of text that my applications scan with OpenToken are small. I have no idea of the scanning speed; it has been quite enough for my needs. -- Niklas Holsti Tidorum Ltd niklas holsti tidorum fi . @ .