From: Ted Dennison <dennison@telepath.com>
Subject: Announce: OpenToken 2.0 released
Date: 2000/01/27
Date: 2000-01-27T00:00:00+00:00 [thread overview]
Message-ID: <3890C62B.18309585@telepath.com> (raw)
Release 2.0 of OpenToken has now been placed on the website (
http://www.telepath.com/dennison/Ted/OpenToken/OpenToken.html ).
Highlights of this new version include an LALR(1) parser and an html
analyzer submitted by the ever-helpful Christoph Green. This is the
first version to include parsing capablity. The existing packages
underwent a major reorganization to accomodate the new
functionality. As some of the restructuring that was done is
incompatable with old code, the major revision has been bumped up to 2.
A partial list of changes is below:
* Renamed the top level of the hierarchy from Token to OpenToken.
* Moved the analyzer underneath the new OpenToken.Token hierarchy.
* Renamed the Token recognizers from Token.* to
OpenToken.Recognizer.*
* Changed the text feeder procedure pointer into a text feeder
object. This will allow full re-entrancy in analyzers that was
thwarted by those global text feeders previously.
* Updated the SLOC counter to read a list of files to process from a
file. It also handles files with errors in them a bit better.
* Added lalr(1) parsing capability and numerous packages to support
it. A structure is in place to build other parsers as well.
* Created a package hierarchy to support parse tokens. The word
"Token" in OpenToken now refers to objects of this type, rather
than to token recognizers.
* An HTML lexer has been added to the language lexers
* .Recognizer.Bracketed_Comment now works properly with
single-character terminators.
* Rewrote the text feeer and analyzer to minimize data copying.
With this release OpenToken now gains status as a viable replacement for
lex/yacc. In many ways it is more powerful, and there are plans to add
even more power to it. For those of you not already familiar with
OpenToken, I encourage you to vist the website and look around. But in
the meantime, here's a blurb about it from the readme file:
The OpenToken package is a facility for performing token
analysis and parsing within the Ada language. It is designed
to provide all the functionality of a traditional lexical
analyzer/parser generator, such as lex/yacc. But due to the
magic of inheritance and runtime polymorphism it is
implemented entirely in Ada as withed-in code. No
precompilation step is required, and no messy tool-generated
source code is created.
Additionally, the technique of using classes of recognizers
promises to make most token specifications as simple as making
an easy to read procedure call. The most error prone part of
generating analyzers, the token pattern matching, has been
taken from the typical user's hands and placed into reusable
classes. Over time I hope to see the addition of enough
reusable recognizer classes that very few users will ever need
to write a custom one. Parse tokens themselves also use this
technique, so they ought to be just as reusable in principle,
athough there currently aren't a lot of predefined parse
tokens included in OpenToken.
Ada's type safety features should also make misbehaving
analyzers and parsers easier to debug. All this will hopefully
add up to token analyzers and parsers that are much simpler
and faster to create, easier to get working properly, and
easier to understand.
--
T.E.D.
Home - mailto:dennison@telepath.com Work - mailto:dennison@ssd.fsi.com
WWW - http://www.telepath.com/dennison/Ted/TED.html ICQ - 10545591
next reply other threads:[~2000-01-27 0:00 UTC|newest]
Thread overview: 33+ messages / expand[flat|nested] mbox.gz Atom feed top
2000-01-27 0:00 Ted Dennison [this message]
2000-01-28 0:00 ` Announce: OpenToken 2.0 released Jürgen Pfeifer
2000-01-28 0:00 ` Ted Dennison
2000-01-31 0:00 ` Hyman Rosen
2000-02-01 0:00 ` Ted Dennison
2000-02-01 0:00 ` Hyman Rosen
2000-02-01 0:00 ` David Starner
2000-02-01 0:00 ` Brian Rogoff
2000-02-01 0:00 ` Hyman Rosen
2000-02-01 0:00 ` Brian Rogoff
2000-02-02 0:00 ` Hyman Rosen
2000-02-02 0:00 ` Jeff Carter
2000-02-02 0:00 ` Vladimir Olensky
2000-02-01 0:00 ` Hyman Rosen
2000-02-02 0:00 ` Ted Dennison
2000-02-04 0:00 ` Ted Dennison
2000-02-05 0:00 ` Ehud Lamm
2000-02-04 0:00 ` Florian Weimer
2000-02-07 0:00 ` Hyman Rosen
2000-02-07 0:00 ` Florian Weimer
2000-02-07 0:00 ` Hyman Rosen
2000-02-09 0:00 ` Robert A Duff
2000-02-09 0:00 ` Hyman Rosen
2000-02-09 0:00 ` Larry Kilgallen
2000-02-17 0:00 ` Robert A Duff
2000-02-17 0:00 ` Hyman Rosen
2000-02-17 0:00 ` Robert A Duff
2000-02-17 0:00 ` Hyman Rosen
2000-02-17 0:00 ` Robert A Duff
2000-02-17 0:00 ` Hyman Rosen
[not found] ` <88iuk2$s6d3@ftp.kvaerner.com>
2000-03-05 0:00 ` [OT] C and in-band signalling (was: Re: Announce: OpenToken 2.0 released) Florian Weimer
2000-03-06 0:00 ` Tarjei T. Jensen
2000-03-06 0:00 ` Keith Thompson
replies disabled
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox