OpenToken 3.0b release

OpenToken 3.0b release

[Ted Dennison]

OpenToken 3.0b has just been released. This version contains a somewhat
experimental facility for recursive-decent parsing inspired by ANTLR,
which I hope to make the default method.

Attached below is the Readme file (minus some old history).


                          OpenToken Package Readme

                                Version 3.0b

The OpenToken package is a facility for performing token analysis and
parsing within the Ada language. It is designed to provide all the
functionality of a traditional lexical analyzer/parser generator, such
as lex/yacc. But due to the magic of inheritance and runtime
polymorphism it is implemented entirely in Ada as withed-in code. No
precompilation step is required, and no messy tool-generated source code
is created.

Additionally, the technique of using classes of recognizers promises to
make most token specifications as simple as making an easy to read
procedure call. The most error prone part of generating analyzers, the
token pattern matching, has been taken from the typical user's hands and
placed into reusable classes. Over time I hope to see the addition of
enough reusable recognizer classes that very few users will ever need to
write a custom one. Parse tokens themselves also use this technique, so
they ought to be just as reusable in principle, athough there currently
aren't a lot of predefined parse tokens included in OpenToken.

Ada's type safety features should also make misbehaving analyzers and
parsers easier to debug. All this will hopefully add up to token
analyzers and parsers that are much simpler and faster to create, easier
to get working properly, and easier to understand.

History

Version 3.0b

This version contains another code reorganization to go with another new
parsing facility. This time it is recursive decent parsing. The new
method has the following advantages over table-driven parsers:

   * Its simpler to implement.
   * Its provides many more opportunities for reuse.
   * Its parsers are debugable.
   * There's no expensive parser-generation phase.

The disadvantages are:

   * Its parsers are most likely a bit slower.

Given the above balance, I do intend to make this the standard supported
parsing facility for future versions of OpenToken. The "b" designation
is there to indicate that some things might not be in quite their
permanent form yet, and that there isn't yet the full set of reusable
tokens to support it that I would like to see in a release. I'm hoping
for feedback both in the form of criticisms/suggestions, and reusable
tokens in order to help finalize this facility.

A general list of the changes is below:

   * Renamed OpenToken.Token tree to OpenToken.Token.Enumerated.
   * Created a new (non-enumerated) base token type and base analyzer
type
     in OpenToken.Token.
   * Made a Parse routine and a Could_Parse_To routine primitives of the
     base token type.
   * Created the following predefined nonterminal tokens (both as
straight
     types, and as mixins).
        o List
        o Selection
        o Sequence
   * Fixed a bug in the bracketed comment recognizer.
   * Implemented a (hopefully temporary) work-around for a bug in Gnat
     version 3.13p.
   * Fixed a bug in the string recognizer where it was mishandling octal
     and hex escape sequences.
   * Changed the analyzer and the text feeders to support analyzing
binary
     files.
   * The HTML lexer has been improved to be a bit faster and more
flexible.

--
T.E.D.

http://www.telepath.com/~dennison/Ted/TED.html


Sent via Deja.com http://www.deja.com/
Before you buy.

Re: OpenToken 3.0b release

[Ted Dennison]

In article <8n96nq$4kj$1@nnrp1.deja.com>,
  Ted Dennison <dennison@telepath.com> wrote:
> OpenToken 3.0b has just been released. This version contains a

BTW: It is available for download in both gzip and zipped forms at

http://www.telepath.com/dennison/Ted/OpenToken/OpenToken.html


--
T.E.D.

http://www.telepath.com/~dennison/Ted/TED.html


Sent via Deja.com http://www.deja.com/
Before you buy.