Re: OpenToken: Handling the empty word token

["Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de>]

On Fri, 27 Jan 2012 08:22:12 -0800 (PST), mtrenkmann wrote:

> Is there a way to instrument the parser to silently accept the epsilon
> token whenever it expects it without consuming a token from the lexer,
> or is it a common convention to translate each grammar into a epsilon-
> free representation?

I use neither explicit grammars nor OpenToken, so it is possible that I
didn't really understand the problem you have.

For a table-driven parser, which I am using, there are two ways to handle
implied tokens, which, I suppose, is what you wanted:

1. You could put an empty string into the table and let that be recognized.
It is a straightforward approach which has the disadvantage that empty
string is always matched, so when you have a recursive grammar you might
run into an endless loop of matching the empty string over and over again
if the parser's state is not really changed.

2. A better way is to connect to the semantic callback on "something
expected." For example, to handle for example assumed multiplication in
infix expressions, e.g. A B + C => A*B + C, I connect to "on missing
operation" and push the multiplication onto the operation stack. That is.
This is not really different from how you would handle a missing right
bracket. Once the parser detected that, you would push the bracket onto the
stack and continue (if you allow recovery on such errors).

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de