Re: OpenToken: Handling the empty word token

["Randy Brukardt" <randy@rrsoftware.com>]

"Stephen Leake" <stephen_leake@stephe-leake.org> wrote in message 
news:82obtkgj81.fsf@stephe-leake.org...
> "Randy Brukardt" <randy@rrsoftware.com> writes:
>
>> "Stephen Leake" <stephen_leake@stephe-leake.org> wrote in message
>> news:82ehuibdwt.fsf@stephe-leake.org...
...
> I'm also on somewhat shakey ground on parser theory, but I did read (and
> understand :) "the red dragon book"
> (http://en.wikipedia.org/wiki/Principles_of_Compiler_Design), when I
> took over OpenToken and fixed some major bugs.
>
> The parser is an LALR parser.
>
> But I think the implementation is not complete. There are no examples of
> grammars containing epsilon in the OpenToken tests. I don't remember
> what the dragon book has to say about epsilon handling.

Well, you don't need to do anything special to handle "epsilon". Unless you 
actually want to write "epsilon", which is rather silly IMHO, as it 
represents nothing at all, and it is easier to write nothing at all.

As I recall, the only problem I had with our generator was with an ambiguity 
in recognizing the ends of productions in the input; in our case that was 
solved by requiring an action number at the end even if no action was 
needed. An alternative way to handle that would be to change the input 
grammar to have "epsilon" stand for absoluty nothing (that might be easier 
if there are already some sort of reserved words in the grammar source).

                                       Randy.