Re: OpenToken: Handling the empty word token

["Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de>]

On Mon, 30 Jan 2012 08:28:57 -0800 (PST), mtrenkmann wrote:

> I am wondering if that behavior cannot be implemented in the table-
> lookup procedure. That is when I call Action_For with an unexpected
> token, while there is the epsilon as one of the expected tokens, then
> this epsilon could be accepted automatically, the parser changes its
> state accordingly, and the unexpected token is checked again. Don't
> know if that is possible, since I am not yet familiar with the
> internals of table-driven parsing.

You have tables of token for states where there are alternatives. You match
the source against the table:

    Get (Source, Statements_Table, Token, Got_It);
    if Got_It then -- One of the tokens has been matched
       case Token is
           when If_Token =>
               Do_If;
           when For_Token =>
               Do_For;
            when ... =>
               ...
       end case;
    else -- Nothing matched, this is the "epsilon" choice
        ...
    end if;

This is basically a recursive descent parser. You would also have to define
a pair useful subprograms like Skip_Blanks, Get_Identifier, Get_Literal,
Get_Keyword.

The only difficult part is infix expressions. For these there are
ready-to-use means, which match the expression evaluating it or producing a
tree as byproduct.

The problem as you described it just does not exist. You get the keyword,
if it is not there, you do something in reaction. That is.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de