Re: Lexical Conundrum

[Mark A Biggar <mark.a.biggar@lmco.com>]

Robert A Duff wrote:
> 
> In article <01bd3d80$101287c0$LocalHost@xhv46.dial.pipex.com>,
> Nick Roberts <Nick.Roberts@dial.pipex.com> wrote:
> >But, if you look closely at line 6, you will see the sequence
> >
> >   or'a'in
> >
> >in the middle of an expression.
> >
> >Now, from chapter 2 of the RM, one might get the impression that this could
> >be parsed as five lexical elements (three identifiers and two apostrophes).
> 
> I think 2.2(7) makes it clear that the above is three lexical elements,
> not five.

Not really, as section 2.2(7) only talks about cases that require white space
between tokens to be legal.  The real point in this example is the "or" is
a keyword not an identifier, so the following "'" must be the start of
a character literal, not the start of an attribute.  If you look at the 
LM grammar, you will find that there are NO cases where an attribute follows
a key word and also NO cases where a character literal follows an identifier,
so you can always determine the meaning of "'" just by remembering the
classification of the previous token.  This whole problem is one of the
reasons why the Ada95 LM makes such a big deal of the fact
that keywords tokens are NOT a subset of identifier tokens, which was a
problem with the Ada83 LM.  Note that a lexical analyser can fully determine
this on its own with out help from the associated parser.  So any compiler
that doesn't tokenize the above as the 3 tokens "or", "'a'" and "in" is
in violation of the LM.

--
Mark Biggar
mark.a.biggar@lmco.com