OpenToken: Parsing Ada (subset)?

[Jacob Sparre Andersen <sparre@nbi.dk>]

I'm attempting to use OpenToken to parse an Ada subset without much
success.

I've reduced my problem all the way down to attempting to parse a
"dotted" identifier:

   Example           -> Dotted_Identifier EOF
   Dotted_Identifier -> Identifier | Dotted_Identifier "." Identifier

I declare the syntax like this (I've removed the reserved words for now):

   Syntax : constant Tokenizer.Syntax :=
     (Dot_T                 => Tokenizer.Get (OpenToken.Recognizer.Separator.Get (".")),
      Identifier_T          => Tokenizer.Get (OpenToken.Recognizer.Identifier.Get
                                                (Start_Chars => Ada.Strings.Maps.Constants.Letter_Set,
                                                 Body_Chars  => Ada.Strings.Maps.Constants.Alphanumeric_Set)),
      Comment_T             => Tokenizer.Get (OpenToken.Recognizer.Line_Comment.Get ("--")),
      Whitespace_T          => Tokenizer.Get (OpenToken.Recognizer.Character_Set.Get
                                                (OpenToken.Recognizer.Character_Set.Standard_Whitespace)),
      End_Of_File_T         => Tokenizer.Get (OpenToken.Recognizer.End_Of_File.Get));

I declare the token objects (for writing the grammar) like this:

   Dot        : constant Master_Token.Class := Master_Token.Get (Dot_T);
   EOF        : constant Master_Token.Class := Master_Token.Get (End_Of_File_T);
   Identifier : constant Identifiers.Instance'Class := Identifiers.Get (Identifier_T);

   Compilation_Unit  : constant Nonterminal.Class := Nonterminal.Get (Compilation_Unit_T);
   Dotted_Identifier : constant Dotted_Identifiers.Class := Dotted_Identifiers.Get (Dotted_Identifier_T);

I declare the grammar as:

   Grammar : constant Production_List.Instance :=
     Compilation_Unit  <= Dotted_Identifier & EOF and
     Dotted_Identifier <= Dotted_Identifier & Dot & Identifier + Dotted_Identifiers.Join_Dotted_Identifiers and
     Dotted_Identifier <= Identifier;

When I attempt to use this grammar to parse a "dotted" identifier, I run
into problems when OpenToken attempts to convert an identifier token to
a Dotted_Identifier.Instance object.  The problem which occurs is that
the identifier is represented as a plain Master_Token.Instance object,
which doesn't contain the actual identifier - which means that I can't
copy it into the "dotted" identifier.

I'm using OpenToken 6.0b.  The full compilable source (including the
used OpenToken packages) can be downloaded from
<http://jacob.sparre-andersen.dk/temp/ada_parsing_with_opentoken-2015-06-01-a.zip>.
To build the example:

   gnatmake parse_dotted_identifier

To test it:

   echo Ada.Text_IO.Integer_IO | ./parse_dotted_identifier

I'm probably doing something obvious wrong, but what?

Greetings,

Jacob
-- 
"It is very easy to get ridiculously confused about the
 tenses of time travel, but most things can be resolved
 by a sufficiently large ego."