Tokens

Tokens

[ProLogic]

What's the best way to split a string into tokens?
Ada.Strings.Tokenize ?

Problem is, I cannot find very much documentation on this. Where can I find
a complete documentation on ADA programming and it's libraries.

Thanks
ProLogic

Re: Tokens

[Pascal Obry]

"ProLogic" <ProLogic@prologitech.ods.org> writes:

> What's the best way to split a string into tokens?
> Ada.Strings.Tokenize ?
> 
> Problem is, I cannot find very much documentation on this. Where can I find
> a complete documentation on ADA programming and it's libraries.

I don't know since Ada.Strings.Tokensize does not exists. Maybe you wanted to
say Ada.Strings.Get_Token ? In that case just read the RM section:

   http://www.adapower.com/rm95/arm95_226.html#SEC226

> ProLogic

I don't think this is your real name, is it ?

Pascal.

-- 

--|------------------------------------------------------
--| Pascal Obry                           Team-Ada Member
--| 45, rue Gabriel Peri - 78114 Magny Les Hameaux FRANCE
--|------------------------------------------------------
--|         http://perso.wanadoo.fr/pascal.obry
--| "The best way to travel is by means of imagination"
--|
--| gpg --keyserver wwwkeys.pgp.net --recv-key C1082595

Re: Tokens

[ProLogic]

Thank you Pascal.

And no, it is not my real name. I have never used my real name on the
internet except for people I know.

ProLogic

"Pascal Obry" <p.obry@wanadoo.fr> wrote in message
news:un0uwrtcj.fsf@wanadoo.fr...
>
> "ProLogic" <ProLogic@prologitech.ods.org> writes:
>
> > What's the best way to split a string into tokens?
> > Ada.Strings.Tokenize ?
> >
> > Problem is, I cannot find very much documentation on this. Where can I
find
> > a complete documentation on ADA programming and it's libraries.
>
> I don't know since Ada.Strings.Tokensize does not exists. Maybe you wanted
to
> say Ada.Strings.Get_Token ? In that case just read the RM section:
>
>    http://www.adapower.com/rm95/arm95_226.html#SEC226
>
> > ProLogic
>
> I don't think this is your real name, is it ?
>
> Pascal.
>
> --
>
> --|------------------------------------------------------
> --| Pascal Obry                           Team-Ada Member
> --| 45, rue Gabriel Peri - 78114 Magny Les Hameaux FRANCE
> --|------------------------------------------------------
> --|         http://perso.wanadoo.fr/pascal.obry
> --| "The best way to travel is by means of imagination"
> --|
> --| gpg --keyserver wwwkeys.pgp.net --recv-key C1082595

Re: Tokens

[ProLogic]

Could I you show me a working example of this please? I've read that entire
section you have shown me, hmmm

"Pascal Obry" <p.obry@wanadoo.fr> wrote in message
news:un0uwrtcj.fsf@wanadoo.fr...
>
> "ProLogic" <ProLogic@prologitech.ods.org> writes:
>
> > What's the best way to split a string into tokens?
> > Ada.Strings.Tokenize ?
> >
> > Problem is, I cannot find very much documentation on this. Where can I
find
> > a complete documentation on ADA programming and it's libraries.
>
> I don't know since Ada.Strings.Tokensize does not exists. Maybe you wanted
to
> say Ada.Strings.Get_Token ? In that case just read the RM section:
>
>    http://www.adapower.com/rm95/arm95_226.html#SEC226
>
> > ProLogic
>
> I don't think this is your real name, is it ?
>
> Pascal.
>
> --
>
> --|------------------------------------------------------
> --| Pascal Obry                           Team-Ada Member
> --| 45, rue Gabriel Peri - 78114 Magny Les Hameaux FRANCE
> --|------------------------------------------------------
> --|         http://perso.wanadoo.fr/pascal.obry
> --| "The best way to travel is by means of imagination"
> --|
> --| gpg --keyserver wwwkeys.pgp.net --recv-key C1082595

An example was Re: Tokens

[chris.danx]

"ProLogic" <ProLogic@prologitech.ods.org> wrote in message
news:3ce81da1$1@news.comindico.com.au...
> Could I you show me a working example of this please? I've read that
entire
> section you have shown me, hmmm

This is a quicky example I wrote in 10mins.  It uses recursion which I think
is clearer in this example, but if it's not clear what is going on I'll
rewrite it with a loop no bother.

If OE mangles it, send me an email (change spamoff to chris)

HTH,
Chris


with ada.strings.fixed;  -- find_token is here;
with ada.strings.maps;   -- character_set is here;
with ada.text_io;

use ada.strings;         -- outside/inside are here;

procedure token_example is

   -- prints all the "words" in a string, where
   -- delims is the set of characters regarded as whitespace.
   --
   procedure what_words (str    : in string;
                         delims : in maps.character_set) is
      first : positive := 1;
      last  : natural := 0;
   begin
      fixed.find_token (str,
                        delims,
                        ada.strings.outside,
                        first,
                        last);

      -- if last = 0 there are no more tokens, so we stop,
      -- otherwise we recur!
      --
      if last > 0 then
         ada.text_io.put (str(first..last) & " & ");

         what_words (str (last + 1..str'last),
                     delims);
      end if;
   end what_words;

   -- a simple character set!
   --
   something_simple : maps.character_set := maps.to_set (" !?,");

begin
   what_words ("My name is bob! And I like skinny dipping parties how about
you?",
               something_simple);
end token_example;

Re: An example was Re: Tokens

[chris.danx]

"chris.danx" <spamoff.danx@ntlworld.com> wrote in message
news:dkWF8.34770$Iv.4001930@news6-win.server.ntlworld.com...

Hi,

Is there a cheap way to prevent the last "&" from being printed in the code
below?  I can think of several options but all of them require an additional
call to find_token or switching to a loop.  Is there another way that
doesn't have a repeated call to find token?

Since it's just an example there's no real harm done, it's just annoying
that's all!!!


Chris

>    procedure what_words (str    : in string;
>                          delims : in maps.character_set) is
>       first : positive := 1;
>       last  : natural := 0;
>    begin
>       fixed.find_token (str,
>                         delims,
>                         ada.strings.outside,
>                         first,
>                         last);
>
>       -- if last = 0 there are no more tokens, so we stop,
>       -- otherwise we recur!
>       --
>       if last > 0 then
>          ada.text_io.put (str(first..last) & " & ");
>
>          what_words (str (last + 1..str'last),
>                      delims);
>       end if;
>    end what_words;

Re: An example was Re: Tokens

[ProLogic]

Thanks chris. Recusion is fine.

"chris.danx" <spamoff.danx@ntlworld.com> wrote in message
news:dkWF8.34770$Iv.4001930@news6-win.server.ntlworld.com...
>
> "ProLogic" <ProLogic@prologitech.ods.org> wrote in message
> news:3ce81da1$1@news.comindico.com.au...
> > Could I you show me a working example of this please? I've read that
> entire
> > section you have shown me, hmmm
>
> This is a quicky example I wrote in 10mins.  It uses recursion which I
think
> is clearer in this example, but if it's not clear what is going on I'll
> rewrite it with a loop no bother.
>
> If OE mangles it, send me an email (change spamoff to chris)
>
> HTH,
> Chris
>
>
> with ada.strings.fixed;  -- find_token is here;
> with ada.strings.maps;   -- character_set is here;
> with ada.text_io;
>
> use ada.strings;         -- outside/inside are here;
>
> procedure token_example is
>
>    -- prints all the "words" in a string, where
>    -- delims is the set of characters regarded as whitespace.
>    --
>    procedure what_words (str    : in string;
>                          delims : in maps.character_set) is
>       first : positive := 1;
>       last  : natural := 0;
>    begin
>       fixed.find_token (str,
>                         delims,
>                         ada.strings.outside,
>                         first,
>                         last);
>
>       -- if last = 0 there are no more tokens, so we stop,
>       -- otherwise we recur!
>       --
>       if last > 0 then
>          ada.text_io.put (str(first..last) & " & ");
>
>          what_words (str (last + 1..str'last),
>                      delims);
>       end if;
>    end what_words;
>
>    -- a simple character set!
>    --
>    something_simple : maps.character_set := maps.to_set (" !?,");
>
> begin
>    what_words ("My name is bob! And I like skinny dipping parties how
about
> you?",
>                something_simple);
> end token_example;
>
>
>
>
>

Re: An example was Re: Tokens

[David C. Hoos, Sr.]

----- Original Message -----
From: "chris.danx" <spamoff.danx@ntlworld.com>
Newsgroups: comp.lang.ada
To: <comp.lang.ada@ada.eu.org>
Sent: May 19, 2002 6:30 PM
Subject: Re: An example was Re: Tokens


>
> "chris.danx" <spamoff.danx@ntlworld.com> wrote in message
> news:dkWF8.34770$Iv.4001930@news6-win.server.ntlworld.com...
>
> Hi,
>
> Is there a cheap way to prevent the last "&" from being printed in the code
> below?  I can think of several options but all of them require an additional
> call to find_token or switching to a loop.  Is there another way that
> doesn't have a repeated call to find token?

Here is a more elegant example that I assert is more generally useful, as it
simply returns an array containing the first and last characters of each
token meeting the criterion.  The result can then be used to access the
tokens individually.

with Ada.Strings.Fixed;
with Ada.Strings.Maps;
with Ada.Text_IO;
procedure Test_Token_Limits is

   type Limits is record
      First : Positive;
      Last  : Natural;
   end record;

   type Limits_Array is array (Positive range <>) of Limits;

   function Token_Limits
     (Source     : String;
      Delimiters : Ada.Strings.Maps.Character_Set)
     return Limits_Array is
      The_Limits : Limits;
   begin
      Ada.Strings.Fixed.Find_Token
        (Source,
         Delimiters,
         Ada.Strings.Outside,
         The_Limits.First,
         The_Limits.Last);
      if The_Limits.Last /= 0 then
         return The_Limits & Token_Limits
           (Source (The_Limits.Last + 1 .. Source'Last), Delimiters);
      else
         return Limits_Array'(1 .. 0 => The_Limits);
      end if;
   end Token_Limits;

   Delimiters : constant Ada.Strings.Maps.Character_Set :=
     Ada.Strings.Maps.To_Set (" !?,:;.");

   Source : constant String := "Now is the time for all good men to " &
     "come to the aid of their country.";

   Word_Limits : constant Limits_Array := Token_Limits
     (Source     => Source,
      Delimiters => Delimiters);
begin
   for W in Word_Limits'Range loop
      Ada.Text_IO.Put_Line
        ("Word no." & W'Img & " is """ &
         Source (Word_Limits (W).First .. Word_Limits (W).Last) & """."
        );
   end loop;
end Test_Token_Limits;