Parsing a line into strings

Parsing a line into strings

[C N]

Hi all,

    I've been looking for a comand in Ada that is equivilent to C's
"strtok" .  So far - no luck.
I need to break up a line that is read in from a file with the strings
delimited by comas and spaces.

    If this command does'nt exhist, would anyone be willing to share a
chunk of code they've developed that performs this operation?

    Any and all responses are greatly appreaciated,

-=Chris=-

-=// "Even a blind squrrel finds a nut once in a while." \\=-

Re: Parsing a line into strings

[Samuel Tardieu]

>>>>> "Chris" == C N <netzelc@oit.edu> writes:

Chris> If this command does'nt exhist, would anyone be willing to
Chris> share a chunk of code they've developed that performs this
Chris> operation?

If you're using GNAT, the package GNAT.Spitbol has such a feature.

  Sam
-- 
Samuel Tardieu -- sam@ada.eu.org

Re: Parsing a line into strings

[C N]

    Oh, sorry, I forgot to mention that I am using "ObjectAda - learning
edition 7".  I plan on getting gnat soon ( I heard it's much better ),
but I have to make-do with this for now.

    Thanks for the info - I will keep that in mind when I swich
compilers,

    -=Chris=-

Re: Parsing a line into strings

[Brian Rogoff]

On Thu, 9 Jul 1998, Dmitriy Anisimkov wrote:
> C N wrote:
> > 
> >     Oh, sorry, I forgot to mention that I am using "ObjectAda - learning
> > edition 7".  I plan on getting gnat soon ( I heard it's much better ),
> > but I have to make-do with this for now.
> > 
> >     Thanks for the info - I will keep that in mind when I swich
> 
> It is posable to compile GNAT modules under the 
> Aonix "ObjectAda - learning edition 7". 
> I did it for generic sorting packages.

It is not possible to easily port the GNAT Spitbol packages to any Ada 95
compiler which does not support the Unrestricted_Access attribute
(amongst others) or some similar functionality. Of course, since downward 
closures are easily ;-) simulated by generics or objects, someone will  
of course rewrite this in portable Ada 95 to prove me wrong.

This brings up the interesting question, related to the resuscitated
downward funarg thread, of which Ada 95 compilers support
Unrestricted_Access (under any name) or a similar feature. Any vendors
care to comment?

-- Brian

Re: Parsing a line into strings

[Robert Dewar]

Brian said

<<It is not possible to easily port the GNAT Spitbol packages to any Ada 95
compiler which does not support the Unrestricted_Access attribute
(amongst others) or some similar functionality. Of course, since downward
closures are easily ;-) simulated by generics or objects, someone will
of course rewrite this in portable Ada 95 to prove me wrong.
<<

In fact GNAT.Spitbol uses Unrestricted_Access in a pretty fundamental
way, having nothing at all to do with downward funargs!

Re: Parsing a line into strings

[nabbasi]

In article <Pine.BSF.3.96.980708161520.6018A-100000@shell5.ba.best.com>, Brian
says...

>It is not possible to easily port the GNAT Spitbol packages to any Ada 95
>compiler which does not support the Unrestricted_Access attribute
>(amongst others) or some similar functionality. 

one of things I like about the Ada language is its portability. I hope
we dont get to the situation where some Ada 95 code compile on one 
compiler but not on another. may be this unrestricted_access attribute
should be implemented by other compiler vendors to match GNAT.

Nasser

Re: Parsing a line into strings

[Dmitriy Anisimkov]

C N wrote:
> 
>     Oh, sorry, I forgot to mention that I am using "ObjectAda - learning
> edition 7".  I plan on getting gnat soon ( I heard it's much better ),
> but I have to make-do with this for now.
> 
>     Thanks for the info - I will keep that in mind when I swich

It is posable to compile GNAT modules under the 
Aonix "ObjectAda - learning edition 7". 
I did it for generic sorting packages.

Re: Parsing a line into strings

[Dale Stanbrough]

"If you're using GNAT, the package GNAT.Spitbol has such a feature."

Spitbol is a rather large hammer to crack a rather small nut.
I have a package that will do what you want to do. It is at...

    http://www.cs.rmit.edu.au/~dale/software/index.html


From the package spec...

  function Select_Field (
      Item            : String;
      Field_No        : Integer;
      Field_Separator : Character) return String;

   --  Returns a string that represents the nth string in the field.
   --  The 'first of the return string is always set to one
   --  Fields are separated by the supplied character
   --  E.g.
   --
   --     Field ("cat:dog:mouse", 3, ':')
   --
   --  would result in the string
   --
   --     "mouse"
   --
   --  A -ve value indicates searching from the RHS.
   --
   --  E.g.
   --
   --     Field ("cat:dog:mouse", -1, ':')
   --
   --  would result in the string
   --     "mouse"


Dale

Re: Parsing a line into strings

[Robert Dewar]

Nasser says

<<one of things I like about the Ada language is its portability. I hope
we dont get to the situation where some Ada 95 code compile on one
compiler but not on another. may be this unrestricted_access attribute
should be implemented by other compiler vendors to match GNAT.
>>

The Ada language is not portable, rather the Ada language facilitates
writing portable code. If portability is a requirement, then if you
take appropriate care, Ada is a particularly good language to meet
this requirement.

However, it is most certainly possible to write non-portable code, and
this is not necessarily a bad thing, it depends on whether or not portability
is a requirement.

For the GNAT library, the only requirement is that it be portable to
any implementation of GNAT, it is simply not a requirement that it
be able to be compiled on other compilers. This may be possible for
some of the simpler packages, and if so fine, but the requirements
for these routines are performance, functionality, and a convenient
interface -- portability is simply not a requirement, so we feel
absolutely free to use non-portable attributes and pragmas when
implementing these packages.

You can't expect all other vendors to implement Urestricted_Access.
Remember that several vendors fought furiously any attempt to provide
features of this kind, because they are not display-friendly. Since at
least one Ada 95 compiler uses displays, you can anticipate that this
implementation at least is unlikely to implement Urestricted_Access,
and that situation is unlikely to change.

At the same time, we definitely feel free to add useful pragmas and
attributes to GNAT (see the GNAT programmers guide for a list). Of
course if portability to other Ada compilers (as opposed to portability
to other GNAT implementations) is a consideration, then you should not
use these implementation dependent attributes and pragmas.

One extra features that GNAT implements that I *do* think all vendors
should implement is two extra Restriction identifiers:

   pragma Restrictions (No_Implementation_Pragmas);
   pragma Restrictions (No_Implementation_Attributes);

which respectively prevent any use of implementation dependent
pragmas or attributes.

Re: Parsing a line into strings

[Michael F Brenner]

Some code was posted to this list last year which might do some of the
functionality you wish (depending on what that functionality is).

EXTRACT FROM THIS NEWSGROUP FROM LAST YEAR:
----------------------------------------------------------------------

package fielder is
  pragma pure (fielder);

  -- Sets left_field and right_field to the column pointers dividing
  -- BUFFER into fields.
  --
  -- The field boundaries are any noticed characters (currently eoln & blank)
  -- Computes the number_of_fields. Field zero means the entire buffer
  -- (all fields).
  --
  -- Suggested usage:
  --
  -- procedure try_fielder is
  --   fielding:       fieldings;
  --   message:        string (1..80);
  --   message_length: natural := 49; -- the characters used in MESSAGE
  -- begin
  --   break_into_fields (message, message_length, fielding);
  --   print ("field (1)=" &
  --          message (fielding.left_field(1) .. fielding.right_field(1));
  -- end try_fielder;
  --
type field_lists  is array (0..255) of natural;

  type fieldings is record
    left_field, right_field: field_lists;
    number_of_fields:        natural;
  end record;

  procedure break_into_fields (buffer:          string;
                               length:          natural;
                               fielding: in out fieldings;
                               blank1:          character := ' ';
                               blank2:          character := ascii.ht);

                               -- After break_into_fields, fielding contains
                               -- the left and right boundaries of each
                               -- of the fields. Used for efficient access
                               -- to those boundaries in the long form of
                               -- field.

  function field              (field_no:        natural;
                               buffer:          string;
                               length:          natural;
                               fielding:        fieldings) return string;

                               -- The long form of FIELD returns the
                               -- desired field as broken up in FIELDING.
                               -- This breaks is up once and returns
                               -- many fields without breaking it up again.

  function field              (field_no:        natural;
                               buffer:          string) return string;

                               -- The short form of FIELD returns the
                               -- desired field, breaking the buffer up
                               -- into fields each time it is called.
                               -- Good for one-time calls, but bad for loops.

procedure nullify           (field_no:        natural;
                               buffer:   in out string; -- length doesnt change
                               length:   in out natural;
                               fielding: in out fieldings);

                               -- After NULLIFY, the desired field is
                               -- all blanks and fielding reflects that

 procedure set_field (field_no: natural;           -- field to be replaced
                      buffer:   in out string;     -- length doesnt change
                      length:   in out natural;    -- changes
                      fielding: in out fieldings;  -- changes
                      new_data: string);           -- is inserted

end fielder;





package body fielder is

procedure break_into_fields (buffer: string;
                              length: natural;
                              fielding: in out fieldings;
                              blank1:          character := ' ';
                              blank2:          character := ascii.ht) is

   -- After the procedure BREAK_INTO_FIELDS is called, the FIELDING
   -- contains updated LEFT AND RIGHT. LEFT(1) points to the leftmost
   -- character in the first field. FIELD(1) returns the first field.
   -- FIELD(0) returns buffer (1..length). Nullify sets a field to
   -- null, shifting the other arguments down a position, like $2="" in awk.

   end_of_line_character: constant character := character'val (10);

   type bit_lists is array (character) of boolean;
   pragma pack (bit_lists);
   noticed: bit_lists := (others => false);

   column_upto: natural   := buffer'first;
   ch:          character := ' ';
   fielder_len_gt_length: exception;
   last: constant natural := buffer'first - 1 + length;

   procedure get_next_character is
   begin
     if column_upto > last then
       ch := end_of_line_character;
     else
       ch := buffer (column_upto);
     end if;
     column_upto := column_upto + 1;
   end get_next_character;

   procedure find_a_non_blank is
   begin
     loop
       get_next_character;
       exit when not noticed (ch);
       exit when ch = end_of_line_character;
     end loop;
   end find_a_non_blank;

 begin
   noticed (end_of_line_character) := true;
   noticed (blank1) := true;
   noticed (blank2) := true;
   fielding.number_of_fields := 0;
   for j in 1..fielding.left_field'last loop
     fielding.left_field(j)  := 0;
     fielding.right_field(j) := 0;
   end loop; -- Permits user to ask for field 2 when there is only 1 field.
   fielding.left_field (0) := buffer'first;
   fielding.right_field(0) := last;
   if length>buffer'length then
     raise fielder_len_gt_length;
   end if;

   loop
     find_a_non_blank;
     exit when ch = end_of_line_character;
     fielding.number_of_fields := fielding.number_of_fields+1;
     fielding.left_field(fielding.number_of_fields) := column_upto-1;
     if fielding.number_of_fields = field_lists'last then
       fielding.right_field (fielding.number_of_fields) := buffer'last;
       exit;
     end if;
     loop
       get_next_character;
       exit when noticed (ch);
     end loop;
     fielding.right_field(fielding.number_of_fields) := column_upto - 2;
     exit when ch = end_of_line_character;
   end loop;
 end break_into_fields;

 function field (field_no: natural;
                 buffer:   string;
                 length:   natural;
                 fielding: fieldings) return string is

   lf: natural renames fielding.left_field (field_no);
   rf: natural renames fielding.right_field (field_no);
   n:  natural renames fielding.number_of_fields;
 begin
   if field_no = 0 then
     return buffer (1..length);
   elsif length = 0 or else field_no > N or else rf < lf then
     return "";
   else
     declare
       subtype desired_columns is natural range LF .. RF;
       size: constant integer := RF + 1 - LF;
       alternate_buffer: constant string (1..size) := buffer (desired_columns);
     begin
       return alternate_buffer;
     end;
   end if;
 end field;

 function field (field_no: natural;
                 buffer:   string) return string is
   fielding: fieldings;
   aligned_buffer: constant string (1..buffer'length) := buffer;
 begin
   break_into_fields (aligned_buffer, buffer'length, fielding);
   return field (field_no, aligned_buffer, buffer'length, fielding);
 end field;

procedure nullify (field_no: natural;
                    buffer:   in out string;  -- length not changed
                    length:   in out natural;
                    fielding: in out fieldings) is

   -- NULLIFY removes a field, shifting all the other fields left one field.
   -- If the fields were (1=> "AA", 2=> "BC", 3=> "ZZ"), then after giving
   -- NULLIFY (2), the fields are (1=> "AA", 2=> "ZZ"). Nullify(0) nullfies
   -- all fields.

 begin
   if field_no=0 then
     fielding.left_field(0) := 0;
     fielding.right_field(0) := 0;
     fielding.number_of_fields := 0;
     for c in buffer'range loop
       buffer (c) := ' ';
     end loop;
     buffer(buffer'first) := ascii.nul;

       -- All fields will be nullified by the loop below.

   elsif field_no > fielding.number_of_fields then
     null;

     -- Nothing is changed by the loop below.

   elsif fielding.number_of_fields > 0 then
     for c in fielding.left_field(field_no)..
              fielding.right_field(field_no) loop
       buffer(c) := ' ';
     end loop;

     for j in field_no..fielding.number_of_fields - 1 loop
       fielding.left_field(j)  := fielding.left_field(j+1);
       fielding.right_field(j) := fielding.right_field(j+1);
     end loop;
     fielding.number_of_fields := fielding.number_of_fields - 1;

     -- One field is nullified by the loop below;

   end if;

   for j in fielding.number_of_fields+1 ..
            fielding.left_field'last loop
     fielding.left_field(j) := 0;
     fielding.right_field(j) := 0;
   end loop;
   length := fielding.right_field (fielding.number_of_fields);

   -- Unused fields are set to zero length

 end nullify;

 procedure set_field (field_no: natural;           -- field to be replaced
                      buffer:   in out string;     -- length does not change
                      length:   in out natural;    -- changes
                      fielding: in out fieldings;  -- changes
                      new_data: string) is         -- is inserted

   -- The message is broken into three parts: A B C:
   --   @ A contains the fields to the left of field (field_no)
   --   @ B contains the new field (field_no) which MAY CONTAIN BLANKS
   --   @ C contains the fields to the right of field (field_no)

   A: constant string := buffer(1..fielding.left_field(field_no)-1);
   B: string renames new_data;
   C: constant string := buffer(fielding.right_field(field_no)+1..length);
   set_field_too_long: exception;

   procedure set_buffer (message: string) is
   begin
     if message'length > buffer'length then
       raise set_field_too_long;
     end if;
     buffer (1..message'length) := message;
     for i in natural(message'length+1)..buffer'length loop
       buffer (i) := ' ';
     end loop;
     length := message'length;
   end set_buffer;

 begin
   if    A'length=0 and C'length=0 then
     set_buffer (B);
   elsif A'length=0 then
     set_buffer (B & ' ' & C);
   elsif B'length=0 then
     set_buffer (A & ' ' & B);
   else
     set_buffer (A & ' ' & B & ' ' & C);
   end if;
   break_into_fields (buffer, length, fielding);
 end set_field;

end fielder;

with fielder;
procedure ttfield is
  x: constant string      := "    abc def ghi";
  L: natural              := x'length;
  y: string (1..12) := (others => 'A');
  yl: natural := 0;
  tsfield_1: exception;
  tsfield_2: exception;
  use fielder;
  fielding: fieldings;

  procedure assert(condition: boolean) is
    ttfield_failed: exception;
  begin
    if not condition then
      raise ttfield_failed;
    end if;
  end assert;
begin
  break_into_fields (x, L, fielding);
  if fielding.number_of_fields /= 3     or else
     field (1, x, L, fielding) /= "abc" or else
     field (2, x, L, fielding) /= "def" or else
     field (3, x, L, fielding) /= "ghi" then
    raise tsfield_1;
  end if;
  if fielder.key ("a b c", 3, 1) /= "c~a" then
    raise tsfield_2;
  end if;
  fielder.break_into_fields (y, yl, fielding);
  fielder.set_field (0, y, yl, fielding, "");
  assert(yl=0);
  y(1..5):= "a b c";
  yl:=5;
  fielder.break_into_fields (y, yl, fielding);
  assert(yl=5);
  fielder.set_field (0, y, yl, fielding, "");
  assert(yl=0);
end ttfield;
-------------------------------------------------------

Re: Parsing a line into strings

[dennison]

In article <35A3A199.D55C3153@oit.edu>,
  C N <netzelc@oit.edu> wrote:
> Hi all,
>
>     I've been looking for a comand in Ada that is equivilent to C's
> "strtok" .  So far - no luck.
> I need to break up a line that is read in from a file with the strings
> delimited by comas and spaces.
>

That command would be "strtok".  :-)

strtok is NOT a C command. It is a library function. If you have the library,
you can call it from C or from Ada.

Of course if this is a class assignment, you teacher may be a bit
dissappointed that you didn't write it yourself...

T.E.D.

-----== Posted via Deja News, The Leader in Internet Discussion ==-----
http://www.dejanews.com/rg_mkgrp.xp   Create Your Own Free Member Forum

Re: Parsing a line into strings

[Brian Rogoff]

On 8 Jul 1998, Robert Dewar wrote:
> 
> In fact GNAT.Spitbol uses Unrestricted_Access in a pretty fundamental
> way, having nothing at all to do with downward funargs!

That sound you hear is me removing my foot from my mouth :-)

In order to do penance, I have to post Ada code. Since the original poster 
was asking for the functionality of strtok in Ada, let me take a stab at
it; note that I won't diddle the state of the string like strtok does, but 
this is pretty much it. Since the caller knows the start position its easy
to use slices to extract the "tokens". Hacked and not tested, as usual :-)

-- Brian

with Ada.Strings.Maps;
use Ada.Strings.Maps;

procedure String_Separator_Position (Input : in String ; 
                                     Separators : in Character_Set;
                                     Start : in out Integer) is 
    Pos : Integer := Start;
begin
    if Start < Input'First or Start > Input'Last then 
        return; -- or raise an exception, or whatever...
    end if;

    for I in Start .. Input'Last loop
        if Is_In(Input(I),Separators) then 
            Start := I;
            return;
        end if;
    end loop;
end String_Separator_Position;


    

Re: Parsing a line into strings

[Darren Davenport]

C N <netzelc@oit.edu> wrote:
: Hi all,

:     I've been looking for a comand in Ada that is equivilent to C's
: "strtok" .  So far - no luck.
: I need to break up a line that is read in from a file with the strings
: delimited by comas and spaces.

:     If this command does'nt exhist, would anyone be willing to share a
: chunk of code they've developed that performs this operation?

:     Any and all responses are greatly appreaciated,

: -=Chris=-

: -=// "Even a blind squrrel finds a nut once in a while." \\=-

Why wouldn't Find_Token from one of the Ada.Strings.* packages work?  Seems to
me like it's fairly equivalent and doesn't require roll-your-own code or the
non-portable Spitbol.

darren

(Remove NOSPAM for email address)

Re: Parsing a line into strings

[Robert I. Eachus]

In article <35a52ade.0@news-in.dtc.net> Darren Davenport <dcd2@NOSPAMnov.diac.com> writes:

  > Why wouldn't Find_Token from one of the Ada.Strings.* packages
  > work?  Seems to me like it's fairly equivalent and doesn't require
  > roll-your-own code or the non-portable Spitbol.

   I was wondering how long it would take for someone to suggest this.
Note that Find_Token is a little more general that strtok, since you
can test for (in this case) tokens delimited by comma or space, or
better, test by enumerating all the characters permitted in tokens.
--

					Robert I. Eachus

with Standard_Disclaimer;
use  Standard_Disclaimer;
function Message (Text: in Clever_Ideas) return Better_Ideas is...

Re: Parsing a line into strings

[david.c.hoos.sr]

In article <35A3A199.D55C3153@oit.edu>,
  C N <netzelc@oit.edu> wrote:
> Hi all,
>
>     I've been looking for a comand in Ada that is equivilent to C's
> "strtok" .  So far - no luck.
> I need to break up a line that is read in from a file with the strings
> delimited by comas and spaces.
>
>     If this command does'nt exhist, would anyone be willing to share a
> chunk of code they've developed that performs this operation?
>
>     Any and all responses are greatly appreaciated,
>
> -=Chris=-
>
> -=// "Even a blind squrrel finds a nut once in a while." \\=-
>

The program at the end of this message will parse the string submitted
as the first command line argument into tokens delimited by any of the
characters in the string submitted as the second command line
argument, and print the tokens one at a time to the standard output.

--- begin program source code ----
with Ada.Command_Line;
with Ada.Strings.Fixed;
with Ada.Strings.Maps;
with Ada.Text_Io;
procedure Tokenize is
begin
   if Ada.Command_Line.Argument_Count /= 2 then
      Ada.Text_Io.Put_Line (Ada.Text_Io.Standard_Error,
         "USAGE:" &
         Ada.Command_Line.Command_Name &
         " <string to be tokenized> <delimiters>");
      Ada.Command_Line.Set_Exit_Status (Ada.Command_Line.Failure);
      return;
   end if;
   declare
      The_String : constant String := Ada.Command_Line.Argument (1);
      The_Delimiters : constant Ada.Strings.Maps.Character_Set :=
      Ada.Strings.Maps.To_Set (Ada.Command_Line.Argument(2));
      First : Positive;     -- Index of first character in token
      Last : Natural := 0;  -- Index of last character in token (also used
      -- incremented by 1 -- as the starting point for next search).
   begin
      loop
         Ada.Strings.Fixed.Find_Token
         (Source => The_String (Last + 1 .. The_String'Last),
            Set => The_Delimiters,
            Test => Ada.Strings.Outside,
            First => First,
            Last => Last);
         exit when Last = 0;
         Ada.Text_Io.Put_Line
         ("Token => """ & The_String (First .. Last) & """");
      end loop;
   end;
end Tokenize;
--- end program source code ----


-----== Posted via Deja News, The Leader in Internet Discussion ==-----
http://www.dejanews.com/rg_mkgrp.xp   Create Your Own Free Member Forum

Re: Parsing a line into strings

[Norman H. Cohen]

Robert Dewar wrote:

> One extra features that GNAT implements that I *do* think all vendors
> should implement is two extra Restriction identifiers:
> 
>    pragma Restrictions (No_Implementation_Pragmas);
>    pragma Restrictions (No_Implementation_Attributes);
> 
> which respectively prevent any use of implementation dependent
> pragmas or attributes.

No_Implementation_Pragmas rules out implementation-defined pragmas, but
not implementation-defined arguments to language-defined pragmas.  For
that, GNAT would have to define

	pragma Restrictions (No_Implementation_Pragma_Arguments);

-- 
Norman H. Cohen

Re: Parsing a line into strings

[Robert I. Eachus]

In article <35AB96E3.2338@watson.ibm.com> "Norman H. Cohen" <ncohen@watson.ibm.com> writes:

  > No_Implementation_Pragmas rules out implementation-defined pragmas, but
  > not implementation-defined arguments to language-defined pragmas.  For
  > that, GNAT would have to define

	   pragma Restrictions (No_Implementation_Pragma_Arguments);

   Shouldn't that be:

	   pragma Restrictions (No_Other_Implementation_Pragma_Arguments);

  ...or am i just being too literal minded? ;-)


--

					Robert I. Eachus

with Standard_Disclaimer;
use  Standard_Disclaimer;
function Message (Text: in Clever_Ideas) return Better_Ideas is...

Re: Parsing a line into strings

[Robert Dewar]

<<           pragma Restrictions (No_Implementation_Pragma_Arguments);

   Shouldn't that be:

           pragma Restrictions (No_Other_Implementation_Pragma_Arguments);

  ...or am i just being too literal minded? ;-)
>>

YOu are being too literal minded!