Reading the while standard input into a String

Reading the while standard input into a String

[Natasha Kerensikova]

Hello,

I have encountered what was a very easily solved problem in C: dump the
whole contents of the standard input into some place in memory, in order
to process it afterwards.

I was quite unhappy with the preprocessing performed by Ada.Text_IO, so
I went on using the text stream interface. However I still read
character by character into a temporary buffer, and it feels ugly. Here
is the code:

with Ada.Strings.Unbounded;
with Ada.Text_IO.Text_Streams;

   Source : Ada.Text_IO.Text_Streams.Stream_Access
     := Ada.Text_IO.Text_Streams.Stream (Ada.Text_IO.Standard_Input);

   procedure Read_Chunk (To : out String; Last : out Natural) is
   begin
      Last := To'First - 1;
      for J in To'Range loop
         Character'Read (Source, To (J));
         Last := J;
      end loop;
   exception
      when Ada.Text_IO.End_Error => Null;
   end Read_Chunk;


   procedure The_Program is
      -- some tuff
      Input : Ada.Strings.Unbounded.Unbounded_String;
   begin
      declare
         Chunk : String (1 .. 256);
         Last : Natural;
      begin
         loop
            Read_Chunk (Chunk, Last);
            Ada.Strings.Unbounded.Append (Input, Chunk (1 .. Last));
            exit when Last < Chunk'Last;
         end loop;
      end;

      -- Process Input or To_String (Input);
   end The_Program;

Is there a better/smarter/less ugly way to achieve the same result?


Thanks for your help,
Natasha

Re: Reading the while standard input into a String

[robin]

"Natasha Kerensikova" <lithiumcat@gmail.com> wrote in message news:slrniunb6n.i18.lithiumcat@sigil.instinctive.eu...
| Hello,
|
| I have encountered what was a very easily solved problem in C: dump the
| whole contents of the standard input into some place in memory, in order
| to process it afterwards.

It's also trivial in PL/I:

do forever;
    allocate line;
    get edit (line) (L);
end;

or read each line into an array element or whatever.

| I was quite unhappy with the preprocessing performed by Ada.Text_IO, so
| I went on using the text stream interface. However I still read
| character by character into a temporary buffer, and it feels ugly. Here
| is the code:
|
| with Ada.Strings.Unbounded;
| with Ada.Text_IO.Text_Streams;
|
|   Source : Ada.Text_IO.Text_Streams.Stream_Access
|     := Ada.Text_IO.Text_Streams.Stream (Ada.Text_IO.Standard_Input);
|
|   procedure Read_Chunk (To : out String; Last : out Natural) is
|   begin
|      Last := To'First - 1;
|      for J in To'Range loop
|         Character'Read (Source, To (J));
|         Last := J;
|      end loop;
|   exception
|      when Ada.Text_IO.End_Error => Null;
|   end Read_Chunk;
|
|
|   procedure The_Program is
|      -- some tuff
|      Input : Ada.Strings.Unbounded.Unbounded_String;
|   begin
|      declare
|         Chunk : String (1 .. 256);
|         Last : Natural;
|      begin
|         loop
|            Read_Chunk (Chunk, Last);
|            Ada.Strings.Unbounded.Append (Input, Chunk (1 .. Last));
|            exit when Last < Chunk'Last;
|         end loop;
|      end;
|
|      -- Process Input or To_String (Input);
|   end The_Program;
|
| Is there a better/smarter/less ugly way to achieve the same result? 

Re: Reading the while standard input into a String

[Dmitry A. Kazakov]

On Sun, 5 Jun 2011 16:20:39 +0000 (UTC), Natasha Kerensikova wrote:

> However I still read
> character by character

You have to, because the definition of line end is language/OS/encoding
dependent, so in order to detect line ends properly you need to scan
characters one by one, maybe recoding them into the encoding used by the
parser (e.g. UTF-8). It does not make much sense to read input by arbitrary
size chunks. Read it line by line. If parser needs returns over the line
margin (unlikely), then keep read lines cached.

>  into a temporary buffer,

Read it into the destination buffer. Don't use Unbounded_String; that is a
bad idea in almost all cases, this one included.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: Reading the while standard input into a String

[stefan-lucks]

> I have encountered what was a very easily solved problem in C: dump the
> whole contents of the standard input into some place in memory, in order
> to process it afterwards.

With good the good old function Ada.Text_IO.Get_Line, you can read the 
input line by line:

with Ada.Text_IO; with Ada.Strings.Unbounded;

procedure Process_File is

   use Ada.Strings.Unbounded;

   function Input_To_String(Between_Lines: String)
                           return String is
      Representative_Of_File: Unbounded_String;
      U: Unbounded_String;
   begin
      begin
         loop
            declare
               Chunk: String := Ada.Text_IO.Get_Line;
            begin
               Append(U, Between_Lines & Chunk);
            end;
         end loop;
      exception
         when Ada.Text_IO.End_Error => null;
      end;
      return To_String(U);
   end Input_To_String;

begin
   Ada.Text_IO.Put_Line("<p>" & Input_To_String("</p> <p>") & "</p>");
end Process_File;

This works nicely. But, due to the usage of Ada.Text_IO and 
Ada.Strings.Unbounded, the efficiency is likely to be mediocre only. (I 
think, gnat will reallocate a larger unbounded string, copy the old string 
into the new one, and then free the old string, each time it appends 
something to U.) Running the program on a sample 4-line input file gives

<p></p> <p>Line 1 is good.</p> <p>Line 2 is better!</p> <p></p> <p>Line 4 
follows an empty line. </p>

-- 
------ Stefan Lucks   --  Bauhaus-University Weimar  --   Germany  ------
               Stefan dot Lucks at uni minus weimar dot de
------  I  love  the  taste  of  Cryptanalysis  in  the  morning!  ------

Re: Reading the while standard input into a String

[Ludovic Brenta]

Natasha Kerensikova wrote on comp.lang.ada:
> I have encountered what was a very easily solved problem in C: dump the
> whole contents of the standard input into some place in memory, in order
> to process it afterwards.

Personally I think this is a fundamentally bad idea.  Consider:

$ my_program < /dev/random

Do you consider it acceptable to use all memory and all swap space
before finally bailing out with a Storage_Error and not doing any
actual processing?

I'd rather process the input one fixed-length chunk at a time and
discard each chunk after processing (possibly by recycling the in-
memory buffer for the next chunk).

Note that reading one character at a time from a stream is quite fast
since streams normally use the operating system's buffered IO. Adding
a second layer of buffering inside your program is probably counter-
productive.

--
Ludovic Brenta.

Re: Reading the while standard input into a String

[Georg Bauhaus]

On 05.06.11 18:20, Natasha Kerensikova wrote:
> Hello,
> 
> I have encountered what was a very easily solved problem in C: dump the
> whole contents of the standard input into some place in memory, in order
> to process it afterwards.

This article by Robert Dewar may interest you,

"Accessing Memory as a String"

http://www.adapower.com/index.php?Command=Class&ClassID=Advanced&CID=213

Re: Reading the while standard input into a String

[Natasha Kerensikova]

Hello,

On 2011-06-06, Ludovic Brenta <ludovic@ludovic-brenta.org> wrote:
> Natasha Kerensikova wrote on comp.lang.ada:
>> I have encountered what was a very easily solved problem in C: dump the
>> whole contents of the standard input into some place in memory, in order
>> to process it afterwards.
>
> Personally I think this is a fundamentally bad idea.  Consider:
>
> $ my_program < /dev/random
>
> Do you consider it acceptable to use all memory and all swap space
> before finally bailing out with a Storage_Error and not doing any
> actual processing?

Actually, in principle, yes. I mean, I want to be able to write code
that behave that way as far as the "reading the whole standard input"
goes, because that's this part I'm trying to learn about.

Of course, I don't want a real program to behave that way, so I would
probably add a "reasonable" limit (like 16 MB) that interrupt the loop,
but that's beyond the scope of my question. And anyway, I don't trust
any program to not use all memory and all swap space, so I have posix
ulimits set accordingly.

> I'd rather process the input one fixed-length chunk at a time and
> discard each chunk after processing (possibly by recycling the in-
> memory buffer for the next chunk).

I would do so if online processing was an option. However in the program
containing code similar to what I posted, I cannot do any processing
before reaching the end of input, because the processed format allows
forward references, so references have to be gathered in a first pass on
the whole input, before doing a second pass where dereference can take
place.

> Note that reading one character at a time from a stream is quite fast
> since streams normally use the operating system's buffered IO. Adding
> a second layer of buffering inside your program is probably counter-
> productive.

That second layer of buffer was actually meant to reduce the number of
Ada.Strings.Unbounded.Append calls (and the expensive reallocations that
go with them).

However I thought that something similar to C's fread(), which can use
efficient bulk memory copy, would be better than a loop of single
character read (even when buffered in memory).


Thanks for your comments,
Natasha

Re: Reading the while standard input into a String

[Ludovic Brenta]

Natasha Kerensikova wrote on comp.lang.ada:
> On 2011-06-06, Ludovic Brenta <ludo...@ludovic-brenta.org> wrote:
>> I'd rather process the input one fixed-length chunk at a time and
>> discard each chunk after processing (possibly by recycling the in-
>> memory buffer for the next chunk).
>
> I would do so if online processing was an option. However in the program
> containing code similar to what I posted, I cannot do any processing
> before reaching the end of input, because the processed format allows
> forward references, so references have to be gathered in a first pass on
> the whole input, before doing a second pass where dereference can take
> place.

OK, fair enough.

>> Note that reading one character at a time from a stream is quite fast
>> since streams normally use the operating system's buffered IO. Adding
>> a second layer of buffering inside your program is probably counter-
>> productive.
>
> That second layer of buffer was actually meant to reduce the number of
> Ada.Strings.Unbounded.Append calls (and the expensive reallocations that
> go with them).

Actually, Ada.Strings.Unbounded.Append itself has mechanisms to reduce
memory reallocations.  But I see your point.

> However I thought that something similar to C's fread(), which can use
> efficient bulk memory copy, would be better than a loop of single
> character read (even when buffered in memory).

How about:

procedure Read_In_Chunks (Stream : in out
Ada.Streams.Root_Stream_Type'Class;
                          Result : out
Ada.Strings.Unbounded.Unbounded_String) is
   Chunk : Ada.Streams.Stream_Element_Array (1 .. 1024);
   Chunk_As_String : String (1 .. 1024);
   for Chunk_As_String'Address use Chunk'Address;
   Last : Natural;
begin
   loop
      Ada.Streams.Read (Stream => Standard_Input,
                        Item   => Chunk,
                        Last   => Last);
      exit when Last = Chunk'First - 1; -- end of stream reached
      Ada.Strings.Unbounded.Append (Result, Chunk_As_String (1 ..
Last));
      exit when Last < Chunk'Last; -- end of stream reached
   end loop;
end Read_In_Chunks;

--
Ludovic Brenta.

Re: Reading the while standard input into a String

[Ludovic Brenta]

Ludovic Brenta wrote on comp.lang.ada:
> Natasha Kerensikova wrote on comp.lang.ada:
>> However I thought that something similar to C's fread(), which can use
>> efficient bulk memory copy, would be better than a loop of single
>> character read (even when buffered in memory).
>
> How about:
>
> procedure Read_In_Chunks (Stream : in out
> Ada.Streams.Root_Stream_Type'Class;
>                           Result : out
> Ada.Strings.Unbounded.Unbounded_String) is
[...]

I think I'm guilty of premature optimization.  I'd be curious to know
whether Read_In_Chunks is actually faster than simply reading one
character at a time and appending it to Result.  Only a benchmark
could tell for sure.

--
Ludovic Brenta.

Re: Reading the while standard input into a String

[Natasha Kerensikova]

Hello,

On 2011-06-06, Dmitry A. Kazakov <mailbox@dmitry-kazakov.de> wrote:
> On Sun, 5 Jun 2011 16:20:39 +0000 (UTC), Natasha Kerensikova wrote:
>
>> However I still read
>> character by character
>
> You have to, because the definition of line end is language/OS/encoding
> dependent, so in order to detect line ends properly you need to scan
> characters one by one, maybe recoding them into the encoding used by the
> parser (e.g. UTF-8). It does not make much sense to read input by arbitrary
> size chunks. Read it line by line. If parser needs returns over the line
> margin (unlikely), then keep read lines cached.

The line end detection problem is exactly why I wanted unprocessed input
bytes. Each instance of the parser code can get at least LF-ended lines
(from unix files) or CR&LF-ended lines (from web form), so unless
Ada.Text_IO can deals with this (but I guess I cannot really count on
it), I have to do it in my own code.

>>  into a temporary buffer,
>
> Read it into the destination buffer.

Well the destination buffer for the processed text is a very inefficient
place to store input text, because the processing involves a lot of
insertions.

Moreover, because of the forward reference issue I detailed in another
post, I cannot see how I can escape the schema:
input stream --> temporary buffer --> output stream/buffer/storage

> Don't use Unbounded_String; that is a
> bad idea in almost all cases, this one included.

Would you explain why?

Unless there is a way to predict the left of the input, I need some
text container able to grow as much as needed while reading.

I will also need a similar container during the processing, and even
GNAT's improved Unbounded_String still does a lot of a reallocations in
the process. So I was considering implementing my own container,
something like Chunked_Unbounded_String, which would allocate memory by
chunks of fixed size (probably provided by a generic package parameter,
using a few kilobytes) and thereby improve a lot performance of lots of
small Appends. But I guess you weren't calling using Unbounded_String a
bad idea only because of performance, were you?


Thanks for your comments,
Natasha

Re: Reading the while standard input into a String

[Georg Bauhaus]

On 05.06.11 18:20, Natasha Kerensikova wrote:
> Hello,
> 
> I have encountered what was a very easily solved problem in C: dump the
> whole contents of the standard input into some place in memory, in order
> to process it afterwards.

To use something like C's fread(1), I'd do the same in Ada with
Streams and Storage_Element_Array.  Like fread, which has void*
arguments, the Ada equivalent will read storage elements (bytes,
likely octets).

Very likely, the storage elements can be converted to Character
without check, i.e. using an instance of Unchecked_Conversion,
or simple the 'Address attribute.  So no copying is involved
at this stage.

When processing Input character by character, no I/O is
performed.


I define an index for the characters in the storage area
so filled:

declare
    Buffer : Storage_Element_Array (...);
    Input : String (Buffer'Range);
    pragma Import (Ada, Input);
    for Input'Address use Buffer'Address;

    Last : Natural;  -- index of last octed
    Pos : Positive;  -- points to current character

begin
    --  have standard input flow into Buffer from a text stream
    --  ...
    for Pos in Input'First .. Last loop
       Process (Input (Pos));
       pos := pos + 1;
    end loop;
end;

Re: Reading the while standard input into a String

[Dmitry A. Kazakov]

On Mon, 6 Jun 2011 10:46:20 +0000 (UTC), Natasha Kerensikova wrote:

> Hello,
> 
> On 2011-06-06, Dmitry A. Kazakov <mailbox@dmitry-kazakov.de> wrote:
>> On Sun, 5 Jun 2011 16:20:39 +0000 (UTC), Natasha Kerensikova wrote:
>>
>>> However I still read
>>> character by character
>>
>> You have to, because the definition of line end is language/OS/encoding
>> dependent, so in order to detect line ends properly you need to scan
>> characters one by one, maybe recoding them into the encoding used by the
>> parser (e.g. UTF-8). It does not make much sense to read input by arbitrary
>> size chunks. Read it line by line. If parser needs returns over the line
>> margin (unlikely), then keep read lines cached.
> 
> The line end detection problem is exactly why I wanted unprocessed input
> bytes.

There is no such thing as unprocessed input.

>>>  into a temporary buffer,
>>
>> Read it into the destination buffer.
> 
> Well the destination buffer for the processed text is a very inefficient
> place to store input text, because the processing involves a lot of
> insertions.

A temporary buffer makes it only slower.

> Moreover, because of the forward reference issue I detailed in another
> post, I cannot see how I can escape the schema:
> input stream --> temporary buffer --> output stream/buffer/storage

Input stream ---recoding---> Line buffer

(The line buffer could be bounded from above as Ludovic suggested.)

>> Don't use Unbounded_String; that is a
>> bad idea in almost all cases, this one included.
> 
> Would you explain why?

Because they are inefficient and do not have array view (lack indexing,
slicing, constraining).

> Unless there is a way to predict the left of the input, I need some
> text container able to grow as much as needed while reading.

String is such a container.

> But I guess you weren't calling using Unbounded_String a
> bad idea only because of performance, were you?

Yes, missing array view is the problem. In all cases you know the size in
advance or when the size incrementally grows use String allocated in the
heap. Unbounded_Strings can be used, for example, as members of
non-controlled records.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: Reading the while standard input into a String

[Natasha Kerensikova]

Hello,

On 2011-06-06, Ludovic Brenta <ludovic@ludovic-brenta.org> wrote:
> Ludovic Brenta wrote on comp.lang.ada:
>> Natasha Kerensikova wrote on comp.lang.ada:
>>> However I thought that something similar to C's fread(), which can use
>>> efficient bulk memory copy, would be better than a loop of single
>>> character read (even when buffered in memory).
>>
>> How about:
>>
>> procedure Read_In_Chunks (Stream : in out
>> Ada.Streams.Root_Stream_Type'Class;
>>                           Result : out
>> Ada.Strings.Unbounded.Unbounded_String) is
> [...]
>
> I think I'm guilty of premature optimization.  I'd be curious to know
> whether Read_In_Chunks is actually faster than simply reading one
> character at a time and appending it to Result.  Only a benchmark
> could tell for sure.

The problem with benchmarks is that they are highly dependant on platform
and situation.

For what it's worth, I tried on my development system (Ubuntu Lucid),
using 1 KiB chunk size to read 10 MiB on random data, alternating calls
to each test program, 10 times each discarding the result (to warm up
caches) and then 10 times each gathering the results.

I got 100-160 ms spent in system mode for both, 90-110 ms in user mode
for your reader, and 440-490 ms in user mode for mine.

Using on-the-fly concatenation of 10 times the original 10 MiB file,
piped into the test programs, I get roughly the same ranges times ten.

So it looks like your optimization was justiified after all.



Thanks a lot for the idea,
Natasha

Re: Reading the while standard input into a String

[Dmitry A. Kazakov]

On Mon, 06 Jun 2011 13:16:03 +0200, Georg Bauhaus wrote:

> Very likely, the storage elements can be converted to Character
> without check, i.e. using an instance of Unchecked_Conversion,
> or simple the 'Address attribute.  So no copying is involved
> at this stage.

Well, if you *know* that you can read directly into string. Declare fread
with Address argument and pass the first element's address to it.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: Reading the while standard input into a String

[Georg Bauhaus]

On 06.06.11 14:11, Dmitry A. Kazakov wrote:
> On Mon, 06 Jun 2011 13:16:03 +0200, Georg Bauhaus wrote:
> 
>> Very likely, the storage elements can be converted to Character
>> without check, i.e. using an instance of Unchecked_Conversion,
>> or simple the 'Address attribute.  So no copying is involved
>> at this stage.
> 
> Well, if you *know* that you can read directly into string. Declare fread
> with Address argument and pass the first element's address to it.

In Ada? Why? :-)

Re: Reading the while standard input into a String

[Dmitry A. Kazakov]

On Mon, 06 Jun 2011 15:32:31 +0200, Georg Bauhaus wrote:

> On 06.06.11 14:11, Dmitry A. Kazakov wrote:
>> On Mon, 06 Jun 2011 13:16:03 +0200, Georg Bauhaus wrote:
>> 
>>> Very likely, the storage elements can be converted to Character
>>> without check, i.e. using an instance of Unchecked_Conversion,
>>> or simple the 'Address attribute.  So no copying is involved
>>> at this stage.
>> 
>> Well, if you *know* that you can read directly into string. Declare fread
>> with Address argument and pass the first element's address to it.
> 
> In Ada? Why? :-)

Because that would be far easier to read and write:

   Input : String (1..1024);
   Count : size_t;
begin
   Count := fread (Input'Address, 1, Input'Length, File);

Or are you asking why anybody should use fread?

BTW, GtkAda binding consistently uses Address instead of storage elements
array. So it was not an idea of mine. (:-))

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: Reading the while standard input into a String

[Georg Bauhaus]

On 06.06.11 16:06, Dmitry A. Kazakov wrote:

> Or are you asking why anybody should use fread?

Yes, that's the point.
Ada has its own stream reading subprograms (that will
read into an array of "bytes").

Re: Reading the while standard input into a String

[Maciej Sobczak]

On Jun 6, 10:33 am, Ludovic Brenta <ludo...@ludovic-brenta.org> wrote:

> Personally I think this is a fundamentally bad idea.  Consider:
>
> $ my_program < /dev/random

If the program has the concept of reading its input, then apparently
it is supposed to run in a bigger context (a system) which we know
nothing about. If that system's specification puts upper bounds on
what is fed on each component's input, then yes, this is a perfectly
good idea and your example just does not represent any intended use
case.
What I'm up to is that reliability guarantees might come from
different sources and scopes and your "problem" might have been
already solved. This is not necessarily just another version of cat or
grep.

> Do you consider it acceptable to use all memory

I find it acceptable to read all input before processing it, if I know
the other programs that will produce the input data for this program.

> I'd rather process the input one fixed-length chunk at a time

This is a can of warms - how big should be that chunk? :-)

> Note that reading one character at a time from a stream is quite fast
> since streams normally use the operating system's buffered IO. Adding
> a second layer of buffering inside your program is probably counter-
> productive.

Or not. System calls are relatively expensive - typically much more
expensive than copying data around. For this reason wisely used
additional buffers can improve performance of I/O operations.

--
Maciej Sobczak * http://www.msobczak.com * http://www.inspirel.com

Re: Reading the while standard input into a String

[Jeffrey Carter]

On 06/06/2011 05:05 AM, Dmitry A. Kazakov wrote:
> On Mon, 6 Jun 2011 10:46:20 +0000 (UTC), Natasha Kerensikova wrote:
>
>> On 2011-06-06, Dmitry A. Kazakov<mailbox@dmitry-kazakov.de>  wrote:
>
>>> Don't use Unbounded_String; that is a
>>> bad idea in almost all cases, this one included.
>>
>> Would you explain why?
>
> Because they are inefficient ...

"Efficiency" is a function of timing requirements. GNAT's Unbounded_String is so 
inefficient that we have a soft real-time application that makes extensive use 
of Unbounded_String and has no problem meeting its timing requirements.

Unbounded_String is perfectly reasonable for many situations; worrying about 
"efficiency" before encountering timing constraints is premature optimization.

-- 
Jeff Carter
"I fart in your general direction."
Monty Python & the Holy Grail
05

Re: Reading the while standard input into a String

[Pascal Obry]

Jeffrey,

> "Efficiency" is a function of timing requirements. GNAT's
> Unbounded_String is so inefficient that we have a soft real-time
> application that makes extensive use of Unbounded_String and has no
> problem meeting its timing requirements.

And will be even more efficient with the rewrite of the controlled 
object support in recent version of GNAT.

Pascal.

-- 

--|------------------------------------------------------
--| Pascal Obry                           Team-Ada Member
--| 45, rue Gabriel Peri - 78114 Magny Les Hameaux FRANCE
--|------------------------------------------------------
--|    http://www.obry.net  -  http://v2p.fr.eu.org
--| "The best way to travel is by means of imagination"
--|
--| gpg --keyserver keys.gnupg.net --recv-key F949BD3B

Re: Reading the while standard input into a String

[Pascal Obry]

Jeffrey,

> "Efficiency" is a function of timing requirements. GNAT's
> Unbounded_String is so inefficient that we have a soft real-time
> application that makes extensive use of Unbounded_String and has no
> problem meeting its timing requirements.

And will be even more efficient with the rewrite of the controlled 
object support in recent version of GNAT.

Pascal.

-- 

--|------------------------------------------------------
--| Pascal Obry                           Team-Ada Member
--| 45, rue Gabriel Peri - 78114 Magny Les Hameaux FRANCE
--|------------------------------------------------------
--|    http://www.obry.net  -  http://v2p.fr.eu.org
--| "The best way to travel is by means of imagination"
--|
--| gpg --keyserver keys.gnupg.net --recv-key F949BD3B

Re: Reading the while standard input into a String

[John B. Matthews]

In article <slrniunb6n.i18.lithiumcat@sigil.instinctive.eu>,
 Natasha Kerensikova <lithiumcat@gmail.com> wrote:

> I have encountered what was a very easily solved problem in C: dump 
> the whole contents of the standard input into some place in memory, 
> in order to process it afterwards.
> 
> I was quite unhappy with the preprocessing performed by Ada.Text_IO, 
> so I went on using the text stream interface. However I still read 
> character by character into a temporary buffer, and it feels ugly.

I usually just loop on Character'Read, as shown here:

<https://sites.google.com/site/drjohnbmatthews/hexdump>

But I keep a link to this article around for reference:

Gem #39: Efficient Stream I/O for Array Types

<http://www.adacore.com/2008/06/09/gem-39/>

[...]

-- 
John B. Matthews
trashgod at gmail dot com
<http://sites.google.com/site/drjohnbmatthews>

Re: Reading the while standard input into a String

[Dmitry A. Kazakov]

On Mon, 6 Jun 2011 08:18:44 -0700 (PDT), Maciej Sobczak wrote:

> On Jun 6, 10:33�am, Ludovic Brenta <ludo...@ludovic-brenta.org> wrote:
> 
>> I'd rather process the input one fixed-length chunk at a time
> 
> This is a can of warms - how big should be that chunk? :-)

For a compiler it is the maximal length of look ahead / roll-back,
normally, less than just one line. The line length can be reasonably
limited to say 10_000 characters (usually the language defines the maximal
line length).

>> Note that reading one character at a time from a stream is quite fast
>> since streams normally use the operating system's buffered IO. Adding
>> a second layer of buffering inside your program is probably counter-
>> productive.
> 
> Or not. System calls are relatively expensive - typically much more
> expensive than copying data around. For this reason wisely used
> additional buffers can improve performance of I/O operations.

The opposite is also possible, unless the OS driver uses user-space
buffers. So I would follow Ludovic's advise, especially in the case of a
compiler where reading overhead is negligible comparing to other
computations per character.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: Reading the while standard input into a String

[Dmitry A. Kazakov]

On Mon, 06 Jun 2011 09:55:39 -0700, Jeffrey Carter wrote:

> Unbounded_String is perfectly reasonable for many situations; worrying about 
> "efficiency" before encountering timing constraints is premature optimization.

Only if Unbounded_String had array interface. Without that Unbounded_String
introduces additional complexity to the design.

I see no reason to use Unbounded_String for an input buffer, texts which
never change etc.

There exist cases for Unbounded_String, but not that many.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: Reading the while standard input into a String

[Maciej Sobczak]

On Jun 6, 8:18 pm, "Dmitry A. Kazakov" <mail...@dmitry-kazakov.de>
wrote:

> > This is a can of warms - how big should be that chunk? :-)
>
> For a compiler it is the maximal length of look ahead / roll-back,
> normally, less than just one line.

This is wrong even for Ada. The compiler cannot drop what it read so
far (except maybe for comments) - it has to retain this information,
even if not literally in its original form. This means that "dropping"
data is not possible and in fact the internal data structures will
grow while the file is being processed - but then the compiler might
as well just read the whole source file into memory and I bet this
will be much smaller that the data structures that are built from it.

BTW - what about generating debug version of the executable with
source embedded? Oops. You'd better keep it around, it's cheap when
compared to anything else.

> > Or not. System calls are relatively expensive - typically much more
> > expensive than copying data around. For this reason wisely used
> > additional buffers can improve performance of I/O operations.
>
> The opposite is also possible, unless the OS driver uses user-space
> buffers. So I would follow Ludovic's advise, especially in the case of a
> compiler where reading overhead is negligible comparing to other
> computations per character.

There is no point in following this advice, because compiler is not
cat or grep. In particular, the compiler does not work on potentially
infinite streams (unlike cat and grep) and there is a clear concept of
a program structure that has its end. This means that reading the
whole source file into memory is a perfectly valid approach.

--
Maciej Sobczak * http://www.msobczak.com * http://www.inspirel.com

Re: Reading the while standard input into a String

[Dmitry A. Kazakov]

On Mon, 6 Jun 2011 11:36:00 -0700 (PDT), Maciej Sobczak wrote:

> On Jun 6, 8:18�pm, "Dmitry A. Kazakov" <mail...@dmitry-kazakov.de>
> wrote:
> 
>>> This is a can of warms - how big should be that chunk? :-)
>>
>> For a compiler it is the maximal length of look ahead / roll-back,
>> normally, less than just one line.
> 
> This is wrong even for Ada.

Of course it is possible for Ada, of which syntax requires neither look
ahead or roll-back.

> BTW - what about generating debug version of the executable with
> source embedded?

What for? All you need is source location tags.

>>> Or not. System calls are relatively expensive - typically much more
>>> expensive than copying data around. For this reason wisely used
>>> additional buffers can improve performance of I/O operations.
>>
>> The opposite is also possible, unless the OS driver uses user-space
>> buffers. So I would follow Ludovic's advise, especially in the case of a
>> compiler where reading overhead is negligible comparing to other
>> computations per character.
> 
> There is no point in following this advice, because compiler is not
> cat or grep.

That is another argument, first we agree that performance is not an issue
here.

> In particular, the compiler does not work on potentially
> infinite streams (unlike cat and grep) and there is a clear concept of
> a program structure that has its end. This means that reading the
> whole source file into memory is a perfectly valid approach.

Wrong. A good example is GNAT compiler which runs out of Windows' 1-2GB
limit when compiling my generics. Macros tend to explode (:-))

Memory is cheap, but that does not mean you should waste it. You just do
not need to keep anything but the current source line.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: Reading the while standard input into a String

[J-P. Rosen]

Le 06/06/2011 20:18, Dmitry A. Kazakov a �crit :
> The opposite is also possible, unless the OS driver uses user-space
> buffers. So I would follow Ludovic's advise, especially in the case of a
> compiler where reading overhead is negligible comparing to other
> computations per character.
> 
NEVER assess anything about efficiency without measurements!

I once doubled the speed of a compiler by rewriting the main character
reading procedure in assembly. A compiler spends more time than you
would think simply skipping spaces...
(Yes, I did measure that - TBH, it was quite a long time ago, on a
computer that was very different from what we have today).
-- 
---------------------------------------------------------
           J-P. Rosen (rosen@adalog.fr)
Adalog a d�m�nag� / Adalog has moved:
2 rue du Docteur Lombard, 92441 Issy-les-Moulineaux CEDEX
Tel: +33 1 45 29 21 52, Fax: +33 1 45 29 25 00

Re: Reading the while standard input into a String

[J-P. Rosen]

Le 06/06/2011 20:53, Dmitry A. Kazakov a �crit :
> Of course it is possible for Ada, of which syntax requires neither look
> ahead or roll-back.
> 
Strictly speaking, it does need some look ahead for oddities like
character'(''')
(and of course, you can have as many comment lines as you want between
the various ticks!)
-- 
---------------------------------------------------------
           J-P. Rosen (rosen@adalog.fr)
Adalog a d�m�nag� / Adalog has moved:
2 rue du Docteur Lombard, 92441 Issy-les-Moulineaux CEDEX
Tel: +33 1 45 29 21 52, Fax: +33 1 45 29 25 00

Re: Reading the while standard input into a String

[Dmitry A. Kazakov]

On Mon, 06 Jun 2011 21:10:11 +0200, J-P. Rosen wrote:

> Le 06/06/2011 20:53, Dmitry A. Kazakov a �crit :
>> Of course it is possible for Ada, of which syntax requires neither look
>> ahead or roll-back.
>> 
> Strictly speaking, it does need some look ahead for oddities like
> character'(''')

Not really if the scanner is aware of the context. A character literal
(operand) cannot appear in the context of an infix operation and thus need
not to be tried after an identifier. On its part an attribute's apostrophe
cannot appear in the prefix context (e.g. after the bracket, where an
operand or prefix operation is expected). I don't use look-ahead in my
parser.

Ada's grammar is wonderfully simple for a table driven parser. It only
looks awfully complex when written in BNF.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: Reading the while standard input into a String

[Dmitry A. Kazakov]

On Mon, 06 Jun 2011 21:06:12 +0200, J-P. Rosen wrote:

> Le 06/06/2011 20:18, Dmitry A. Kazakov a �crit :
>> The opposite is also possible, unless the OS driver uses user-space
>> buffers. So I would follow Ludovic's advise, especially in the case of a
>> compiler where reading overhead is negligible comparing to other
>> computations per character.
>> 
> NEVER assess anything about efficiency without measurements!

I definitely agree, that is why I side with Ludovic. I would first
implement his schema and then measure if the performance is not
satisfactory, before trying kludges which effect on performance is unknown.

> I once doubled the speed of a compiler by rewriting the main character
> reading procedure in assembly. A compiler spends more time than you
> would think simply skipping spaces...

OK, but buffering won't help with spaces anyway, there is no OS call to
skip spaces...

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: Reading the while standard input into a String

[Robert A Duff]

Ludovic Brenta <ludovic@ludovic-brenta.org> writes:

> Personally I think this is a fundamentally bad idea.  Consider:
>
> $ my_program < /dev/random

What do you expect

    sort /dev/random

to do?  Or

    tail /dev/random

?

;-) ;-)

> Note that reading one character at a time from a stream is quite fast
> since streams normally use the operating system's buffered IO. Adding
> a second layer of buffering inside your program is probably counter-
> productive.

Yes, probably.

- Bob

Re: Reading the while standard input into a String

[Shark8]

> Of course it is possible for Ada, of which syntax requires neither look
> ahead or roll-back.

How are Strings handled? In particular, embedding the double-quote.
How does one parse "Bill said, ""I don't care.""" w/o a look-ahead (or
roll back)?
(Especially if, as seems to be the case here, he wants to do such via
streams-of-some-sort.)

Re: Reading the while standard input into a String

[Randy Brukardt]

"Shark8" <onewingedshark@gmail.com> wrote in message 
news:b6e5b2c3-4b9c-4e47-96d0-ad9add77f7fc@d28g2000yqf.googlegroups.com...
>> Of course it is possible for Ada, of which syntax requires neither look
>> ahead or roll-back.
>
> How are Strings handled? In particular, embedding the double-quote.
> How does one parse "Bill said, ""I don't care.""" w/o a look-ahead (or
> roll back)?
> (Especially if, as seems to be the case here, he wants to do such via
> streams-of-some-sort.)

Ada source text requires one CHARACTER of lookahead (assuming a bit of 
context is saved). That doesn't not require a lot of buffering. ;-) 
(Ada.Text_IO.Get_Line requires more lookahead than Ada syntax does. Go 
figure.)

                    Randy.

Re: Reading the while standard input into a String

[Randy Brukardt]

"Maciej Sobczak" <see.my.homepage@gmail.com> wrote in message 
news:a5b07c48-756b-41fd-bd85-c95016702a73@hv8g2000vbb.googlegroups.com...
On Jun 6, 8:18 pm, "Dmitry A. Kazakov" <mail...@dmitry-kazakov.de>
wrote:

>> > This is a can of warms - how big should be that chunk? :-)
>>
>> For a compiler it is the maximal length of look ahead / roll-back,
>> normally, less than just one line.
>
>This is wrong even for Ada. The compiler cannot drop what it read so
>far (except maybe for comments) - it has to retain this information,
>even if not literally in its original form. This means that "dropping"
>data is not possible and in fact the internal data structures will
>grow while the file is being processed - but then the compiler might
>as well just read the whole source file into memory and I bet this
>will be much smaller that the data structures that are built from it.

Dmitry is right. Janus/Ada uses a small text buffer for the incoming source 
(back in the CP/M days is was 128 bytes, which matched the I/O size of the 
OS - it's somewhat larger now). We don't check any source beyond that; the 
source is converted into tokens which are far smaller than the original 
source in most cases. (Tokens for numeric literals might be larger than the 
corresponding literal.) And we don't keep that in memory, either; the entire 
set of tokens is written into a parsing stream which drives other compiler 
passes. This is all heavily encoded and optimized (the design was necessary 
to get the compiler to run at all on the small CP/M and MS-DOS systems) --  
limiting the compiler to compiling programs of less than 8K of source would 
never have been a realistic option.

>BTW - what about generating debug version of the executable with
>source embedded? Oops. You'd better keep it around, it's cheap when
>compared to anything else.

Line/position information is far smaller than the source code (and is even 
smaller than a pointer to the source code), and that's needed anyway to make 
decent error messages. (Janus/Ada limits the source lines in a file to 
65000, line lengths to 250).

...
>There is no point in following this advice, because compiler is not
>cat or grep. In particular, the compiler does not work on potentially
>infinite streams (unlike cat and grep) and there is a clear concept of
>a program structure that has its end. This means that reading the
>whole source file into memory is a perfectly valid approach.

That wouldn't be a practical approach -- source files can be very large 
(especially when generated by a tool). I've worried about that limit of 
65000 lines (not that I've personally seen a file that exceeded it), as I've 
seen tools that turn very large data structures into Ada code.

                                                       Randy.

Re: Reading the while standard input into a String

[Randy Brukardt]

"J-P. Rosen" <rosen@adalog.fr> wrote in message 
news:isj8f2$485$1@dont-email.me...
...
> I once doubled the speed of a compiler by rewriting the main character
> reading procedure in assembly. A compiler spends more time than you
> would think simply skipping spaces...
> (Yes, I did measure that - TBH, it was quite a long time ago, on a
> computer that was very different from what we have today).

System calls are expensive on every machine I've ever seen. I've yet to see 
a case where it is better to let the system buffer the input rather than do 
it yourself. I ended up moving that into Ada.Text_IO after dealing with 
customers that were not willing to believe that fact.

As far as compilers go, I recall that most of the syntax pass was coded in 
assembler in the early days of Janus/Ada; and it far more than doubled the 
speed over the Ada version (it was closer to a factor of 10). We maintained 
both versions for many years. On more modern machines, with compilers that 
have much better optimization, I've been able to get rid of that assembler 
code.

                                               Randy.

Re: Reading the while standard input into a String

[Randy Brukardt]

"Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> wrote in message 
news:1qweahw1opg3q$.qlezbqnr1n6w.dlg@40tude.net...
...
> BTW, GtkAda binding consistently uses Address instead of storage elements
> array. So it was not an idea of mine. (:-))

Good. That's an Ada83ism, any Ada 95 code that uses 'Address (other than 
address clauses for hardware) is broken in my view. It is much better to use 
some version of 'Access and not lose type checking.

And in this case, use the stream operations, not bastard non-portable junk.

                            Randy.

Re: Reading the while standard input into a String

[Dmitry A. Kazakov]

On Mon, 6 Jun 2011 16:37:03 -0700 (PDT), Shark8 wrote:

>> Of course it is possible for Ada, of which syntax requires neither look
>> ahead or roll-back.
> 
> How are Strings handled? In particular, embedding the double-quote.
> How does one parse "Bill said, ""I don't care.""" w/o a look-ahead (or
> roll back)?

loop
   if Source (Index) = '"' then
      Index := Index + 1;
      exit when Source (Index) /= '"'; -- Does not belong to the literal
   end if;
   -- Process a new literal's character in Source (Index)
   Index := Index + 1;
end loop;

Yes, you always need a buffer for the current character, which some may
consider as a look-ahead, but I don't. (:-))

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: Reading the while standard input into a String

[Maciej Sobczak]

On Jun 6, 8:53 pm, "Dmitry A. Kazakov" <mail...@dmitry-kazakov.de>
wrote:

> > BTW - what about generating debug version of the executable with
> > source embedded?
>
> What for? All you need is source location tags.

Which only grow in memory. This means that the concept of dropping the
chunks that were already processed is a fiction - it just does not
happen, the data is still there in memory, just in some other form.

> > In particular, the compiler does not work on potentially
> > infinite streams (unlike cat and grep) and there is a clear concept of
> > a program structure that has its end. This means that reading the
> > whole source file into memory is a perfectly valid approach.
>
> Wrong. A good example is GNAT compiler which runs out of Windows' 1-2GB
> limit when compiling my generics. Macros tend to explode (:-))

They will explode whether you will keep the original source or not.
Let me guess - your source code is smaller than 1-2GB, right? Frankly
- how big it is in terms of percentage of the total memory used? 1%?
Dropping this amount of data (while at the same time complicating the
process with additional tags, etc.) as a means of preserving memory is
pointless. I bet there are much bigger optimization opportunities in
resulting internal data structures.

> Memory is cheap, but that does not mean you should waste it. You just do
> not need to keep anything but the current source line.

And now I would very welcome somebody from existing compiler vendors
to hop in and confess how this is *actually* done. :-)

--
Maciej Sobczak * http://www.msobczak.com * http://www.inspirel.com

Re: Reading the while standard input into a String

[Dmitry A. Kazakov]

On Tue, 7 Jun 2011 00:32:15 -0700 (PDT), Maciej Sobczak wrote:

> On Jun 6, 8:53�pm, "Dmitry A. Kazakov" <mail...@dmitry-kazakov.de>
> wrote:
> 
>>> BTW - what about generating debug version of the executable with
>>> source embedded?
>>
>> What for? All you need is source location tags.
> 
> Which only grow in memory.

Not as fast as the source does.

> the data is still there in memory, just in some other form.

That is why compilers are also called "translator."

>>> In particular, the compiler does not work on potentially
>>> infinite streams (unlike cat and grep) and there is a clear concept of
>>> a program structure that has its end. This means that reading the
>>> whole source file into memory is a perfectly valid approach.
>>
>> Wrong. A good example is GNAT compiler which runs out of Windows' 1-2GB
>> limit when compiling my generics. Macros tend to explode (:-))
> 
> They will explode whether you will keep the original source or not.

Nope. Ada was designed for separate compilation.

> Let me guess - your source code is smaller than 1-2GB, right? Frankly
> - how big it is in terms of percentage of the total memory used? 1%?

I suppose that is not the case for GNAT, but considering what you proposed
together with inlining, compilation of generic instantiations would require
keeping expanded sources. I am using generics to emulate MI and delegation.
I.e. there are long paths of generics using other generics (as parameters
or parents), which in turn use other generics etc. The whole "inheritance"
tree must be instantiated and so expanded.

> I bet there are much bigger optimization opportunities in
> resulting internal data structures.

That does not justify keeping more than one source line in the memory.
 
-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: Reading the while standard input into a String

[Martin]

On Jun 6, 7:14 pm, "John B. Matthews" <nos...@nospam.invalid> wrote:
[snip]
> Gem #39: Efficient Stream I/O for Array Types
>
> <http://www.adacore.com/2008/06/09/gem-39/>


I wish someone (Bob? Randy?) would answer the question I posted on
that!! :-)

-- Martin

Re: Reading the while standard input into a String

[John B. Matthews]

In article 
<44ee8f17-40fb-4a17-aaad-38d93f36cbaf@gh5g2000vbb.googlegroups.com>,
 Martin <martin.dowie@btopenworld.com> wrote:

> On Jun 6, 7:14 pm, "John B. Matthews" <nos...@nospam.invalid> wrote:
> [snip]
> > Gem #39: Efficient Stream I/O for Array Types
> >
> > <http://www.adacore.com/2008/06/09/gem-39/>
> 
> 
> I wish someone (Bob? Randy?) would answer the question I posted on 
> that!! :-)

I see that B.2(8) says "at least the size of a storage element," while 
GNAT uses the 'Size clause you mention in Interfaces (interfac.ads).

<http://www.adaic.org/resources/add_content/standards/05rm/html/RM-B-2.html>

Not that I know the answer; I'm just a language citizen. :-)

-- 
John B. Matthews
trashgod at gmail dot com
<http://sites.google.com/site/drjohnbmatthews>

Re: Reading the while standard input into a String

[Robert A Duff]

Martin <martin.dowie@btopenworld.com> writes:

> I wish someone (Bob? Randy?) would answer the question I posted on
> that!! :-)

What question?  I don't read this newsgroup very often,
so I probably missed it.

- Bob

Re: Reading the while standard input into a String

[John B. Matthews]

In article <wccei35lfo6.fsf@shell01.TheWorld.com>,
 Robert A Duff <bobduff@shell01.TheWorld.com> wrote:

> Martin <martin.dowie@btopenworld.com> writes:
> 
> > I wish someone (Bob? Randy?) would answer the question I posted on
> > that!! :-)
> 
> What question?  I don't read this newsgroup very often,
> so I probably missed it.

It's at the end of the article:

<http://www.adacore.com/2008/06/09/gem-39/>

Sorry, I should have said so in my followup.

-- 
John B. Matthews
trashgod at gmail dot com
<http://sites.google.com/site/drjohnbmatthews>

Re: Reading the while standard input into a String

[Robert A Duff]

"John B. Matthews" <nospam@nospam.invalid> writes:

> In article <wccei35lfo6.fsf@shell01.TheWorld.com>,
>  Robert A Duff <bobduff@shell01.TheWorld.com> wrote:
>
>> Martin <martin.dowie@btopenworld.com> writes:
>> 
>> > I wish someone (Bob? Randy?) would answer the question I posted on
>> > that!! :-)
>> 
>> What question?  I don't read this newsgroup very often,
>> so I probably missed it.
>
> It's at the end of the article:
>
> <http://www.adacore.com/2008/06/09/gem-39/>
>
> Sorry, I should have said so in my followup.

Maybe a 'Component_Size clause would be in order.

- Bob

Re: Reading the while standard input into a String

[Egil Høvik]

To avoid padding/alignment, I would add a "pragma Pack(Buffer);"

-- 
~egilhh

Re: Reading the while standard input into a String

[Martin]

On Jun 8, 8:38 am, Egil Høvik <egilho...@hotmail.com> wrote:
> To avoid padding/alignment, I would add a "pragma Pack(Buffer);"
>
> --
> ~egilhh

Pragma's can be ignored...although in this case you'd be on a fairly
unusual platform if it couldn't support this packing!

-- Martin

Re: Reading the while standard input into a String

[Egil Høvik]

actually, only unrecognized pragmas can be ignored as stated in RM 2.8(11).
I certainly wouldn't use a compiler that didn't recognize pragmas such as
Storage_Size, Pure, Elaborate. These and others are clearly defined by the 
language, and must be recognized.

The same goes for pragma Pack. A compiler that does not recognize (and ignores)
such a pragma is not an Ada compiler. If it cannot comply with the packing 
request, fine, but that's an error, and cannot be silently ignored.

(of course, the exception to this is pragma Inline, which is explicitly 
permitted to be ignored RM 6.3.2(6))


-- 
~egilhh

Re: Reading the while standard input into a String

[Niklas Holsti]

Egil H�vik wrote:
> actually, only unrecognized pragmas can be ignored as stated in RM 2.8(11).
> I certainly wouldn't use a compiler that didn't recognize pragmas such as
> Storage_Size, Pure, Elaborate. These and others are clearly defined by the 
> language, and must be recognized.
> 
> The same goes for pragma Pack. A compiler that does not recognize (and ignores)
> such a pragma is not an Ada compiler. If it cannot comply with the packing 
> request, fine, but that's an error, and cannot be silently ignored.

The compiler can ignore pragma Pack in the sense that the compiler can 
allocate memory for the data type in the same was as if there were no 
pragma Pack. That is, the pragma does not have to result in a smaller 
memory lay-out. The pragma only asks the compiler to *try* to minimize 
the size, but the compiler does not have to try very hard... or at all.

If you want to be sure that pragma Pack has some (desired) effect, you 
should define the 'Size attribute of the type. If the compiler cannot 
pack the type into the given size, it must report a compile-time error.

(Egil, please quote the relevant parts of the post to which you are 
replying! In the common newsreaders the context of your reply is lost if 
you don't do that.)

-- 
Niklas Holsti
Tidorum Ltd
niklas holsti tidorum fi
       .      @       .

Re: Reading the while standard input into a String

[Randy Brukardt]

"Niklas Holsti" <niklas.holsti@tidorum.invalid> wrote in message 
news:959s61F40sU1@mid.individual.net...
...
> The compiler can ignore pragma Pack in the sense that the compiler can 
> allocate memory for the data type in the same was as if there were no 
> pragma Pack. That is, the pragma does not have to result in a smaller 
> memory lay-out. The pragma only asks the compiler to *try* to minimize the 
> size, but the compiler does not have to try very hard... or at all.
>
> If you want to be sure that pragma Pack has some (desired) effect, you 
> should define the 'Size attribute of the type. If the compiler cannot pack 
> the type into the given size, it must report a compile-time error.

In this particular case, where a specific representation of an array is 
desired, you have to specify 'Component_Size on the array type. Pack doesn't 
necessarily specify a particular representation, and even when it does, it 
is much better to explicitly state that you want 16 bit components (rather 
than just "smaller" components).

In the original example, the type should have been declared:

    type Buffer is array (1..32) of Interfaces.Unsigned_16
        with Component_Size => 16;

presuming you have an Ada 2012 compiler. If not, fall back on those ugly 
attribute definition clauses:

    type Buffer is array (1..32) of Interfaces.Unsigned_16;
    for Buffer'Component_Size use 16;

(Ugly because you have to repeat the type name and because they can get 
separated from the type.)

                         Randy.

Re: Reading the while standard input into a String

[Martin]

On Jun 9, 1:46 am, "Randy Brukardt" <ra...@rrsoftware.com> wrote:
> "Niklas Holsti" <niklas.hol...@tidorum.invalid> wrote in message
>
> news:959s61F40sU1@mid.individual.net...
> ...
>
> > The compiler can ignore pragma Pack in the sense that the compiler can
> > allocate memory for the data type in the same was as if there were no
> > pragma Pack. That is, the pragma does not have to result in a smaller
> > memory lay-out. The pragma only asks the compiler to *try* to minimize the
> > size, but the compiler does not have to try very hard... or at all.
>
> > If you want to be sure that pragma Pack has some (desired) effect, you
> > should define the 'Size attribute of the type. If the compiler cannot pack
> > the type into the given size, it must report a compile-time error.
>
> In this particular case, where a specific representation of an array is
> desired, you have to specify 'Component_Size on the array type. Pack doesn't
> necessarily specify a particular representation, and even when it does, it
> is much better to explicitly state that you want 16 bit components (rather
> than just "smaller" components).
>
> In the original example, the type should have been declared:
>
>     type Buffer is array (1..32) of Interfaces.Unsigned_16
>         with Component_Size => 16;
>
> presuming you have an Ada 2012 compiler. If not, fall back on those ugly
> attribute definition clauses:
>
>     type Buffer is array (1..32) of Interfaces.Unsigned_16;
>     for Buffer'Component_Size use 16;
>
> (Ugly because you have to repeat the type name and because they can get
> separated from the type.)
>
>                          Randy.

Randy / Bob,

Thanks for these answers...looking forward to Ada2012 even more
now! :-)

-- Martin