GNAT Ada.Streams Bug?

GNAT Ada.Streams Bug?

[Warren W. Gay VE3WWG]

This is on gnat-3.14p on debian (debian's package library doesn't
seem to provide 3.15p), so this may be fixed by now:

I am running into a Ada.Streams problem with 24 bit values (3 bytes).
For example, if you use Ada.Streams to read in a partition table,
defined as follows, with 3.14p, you'll read more bytes than needed
(thus messing things up):

    type chs_t is mod 2**24;               -- Cylinder/Head/Sector
    for chs_t'size use 24;

    -- All other types below are 8, 16 or 32 bits wide

    type Partition_Type is
       record
          Flags :        part_flags_t;     -- Partition flags
          CHS_Start :    chs_t;            -- Cyl/Head/Sector Start
          Kind :         part_kind_t;      -- Partition Type
          CHS_End :      chs_t;            -- Cyl/Head/Sector End
          LBA_Start :    part_offset_t;    -- LBA Offset
          LBA_Size :     part_size_t;      -- LBA Size
       end record;

    for Partition_Type use
       record
          Flags          at 0 range 0..7;
          CHS_Start      at 1 range 0..23;
          Kind           at 4 range 0..7;
          CHS_End        at 5 range 0..23;
          LBA_Start      at 8 range 0..31;
          LBA_Size       at 12 range 0..31;
       end record;

    type Partition_Array is array(Natural range 1..4)
       of Partition_Type;

When Partition_Type'Read(Stream,Part_Table) didn't work correctly, I
put debugging statements into the 'Read routine, and
saw to my horror that the CHS_Start and CHS_End values are read in
as 4 byte values, instead of 3! So when chs_t'Read is called,
it wants STREAM_ELEMENT_ARRAY(1..4) filled, instead of the expected
1..3 slice.

Questions for the Group:

1. Can I assume this is a bug? (I am not a language lawyer)

2. Does anyone know if this has been fixed in gnat-3.15p or later?

It seems that if I stick to powers of 2 for data element sizes, things
work as expected. The work around is to divide up the chs_t type.

-- 
Warren W. Gay VE3WWG
http://home.cogeco.ca/~ve3wwg

Re: GNAT Ada.Streams Bug?

[Jeff C r e e.m]

"Warren W. Gay VE3WWG" <ve3wwg@NoSPAM.cogeco.ca> wrote in message 
news:fOopd.59177$Le1.1221311@news20.bellglobal.com...
> This is on gnat-3.14p on debian (debian's package library doesn't
> seem to provide 3.15p), so this may be fixed by now:
>
> I am running into a Ada.Streams problem with 24 bit values (3 bytes).
> For example, if you use Ada.Streams to read in a partition table,
> defined as follows, with 3.14p, you'll read more bytes than needed
> (thus messing things up):
>

These might be helpful

http://groups.google.com/groups?hl=en&lr=&threadm=t3f6rg54jlkfee%40corp.supernews.com&rnum=2&prev=/groups%3Fq%3Dstreams%2Bcreem%2Bsize%26hl%3Den%26lr%3D%26selm%3Dt3f6rg54jlkfee%2540corp.supernews.com%26rnum%3D2

http://groups.google.com/groups?q=streams+creem+size&hl=en&lr=&selm=t739enmlcvuo2d%40corp.supernews.com&rnum=1

http://www.ada-auth.org/cgi-bin/cvsweb.cgi/AIs/AI-00270.TXT?rev=1.6


In any case, you can "work around" this and get more efficient code to boot 
by writing your own 'read and 'write for  your record to read the whole
"hunk" of data in at once (based on the 'size of the record in terms of 
stream elements).

Re: GNAT Ada.Streams Bug?

[Warren W. Gay VE3WWG]

Jeff C r e e.m wrote:
> "Warren W. Gay VE3WWG" <ve3wwg@NoSPAM.cogeco.ca> wrote in message 
> news:fOopd.59177$Le1.1221311@news20.bellglobal.com...
> 
>>This is on gnat-3.14p on debian (debian's package library doesn't
>>seem to provide 3.15p), so this may be fixed by now:
>>
>>I am running into a Ada.Streams problem with 24 bit values (3 bytes).
>>For example, if you use Ada.Streams to read in a partition table,
>>defined as follows, with 3.14p, you'll read more bytes than needed
>>(thus messing things up):
> 
> These might be helpful
> 
> http://groups.google.com/groups?hl=en&lr=&threadm=t3f6rg54jlkfee%40corp.supernews.com&rnum=2&prev=/groups%3Fq%3Dstreams%2Bcreem%2Bsize%26hl%3Den%26lr%3D%26selm%3Dt3f6rg54jlkfee%2540corp.supernews.com%26rnum%3D2
> 
> http://groups.google.com/groups?q=streams+creem+size&hl=en&lr=&selm=t739enmlcvuo2d%40corp.supernews.com&rnum=1
> 
> http://www.ada-auth.org/cgi-bin/cvsweb.cgi/AIs/AI-00270.TXT?rev=1.6

Interesting. So the bug is in the RM ;-)

> In any case, you can "work around" this and get more efficient code to boot 
> by writing your own 'read and 'write for  your record to read the whole
> "hunk" of data in at once (based on the 'size of the record in terms of 
> stream elements).

I am not concerned at all about the efficiency of this case, as you
read the partition 4 times at boot time (less if you have fewer disks).
I know however, that you were stating the more general case.

It is disappointing to learn that basically Ada95 is "portability
busted" when it comes to Ada.Streams. As Randy (I think) mentioned
in the AI link, it is very difficult to adapt a stream to external
formats - more impossible to be compiler independent at the same
time. Since I am trying to interface a stream to the boot sector,
this becomes a real problem that I'll have to "work around".

This is real disappointing for someone doing O/S work.
C code bypasses any difficulty here, because you just
give read(2) a pointer (essentially Var'Address) and tell it
how many bytes to deposit there. Type is not checked, lengths
not checked etc. Dangerous, and of course you could resort
to this same technique in Ada as well.

In Ada95 however, you don't want a custom Read routine for
every O/S type you create (else you have to resign
yourself to a small set of unsigned types and dispense
with one of advantages of strong typing). You also don't
want to do "conversions" all over the place. Finally,
generic adapters would be too many and Unchecked_Conversion
is not efficient or pleasant. So what to do?

STREAMS lets you do your I/O in a safe, but strongly typed
way. It seems like the right thing to do. It seems that
for GNAT and Intel, you can work around it if you are
careful. For some platforms however, where the base
types for all types are 32 bits, this sounds impossible.

Based upon the links you provided, THERE IS HOPE in
Ada200Y for Streams. I welcome those changes.

The point about efficiency is also a good one to raise. For
example, if you String'Read(Stream,Var), GNAT calls the
'Read routine for each and every Character of the string.
One pines for something more efficient than this, and I
hope for the day when this improves in GNAT. (hint)

Thanks for the links.

wwg.
--
Warren W. Gay VE3WWG
http://home.cogeco.ca/~ve3wwg

Re: GNAT Ada.Streams Bug?

[Georg Bauhaus]

Warren W. Gay VE3WWG <ve3wwg@nospam.cogeco.ca> wrote:
: The point about efficiency is also a good one to raise. For
: example, if you String'Read(Stream,Var), GNAT calls the
: 'Read routine for each and every Character of the string.
: One pines for something more efficient than this, and I
: hope for the day when this improves in GNAT. (hint)

While not a stream of elements, perhaps Fast_Read in
the XML/Ada sources is of interest?


-- Georg

Re: GNAT Ada.Streams Bug?

[Warren W. Gay VE3WWG]

Georg Bauhaus wrote:
> Warren W. Gay VE3WWG <ve3wwg@nospam.cogeco.ca> wrote:
> : The point about efficiency is also a good one to raise. For
> : example, if you String'Read(Stream,Var), GNAT calls the
> : 'Read routine for each and every Character of the string.
> : One pines for something more efficient than this, and I
> : hope for the day when this improves in GNAT. (hint)
> 
> While not a stream of elements, perhaps Fast_Read in
> the XML/Ada sources is of interest?

All kinds of possibilities are available when _you_ control
the format. But when dealing with things like partition
tables, and existing file formats, I find it disappointing
that Ada95 struggles with this. I am happy to see that this
problem is recognized, and will in some way be addressed in
the Ada200Y standard.
-- 
Warren W. Gay VE3WWG
http://home.cogeco.ca/~ve3wwg

Re: GNAT Ada.Streams Bug?

[Simon Wright]

"Warren W. Gay VE3WWG" <ve3wwg@NoSpam.cogeco.ca> writes:

> I am not concerned at all about the efficiency of this case, as you
> read the partition 4 times at boot time (less if you have fewer
> disks).  I know however, that you were stating the more general
> case.
> 
> It is disappointing to learn that basically Ada95 is "portability
> busted" when it comes to Ada.Streams. As Randy (I think) mentioned
> in the AI link, it is very difficult to adapt a stream to external
> formats - more impossible to be compiler independent at the same
> time. Since I am trying to interface a stream to the boot sector,
> this becomes a real problem that I'll have to "work around".

So the AI lets you make the streaming portable provided that
everything is a multiple of Stream_Element'Size (and, I guess, all the
participants are of the same endianness).

If those conditions are met, you have what looks like a more
convenient way of expressing representations. Otherwise, it'll have to
be the hard way .. I must say it wouldn't have occurred to me to
expect a Stream to meet your requirements. But we progress by meeting
needs, and someone has to express them!

-- 
Simon Wright                               100% Ada, no bugs.

Re: GNAT Ada.Streams Bug?

[Martin Krischik]

Warren W. Gay VE3WWG wrote:

> Georg Bauhaus wrote:
>> Warren W. Gay VE3WWG <ve3wwg@nospam.cogeco.ca> wrote:
>> : The point about efficiency is also a good one to raise. For
>> : example, if you String'Read(Stream,Var), GNAT calls the
>> : 'Read routine for each and every Character of the string.
>> : One pines for something more efficient than this, and I
>> : hope for the day when this improves in GNAT. (hint)
>> 
>> While not a stream of elements, perhaps Fast_Read in
>> the XML/Ada sources is of interest?
> 
> All kinds of possibilities are available when _you_ control
> the format. But when dealing with things like partition
> tables, and existing file formats, I find it disappointing
> that Ada95 struggles with this. I am happy to see that this
> problem is recognized, and will in some way be addressed in
> the Ada200Y standard.

True. When you do low level programming than you migh want to consider some
of Ada's low level features - and not a very high level IO package.

In your case: Read the data into an array of Short_Short_Integer and then
use either Unchecked_Convertion or for X'Address use Y'Address to convert
it to your structure.

You should consider one of the lower level IO package to read the data. Ada
has 4 IO packages and the most difficult part about them is to choose the
right one.

With Regards

Martin
-- 
mailto://krischik@users.sourceforge.net
http://www.ada.krischik.com

Re: GNAT Ada.Streams Bug?

[Warren W. Gay VE3WWG]

Simon Wright wrote:
> "Warren W. Gay VE3WWG" <ve3wwg@NoSpam.cogeco.ca> writes:
>>I am not concerned at all about the efficiency of this case, as you
>>read the partition 4 times at boot time (less if you have fewer
>>disks).  I know however, that you were stating the more general
>>case.
>>
>>It is disappointing to learn that basically Ada95 is "portability
>>busted" when it comes to Ada.Streams. As Randy (I think) mentioned
>>in the AI link, it is very difficult to adapt a stream to external
>>formats - more impossible to be compiler independent at the same
>>time. Since I am trying to interface a stream to the boot sector,
>>this becomes a real problem that I'll have to "work around".
> 
> So the AI lets you make the streaming portable provided that
> everything is a multiple of Stream_Element'Size (and, I guess, all the
> participants are of the same endianness).

I suppose I have always considered everything to be a multiple of
Stream_Element'Size anyway. But you are right in that you could
have types in odd bitwise sizes. My expectations were
set lower than yours ;-)

> If those conditions are met, you have what looks like a more
> convenient way of expressing representations. Otherwise, it'll have to
> be the hard way .. I must say it wouldn't have occurred to me to
> expect a Stream to meet your requirements. But we progress by meeting
> needs, and someone has to express them!

With GNAT, I simply need to stick to multiples of 2,4,8 etc.,
and then their streams I/O works fine (apart from arrays of
bytes).

I got around my problem by separating the head from sector
& cylinder (24 bits => 8 + 16). This works fine, since we're
dealing with a 1 and 2 byte values.

Warren.

Re: GNAT Ada.Streams Bug?

[Simon Wright]

"Warren W. Gay VE3WWG" <ve3wwg@NoSpam.cogeco.ca> writes:

> Simon Wright wrote:

> > So the AI lets you make the streaming portable provided that
> > everything is a multiple of Stream_Element'Size (and, I guess, all the
> > participants are of the same endianness).
> 
> I suppose I have always considered everything to be a multiple of
> Stream_Element'Size anyway. But you are right in that you could have
> types in odd bitwise sizes. My expectations were set lower than
> yours ;-)

Someone decided that all ethernet messages in our system should have a
header so he could debug at the network level, then decided to squash
all the bits in as tightly as possible so (I suppose) as to save
space. Even more fun when you need XDR versions .. not expectations so
much as bitter experience.

-- 
Simon Wright                               100% Ada, no bugs.

Re: GNAT Ada.Streams Bug?

[Warren W. Gay VE3WWG]

Martin Krischik wrote:
> Warren W. Gay VE3WWG wrote:
>>Georg Bauhaus wrote:
>>>Warren W. Gay VE3WWG <ve3wwg@nospam.cogeco.ca> wrote:
>>>: The point about efficiency is also a good one to raise. For
>>>: example, if you String'Read(Stream,Var), GNAT calls the
>>>: 'Read routine for each and every Character of the string.
>>>: One pines for something more efficient than this, and I
>>>: hope for the day when this improves in GNAT. (hint)
>>>
>>>While not a stream of elements, perhaps Fast_Read in
>>>the XML/Ada sources is of interest?
>>
>>All kinds of possibilities are available when _you_ control
>>the format. But when dealing with things like partition
>>tables, and existing file formats, I find it disappointing
>>that Ada95 struggles with this. I am happy to see that this
>>problem is recognized, and will in some way be addressed in
>>the Ada200Y standard.
> 
> True. When you do low level programming than you migh want to consider some
> of Ada's low level features - and not a very high level IO package.
> 
> In your case: Read the data into an array of Short_Short_Integer and then
> use either Unchecked_Convertion or for X'Address use Y'Address to convert
> it to your structure.

I already mentioned those. The for X'Address approach, you have
to be very sure you have the sizes right (for example, it is
easy to get sizes wrong based on words vs bytes etc.) This
approach BTW, is probably scarey stuff for safety critical
code (I'll bet it gets a lot of scrutiny!)

Unchecked_Conversion works ok for small things, but is a big
CPU waste if you have to do it a lot, or for large items. Not
anywhere near the ideal approach IMO.

> You should consider one of the lower level IO package to read the data. Ada
> has 4 IO packages and the most difficult part about them is to choose the
> right one.

Well, I don't know what other packages would be "right" for
reading in a partition table from an IDE controller. In this
example, I extended Ada.Streams.Root_Stream_Type'Class, using
code that calls Asm() code to do input/output instructions.
This felt like the most elegant way to write the code, and
it sure is more elegant than the other approaches I have
tried in Ada.

Streams works great (except possibly for overhead), with the
exception of 3 byte values. So I fixed it by splitting the
chs_t type into a separate byte sized head_t and a separate
seccyl_t type (16 bits). Then everything worked as expected.

Like I said, you _can_ do things like:

    Status : ide_status_t;
begin
    loop
       Status := ide_status_t(Input_1(Handle,Status_Port));
       ...
    end loop;
    Output_1(Handle,Cmd_Port,ide_cmd_t(IDE_IDENTIFY));

_BUT_ the ide_status_t() conversions and friends
all over the place are very tedious (try
it sometime). Ignoring the C convention type names,
it starts to be like C programming with casts all
over the place ;-)

Don't forget that in a module that does the disk
I/O you are dealing with 50+ data types, if
you subscribe to the strong type model.
That leaves genrics out, unless you want to pay the
price of all of those instantiations.

In applications like this, it seems to me that the Ada
streams is meant to be the correct approach (strong type
and convenience).  However, for some low level work,
I recognize that the for X'Address is the only
sensible thing to do. But it must be done with
great care, or you could be finding yourself looking
at memory corruption bugs.
--
Warren W. Gay VE3WWG
http://home.cogeco.ca/~ve3wwg

Re: GNAT Ada.Streams Bug?

[tmoran]

>All kinds of possibilities are available when _you_ control the format.
  Anytime you need a certain external form you should plan on writing
your own 'Read et al.  It may happen to work out that the default
routines supplied by the compiler do what you want, but that shouldn't
be your starting assumption.  Things with odd sizes likely won't work,
and composite objects will be inefficient.  If you use a record rep
clause to allow you to do IO on a multibyte object, you ought to write
your own 'Read et al for that object.  eg:

    for Partition_Type use
       record
          Flags          at 0 range 0..7;
          CHS_Start      at 1 range 0..23;
          Kind           at 4 range 0..7;
          CHS_End        at 5 range 0..23;
          LBA_Start      at 8 range 0..31;
          LBA_Size       at 12 range 0..31;
       end record;

  subtype External_Partition_Type
    is Ada.Streams.Stream_Element_Array(1 .. Partition_Type'size/8);
  function Internalize is new Ada.Unchecked_Conversion
    (Source=>External_Partition_Type, Target=>Partition_Type);
  function Externalize is new Ada.Unchecked_Conversion
    (Source=>Partition_Type, Target=>External_Partition_Type);

    procedure Partition_Type_Read
                (Stream : access Ada.Streams.Root_Stream_Type'class;
                 Item   : out Partition_Type) is
        External : External_Partition_Type;
        Last : Ada.Streams.Stream_Element_Offset;
    begin
        Read(Stream.all, External, Last);
        if Last /= External'last then
            raise ...
        end if;
        Item := Internalize(External);
    end Partition_Type_Read;

    for Partition_Type'Read use Partition_Type_Read;

IMHO

Re: GNAT Ada.Streams Bug?

[Warren W. Gay VE3WWG]

tmoran@acm.org wrote:
>>All kinds of possibilities are available when _you_ control the format.
> 
>   Anytime you need a certain external form you should plan on writing
> your own 'Read et al.  It may happen to work out that the default
> routines supplied by the compiler do what you want, but that shouldn't
> be your starting assumption.  

I believe this to be generally true, but I prefer not to have to
do this for all records. However, efficiency is one good reason
to do it (in this case, the partition table is only read at
boot time).

> Things with odd sizes likely won't work,
> and composite objects will be inefficient.  

Yes. I solved my problem by avoiding the 24 bit type, but yes,
I could have worked around it with a routine like you suggested.

> If you use a record rep
> clause to allow you to do IO on a multibyte object, you ought to write
> your own 'Read et al for that object.  eg:
> 
>     for Partition_Type use
>        record
>           Flags          at 0 range 0..7;
>           CHS_Start      at 1 range 0..23;
>           Kind           at 4 range 0..7;
>           CHS_End        at 5 range 0..23;
>           LBA_Start      at 8 range 0..31;
>           LBA_Size       at 12 range 0..31;
>        end record;
> 
>   subtype External_Partition_Type
>     is Ada.Streams.Stream_Element_Array(1 .. Partition_Type'size/8);
>   function Internalize is new Ada.Unchecked_Conversion
>     (Source=>External_Partition_Type, Target=>Partition_Type);
>   function Externalize is new Ada.Unchecked_Conversion
>     (Source=>Partition_Type, Target=>External_Partition_Type);
> 
>     procedure Partition_Type_Read
>                 (Stream : access Ada.Streams.Root_Stream_Type'class;
>                  Item   : out Partition_Type) is
>         External : External_Partition_Type;
>         Last : Ada.Streams.Stream_Element_Offset;
>     begin
>         Read(Stream.all, External, Last);
>         if Last /= External'last then
>             raise ...
>         end if;
>         Item := Internalize(External);
>     end Partition_Type_Read;
> 
>     for Partition_Type'Read use Partition_Type_Read;
> 
> IMHO

The Unchecked_Conversion is good for safety, but I hate
the inefficiency aspect of it (for partitions, as before,
this is not an issue). I would probably resort to the
for X'Address approach, and then use an assertion that
checks that the sizes agree.

Stepping back however, I still quibble that this "should"
be unnecessary, unless "efficiency was the law" in that
particular application.
-- 
Warren W. Gay VE3WWG
http://home.cogeco.ca/~ve3wwg