Controlling endian-ness?

Controlling endian-ness?

[Peter C. Chapin]

I'm trying to read/write 32 bit quantities from/to a binary file. The 
file has a format defined by a specification that is outside my control. 
Some of the quantities are stored in the file in big endian order. 
However, my machine is a little endian machine. The file in question 
also contains various values that are most naturally represented with 
different data types. Thus I'm looking at Stream_IO as a way to deal 
with it.

As an experiment I wrote a small program that defines a 32 bit unsigned 
integer type and then overrides the 'Write attribute to write the value 
in big endian form. Here is what I have:

with Ada.Streams;              use Ada.Streams;
with Ada.Streams.Stream_IO;    use Ada.Streams.Stream_IO;
with Interfaces;               use Interfaces;

procedure Stream_Check is
    pragma Assert(Stream_Element'Size = 8);

    type Word is mod 2**32;
    procedure Word_Write
      (Stream : access Root_Stream_Type'Class; Item : in Word);
    for Word'Size  use 32;
    for Word'Write use Word_Write;

    procedure Word_Write
      (Stream : access Root_Stream_Type'Class; Item : in Word) is
       Buffer    : Stream_Element_Array(0..3);
       Workspace : Unsigned_32 := Unsigned_32(Word);  -- ERROR HERE!
    begin
       Buffer(0) :=
           Stream_Element(Shift_Right((Workspace and 16#FF000000#), 24));
       Buffer(1) :=
           Stream_Element(Shift_Right((Workspace and 16#00FF0000#), 16));
       Buffer(2) :=
           Stream_Element(Shift_Right((Workspace and 16#0000FF00#),  8));
       Buffer(3) :=
           Stream_Element(Shift_Right((Workspace and 16#000000FF#),  0));
       Write(Stream.all, Buffer);
    end Word_Write;

    Output_File    : File_Type;
    Stream_Pointer : Stream_Access;

    W : Word := 16#0000FFFF#;
begin
    Create(Output_File, Out_File, "test.bin");
    Stream_Pointer := Stream(Output_File);
    Word'Write(Stream_Pointer, W);
    Close(Output_File);
end Stream_Check;

I'm using GNAT GPL 2008. It produces an error on the indicated line 
saying, "invalid use of subtype mark in expression or call." Apparently 
it doesn't like me trying to convert a Word to an Unsigned_32, but I 
don't understand why (am I doing that conversion right?).

I want to use Unsigned_32 so that I can use Shift_Right. I can't 
override the 'Write attribute for Unsigned_32 directly because it's in a 
different package (right?).

Overall I have a feeling that there is probably a much better way to do 
this.

Peter

Re: Controlling endian-ness?

[Steve]

"Peter C. Chapin" <pcc482719@gmail.com> wrote in message 
news:4897b7f5$0$19705$4d3efbfe@news.sover.net...
> I'm trying to read/write 32 bit quantities from/to a binary file. The file 
> has a format defined by a specification that is outside my control. Some 
> of the quantities are stored in the file in big endian order. However, my 
> machine is a little endian machine. The file in question also contains 
> various values that are most naturally represented with different data 
> types. Thus I'm looking at Stream_IO as a way to deal with it.
>
> As an experiment I wrote a small program that defines a 32 bit unsigned 
> integer type and then overrides the 'Write attribute to write the value in 
> big endian form. Here is what I have:
>
> with Ada.Streams;              use Ada.Streams;
> with Ada.Streams.Stream_IO;    use Ada.Streams.Stream_IO;
> with Interfaces;               use Interfaces;
>
> procedure Stream_Check is
>    pragma Assert(Stream_Element'Size = 8);
>
>    type Word is mod 2**32;
>    procedure Word_Write
>      (Stream : access Root_Stream_Type'Class; Item : in Word);
>    for Word'Size  use 32;
>    for Word'Write use Word_Write;
>
>    procedure Word_Write
>      (Stream : access Root_Stream_Type'Class; Item : in Word) is
>       Buffer    : Stream_Element_Array(0..3);
>       Workspace : Unsigned_32 := Unsigned_32(Word);  -- ERROR HERE!
>    begin
>       Buffer(0) :=
>           Stream_Element(Shift_Right((Workspace and 16#FF000000#), 24));
>       Buffer(1) :=
>           Stream_Element(Shift_Right((Workspace and 16#00FF0000#), 16));
>       Buffer(2) :=
>           Stream_Element(Shift_Right((Workspace and 16#0000FF00#),  8));
>       Buffer(3) :=
>           Stream_Element(Shift_Right((Workspace and 16#000000FF#),  0));
>       Write(Stream.all, Buffer);
>    end Word_Write;
>
>    Output_File    : File_Type;
>    Stream_Pointer : Stream_Access;
>
>    W : Word := 16#0000FFFF#;
> begin
>    Create(Output_File, Out_File, "test.bin");
>    Stream_Pointer := Stream(Output_File);
>    Word'Write(Stream_Pointer, W);
>    Close(Output_File);
> end Stream_Check;
>
> I'm using GNAT GPL 2008. It produces an error on the indicated line 
> saying, "invalid use of subtype mark in expression or call." Apparently it 
> doesn't like me trying to convert a Word to an Unsigned_32, but I don't 
> understand why (am I doing that conversion right?).
>
> I want to use Unsigned_32 so that I can use Shift_Right. I can't override 
> the 'Write attribute for Unsigned_32 directly because it's in a different 
> package (right?).
>
> Overall I have a feeling that there is probably a much better way to do 
> this.
>
> Peter

I don't know if this will be useful to you, but here is the technique I used 
for reversing the byte order on a 32 bit integer.  Actually I was going for 
"network byte order" of a float (s_float is a 32 bit float, u_long is a 
32bit unsigned integer).

      TYPE aByte IS MOD 256;
      FOR aByte'SIZE USE 8;
      TYPE aByteArray IS ARRAY( Positive RANGE <> ) OF aByte;

      FUNCTION nltohf( value : u_long ) RETURN s_float IS
         TYPE aFourBytes IS NEW aByteArray(1..4);
         FUNCTION Conv IS
           NEW Ada.Unchecked_Conversion( aFourBytes, s_float );
         FUNCTION Conv IS
           NEW Ada.Unchecked_Conversion( u_long, aFourBytes );
         temp : aFourBytes := Conv( value );
      BEGIN
         RETURN Conv( aFourBytes'( temp(4), temp(3), temp(2), temp(1) ) );
      END nltohf;

It isn't endian neutral (or particularly pretty), but it got the job done.

Regards,
Steve

Re: Controlling endian-ness?

[christoph.grein]

I didn't spot it myself first, but it's easy:

>     procedure Word_Write
>       (Stream : access Root_Stream_Type'Class; Item : in Word) is
                                                 |
>        Buffer    : Stream_Element_Array(0..3);
>        Workspace : Unsigned_32 := Unsigned_32(Word);  -- ERROR HERE!
                                                |should be Item here.

Re: Controlling endian-ness?

[Peter C. Chapin]

christoph.grein@eurocopter.com wrote:

> I didn't spot it myself first, but it's easy:
> 
>>     procedure Word_Write
>>       (Stream : access Root_Stream_Type'Class; Item : in Word) is
>                                                  |
>>        Buffer    : Stream_Element_Array(0..3);
>>        Workspace : Unsigned_32 := Unsigned_32(Word);  -- ERROR HERE!
>                                                 |should be Item here.

Of course! Thanks... and sorry about consuming bandwidth for something 
so silly. Now I know why I usually suffix type names with "_Type". If I 
had called it "Word_Type" I probably would not have made the error in 
the first place. I guess that's what I get for trying to write something 
quick and dirty!

Peter

Re: Controlling endian-ness?

[Peter C. Chapin]

Steve wrote:

> It isn't endian neutral (or particularly pretty), but it got the job done.

Hmm. Unchecked_Conversion, huh? That's an approach I didn't consider.

Peter

Re: Controlling endian-ness?

[anon]

For using Unsigned_32. Why not just override the built-in Write attributes 
routines for Unsigned_32 by re-writing the attributes routines. This may at 
most require a new package with a few extra attributes routines that may 
be needed. Also, this will also allow you to use "Shift_Left/Shift_Right" 
directly with Unsigned_32.  

So in your program you could use:

   Data : Unsigned_32 ;
  ...
   Unsigned_32'Write ( Stream_Pointer, Data ) ;


In GNAT you can not add new attributes, but you can override the built-in 
versions with a new set of attributes routines. 



In <4897b7f5$0$19705$4d3efbfe@news.sover.net>, "Peter C. Chapin" <pcc482719@gmail.com> writes:
>I'm trying to read/write 32 bit quantities from/to a binary file. The 
>file has a format defined by a specification that is outside my control. 
>Some of the quantities are stored in the file in big endian order. 
>However, my machine is a little endian machine. The file in question 
>also contains various values that are most naturally represented with 
>different data types. Thus I'm looking at Stream_IO as a way to deal 
>with it.
>
>As an experiment I wrote a small program that defines a 32 bit unsigned 
>integer type and then overrides the 'Write attribute to write the value 
>in big endian form. Here is what I have:
>
>with Ada.Streams;              use Ada.Streams;
>with Ada.Streams.Stream_IO;    use Ada.Streams.Stream_IO;
>with Interfaces;               use Interfaces;
>
>procedure Stream_Check is
>    pragma Assert(Stream_Element'Size = 8);
>
>    type Word is mod 2**32;
>    procedure Word_Write
>      (Stream : access Root_Stream_Type'Class; Item : in Word);
>    for Word'Size  use 32;
>    for Word'Write use Word_Write;
>
>    procedure Word_Write
>      (Stream : access Root_Stream_Type'Class; Item : in Word) is
>       Buffer    : Stream_Element_Array(0..3);
>       Workspace : Unsigned_32 := Unsigned_32(Word);  -- ERROR HERE!
>    begin
>       Buffer(0) :=
>           Stream_Element(Shift_Right((Workspace and 16#FF000000#), 24));
>       Buffer(1) :=
>           Stream_Element(Shift_Right((Workspace and 16#00FF0000#), 16));
>       Buffer(2) :=
>           Stream_Element(Shift_Right((Workspace and 16#0000FF00#),  8));
>       Buffer(3) :=
>           Stream_Element(Shift_Right((Workspace and 16#000000FF#),  0));
>       Write(Stream.all, Buffer);
>    end Word_Write;
>
>    Output_File    : File_Type;
>    Stream_Pointer : Stream_Access;
>
>    W : Word := 16#0000FFFF#;
>begin
>    Create(Output_File, Out_File, "test.bin");
>    Stream_Pointer := Stream(Output_File);
>    Word'Write(Stream_Pointer, W);
>    Close(Output_File);
>end Stream_Check;
>
>I'm using GNAT GPL 2008. It produces an error on the indicated line 
>saying, "invalid use of subtype mark in expression or call." Apparently 
>it doesn't like me trying to convert a Word to an Unsigned_32, but I 
>don't understand why (am I doing that conversion right?).
>
>I want to use Unsigned_32 so that I can use Shift_Right. I can't 
>override the 'Write attribute for Unsigned_32 directly because it's in a 
>different package (right?).
>
>Overall I have a feeling that there is probably a much better way to do 
>this.
>
>Peter

Re: Controlling endian-ness?

[Peter C. Chapin]

anon wrote:

> For using Unsigned_32. Why not just override the built-in Write attributes 
> routines for Unsigned_32 by re-writing the attributes routines. This may at 
> most require a new package with a few extra attributes routines that may 
> be needed. Also, this will also allow you to use "Shift_Left/Shift_Right" 
> directly with Unsigned_32.  
> 
> So in your program you could use:
> 
>    Data : Unsigned_32 ;
>   ...
>    Unsigned_32'Write ( Stream_Pointer, Data ) ;

I actually tried something like this but the compiler told me that I 
couldn't redefine attributes for that type (or something like that). I 
think the problem is that since the type is defined in another package 
it is already frozen or some such. It could also be that I did it wrong.

Peter

Re: Controlling endian-ness?

[Adam Beneschan]

On Aug 5, 3:14 pm, "Peter C. Chapin" <pcc482...@gmail.com> wrote:
> anon wrote:
> > For using Unsigned_32. Why not just override the built-in Write attributes
> > routines for Unsigned_32 by re-writing the attributes routines. This may at
> > most require a new package with a few extra attributes routines that may
> > be needed. Also, this will also allow you to use "Shift_Left/Shift_Right"
> > directly with Unsigned_32.
>
> > So in your program you could use:
>
> >    Data : Unsigned_32 ;
> >   ...
> >    Unsigned_32'Write ( Stream_Pointer, Data ) ;
>
> I actually tried something like this but the compiler told me that I
> couldn't redefine attributes for that type (or something like that). I
> think the problem is that since the type is defined in another package
> it is already frozen or some such. It could also be that I did it wrong.

Yes.  You can't apply any "for T'attribute use..." clause to something
defined in another package.  (It sounds to me like this is what you
tried... if I guessed wrong, sorry.)  You might consider defining your
own type (perhaps *derived* from Unsigned_32) and using your new type
when declaring a variable or component that needs to be written using
a special routine.

     type My_Unsigned_32 is new Unsigned_32;
     procedure My_Write
       (Stream : not null access Ada.Streams.Root_Stream_Type'Class;
        Item   : in My_Unsigned_32);
     for My_Unsigned_32'Write use My_Write;

                                    -- Adam
                                -- Adam

Re: Controlling endian-ness?

[Peter C. Chapin]

Adam Beneschan wrote:

>      type My_Unsigned_32 is new Unsigned_32;
>      procedure My_Write
>        (Stream : not null access Ada.Streams.Root_Stream_Type'Class;
>         Item   : in My_Unsigned_32);
>      for My_Unsigned_32'Write use My_Write;

You're right that works. I actually thought I tried using a derived type 
before, but evidently I didn't do it right. Anyway thanks for the 
suggestion.

Peter

Re: Controlling endian-ness? (Example: Attribute Overriding)

[anon]

--  File: newprog.adb
--
-- An example of how to Override an built-in Attribute:
-- Based on examples from RM 13.3 ( 84 ) and 13.13.2 ( 40 ) ;
--
with Interfaces ;
use  Interfaces ;

with Ada.Text_IO ;
use  Ada.Text_IO ;

with Ada.Streams.Stream_IO;

procedure NewProg is

  -----------------------------------------------------------

   type Unsigned_32 is new Interfaces.Unsigned_32 ; 

  --
  -- Define New Attribute Specifications
  --
   procedure newWrite
     ( Stream : access Ada.Streams.Root_Stream_Type'Class;
       Item   : in Unsigned_32 ) ;
                                           -- Performs Overriding of attribute
    for Unsigned_32'Write use newWrite ; 

  -----------------------------------------------------------

  --
  -- Define New Attribute Body
  --
   procedure newWrite
     ( Stream : access Ada.Streams.Root_Stream_Type'Class;
       Item   : in Unsigned_32 ) is
     begin
       --
       -- Replace with your "Write code"
       --
       -- test message only, just to prove that we are using this Attribute 
       --
       Put_Line ( "New Write Attribute" ) ; 
     end newWrite ;

  -----------------------------------------------------------

   Data_File      : Ada.Streams.Stream_IO.File_Type;
   Data_Stream    : Ada.Streams.Stream_IO.Stream_Access;
   The_Filename   : constant String :=  "Test.Dat" ;

  Test_Data : Unsigned_32 := 9 ;

begin

  Ada.Streams.Stream_IO.Create ( Data_File, 
                                 Ada.Streams.Stream_IO.Out_File,
                                 The_Filename ) ;

  Data_Stream := Ada.Streams.Stream_IO.Stream ( Data_File ) ;


  Put_Line ( "Testing: New Atrribute: Write" ) ;
  New_Line ;

  Unsigned_32'Write ( Data_Stream, Test_Data ) ;

  Ada.Streams.Stream_IO.Close ( Data_File ) ;

end NewProg ;

In <4898d0ca$0$19680$4d3efbfe@news.sover.net>, "Peter C. Chapin" <pcc482719@gmail.com> writes:
>anon wrote:
>
>> For using Unsigned_32. Why not just override the built-in Write attributes 
>> routines for Unsigned_32 by re-writing the attributes routines. This may at 
>> most require a new package with a few extra attributes routines that may 
>> be needed. Also, this will also allow you to use "Shift_Left/Shift_Right" 
>> directly with Unsigned_32.  
>> 
>> So in your program you could use:
>> 
>>    Data : Unsigned_32 ;
>>   ...
>>    Unsigned_32'Write ( Stream_Pointer, Data ) ;
>
>I actually tried something like this but the compiler told me that I 
>couldn't redefine attributes for that type (or something like that). I 
>think the problem is that since the type is defined in another package 
>it is already frozen or some such. It could also be that I did it wrong.
>
>Peter

Re: Controlling endian-ness?

[Simon Wright]

"Peter C. Chapin" <pcc482719@gmail.com> writes:

> anon wrote:
>
>> For using Unsigned_32. Why not just override the built-in Write
>> attributes routines for Unsigned_32 by re-writing the attributes
>> routines. This may at most require a new package with a few extra
>> attributes routines that may be needed. Also, this will also allow
>> you to use "Shift_Left/Shift_Right" directly with Unsigned_32.  
>>
>> So in your program you could use:
>>
>>    Data : Unsigned_32 ;
>>   ...
>>    Unsigned_32'Write ( Stream_Pointer, Data ) ;
>
> I actually tried something like this but the compiler told me that I
> couldn't redefine attributes for that type (or something like that). I
> think the problem is that since the type is defined in another package
> it is already frozen or some such. It could also be that I did it
> wrong.

(I'm not sure if this is in GNAT-GPL-2008, which I wasn't able to build
for Intel Mac):

(1) find s-strxdr.adb in the Ada library source directory
.../adainclude/

(2) copy it to s-stratt.adb in your compilation path

(3) give gnatmake the -a flag.

s-strxdr.adb contains an alternate body for System.Stream_Attributes
which streams to/from XDR (eXternal Data Representation) format.

You might think you would only need to do this on the little-endian
side, but there was a problem with Wide_Character in older GNATs (2
bytes or 4?) so checking the source would be good. Also, of course, you
might have problems with floats: no problems found by us where it's IEE
hardware both sides, eg PowerPC/i386.