Ada records and byte order

Ada records and byte order

[Karl Ran]

Hi there,

I've a question about Ada records:

I have to read a binary file from disk which was written by another program.

The record size is fixed: 3 bytes
The file structure looks like this:
      

|byte 1 | |byte 2 | |byte 3 |   |byte 4...
aaaa.aaaa.bbbb.bbbb.bbbb.cccc | aaaa.a.....
M       L M            L M  L
S       S S            S S  S
B       B B            B B  B

a:  8 bits
b: 12 bits
c:  4 bits

Here is the code ...

procedure test is

   subtype BYTE     is Integer range 0 .. 2 **   8 - 1;
   subtype WORD12   is integer range 0 .. 2 **  12 - 1;
   subtype WORD4    is integer range 0 .. 2 **   4 - 1;

   type My_rec is
      record
         A : BYTE;
         B : WORD12;
         C : WORD4;
      end record;
   
   for My_rec use
      record
         A at 0 range 0 ..  7;
         B at 1 range 0 .. 11;
         C at 2 range 0 ..  3;
      end record;   

   Abc : My_Rec; 

begin
   Abc.b := 16#123#;
   ...
end test;
   
... which fails to compile whith GNAT on a (low-endian) i386:
test.adb:31:10: component "C" overlaps "B" at line 30
         
Is there an Ada like (TM) solution for this kind of (byte order) problem?

   
Thanks,
Karl

Re: Ada records and byte order

[Carbonne Damien]

Karl Ran a �crit :

>
> Here is the code ...
>
> procedure test is
>
>    subtype BYTE     is Integer range 0 .. 2 **   8 - 1;

Add:
for BYTE'size use 8;

>
>    subtype WORD12   is integer range 0 .. 2 **  12 - 1;

Add
for WORD'Size use 12;

>
>    subtype WORD4    is integer range 0 .. 2 **   4 - 1;

Add:
for WORD4'Size use 4;

>
>
>    type My_rec is
>       record
>          A : BYTE;
>          B : WORD12;
>          C : WORD4;
>       end record;
>
>    for My_rec use
>       record
>          A at 0 range 0 ..  7;
>          B at 1 range 0 .. 11;
>          C at 2 range 0 ..  3;

Change previous line to:
C at 1 range 12 .. 15;
or
C at 2 range 4 .. 7;

>
>       end record;

Add:
for My_rec'Size use 24;

>
>
>    Abc : My_Rec;
>
> begin
>    Abc.b := 16#123#;
>    ...
> end test;
>
> ... which fails to compile whith GNAT on a (low-endian) i386:
> test.adb:31:10: component "C" overlaps "B" at line 30
>

The message seems quite explicit here !

>
> Is there an Ada like (TM) solution for this kind of (byte order) problem?

What you have is not a byte order problem.
You may have one if the program with which you wrote the file and the one you
are
writing in Ada are run on targets with different byte orders.
This can not be deduced from your message.

For Byte order problem, I remember having seen something (quite complexe) on
this thread some times ago, but I don't remember when and who posted it.

Anyway,  a compiler may refuse certain representation clauses and thus, there
may be some portability problems in this area.
GNAT should accept what you want in that case, unless there is a real error like
here.
Pragmatically, from what I have read on thsi subject, major compilers accept
standard size clauses (powers of 2). There are always limits !

The Ada syntax is different from C/C++ one for bitfields where you only specify
the size (length) of each field, whereas in Ada you specify
bit bounds (first and last) counted from an offset (defined in Memory_Unit I
think).


Regards

Damien

Re: Ada records and byte order

[Jeffrey Carter]

Karl Ran wrote:
> 
> Hi there,
> 
> I've a question about Ada records:
> 
> I have to read a binary file from disk which was written by another program.
> 
> The record size is fixed: 3 bytes
> The file structure looks like this:
> 
> 
> |byte 1 | |byte 2 | |byte 3 |   |byte 4...
> aaaa.aaaa.bbbb.bbbb.bbbb.cccc | aaaa.a.....
> M       L M            L M  L
> S       S S            S S  S
> B       B B            B B  B
> 
> a:  8 bits
> b: 12 bits
> c:  4 bits
> 
> Here is the code ...
> 
> procedure test is
> 
>    subtype BYTE     is Integer range 0 .. 2 **   8 - 1;
>    subtype WORD12   is integer range 0 .. 2 **  12 - 1;
>    subtype WORD4    is integer range 0 .. 2 **   4 - 1;
> 
>    type My_rec is
>       record
>          A : BYTE;
>          B : WORD12;
>          C : WORD4;
>       end record;
> 
>    for My_rec use
>       record
>          A at 0 range 0 ..  7;
>          B at 1 range 0 .. 11;
>          C at 2 range 0 ..  3;
>       end record;
> 
>    Abc : My_Rec;
> 
> begin
>    Abc.b := 16#123#;
>    ...
> end test;
> 
> ... which fails to compile whith GNAT on a (low-endian) i386:
> test.adb:31:10: component "C" overlaps "B" at line 30
> 
> Is there an Ada like (TM) solution for this kind of (byte order) problem?

On a little-endian machine, you are placing B in all of byte 3 and the 4
LSBs of byte 2. Since you are also placing C in the 4 LSBs of byte 3,
you are attempting to overlap these components.

The basic problem is that B is "at 1 range 4 .. 15" on a little-endian
machine. 15 is the MSB, and 0 the LSB, of a 16-bit word, on a
little-endian machine. Your numbering would be appropriate on a
big-endian machine. You can use System.Default_Bit_Order to obtain
endian-independent bit numbers, since it is now required to be static.

You might also want to use modular types rather than subtypes of
Integer, such as

subtype Byte is Interfaces.Unsigned_8;
type Word_12 is mod 2 ** 12;
type Nibble  is mod 2 **  4;

You might also want to include 'Size clauses on your types. The objects
(variables) that you use to read these values from the file should be
explicitly set to 24 bits to help ensure that you don't read 32 bits and
throw away 8.

-- 
Jeff Carter
"Perfidious English mouse-dropping hoarders."
Monty Python & the Holy Grail

Re: Ada records and byte order

[David C. Hoos, Sr.]

[-- Attachment #1: Type: text/plain, Size: 3459 bytes --]

The problem is that on a little-endian machine, your 12-bit record component
indicated by the bits bbbb bbbb bbbb are not contiguous in memory if read from
the file stream by ordinary means.

Ada Stream_IO provides a means of dealing with this problem, but it has the
added complication that when parts of two or more record components
occupy the same byte, the default stream attributes cannot be used, because
that byte will be written or read more than once (i.e., once for each
component
that occupies it).  To see why this is true, refer to LRM 95 13.13.2 (9),
viz.:

For elementary types, the representation in terms of stream elements is
implementation defined. For composite types, the Write or Read attribute
for each component is called in a canonical order. The canonical order of
components is last dimension varying fastest for an array, and positional
aggregate order for a record. Bounds are not included in the stream if T is an
array type. If T is a discriminated type, discriminants are included only if
they
have defaults. If T is a tagged type, the tag is not included.

Given all of this, I have written a small demonstration program that will
write
out a file such as you describe.  Normally, of course, the types would be
declared in their own package, and the stream-oriented attributes would be
implemented in the body of that package, but I have combined them all here
to keep things to a single file.

The source code for the demonstration program is attached.

I do not have access to a big-endian machine today, so this program has
only been tested on a little-endian machine.

Hoe this helps to understand the problem.

There have been many questions of similar nature on this newsgroup, so
I thought I'd seize upon a real example, and show how I've been solving
this kind of problem.  If anyone has a better way to do it, I'm all ears.

----- Original Message -----
From: "Karl Ran" <karlran1234@yahoo.com>
Newsgroups: comp.lang.ada
To: <comp.lang.ada@ada.eu.org>
Sent: June 23, 2001 5:15 AM
Subject: Ada records and byte order


> Hi there,
>
> I've a question about Ada records:
>
> I have to read a binary file from disk which was written by another program.
>
> The record size is fixed: 3 bytes
> The file structure looks like this:
>
>
> |byte 1 | |byte 2 | |byte 3 |   |byte 4...
> aaaa.aaaa.bbbb.bbbb.bbbb.cccc | aaaa.a.....
> M       L M            L M  L
> S       S S            S S  S
> B       B B            B B  B
>
> a:  8 bits
> b: 12 bits
> c:  4 bits
>
> Here is the code ...
>
> procedure test is
>
>    subtype BYTE     is Integer range 0 .. 2 **   8 - 1;
>    subtype WORD12   is integer range 0 .. 2 **  12 - 1;
>    subtype WORD4    is integer range 0 .. 2 **   4 - 1;
>
>    type My_rec is
>       record
>          A : BYTE;
>          B : WORD12;
>          C : WORD4;
>       end record;
>
>    for My_rec use
>       record
>          A at 0 range 0 ..  7;
>          B at 1 range 0 .. 11;
>          C at 2 range 0 ..  3;
>       end record;
>
>    Abc : My_Rec;
>
> begin
>    Abc.b := 16#123#;
>    ...
> end test;
>
> ... which fails to compile whith GNAT on a (low-endian) i386:
> test.adb:31:10: component "C" overlaps "B" at line 30
>
> Is there an Ada like (TM) solution for this kind of (byte order) problem?
>
>
> Thanks,
> Karl
> _______________________________________________
> comp.lang.ada mailing list
> comp.lang.ada@ada.eu.org
> http://ada.eu.org/mailman/listinfo/comp.lang.ada
>


[-- Attachment #2: ran.adb --]
[-- Type: application/octet-stream, Size: 3553 bytes --]

with Ada.Text_IO;
with Ada.Streams.Stream_IO;
with System;
procedure Ran is
   type Unsigned_4 is mod 2 **4;
   for Unsigned_4'Size use 4;

   type Unsigned_8 is mod 2 ** 8;
   for Unsigned_8'Size use 8;

   type Unsigned_12 is mod 2 ** 12;
   for Unsigned_12'Size use 12;

   -- Because a single byte is occupied by Both part of B and all of C,
   -- We combine them into a record so we can define stream-oriented
   -- attributes such that they can be read and written from/to a stream
   -- properly, regardless of machine endianness.

   type B_And_C is
      record
        B : Unsigned_12;
        C : Unsigned_4;
      end record;

   -- Use this representation clause on a little-endian machine.
   for B_And_C use
     record at mod 1;
        B at 0 range 4 .. 15;
        C at 0 range 0 .. 3;
     end record;

   -- Use this representation clause on a big-endian machine.
--    for B_and_C use
--      record at mod 1;
--         B at 0 range 0 .. 11;
--         C at 0 range 12 .. 15;
--      end record;

   for B_And_C'Size use 16;

   -- We now procedd to declare the stream-orientd attributes
   procedure B_And_C_Read
     (Stream : access Ada.Streams.Root_Stream_Type'Class;
      Item   : out     B_And_C);

   procedure B_And_C_Write
     (Stream : access Ada.Streams.Root_Stream_Type'Class;
      Item   : in     B_And_C);

   for B_And_C'Read use B_And_C_Read;

   for B_And_C'Write use B_And_C_Write;

   type My_Record is
     record
        A  : Unsigned_8;
        BC : B_And_C;
     end record;

   type Byte_Array is array (Positive range <>) of Unsigned_8;

   -- This procedure reverses the oder of the bytes in its argument.
   procedure Swap (The_Bytes : in out Byte_Array) is
      Temp : Unsigned_8;
   begin
      for B in 1 .. The_Bytes'Last / 2 loop
         Temp := The_Bytes (B);
         The_Bytes (B) := The_Bytes (The_Bytes'Last - B + 1);
         The_Bytes (The_Bytes'Last - B + 1) := Temp;
      end loop;
   end Swap;

   -- These porocedures implement the stream-oriented attributes.
   procedure B_And_C_Read
     (Stream : access Ada.Streams.Root_Stream_Type'Class;
      Item   : out     B_And_C) is
      The_Bytes : Byte_Array (1 .. Item'Size / Unsigned_8'Size);
      for The_Bytes'Address use Item'Address;
      use type System.Bit_Order;
   begin
      Byte_Array'Read (Stream, The_Bytes);
      if System.Default_Bit_Order = System.Low_Order_First then
        Swap (The_Bytes);
      end if;
   end B_And_C_Read;

   procedure B_And_C_Write
     (Stream : access Ada.Streams.Root_Stream_Type'Class;
      Item   : in     B_And_C) is
      The_Bytes : Byte_Array (1 .. Item'Size / Unsigned_8'Size);
      for The_Bytes'Address use Item'Address;
      use type System.Bit_Order;
   begin
      if System.Default_Bit_Order = System.Low_Order_First then
        Swap (The_Bytes);
      end if;
      Byte_Array'Write (Stream, The_Bytes);
   end B_And_C_Write;

   -- Define the object with all six nybbles unique, so we can
   -- demonstrate that they appear in correct order in the file.
   Item : My_Record := (16#9A#, (16#B0D#, 16#C#));

   File : Ada.Streams.Stream_IO.File_Type;

   Stream : Ada.Streams.Stream_IO.Stream_Access;
begin
   Ada.Streams.Stream_IO.Create
     (Name => "ran.dat",
      File => File,
      Mode => Ada.Streams.Stream_IO.Out_File);

   -- Associate a stream with the opened/created file
   Stream := Ada.Streams.Stream_IO.Stream (File);

   for I in 1 .. 4 loop

      My_Record'Write (Stream, Item);

   end loop;

   Ada.Streams.Stream_IO.Close (File);

end Ran;

Re: Ada records and byte order

[Jeffrey Carter]

"David C. Hoos, Sr." wrote:
> 
> The problem is that on a little-endian machine, your 12-bit record component
> indicated by the bits bbbb bbbb bbbb are not contiguous in memory if read from
> the file stream by ordinary means.

Oops! I guess I've been spending too much time dealing with big-endian
machines. David is right, so disregard my previous post.

-- 
Jeff Carter
"Perfidious English mouse-dropping hoarders."
Monty Python & the Holy Grail

Re: Ada records and byte order

[Karl Ran]

Thanks everyone how conributed to solve my problem.
Special thanks goes to David C. Hoos for a ready to compile-and-run example.

Karl.