GNAT 4.4.5 Record Bit_Order Endian issues

GNAT 4.4.5 Record Bit_Order Endian issues

[awdorrin]

I have a record with specification definition that starts, using big endian notation, with:

for TRACK_RECORD_TYPE use
 record
  TRACK_ID at 0 range 0 .. 11;
  TRACK_ID_CNT at 1 range 4 .. 9;
   ...


The meaning being that the first 12 bits hold a track id and the next 6 bits hold the Count value.

I have had success in the past, using the gnat 4.4.5 compiler under Debian 6.0.6, to use the directive: for TRACK_RECORD_TYPE'Bit_Order use System.High_Order_First;
In order to continue using the definition in the big endian notation.

From the build output, the Bit_Order flag at first appears to be processed correctly, with the built compiler flags displaying to translated little endian definition as:

for TRACK_RECORD_TYPE use
 record
  TRACK_ID at 0 range 4 .. 15;
  TRACK_ID_CNT at 1 range 11 .. 6;
   ...

But the TRACK_ID_CNT range does not provide the proper definition.
I believe it should be: TRACK_ID_CNT at 1 range 5 .. 0

Using the following code:

TRK_REC.TRACK_ID := 1;
Text_IO.Put_Line("A: TRK_REC.TRACK_ID = " & Int32'Image(TRK_REC.TRACK_ID));
TRK_REC.TRACK_ID_CNT := 1;
Text_IO.Put_Line("B: TRK_REC.TRACK_ID = " & Int32'Image(TRK_REC.TRACK_ID));

The output displayed is:

A: TRK_REC.TRACK_ID = 1
B: TRK_REC.TRACK_ID = 1025

Apparently because the TRACK_ID field is being read as a 16-bit value where:

"0000 0100 0000 0001" -> "BBBB CCAA AAAA AAAA"
where the 'A' bits are the TRACK_ID and the B bits are part of the TRACK_ID_CNT field, and the 'C' bits are an overlap of the second variable, due to the improper definition translation.

Not sure if this 'bug' would be fixed in a newer version of the GNAT compiler or not.

Re: GNAT 4.4.5 Record Bit_Order Endian issues

[Randy Brukardt]

"awdorrin" <awdorrin@gmail.com> wrote in message 
news:20bda3de-b033-4b4e-8298-2ac47701b814@googlegroups.com...
...
> Not sure if this 'bug' would be fixed in a newer version of the GNAT 
> compiler or not.

The RM has a strict definition of what 'Bit_Order means; the only variable 
is the size of a "machine word". So I think GNAT has very limited 
flexibility in what the result is for such a transformation. I didn't try to 
figure out if they got it wrong or not, but if not, they probably could not 
change the results (in the absense of some implementation-defined pragma or 
aspect).

                                         Randy.

Re: GNAT 4.4.5 Record Bit_Order Endian issues

[Adam Beneschan]

On Monday, January 14, 2013 9:43:51 AM UTC-8, awdorrin wrote:
> I have a record with specification definition that starts, using big endian notation, with:
> 
> for TRACK_RECORD_TYPE use
>  record
>   TRACK_ID at 0 range 0 .. 11;
>   TRACK_ID_CNT at 1 range 4 .. 9;
>    ...
> 
> 
> The meaning being that the first 12 bits hold a track id and the next 6 bits hold the Count value.
> 
> I have had success in the past, using the gnat 4.4.5 compiler under Debian 6.0.6, to use the directive: for TRACK_RECORD_TYPE'Bit_Order use System.High_Order_First;
> 
> In order to continue using the definition in the big endian notation.
> 
> From the build output, the Bit_Order flag at first appears to be processed correctly, with the built compiler flags displaying to translated little endian definition as:
> 
> 
> 
> for TRACK_RECORD_TYPE use
>  record
>   TRACK_ID at 0 range 4 .. 15;
>   TRACK_ID_CNT at 1 range 11 .. 6;
>    ...
> 
> But the TRACK_ID_CNT range does not provide the proper definition.
> I believe it should be: TRACK_ID_CNT at 1 range 5 .. 0
> 
> Using the following code:
> 
> TRK_REC.TRACK_ID := 1;
> Text_IO.Put_Line("A: TRK_REC.TRACK_ID = " & Int32'Image(TRK_REC.TRACK_ID));
> TRK_REC.TRACK_ID_CNT := 1;
> Text_IO.Put_Line("B: TRK_REC.TRACK_ID = " & Int32'Image(TRK_REC.TRACK_ID));
> 
> The output displayed is:
> 
> A: TRK_REC.TRACK_ID = 1
> B: TRK_REC.TRACK_ID = 1025
> 
> Apparently because the TRACK_ID field is being read as a 16-bit value where:
> 
> "0000 0100 0000 0001" -> "BBBB CCAA AAAA AAAA"
> where the 'A' bits are the TRACK_ID and the B bits are part of the TRACK_ID_CNT field, and the 'C' bits are an overlap of the second variable, due to the improper definition translation.
> 
> Not sure if this 'bug' would be fixed in a newer version of the GNAT compiler or not.

Well, it looks to me like there's definitely a bug in there, unless your record is an Unchecked_Union.  If the fields really do overlap, the program should not compile; the language doesn't allow overlapping fields except in variant record cases (13.5.1(11)).  And if they don't overlap, then of course setting one component should not affect the value of the other one.

                               -- Adam

Re: GNAT 4.4.5 Record Bit_Order Endian issues

[AdaMagica]

> for TRACK_RECORD_TYPE use record
>   TRACK_ID     at 0 range 0 .. 11;
>   TRACK_ID_CNT at 1 range 4 .. 9;

Change this to

for TRACK_RECORD_TYPE use record
  TRACK_ID     at 0 range  0 .. 11;
  TRACK_ID_CNT at 0 range 12 .. 17;
for TRACK_RECORD_TYPE'Bit_Order use High_Order_First;

Re: GNAT 4.4.5 Record Bit_Order Endian issues

[Stephen Leake]

awdorrin <awdorrin@gmail.com> writes:

> I have a record with specification definition that starts, using big endian notation, with:
>
> for TRACK_RECORD_TYPE use
>  record
>   TRACK_ID at 0 range 0 .. 11;
>   TRACK_ID_CNT at 1 range 4 .. 9;
>    ...
>
>
> The meaning being that the first 12 bits hold a track id and the next
> 6 bits hold the Count value.

If you are allocating consecutive  bits, then do this:

for TRACK_RECORD_TYPE use
   record
     TRACK_ID     at 0 range  0 .. 11;
     TRACK_ID_CNT at 0 range 12 .. 17;
     ...


-- 
-- Stephe

Re: GNAT 4.4.5 Record Bit_Order Endian issues

[Simon Wright]

Stephen Leake <stephen_leake@stephe-leake.org> writes:

> awdorrin <awdorrin@gmail.com> writes:
>
>> I have a record with specification definition that starts, using big endian notation, with:
>>
>> for TRACK_RECORD_TYPE use
>>  record
>>   TRACK_ID at 0 range 0 .. 11;
>>   TRACK_ID_CNT at 1 range 4 .. 9;
>>    ...
>>
>>
>> The meaning being that the first 12 bits hold a track id and the next
>> 6 bits hold the Count value.
>
> If you are allocating consecutive  bits, then do this:
>
> for TRACK_RECORD_TYPE use
>    record
>      TRACK_ID     at 0 range  0 .. 11;
>      TRACK_ID_CNT at 0 range 12 .. 17;
>      ...

I don't see why this makes a difference? It certainly affects how GNAT
behaves, though I have to say it looks buggy to me. 

OP's way:

awdorrin.ads:13:26: info: reverse bit order in machine scalar of length 16
awdorrin.ads:13:26: info: little-endian range for component "Track_Id" is 4 .. 15
awdorrin.ads:15:06: component "Track_Id_Cnt" overlaps "Track_Id" at line 13
awdorrin.ads:15:30: info: reverse bit order in machine scalar of length 16
awdorrin.ads:15:30: info: little-endian range for component "Track_Id_Cnt" is 6 .. 11
gnatmake: "awdorrin.ads" compilation error

Your way:

awdorrin.ads:13:26: info: reverse bit order in machine scalar of length 32
awdorrin.ads:13:26: info: little-endian range for component "Track_Id" is 20 .. 31
awdorrin.ads:14:30: info: reverse bit order in machine scalar of length 32
awdorrin.ads:14:30: info: little-endian range for component "Track_Id_Cnt" is 14 .. 19

Representation information for unit Awdorrin (spec)
---------------------------------------------------

for Track_Record_Type'Size use 32;
for Track_Record_Type'Alignment use 4;
for Track_Record_Type use record
   Track_Id     at 2 range  4 .. 15;
   Track_Id_Cnt at 1 range  6 .. 11;
end record;

Re: GNAT 4.4.5 Record Bit_Order Endian issues

[AdaMagica]

> I don't see why this makes a difference? It certainly affects how GNAT
> behaves, though I have to say it looks buggy to me. 

No, this is acording to the RM (I don't feel like looking up the para just now).
The model are machine scalars, and the bits are allocated by counting the bits inside the scalar.

In this example, the machine scalar is 16 bits.

Re: GNAT 4.4.5 Record Bit_Order Endian issues

[Simon Wright]

AdaMagica <christ-usch.grein@t-online.de> writes:

>> I don't see why this makes a difference? It certainly affects how GNAT
>> behaves, though I have to say it looks buggy to me. 
>
> No, this is acording to the RM (I don't feel like looking up the para
> just now).  The model are machine scalars, and the bits are allocated
> by counting the bits inside the scalar.
>
> In this example, the machine scalar is 16 bits.

It's http://www.ada-auth.org/standards/12aarm/html/AA-13-5-1.html#p13.4
- and the problem isn't so much that the components overlap as that the
machine scalars that they're allocated in overlap.

I don't think I'd ever have written

   for Track_Record_Type use record
     Track_Id at 0 range 0 .. 11;
     Track_Id_Cnt at 1 range 4 .. 9;

because it's confusing whichever bit order you use!

Re: GNAT 4.4.5 Record Bit_Order Endian issues

[Randy Brukardt]

"Simon Wright" <simon@pushface.org> wrote in message 
news:lyfw21v24l.fsf@pushface.org...
...
>> If you are allocating consecutive  bits, then do this:
>>
>> for TRACK_RECORD_TYPE use
>>    record
>>      TRACK_ID     at 0 range  0 .. 11;
>>      TRACK_ID_CNT at 0 range 12 .. 17;
>>      ...
>
> I don't see why this makes a difference? It certainly affects how GNAT
> behaves, though I have to say it looks buggy to me.

Because of the definition of Bit_Order! Essentially, if you're using the 
native bit order, these two mean the same thing, but they mean completely 
different things for the non-native bit order. Read the RM for details.

GNAT is most likely do thing right thing here, and it has no choice because 
of the definition of the language.

                                  Randy.

Re: GNAT 4.4.5 Record Bit_Order Endian issues

[Stephen Leake]

Simon Wright <simon@pushface.org> writes:

> Stephen Leake <stephen_leake@stephe-leake.org> writes:
>
>> awdorrin <awdorrin@gmail.com> writes:
>>
>>> I have a record with specification definition that starts, using big endian notation, with:
>>>
>>> for TRACK_RECORD_TYPE use
>>>  record
>>>   TRACK_ID at 0 range 0 .. 11;
>>>   TRACK_ID_CNT at 1 range 4 .. 9;
>>>    ...
>>>
>>>
>>> The meaning being that the first 12 bits hold a track id and the next
>>> 6 bits hold the Count value.
>>
>> If you are allocating consecutive  bits, then do this:
>>
>> for TRACK_RECORD_TYPE use
>>    record
>>      TRACK_ID     at 0 range  0 .. 11;
>>      TRACK_ID_CNT at 0 range 12 .. 17;
>>      ...
>
> I don't see why this makes a difference? 

The best explanation I have seen is in
http://www.ada-auth.org/ai-files/grab_bag/bitorder.pdf

That introduces the concept of "machine scalar", which helps a lot.

The problem is what does "the first twelve bits" mean, when the
"natural" size of a chunk of memory is 8 bits?

In this code:

   TRACK_ID at 0 range 0 .. 11;
   TRACK_ID_CNT at 1 range 4 .. 9;

if Storage_Unit = 8, the two fields overlap. I can't figure out a
storage arrangement that makes these fields contiguous; can you draw a
picture? Changing to a compiler/target with a different Storage_Size
changes the meaning.

In this code: 

      TRACK_ID     at 0 range  0 .. 11;
      TRACK_ID_CNT at 0 range 12 .. 17;

it's obvious that the fields are contiguous, and it's up to the compiler
to ensure that happens.

Note that gnat agrees with me:

> OP's way:

> awdorrin.ads:13:26: info: reverse bit order in machine scalar of length 16
> awdorrin.ads:13:26: info: little-endian range for component "Track_Id" is 4 .. 15
> awdorrin.ads:15:06: component "Track_Id_Cnt" overlaps "Track_Id" at line 13
> awdorrin.ads:15:30: info: reverse bit order in machine scalar of length 16
> awdorrin.ads:15:30: info: little-endian range for component "Track_Id_Cnt" is 6 .. 11
> gnatmake: "awdorrin.ads" compilation error

This says that the first code specifies overlapping fields, which is not
what you want.

> Your way:

> awdorrin.ads:13:26: info: reverse bit order in machine scalar of length 32
> awdorrin.ads:13:26: info: little-endian range for component "Track_Id" is 20 .. 31
> awdorrin.ads:14:30: info: reverse bit order in machine scalar of length 32
> awdorrin.ads:14:30: info: little-endian range for component "Track_Id_Cnt" is 14 .. 19

This says the two fields occupy the 16 bits numbered 31 .. 19, and are
consecutive.

Why do you say this is buggy?

-- 
-- Stephe

Re: GNAT 4.4.5 Record Bit_Order Endian issues

[awdorrin]

On Wednesday, January 16, 2013 10:49:31 PM UTC-5, Stephen Leake wrote:

> 
> The best explanation I have seen is in
> http://www.ada-auth.org/ai-files/grab_bag/bitorder.pdf
> That introduces the concept of "machine scalar", which helps a lot.
> 
> The problem is what does "the first twelve bits" mean, when the
> "natural" size of a chunk of memory is 8 bits?

This is where the confusion comes from.

> 
> In this code:
> 
>    TRACK_ID at 0 range 0 .. 11;
>    TRACK_ID_CNT at 1 range 4 .. 9;
> 
> if Storage_Unit = 8, the two fields overlap. I can't figure out a
> storage arrangement that makes these fields contiguous; can you draw a
> picture? Changing to a compiler/target with a different Storage_Size
> changes the meaning.

The code in question was originally developed on a Motorola (Big Endian) CPU, So, given the Motorola 'Bit Order' being 'High_Order_First':

Byte: 0        1        2        3
Bits: 01234567 01234567 01234567 01234567

      00000000 00111111 11112222 22222233
      01234567 89012345 67890123 45678901
     
For a picture using the definition of:
 TRACK_ID at 0 range 0 .. 11;
 TRACK_ID_CNT at 1 range 4 .. 9;

Track Id starts at 'Byte 0, Bit 0 and extends to the 11th bit'

Byte:|0       |1        |2
Bits:|01234567|0123 4567|01234567
     |00000000|0011 1111|11112222
     |01234567|8901 2345|67890123
     |Track ID     |
              |0000 0000 00|11111|
              |0123 4567 89|01234|
                   |Track  |
                   |Id Cnt |


> 
> In this code: 
>       TRACK_ID     at 0 range  0 .. 11;
>       TRACK_ID_CNT at 0 range 12 .. 17;
>
> it's obvious that the fields are contiguous, and it's up to the compiler
> to ensure that happens.

I agree that this notation makes more sense, given Big Endian notation. But the question I have is, when can you start at a byte other than 0?


> Note that gnat agrees with me:
>
> > OP's way:
> > awdorrin.ads:13:26: info: reverse bit order in machine scalar of length 16
> > awdorrin.ads:13:26: info: little-endian range for component "Track_Id" is 4 .. 15
> > awdorrin.ads:15:06: component "Track_Id_Cnt" overlaps "Track_Id" at line 13
> > awdorrin.ads:15:30: info: reverse bit order in machine scalar of length 16
> > awdorrin.ads:15:30: info: little-endian range for component "Track_Id_Cnt" is 6 .. 11
> > gnatmake: "awdorrin.ads" compilation error
> 
> This says that the first code specifies overlapping fields, which is not
> what you want.
> 
> > Your way:
> > awdorrin.ads:13:26: info: reverse bit order in machine scalar of length 32
> > awdorrin.ads:13:26: info: little-endian range for component "Track_Id" is 20 .. 31
> > awdorrin.ads:14:30: info: reverse bit order in machine scalar of length 32
> > awdorrin.ads:14:30: info: little-endian range for component "Track_Id_Cnt" is 14 .. 19
> 
> This says the two fields occupy the 16 bits numbered 31 .. 19, and are
> consecutive.

As I just discovered after upgrading my system from Debian 6/GNAT 4.4.5 to Debian 7/GNAT 4.6; the GNAT 4.6 compiler produces the 'overlaps' error messages and will not compile. 

GNAT 4.4.5 does not produce this error, and compiles the definition into an overlapping field on an Intel/Little Endian target.

>
> Why do you say this is buggy?
>

The first issue is that GNAT 4.4.5 compiles the code without error and overlaps the fields, this is a bug in the 4.4.5 compiler (which I am glad has been caught with the 4.6 compiler.)

Given:
   TRACK_ID     at 0 range 0 .. 11;
   TRACK_ID_CNT at 1 range 4 .. 9; 

And this:
    TRACK_ID     at 0 range  0 .. 11;
    TRACK_ID_CNT at 0 range 12 .. 17; 

The 'Bit_Order = High_Order_First' means that the starting point from the left hand side is Byte 0, Bit 0, so I would expect the above two examples to produce the same code, based on the byte offsets acting as an index into the memory space.

Meaning that the following statements all define the same bit location, from a big endian/High order first notation:

'at 0 range  8.. 8' = 'at 1 range 0..0'
'at 0 range 32..32' = 'at 5 range 0..0'

Now, if the standard wants to say 'all data fields that span a group of bytes must be defined from the same starting byte', that would make sense to me. 

For example:
    TRACK_ID     at 0 range  0 .. 11;
    TRACK_ID_CNT at 0 range 12 .. 17;
    VAR1         at 0 range 18 .. 20;
    VAR2         at 0 range 21 .. 23;
    Byte1        at 3 range  0 .. 7;
    Int32Var     at 4 range  0 .. 31;

As for the 'A Proposal for Endian-Portable Record Representation Clauses' document, I take the proposal to mean that we should be defining the records as a multiple of a 'machine scalar' - which the document states could be 8-bit, 16-bit or 32-bit.

After further thought, I understand this to mean that our record definitions should fit the least common multiple that fits the span.

So, given my above example, defining 'Byte1' at starting at byte 3, would be wrong, since the first 'group' is 3 bytes (24-bits) and instead should be an even multiple (ie. 32-bits), so:

    TRACK_ID     at 0 range  0 .. 11;
    TRACK_ID_CNT at 0 range 12 .. 17;
    VAR1         at 0 range 18 .. 20;
    VAR2         at 0 range 21 .. 23;
    Byte1        at 0 range 24 .. 31;
    Int32Var     at 4 range  0 .. 31;

I would assume, that if the record needed the 'Byte1' and 'Int32Var' elements swapped physically that the definition would need to be bumped from a '32-bit' base, to a 64-bit base:

    TRACK_ID     at 0 range  0 .. 11;
    TRACK_ID_CNT at 0 range 12 .. 17;
    VAR1         at 0 range 18 .. 20;
    VAR2         at 0 range 21 .. 23;
    Int32Var     at 0 range 24 .. 55;
    Byte1        at 0 range 56 .. 63;
    Int32Var2    at 8 range  0 .. 31;

While I understand the rationale of defining the records using this methodology, it requires that the programmer knows that the 'starting byte' has restrictions.

Rather than put this expectation on the programmer, it seems like something best left to the compiler to work out.

So if I write 'Track_ID_Cnt at 1 range 4 .. 7' the compiler should be able to translate that for me to: 'Track_Id_Cnt at 0 range 12 .. 17' (etc)

-Al

Re: GNAT 4.4.5 Record Bit_Order Endian issues

[Stephen Leake]

awdorrin <awdorrin@gmail.com> writes:

> The code in question was originally developed on a Motorola (Big
> Endian) CPU, So, given the Motorola 'Bit Order' being
> 'High_Order_First':
>
> Byte: 0        1        2        3
> Bits: 01234567 01234567 01234567 01234567
>
>       00000000 00111111 11112222 22222233
>       01234567 89012345 67890123 45678901
>      
> For a picture using the definition of:
>  TRACK_ID at 0 range 0 .. 11;
>  TRACK_ID_CNT at 1 range 4 .. 9;
>
> Track Id starts at 'Byte 0, Bit 0 and extends to the 11th bit'
>
> Byte:|0       |1        |2
> Bits:|01234567|0123 4567|01234567
>      |00000000|0011 1111|11112222
>      |01234567|8901 2345|67890123
>      |Track ID     |
>               |0000 0000 00|11111|
>               |0123 4567 89|01234|
>                    |Track  |
>                    |Id Cnt |

I see. I guess I never tried to write rep specs this way; I've always
implicitly used the machine scalar concept.

>> In this code: 
>>       TRACK_ID     at 0 range  0 .. 11;
>>       TRACK_ID_CNT at 0 range 12 .. 17;
>>
>> it's obvious that the fields are contiguous, and it's up to the compiler
>> to ensure that happens.
>
> I agree that this notation makes more sense, given Big Endian
> notation. But the question I have is, when can you start at a byte
> other than 0?

When the field you are describing starts on a byte boundary.

More precisely, on a Storage_Unit boundary.

> As I just discovered after upgrading my system from Debian 6/GNAT
> 4.4.5 to Debian 7/GNAT 4.6; the GNAT 4.6 compiler produces the
> overlaps' error messages and will not compile.

It doesn't help that only very recent versions of GNAT get this right.

> Now, if the standard wants to say 'all data fields that span a group
> of bytes must be defined from the same starting byte', that would make
> sense to me.

Yes, that is what it should say. I agree the ARM is very deficient in
descibing this. It really needs to say what Cohen's paper says.

13.5.1(3) says:

     component_clause ::=
         component_local_name at position range first_bit .. last_bit;

But I could find no definition of "position"!

> For example:
>     TRACK_ID     at 0 range  0 .. 11;
>     TRACK_ID_CNT at 0 range 12 .. 17;
>     VAR1         at 0 range 18 .. 20;
>     VAR2         at 0 range 21 .. 23;

that's a 24 bit machine scalar

>     Byte1        at 3 range  0 .. 7;

that's an 8 bit machine scalar

>     Int32Var     at 4 range  0 .. 31;

that's a 32 bit machine scalar

> As for the 'A Proposal for Endian-Portable Record Representation
> Clauses' document, I take the proposal to mean that we should be
> defining the records as a multiple of a 'machine scalar' - which the
> document states could be 8-bit, 16-bit or 32-bit.

The size of the machine scalar is flexible, it does not have to be the
same thru-out the record.

> After further thought, I understand this to mean that our record
> definitions should fit the least common multiple that fits the span.

That does make things simpler, but it is not required.

> So, given my above example, defining 'Byte1' at starting at byte 3,
> would be wrong, since the first 'group' is 3 bytes (24-bits) and
> instead should be an even multiple (ie. 32-bits), so:
>
>     TRACK_ID     at 0 range  0 .. 11;
>     TRACK_ID_CNT at 0 range 12 .. 17;
>     VAR1         at 0 range 18 .. 20;
>     VAR2         at 0 range 21 .. 23;
>     Byte1        at 0 range 24 .. 31;
>     Int32Var     at 4 range  0 .. 31;

I would probably write it this way, since it is a little easier to
understand.

> I would assume, that if the record needed the 'Byte1' and 'Int32Var'
> elements swapped physically that the definition would need to be
> bumped from a '32-bit' base, to a 64-bit base:
>
>     TRACK_ID     at 0 range  0 .. 11;
>     TRACK_ID_CNT at 0 range 12 .. 17;
>     VAR1         at 0 range 18 .. 20;
>     VAR2         at 0 range 21 .. 23;
>     Int32Var     at 0 range 24 .. 55;
>     Byte1        at 0 range 56 .. 63;
>     Int32Var2    at 8 range  0 .. 31;

That is correct, but not necessary, since Int32Var starts on a byte
boundary. And some compiler might support only 32 bit machine scalars.

> While I understand the rationale of defining the records using this
> methodology, it requires that the programmer knows that the 'starting
> byte' has restrictions.

And the ARM does not help.

> Rather than put this expectation on the programmer, it seems like
> something best left to the compiler to work out.

As Cohen's paper makes clear, the compiler _can_ work it out, but
without the notion of machine scalar, the notation is ambiguous.

> So if I write 'Track_ID_Cnt at 1 range 4 .. 7' the compiler should be
> able to translate that for me to: 'Track_Id_Cnt at 0 range 12 .. 17'
> (etc)

I don't think so, since that '4' is not on a byte boundary.

This is related to the confusion between bit endianness and byte
endianness; they are _not_ the same thing, although they almost always
are correlated. See my other post.

-- 
-- Stephe

Re: GNAT 4.4.5 Record Bit_Order Endian issues

[Robert A Duff]

Stephen Leake <stephen_leake@stephe-leake.org> writes:

>      component_clause ::=
>          component_local_name at position range first_bit .. last_bit;
>
> But I could find no definition of "position"!

The very next paragraph defines the syntax:

    position ::= static_expression

Then 13.5.2 talks about the Position attribute.

I agree that the RM is confusing in this area.
It's partly my fault.  :-(

- Bob

Re: GNAT 4.4.5 Record Bit_Order Endian issues

[Stephen Leake]

Robert A Duff <bobduff@shell01.TheWorld.com> writes:

> Stephen Leake <stephen_leake@stephe-leake.org> writes:
>
>>      component_clause ::=
>>          component_local_name at position range first_bit .. last_bit;
>>
>> But I could find no definition of "position"!
>
> The very next paragraph defines the syntax:
>
>     position ::= static_expression

That doesn't say "position is the number of storage_units from the start
of the record" (there's probably a better definition in Cohen's paper).
For all I can tell from the ARM, "position" could be a color number, or
the number of angels on the head of a pin!

> Then 13.5.2 talks about the Position attribute.

Which just retreives the mysterious "position" number, it doesn't say
what it _means_.

> I agree that the RM is confusing in this area.
> It's partly my fault.  :-(

Ok. What paragraphs do we add? (in five years; better late than never :)

-- 
-- Stephe

Re: GNAT 4.4.5 Record Bit_Order Endian issues

[AdaMagica]

On Saturday, January 19, 2013 2:43:03 AM UTC+1, Stephen Leake wrote:
> Robert A Duff <bobduff@shell01.TheWorld.com> writes:
> > Stephen Leake <stephen_leake@stephe-leake.org> writes:
> >>      component_clause ::=
> >>          component_local_name at position range first_bit .. last_bit;
> >> But I could find no definition of "position"!
> > The very next paragraph defines the syntax:
> >     position ::= static_expression
> That doesn't say "position is the number of storage_units from the start
> of the record" (there's probably a better definition in Cohen's paper).
> For all I can tell from the ARM, "position" could be a color number, or
> the number of angels on the head of a pin!
> Then 13.5.2 talks about the Position attribute.
> Which just retreives the mysterious "position" number, it doesn't say
> what it _means_.
> > I agree that the RM is confusing in this area.
> > It's partly my fault.  :-(
> Ok. What paragraphs do we add? (in five years; better late than never :)

The AARM helps, but this is of course not a binding standard:

13.5.2(2.a/2) Ramification: {AI95-00133-01} Thus, for the default bit order, R.C'Position is the offset of C in storage elements from the beginning of the object, where the first storage element of an object is numbered zero. R'Address + R.C'Position = R.C'Address. For record extensions, the offset is not measured from the beginning of the extension part, but from the beginning of the whole object, as usual.
(2.b) In “R.C'Address – R'Address”, the "–" operator is the one in System.Storage_Elements that takes two Addresses and
returns a Storage_Offset.

Why isn't this in the RM itself? An oversight?

Re: GNAT 4.4.5 Record Bit_Order Endian issues

[Randy Brukardt]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 2293 bytes --]

"AdaMagica" <christ-usch.grein@t-online.de> wrote in message 
news:9dc6d5fe-3382-44dd-8460-e042bd6b808e@googlegroups.com...
On Saturday, January 19, 2013 2:43:03 AM UTC+1, Stephen Leake wrote:
...
>> Then 13.5.2 talks about the Position attribute.
>> Which just retreives the mysterious "position" number, it doesn't say
>> what it _means_.

Huh? 13.5.2(2/2) says in part:
"denotes Denotes the same value as R.C'Address � R'Address". That makes it
pretty clear what it represents (for the Default_Bit_Order).

For the non-default bit order, it just is whatever you specify (and that's 
all we can say,
because it *doesn't* have a clear physical meaning - it defines the machine 
scalars to use,
which makes it a totally confusing mess if you try to work the other way).

>> > I agree that the RM is confusing in this area.
>> > It's partly my fault.  :-(
>> Ok. What paragraphs do we add? (in five years; better late than never :)

I don't think there is anything that would help. We surely tried when we 
created the non-default bit order stuff.

>The AARM helps, but this is of course not a binding standard:
>
>13.5.2(2.a/2) Ramification: {AI95-00133-01} Thus, for the default bit 
>order,
>R.C'Position is the offset of C in storage elements from the beginning of 
>the object,
>where the first storage element of an object is numbered zero. R'Address +
>R.C'Position = R.C'Address. For record extensions, the offset is not 
>measured
>from the beginning of the extension part, but from the beginning of the 
>whole object,
>as usual.
>(2.b) In �R.C'Address � R'Address�, the "�" operator is the one in
>System.Storage_Elements that takes two Addresses and returns a 
>Storage_Offset.
>
>Why isn't this in the RM itself? An oversight?

It's not necessary, it follows directly from the definition of "-", 
'Address, and 'Position as related above, for the default bit order case. In 
the non-default bit order, (2.a/2) isn't even true, so it wouldn't help to 
add such a statement in that case. ('Position defines the machine scalars to 
use in that case, and the meaning defies any simple explanation -- it's 
roughly the same as the meaning in the default-bit-order case, but not 
exactly. The RM can never lie.)

                                                   Randy.

Re: GNAT 4.4.5 Record Bit_Order Endian issues

[Simon Wright]

Stephen Leake <stephen_leake@stephe-leake.org> writes:

> Simon Wright <simon@pushface.org> writes:
>
>> Stephen Leake <stephen_leake@stephe-leake.org> writes:
>>
>>> awdorrin <awdorrin@gmail.com> writes:
>>>
>>>> I have a record with specification definition that starts, using big endian notation, with:
>>>>
>>>> for TRACK_RECORD_TYPE use
>>>>  record
>>>>   TRACK_ID at 0 range 0 .. 11;
>>>>   TRACK_ID_CNT at 1 range 4 .. 9;
>>>>    ...
>>>>
>>>>
>>>> The meaning being that the first 12 bits hold a track id and the next
>>>> 6 bits hold the Count value.
>>>
>>> If you are allocating consecutive  bits, then do this:
>>>
>>> for TRACK_RECORD_TYPE use
>>>    record
>>>      TRACK_ID     at 0 range  0 .. 11;
>>>      TRACK_ID_CNT at 0 range 12 .. 17;
>>>      ...
>>
>> I don't see why this makes a difference? 
>
> The best explanation I have seen is in
> http://www.ada-auth.org/ai-files/grab_bag/bitorder.pdf
>
> That introduces the concept of "machine scalar", which helps a lot.

What that paper leads _me_ to decide is that I will never tell the
compiler to use the non-default bit order. (I have a feeling that Dmitry
would agree with me). The compiler may get it "right", but I don't trust
myself to.

I had stupidly thought that 'use the non-default bit order' would mean
'within ths declaration, behave as though you are on an other-endian
machine'. OP's original declaration worked just fine on the big-endian
machine it was written for.

> The problem is what does "the first twelve bits" mean, when the
> "natural" size of a chunk of memory is 8 bits?
>
> In this code:
>
>    TRACK_ID at 0 range 0 .. 11;
>    TRACK_ID_CNT at 1 range 4 .. 9;
>
> if Storage_Unit = 8, the two fields overlap. I can't figure out a
> storage arrangement that makes these fields contiguous; can you draw a
> picture? Changing to a compiler/target with a different Storage_Size
> changes the meaning.

Certainly a different Storage_Size would screw things up. But which
processor in actual common use - that you're likely to want to
interoperate with - doesn't use 8?

If you were using a PowerPC, the layout would be

Byte    00000000111111112222222233333333
Bit     01234567012345670123456701234567
Element AAAAAAAAAAAABBBBBB--------------

What's so hard about that?

And I'm not sure that the Cohen paper isn't begging the question - in
"Avoiding noncontiguous bit fields" on p7 he says

   "Although bits in different bytes may correspond to adjacent bits in
   a machine scalar (i.e., the bits become adjacent when loaded into a
   register), there is no notion of two bits in different bytes of
   memory being adjacent."

and in the next para

   "Consequently, the notion of a range of bits only makes sense with
   respect to a machine scalar."

> In this code: 
>
>       TRACK_ID     at 0 range  0 .. 11;
>       TRACK_ID_CNT at 0 range 12 .. 17;
>
> it's obvious that the fields are contiguous, and it's up to the compiler
> to ensure that happens.

I would expect the engineer defining the interface to use 16- or 32-bit
'words' and try hard to ensure that fields aren't split across word
boundaries. And I would write field positions in terms of bit offsets
from the 'word'. I think I'm agreeing with you here!

> Why do you say this is buggy?

I now see that GNAT (well, recent GNATs) are behaving according to the
spec. So what I'm saying is that I think the spec (ie, the ARM) is
buggy. At the very least it could say <blink>HERE BE DRAGONS</blink>.

Re: GNAT 4.4.5 Record Bit_Order Endian issues

[Stephen Leake]

Simon Wright <simon@pushface.org> writes:

> Stephen Leake <stephen_leake@stephe-leake.org> writes:
>
>> The best explanation I have seen is in
>> http://www.ada-auth.org/ai-files/grab_bag/bitorder.pdf
>>
>> That introduces the concept of "machine scalar", which helps a lot.
>
> What that paper leads _me_ to decide is that I will never tell the
> compiler to use the non-default bit order. (I have a feeling that Dmitry
> would agree with me). The compiler may get it "right", but I don't trust
> myself to.

It is like learning a new language; it takes a while to get comfortable
with it. And you need a compiler that gets it right :).

> I had stupidly thought that 'use the non-default bit order' would mean
> 'within ths declaration, behave as though you are on an other-endian
> machine'. OP's original declaration worked just fine on the big-endian
> machine it was written for.

That is confusing bit endianness with byte endianness; they are not the
same thing, and the ARM does not make it clear. See my other post.

> And I'm not sure that the Cohen paper isn't begging the question - in
> "Avoiding noncontiguous bit fields" on p7 he says
>
>    "Although bits in different bytes may correspond to adjacent bits in
>    a machine scalar (i.e., the bits become adjacent when loaded into a
>    register), there is no notion of two bits in different bytes of
>    memory being adjacent."

Pointing out the difference between bit and byte endianness (it would
help if he had said that explicitly).

> and in the next para
>
>    "Consequently, the notion of a range of bits only makes sense with
>    respect to a machine scalar."

One of the features of 'machine scalar' is that it allows you to ignore
byte endianness while deciphering a record rep spec.

>> In this code: 
>>
>>       TRACK_ID     at 0 range  0 .. 11;
>>       TRACK_ID_CNT at 0 range 12 .. 17;
>>
>> it's obvious that the fields are contiguous, and it's up to the compiler
>> to ensure that happens.
>
> I would expect the engineer defining the interface to use 16- or 32-bit
> 'words' 

replace 'words' with 'machine scalar', and you've got it. 'words' has
always been a fuzzy concept; 'machine scalar' is much crisper.

> and try hard to ensure that fields aren't split across word
> boundaries. 

Exactly. Because otherwise the meaning is ambiguous.

> And I would write field positions in terms of bit offsets from the
> 'word'. I think I'm agreeing with you here!

Yes.

>> Why do you say this is buggy?
>
> I now see that GNAT (well, recent GNATs) are behaving according to the
> spec. So what I'm saying is that I think the spec (ie, the ARM) is
> buggy. At the very least it could say <blink>HERE BE DRAGONS</blink>.

Yes, the ARM is deficient (I don't think it's actually wrong, just not
as clear/helpful as it could be).

-- 
-- Stephe

Re: GNAT 4.4.5 Record Bit_Order Endian issues

[awdorrin]

So, now on Debian 7 with GNAT 4.6.

Current record definition starts with:

  for MY_RECORD_TYPE'Bit_Order use System.High_Order_First;
  for MY_RECORD_TYPE use
    record
      TRACK_ID_X                  at   0 range  0 .. 11;
      TRACK_ID_USE_COUNTR_X       at   0 range 12 .. 17;
      NORMAL_RAW_DATA_X           at   0 range 18 .. 18;
      TRACK_STATUS_X              at   0 range 19 .. 21;
      DEAD_RECKONING_FLG_X        at   0 range 22 .. 22;
      TRACK_CONVERGENCE_X         at   0 range 23 .. 23;
      BLOCKED_TRACK_FLAG_X        at   0 range 24 .. 24;
      TRACK_INRFRNCE_ALRT_X       at   0 range 25 .. 25;
      MNUVRING_TARGET_FLG_X       at   0 range 26 .. 26;
      DESGNATED_TRACK_FLG_X       at   0 range 27 .. 27;
      SPAREA                      at   0 range 28 .. 31;
   end record;

Within my code, I have something like:
  MY_TRK_REC.TRACK_ID_X := 1;
  MY_TRK_REC.TRACK_ID_USE_COUNTR_X :=1;

Examining things through GDB:

"p MY_TRK_REC.TRACK_ID_X'Address" returns 0xb5cd0cd2
"p MY_TRK_REC.SPAREA'Address" returns 0xb5cd0cd0

The record elements are being laid out backwards in memory, despite the use System.High_Order_First directive.

Looking at the rep-spec output from my compile I get:

for MY_RECORD_TYPE use record
   TRACK_ID_X            at   2 range  4 .. 15;
   TRACK_ID_USE_COUNTR_X at   1 range  6 .. 11;
   NORMAL_RAW_DATA_X     at   1 range  5 ..  5;
   TRACK_STATUS_X        at   1 range  2 ..  4;
   DEAD_RECKONING_FLG_X  at   1 range  1 ..  1;
   TRACK_CONVERGENCE_X   at   1 range  0 ..  0;
   BLOCKED_TRACK_FLAG_X  at   0 range  7 ..  7;
   TRACK_INRFRNCE_ALRT_X at   0 range  6 ..  6;
   MNUVRING_TARGET_FLG_X at   0 range  5 ..  5;
   DESGNATED_TRACK_FLG_X at   0 range  4 ..  4;
   SPAREA                at   0 range  0 ..  3;

So, despite my earlier post today, I'm back to believing that the GNAT4.6 implementation is buggy. (or not documented clearly)

Re: GNAT 4.4.5 Record Bit_Order Endian issues

[awdorrin]

To provide a bit more info from my last post, from the compilation log, when GNAT is processing the Bit_Order clause I see:

info: reverse bit order in machine scalar of length 32
info: little-endian range for component "TRACK_ID_X" is 20 .. 31
info: reverse bit order in machine scalar of length 32
info: little-endian range for component "TRACK_ID_USE_COUNTR_X" is 14 .. 19
info: reverse bit order in machine scalar of length 32
info: little-endian range for component "NORMAL_RAW_DATA_X" is 13 .. 13
info: reverse bit order in machine scalar of length 32
info: little-endian range for component "TRACK_STATUS_X" is 10 .. 12
info: reverse bit order in machine scalar of length 32
info: little-endian range for component "DEAD_RECKONING_FLG_X" is 9 .. 9
info: reverse bit order in machine scalar of length 32
info: little-endian range for component "TRACK_CONVERGENCE_X" is 8 .. 8
info: reverse bit order in machine scalar of length 32
info: little-endian range for component "BLOCKED_TRACK_FLAG_X" is 7 .. 7
info: reverse bit order in machine scalar of length 32
info: little-endian range for component "TRACK_INRFRNCE_ALRT_X" is 6 .. 6
info: reverse bit order in machine scalar of length 32
info: little-endian range for component "MNUVRING_TARGET_FLG_X" is 5 .. 5
info: reverse bit order in machine scalar of length 32
info: little-endian range for component "DESGNATED_TRACK_FLG_X" is 4 .. 4
info: reverse bit order in machine scalar of length 32
info: little-endian range for component "SPAREA" is 0 .. 3

This definition looks correct. For instance the first item in the record, and the first 12 bits of memory, should represent the TRACK_ID_X field.

From the info messages: 
little-endian range for component "TRACK_ID_X" is 20 .. 31

However, as indicated above, when the 'rep spec' is printed out, this is represented as: TRACK_ID_X at 2 range 4 .. 15;

Quite frustrating! :)

What I'd give for some simple conditional compile directives like there are in C...

Re: GNAT 4.4.5 Record Bit_Order Endian issues

[Stephen Leake]

awdorrin <awdorrin@gmail.com> writes:

> Quite frustrating! :)

Yes, it took me a while to really understand all of this

> What I'd give for some simple conditional compile directives like there are in C...

You know how those work, but don't yet know how the Ada way works. The
same was true of tasks and protected types when you started; would you
give those up now?

Give it time to sink in, and focus on the difference between bit and
byte endiannes..

-- 
-- Stephe

Re: GNAT 4.4.5 Record Bit_Order Endian issues

[Simon Wright]

awdorrin <awdorrin@gmail.com> writes:

> So, now on Debian 7 with GNAT 4.6.
>
> Current record definition starts with:
>
>   for MY_RECORD_TYPE'Bit_Order use System.High_Order_First;
>   for MY_RECORD_TYPE use
>     record
>       TRACK_ID_X                  at   0 range  0 .. 11;
>       TRACK_ID_USE_COUNTR_X       at   0 range 12 .. 17;
>       NORMAL_RAW_DATA_X           at   0 range 18 .. 18;
>       TRACK_STATUS_X              at   0 range 19 .. 21;
>       DEAD_RECKONING_FLG_X        at   0 range 22 .. 22;
>       TRACK_CONVERGENCE_X         at   0 range 23 .. 23;
>       BLOCKED_TRACK_FLAG_X        at   0 range 24 .. 24;
>       TRACK_INRFRNCE_ALRT_X       at   0 range 25 .. 25;
>       MNUVRING_TARGET_FLG_X       at   0 range 26 .. 26;
>       DESGNATED_TRACK_FLG_X       at   0 range 27 .. 27;
>       SPAREA                      at   0 range 28 .. 31;
>    end record;
>
> Within my code, I have something like:
>   MY_TRK_REC.TRACK_ID_X := 1;
>   MY_TRK_REC.TRACK_ID_USE_COUNTR_X :=1;
>
> Examining things through GDB:
>
> "p MY_TRK_REC.TRACK_ID_X'Address" returns 0xb5cd0cd2
> "p MY_TRK_REC.SPAREA'Address" returns 0xb5cd0cd0
>
> The record elements are being laid out backwards in memory, despite
> the use System.High_Order_First directive.

That is *because of* the use of the Bit_Order clause.

If you comment-out the Bit_Order clause and do the same, you'll find
that My_Trk_Rec.Track_Id_X'Address is the same as My_Trk_Rec'Address,
and My_Trk_Rec.Sparea'Address is 3 greater.

Re: GNAT 4.4.5 Record Bit_Order Endian issues

[awdorrin]

On Thursday, January 17, 2013 3:58:03 PM UTC-5, Simon Wright wrote:
> 
> That is *because of* the use of the Bit_Order clause.
> 
> If you comment-out the Bit_Order clause and do the same, you'll find
> that My_Trk_Rec.Track_Id_X'Address is the same as My_Trk_Rec'Address,
> and My_Trk_Rec.Sparea'Address is 3 greater.

There appears to be something else going on when trying to address outside of bits 0-7.

In order to get the record defined correctly, I had to:
1. Remove the Bit_Order = System.High_Order_First
2. Define the record as the following:
for TRACK_RECORD_TYPE use record
   TRACK_ID_X            at   0 range  0 .. 11;
   TRACK_ID_USE_COUNTR_X at   1 range  4 ..  9;
   NORMAL_RAW_DATA_X     at   2 range  2 ..  2;
   TRACK_STATUS_X        at   2 range  3 ..  5;
   DEAD_RECKONING_FLG_X  at   2 range  6 ..  6;
   TRACK_CONVERGENCE_X   at   2 range  7 ..  7;
   BLOCKED_TRACK_FLAG_X  at   3 range  0 ..  0;
   TRACK_INRFRNCE_ALRT_X at   3 range  1 ..  1;
   MNUVRING_TARGET_FLG_X at   3 range  2 ..  2;
   DESGNATED_TRACK_FLG_X at   3 range  3 ..  3;
   SPAREA                at   3 range  4 ..  7;

Things do not appear to work out properly when you try to define everything using 'at 0' and ranges 0-31.

Maybe I'm missing something and/or just have myself completely confused at this point, but I am not convinced that the record definition will map to the same locations when compiled on a big endian platform.

Re: GNAT 4.4.5 Record Bit_Order Endian issues

[awdorrin]

Given the previous definition, the mapping to memory is not what I am expecting. (Which is why I thought I needed the Bit_Order = System.High_Order_First)

Given: TRACK_ID_X at 0 range 0 .. 11;

MyTrkRec.Track_Id_X = 2748; -- 0xabc

Using GDB to debug and print the byte values:
On a little endian system, I get: 0xcb 0x0a
On a big endian system, using the original code, I get: 0xab 0xc0

Now, I expect the numbers to be reordered, due to the endian differences.

This is why I thought the Bit_Order flag was used to fix.
On Intel, using Bit_Order = System.High_Order_First,
I would have expected to see: 0xcb 0xa0

Re: GNAT 4.4.5 Record Bit_Order Endian issues

[J-P. Rosen]

Le 17/01/2013 23:16, awdorrin a �crit :
> This is why I thought the Bit_Order flag was used to fix.
> On Intel, using Bit_Order = System.High_Order_First,
> I would have expected to see: 0xcb 0xa0
I didn't follow the discussion closely, but from this remark it seems
you are confusing bit order and byte order.

High_Order_First is only about bit order, i.e. whether, in a 32 bits
value f.e., the leftmost bit is numbered 31 or 0. It has nothing to do
(and no effect) on the order in which the various bytes that make up
this value are stored in memory.

Regarding the use of "at": you can always use 0 and number the bits
starting from the beginning of the record (and bit numbers are not
limited to 31). "at" just saves you a multiplication by the number of
bits in a storage unit. It is there because you normally use
representation clauses in connection with some low level device, and the
idea was to allow to copy directly from your hardware reference manual,
using the same description.

-- 
J-P. Rosen
Adalog
2 rue du Docteur Lombard, 92441 Issy-les-Moulineaux CEDEX
Tel: +33 1 45 29 21 52, Fax: +33 1 45 29 25 00
http://www.adalog.fr

Re: GNAT 4.4.5 Record Bit_Order Endian issues

[Niklas Holsti]

On 13-01-18 08:15 , J-P. Rosen wrote:

> Regarding the use of "at": you can always use 0 and number the bits
> starting from the beginning of the record (and bit numbers are not
> limited to 31).

Except that if you use the non-default bit order, then the bit numbers
are limited by the number of bits in the "largest machine scalar", per
RM 13.5.1(10.2/2), except in a special case (10.3/2).

Which means, for a portable program where you don't know what the
default bit order will be on some platform, nor know what size scalars
that platform has, you had best limit the bit numbers to a very
conservative range.

I have stopped using record representation clauses in portable Ada
programs and prefer to access bit-fields by shifting and masking. Sorry
to say.

-- 
Niklas Holsti
Tidorum Ltd
niklas holsti tidorum fi
      .      @       .

Re: GNAT 4.4.5 Record Bit_Order Endian issues

[Stephen Leake]

awdorrin <awdorrin@gmail.com> writes:

> Given the previous definition, the mapping to memory is not what I am
> expecting. (Which is why I thought I needed the Bit_Order =
> System.High_Order_First)
>
> Given: TRACK_ID_X at 0 range 0 .. 11;
>
> MyTrkRec.Track_Id_X = 2748; -- 0xabc
>
> Using GDB to debug and print the byte values:
> On a little endian system, I get: 0xcb 0x0a
> On a big endian system, using the original code, I get: 0xab 0xc0
>
> Now, I expect the numbers to be reordered, due to the endian
> differences.
>
> On Intel, using Bit_Order = System.High_Order_First,
> I would have expected to see: 0xcb 0xa0

There are two different kinds of "endianness"; bit endianness, and
byte endianness. 

"bit endianness" is about how to interpret the bit numbers in record rep
specs.

"byte endianness" is about how bytes are ordered in memory

Cohen's paper, and the Bit_Order attribute, only address bit endianness,
not byte endianness.

So if you are writing a multi-byte field to a machine register, or to a
network stream, you need to do byte swapping; Bit_Order does not affect
that. 

> This is why I thought the Bit_Order flag was used to fix.

That is a common misconception. And the ARM is not clear here.

-- 
-- Stephe

Re: GNAT 4.4.5 Record Bit_Order Endian issues

[awdorrin]

On Thursday, January 17, 2013 5:16:36 PM UTC-5, awdorrin wrote:
> This is why I thought the Bit_Order flag was used to fix.
> 
> On Intel, using Bit_Order = System.High_Order_First,
> I would have expected to see: 0xcb 0xa0

I had a realization in the shower this morning:

The byte and bit ranges used in the specification clause is not describing memory locations, it is describing the location in the CPU register. The Bit_Order flag is being used to describe which end of the CPU register is bit zero.

So the 'machine scalar' term that was being used earlier is really refering to the size of the memory access used to load the register.

Due to the (byte level) endian differences, it would seem that it is not possible to describe the physical bit locations in memory, since it is the offset in the register that is being described.

In order to get the record memory contents ordered the way I want them to be, I'm going to have to play some byte swap games and translate the record memory contents into a 'big endian buffer'. 

Presumably, if the record contents can be defined in terms of a 32-bit 'machine scalar' - then this endian conversion can be done by starting at 'MyTrkRec'Address' and doing byte swaps on 4-byte (32-bit) groupings.

In C I'd simply make a 'uint32_t' pointer to the record address, and loop through till the end of the record doing 'htonl()' calls. But Ada doesn't make things quite so 'easy' ;-)

Fixing poorly designed legacy code is really not my favorite thing to do...

Re: GNAT 4.4.5 Record Bit_Order Endian issues

[awdorrin]

So, I think I figured out how to solve my problem.

Within the legacy code, the representation clause was used to define the memory locations, then the record was being copied to/from a buffer via an imported 'memcpy' call. It worked since the system was Big Endian, but obviously it is non-portable to a Little Endian system.

What I realize now is, if I define the record using 'Bit_Order = System.High_Order_First; and then define the record elements in terms of a chosen machine scalar type (ie. 32-bit)

Now, the record specification will define 'equivalent' record structures, when endian swapping is taken into account.

The following code is probably not the most elegant way to solve this (and I'm open to suggestions) but here it is:

if Standard'Default_Bit_Order = BIG_ENDIAN then
  -- legacy big endian byte copy
  MEMCPY( Target => Track_Address, Source => Track_Rec'Address, Size => Track_Rec'Size/8);
else
  -- little endian convert to big endian by 32-bit word
  declare
    type EndCvtArray is array(UInt32 range <> ) of UInt32;
    WrdCnt : constant UInt32 := Track_Rec'Size/32;
    Target : EndCvtArray(1..WrdCnt);
    for Target'Address use Track_Address; -- overlay
    Source : EndCvtArray(1..WrdCnt);
    for Source'Address use Track_Rec'Address; -- overlay
  begin
    for i in 1..WrdCnt loop
      Target(i) := SwapUInt32(Source(i));
    end loop;
  end;
end if;

function SwapUIn32 is new GNAT.Byte_Swapping.Swapped4(UInt32);

I'm guessing there might be some cleaner way to do this, perhaps using Stream IO. But I tried the code and it appears to work.

Once I understood that the representation clause wasn't defining the memory locations, this made a lot more sense.

Re: GNAT 4.4.5 Record Bit_Order Endian issues

[Stephen Leake]

awdorrin <awdorrin@gmail.com> writes:

> So, I think I figured out how to solve my problem.

Yes, I agree :)

> The following code is probably not the most elegant way to solve this
> (and I'm open to suggestions) but here it is:
>
> if Standard'Default_Bit_Order = BIG_ENDIAN then
>   -- legacy big endian byte copy
>   MEMCPY( Target => Track_Address, Source => Track_Rec'Address, Size => Track_Rec'Size/8);
> else
>   -- little endian convert to big endian by 32-bit word
>   declare
>     type EndCvtArray is array(UInt32 range <> ) of UInt32;
>     WrdCnt : constant UInt32 := Track_Rec'Size/32;
>     Target : EndCvtArray(1..WrdCnt);
>     for Target'Address use Track_Address; -- overlay
>     Source : EndCvtArray(1..WrdCnt);
>     for Source'Address use Track_Rec'Address; -- overlay
>   begin
>     for i in 1..WrdCnt loop
>       Target(i) := SwapUInt32(Source(i));
>     end loop;
>   end;
> end if;
>
> function SwapUIn32 is new GNAT.Byte_Swapping.Swapped4(UInt32);
>
> I'm guessing there might be some cleaner way to do this, perhaps using Stream IO. But I tried the code and it appears to work.
>
> Once I understood that the representation clause wasn't defining the memory locations, this made a lot more sense.

That's what I do, but I use my own byte_swap package (which I believe I
wrote before GNAT.Byte_Swapping was available).

My code doesn't use address overlays, but that means it makes copies on
the stack. For small data packets, this is not a problem.

-- 
-- Stephe

Re: GNAT 4.4.5 Record Bit_Order Endian issues

[Simon Wright]

awdorrin <awdorrin@gmail.com> writes:

> Presumably, if the record contents can be defined in terms of a 32-bit
> machine scalar' - then this endian conversion can be done by starting
> at 'MyTrkRec'Address' and doing byte swaps on 4-byte (32-bit)
> groupings.
>
> In C I'd simply make a 'uint32_t' pointer to the record address, and
> loop through till the end of the record doing 'htonl()' calls. But Ada
> doesn't make things quite so 'easy' ;-)

Provided all your machine scalars are 32-bits. I've come across layouts
where htons() was needed (twice).

I wrote a toy to give myself some confidence about this - at
http://dl.dropbox.com/u/34783908/Ada/endianness.adb

Re: GNAT 4.4.5 Record Bit_Order Endian issues

[quinot]

On Friday, January 18, 2013 1:24:02 PM UTC+1, awdorrin wrote:
 
> I had a realization in the shower this morning:

That's often where great ideas are hatched! :-)

> The byte and bit ranges used in the specification clause is not describing memory locations, it is describing the location in the CPU register. The Bit_Order flag is being used to describe which end of the CPU register is bit zero.

Right, this is exactly correct.

> Due to the (byte level) endian differences, it would seem that it is not possible to describe the physical bit locations in memory, since it is the offset in the register that is being described.

Correct. If you have a record where some components cross a storage unit boundary, then no standard rep clause can make it so that the record has the same representation in memory on both big endian and little endian platforms.

> In order to get the record memory contents ordered the way I want them to be, I'm going to have to play some byte swap games and translate the record memory contents into a 'big endian buffer'. 
>
> Presumably, if the record contents can be defined in terms of a 32-bit 'machine scalar' - then this endian conversion can be done by starting at 'MyTrkRec'Address' and doing byte swaps on 4-byte (32-bit) groupings.

Yes, that's a correct approach.

> In C I'd simply make a 'uint32_t' pointer to the record address, and loop through till the end of the record doing 'htonl()' calls. But Ada doesn't make things quite so 'easy' ;-)

Well, easy enough :-) You can overlap an array of Interfaces.Unsigned_32 over
your record, and use GNAT.Byte_Swapping.Swap4 on each element.

Also note that the latest development version of GNAT Pro introduces a new implementation-defined attribute Scalar_Storage_Order which precisely aims at relieving programmers from having to do such byte-swapping "by hand" and provide a means to force a record to have the same representation whether the target is big-endian or little-endian (http://www.adacore.com/developers/development-log/NF-71-L207-008-gnat/).


An article about this feature will appear next week in our Ada Gems series (http://www.adacore.com/adaanswers/gems/), and we also have submitted a technical paper on this matter to the upcoming Ada Europe 2013 conference.

Thomas Quinot.

Re: GNAT 4.4.5 Record Bit_Order Endian issues

[quinot]

On Tuesday, January 22, 2013 10:49:23 AM UTC+1, qui...@adacore.com wrote:
 
> An article about this feature will appear next week in our Ada Gems series (http://www.adacore.com/adaanswers/gems/), and we also have submitted a technical paper on this matter to the upcoming Ada Europe 2013 conference.

The gem is now on line:
http://www.adacore.com/adaanswers/gems/gem-140-bridging-the-endianness-gap

Thomas.

Re: GNAT 4.4.5 Record Bit_Order Endian issues

[Stephen Leake]

"Randy Brukardt" <randy@rrsoftware.com> writes:

> "AdaMagica" <christ-usch.grein@t-online.de> wrote in message 
> news:9dc6d5fe-3382-44dd-8460-e042bd6b808e@googlegroups.com...
> On Saturday, January 19, 2013 2:43:03 AM UTC+1, Stephen Leake wrote:
> ...
>>> Then 13.5.2 talks about the Position attribute.
>>> Which just retreives the mysterious "position" number, it doesn't say
>>> what it _means_.
>
> Huh? 13.5.2(2/2) says in part:
> "denotes Denotes the same value as R.C'Address – R'Address". That makes it
> pretty clear what it represents (for the Default_Bit_Order).

The full paragraph is:

2/2
R.C'Position

               {AI95-00133-01AI95-00133-01} If the nondefault bit
               ordering applies to the composite type, and if a
               component_clause specifies the placement of C, denotes
               the value given for the position of the
               component_clause; otherwise, denotes the same value as
               R.C'Address - R'Address. The value of this attribute is
               of the type universal_integer.

Note the 'otherwise'; that's means the value is R.C'Address - R'Address
only if position is _not_ specified in a rep clause.

> For the non-default bit order, it just is whatever you specify (and
> that's all we can say, because it *doesn't* have a clear physical
> meaning - it defines the machine scalars to use, which makes it a
> totally confusing mess if you try to work the other way).

Cohen's paper (http://www.ada-auth.org/ai-files/grab_bag/bitorder.pdf)
gives it a clear meaning in that case.

>>> > I agree that the RM is confusing in this area.
>>> > It's partly my fault.  :-(
>>> Ok. What paragraphs do we add? (in five years; better late than never :)
>
> I don't think there is anything that would help. We surely tried when we 
> created the non-default bit order stuff.

Do you think Cohen's paper makes it clear? I gather not. Why not?

If that's too complex to put in the RM, maybe the implementation advice
could include a reference to a copy of Cohen's paper on the adaic
website (somehow I doubt ISO standards allow web references).

-- 
-- Stephe

Re: GNAT 4.4.5 Record Bit_Order Endian issues

[Jeffrey Carter]

On 01/21/2013 08:21 PM, Stephen Leake wrote:
>
> 2/2
> R.C'Position
>
>                 {AI95-00133-01AI95-00133-01} If the nondefault bit
>                 ordering applies to the composite type, and if a
>                 component_clause specifies the placement of C, denotes
>                 the value given for the position of the
>                 component_clause; otherwise, denotes the same value as
>                 R.C'Address - R'Address. The value of this attribute is
>                 of the type universal_integer.
>
> Note the 'otherwise'; that's means the value is R.C'Address - R'Address
> only if position is _not_ specified in a rep clause.

No. This says

if A and B then
    D;
else
    E;
end if;

A = "the nondefault bit ordering applies to the composite type"
B = "a component_clause specifies the placement of C"
D = "denotes the value given for the position of the component_clause"
E = "denotes the same value as R.C'Address - R'Address"

The negation of "A and B" is "not A or not B". Thus E holds when the default bit 
ordering applies, even if the position is specified.

-- 
Jeff Carter
"To Err is human, to really screw up, you need C++!"
Stéphane Richard
63

Re: GNAT 4.4.5 Record Bit_Order Endian issues

[Stephen Leake]

Jeffrey Carter <spam.jrcarter.not@spam.not.acm.org> writes:

> On 01/21/2013 08:21 PM, Stephen Leake wrote:
>>
>> 2/2
>> R.C'Position
>>
>>                 {AI95-00133-01AI95-00133-01} If the nondefault bit
>>                 ordering applies to the composite type, and if a
>>                 component_clause specifies the placement of C, denotes
>>                 the value given for the position of the
>>                 component_clause; otherwise, denotes the same value as
>>                 R.C'Address - R'Address. The value of this attribute is
>>                 of the type universal_integer.
>>
>> Note the 'otherwise'; that's means the value is R.C'Address - R'Address
>> only if position is _not_ specified in a rep clause.
>
> No. This says
>
> if A and B then
>    D;
> else
>    E;
> end if;

Ah, missed that. I stand corrected. I did take symbolic algebra in
college - my professor would be disappointed ;).

So, this _never_ defines the meaning of the position given in a
component clause; it only defines the meaning of the 'Position
attribute. 

I suppose we could assume that 'Position always means the same thing as
the position specified in the component clause, but it would be better
if this (and 13.5.1) said that explicitly.

Actually, if we assume Cohen's paper applies, I believe it is still
R.C'Address - R'Address in the nondefault bit order case.

-- 
-- Stephe

Re: GNAT 4.4.5 Record Bit_Order Endian issues

[Randy Brukardt]

"Stephen Leake" <stephen_leake@stephe-leake.org> wrote in message 
news:854ni9c319.fsf@stephe-leake.org...
> "Randy Brukardt" <randy@rrsoftware.com> writes:
...
>Note the 'otherwise'; that's means the value is R.C'Address - R'Address
>only if position is _not_ specified in a rep clause.

Right, because if it is specified you use that value. But it *always* means 
R.C'Address - R'Address, whether it is specified or not; it's only that the 
*value* comes from the specification if there is one. So I take it you are 
complaining about a misplaced "otherwise".

I'd suggest you write up an Ada-Comment so it gets onto the agenda of the 
ARG to fix next time. (I personally think it is obvious, so I wouldn't worry 
about it, but people differ.)

...
>>>> > I agree that the RM is confusing in this area.
>>>> > It's partly my fault.  :-(
>>>> Ok. What paragraphs do we add? (in five years; better late than never 
>>>> :)
>>
>> I don't think there is anything that would help. We surely tried when we
>> created the non-default bit order stuff.
>
> Do you think Cohen's paper makes it clear? I gather not. Why not?

I read that paper three or four times and never understood any of it. The 
discussion of "machine scalars" in the AI and in the RM (and especially 
during the ARG meetings) makes more sense to me than anything in Cohen's 
paper -- and in all honesty, that isn't intuitive, either.

In any case, the Cohen paper is far too long to be used as part of the RM, 
and it surely defies condensation into a few easily understandable 
paragraphs. (If someone can do that, I'd welcome adding it to the AARM, I 
surely couldn't and I tried.)

> If that's too complex to put in the RM, maybe the implementation advice
> could include a reference to a copy of Cohen's paper on the adaic
> website (somehow I doubt ISO standards allow web references).

It's referenced from the AI, but as I said, I don't think it helps all that 
much.

                                               Randy.

Re: GNAT 4.4.5 Record Bit_Order Endian issues

[Stephen Leake]

"Randy Brukardt" <randy@rrsoftware.com> writes:

> "Stephen Leake" <stephen_leake@stephe-leake.org> wrote in message 
> news:854ni9c319.fsf@stephe-leake.org...
>> "Randy Brukardt" <randy@rrsoftware.com> writes:
> ...
>>Note the 'otherwise'; that's means the value is R.C'Address - R'Address
>>only if position is _not_ specified in a rep clause.
>
> Right, because if it is specified you use that value. But it *always* means
> R.C'Address - R'Address, whether it is specified or not; it's only that the 
> *value* comes from the specification if there is one. So I take it you are 
> complaining about a misplaced "otherwise".

Yes. 

> I'd suggest you write up an Ada-Comment so it gets onto the agenda of the 
> ARG to fix next time. (I personally think it is obvious, so I wouldn't worry 
> about it, but people differ.)

I'll do that, but I agree it's not a big deal, since R.C'Address -
R'Address is the only reasonable meaning.

>> Do you think Cohen's paper makes it clear? I gather not. Why not?
>
> I read that paper three or four times and never understood any of it. 

I'll have to read it again; its been a while.

> The discussion of "machine scalars" in the AI and in the RM (and
> especially during the ARG meetings) makes more sense to me than
> anything in Cohen's paper -- and in all honesty, that isn't intuitive,
> either.
>
> In any case, the Cohen paper is far too long to be used as part of the RM, 
> and it surely defies condensation into a few easily understandable 
> paragraphs. (If someone can do that, I'd welcome adding it to the AARM, I 
> surely couldn't and I tried.)

I'm thinking it would be sufficient to say something like:

    'position' specifies R.C'Address - R'Address (in units of Storage_Unit)

    If the nondefault bit order applies, each 'position' must specify the start
    of the machine scalar following the machine scalar used in the previous
    component.

But I'll work on it more. A couple of examples would help. And we need
to point out that this does not affect byte endianness.

-- 
-- Stephe

Re: GNAT 4.4.5 Record Bit_Order Endian issues

[Simon Wright]

Stephen Leake <stephen_leake@stephe-leake.org> writes:

> I'm thinking it would be sufficient to say something like:
>
>     'position' specifies R.C'Address - R'Address (in units of
>     Storage_Unit)
>
>     If the nondefault bit order applies, each 'position' must specify
>     the start of the machine scalar following the machine scalar used
>     in the previous component.

Either this, or must be the same as the machine scalar used in the
previous component (but of course this component might increase the size
of the required machine scalar; eg, the previous component addressed bit
0, this one addresses bit 31).

'previous' - although IMO it'd be unusual to order the component clauses
in non-ascending order (position*System.Storage_Unit + first_bit?), I
don't think the ARM forbids it.