Examining individual bytes of an integer

Examining individual bytes of an integer

[Henrik Härkönen]

I'd like to examine individual bytes of larger (in bit size) integer type. In C would probably move byte pointer etc. Also I'd need to have a way to construct the integer from individual bytes.

Would this be "the Ada-way", the Unchecked_Conversion?

---8<---

procedure Main is

   type Byte is mod 256;
   type Word is mod 2 ** 32;

   type E2 is array(1 .. 4) of Byte;
   
   Z: Word;
   Y: E2;
   
   function Convert_To_E2 is new Ada.Unchecked_Conversion (Source => Word,
                                                           Target => E2);

   function Convert_To_Word is new Ada.Unchecked_Conversion (Source => E2,
                                                             Target => Word);

begin
   Y := Convert_To_E2(Word(435343));
   Z := Convert_To_Word(Y);
end Main;

--->8---

At least that looks nice and neat, I could also use record with assigned names to bytes if necessary.

-Henrik

Re: Examining individual bytes of an integer

[Jeffrey R. Carter]

On 10/14/2018 09:15 PM, Henrik Härkönen wrote:
> 
>     type Byte is mod 256;
>     type Word is mod 2 ** 32;
> 
>     type E2 is array(1 .. 4) of Byte;
>     
>     function Convert_To_E2 is new Ada.Unchecked_Conversion (Source => Word,
>                                                             Target => E2);
> 
>     function Convert_To_Word is new Ada.Unchecked_Conversion (Source => E2,
>                                                               Target => Word);

Unchecked_Conversion is the Ada Way. Often it is a good idea to specify the 
sizes of all the types involved to ensure that the conversion is between 
equal-sized hunks of memory.

However, with the array, you don't know which index corresponds to which byte of 
the integer. You can examine the bytes of several values to determine this, but 
on a different platform or with a different compiler the correspondence may be 
different. Usually the lowest index corresponds to the lowest address, but that 
is not guaranteed by the language.

(Similar concerns arise when using a packed array of Boolean to access 
individual bits.)

> At least that looks nice and neat, I could also use record with assigned names to bytes if necessary.

Yes. With a record, you have the ability to use representation clauses to 
indicate which record field corresponds to which memory address. Combined with 
System.Default_Bit_Order, this allows you to portably determine the role of each 
byte in the integer.

-- 
Jeff Carter
"What's the amount of the insult?"
Never Give a Sucker an Even Break
104

Re: Examining individual bytes of an integer

[Niklas Holsti]

On 18-10-14 22:55 , Jeffrey R. Carter wrote:
> On 10/14/2018 09:15 PM, Henrik Härkönen wrote:
>>
>>     type Byte is mod 256;
>>     type Word is mod 2 ** 32;
>>
>>     type E2 is array(1 .. 4) of Byte;
>>         function Convert_To_E2 is new Ada.Unchecked_Conversion (Source
>> => Word,
>>                                                             Target =>
>> E2);
>>
>>     function Convert_To_Word is new Ada.Unchecked_Conversion (Source
>> => E2,
>>                                                               Target
>> => Word);
>
> Unchecked_Conversion is the Ada Way.

Or Interfaces.Unsigned_xx and shifts.

> Often it is a good idea to specify
> the sizes of all the types involved to ensure that the conversion is
> between equal-sized hunks of memory.

Yes, but GNAT normally gives a warning if the sizes differ.

> However, with the array, you don't know which index corresponds to which
> byte of the integer. You can examine the bytes of several values to
> determine this, but on a different platform or with a different compiler
> the correspondence may be different.

Yes: endianness (or a perverse array index order, see below).

> Usually the lowest index
> corresponds to the lowest address, but that is not guaranteed by the
> language.

Not guaranteed, but implied by the "Implementation advice":

- RM 13.3(11): X'address denotes the address of the first of the storage 
elements allocated to X.

- RM 13.3(14): For an array X, X'Address should point at the first 
component of the array [...].

Combining those two, it seems that the first element of an array should 
start at the first storage element (= lowest address) allocated to the 
array.

By reasonable induction  :-)  the second, third, etc. array elements 
follow in increasing address order. But not _guaranteed_, I agree.

-- 
Niklas Holsti
Tidorum Ltd
niklas holsti tidorum fi
       .      @       .

Re: Examining individual bytes of an integer

[AdaMagica]

Am Sonntag, 14. Oktober 2018 23:28:57 UTC+2 schrieb Niklas Holsti:
> Not guaranteed, but implied by the "Implementation advice":
> 
> - RM 13.3(11): X'address denotes the address of the first of the storage 
> elements allocated to X.
> 
> - RM 13.3(14): For an array X, X'Address should point at the first 
> component of the array [...].
> 
> Combining those two, it seems that the first element of an array should 
> start at the first storage element (= lowest address) allocated to the 
> array.
> 
> By reasonable induction  :-)  the second, third, etc. array elements 
> follow in increasing address order. But not _guaranteed_, I agree.

There is the old Dewar rule that no compiler is doing nonsense. It would be a very unfriendly compiler to not follow this advice.

Re: Examining individual bytes of an integer

[Niklas Holsti]

On 18-10-15 13:18 , AdaMagica wrote:
> Am Sonntag, 14. Oktober 2018 23:28:57 UTC+2 schrieb Niklas Holsti:
>> Not guaranteed, but implied by the "Implementation advice":
>>
>> - RM 13.3(11): X'address denotes the address of the first of the
>> storage elements allocated to X.
>>
>> - RM 13.3(14): For an array X, X'Address should point at the first
>>  component of the array [...].
>>
>> Combining those two, it seems that the first element of an array
>> should start at the first storage element (= lowest address)
>> allocated to the array.
>>
>> By reasonable induction  :-)  the second, third, etc. array
>> elements follow in increasing address order. But not _guaranteed_,
>> I agree.
>
> There is the old Dewar rule that no compiler is doing nonsense. It
> would be a very unfriendly compiler to not follow this advice.

As I remember it, Dewar's rule says that the Ada *RM* does not specify 
nonsense, meaning that if some reading of the RM seems to be nonsense, 
it is the wrong reading.

I don't remember Dewar saying anything similar about compilers.

-- 
Niklas Holsti
Tidorum Ltd
niklas holsti tidorum fi
       .      @       .

Re: Examining individual bytes of an integer

[Randy Brukardt]

"Niklas Holsti" <niklas.holsti@tidorum.invalid> wrote in message 
news:g2k5fbF9thnU1@mid.individual.net...
> On 18-10-15 13:18 , AdaMagica wrote:
>> Am Sonntag, 14. Oktober 2018 23:28:57 UTC+2 schrieb Niklas Holsti:
>>> Not guaranteed, but implied by the "Implementation advice":
>>>
>>> - RM 13.3(11): X'address denotes the address of the first of the
>>> storage elements allocated to X.
>>>
>>> - RM 13.3(14): For an array X, X'Address should point at the first
>>>  component of the array [...].
>>>
>>> Combining those two, it seems that the first element of an array
>>> should start at the first storage element (= lowest address)
>>> allocated to the array.
>>>
>>> By reasonable induction  :-)  the second, third, etc. array
>>> elements follow in increasing address order. But not _guaranteed_,
>>> I agree.
>>
>> There is the old Dewar rule that no compiler is doing nonsense. It
>> would be a very unfriendly compiler to not follow this advice.
>
> As I remember it, Dewar's rule says that the Ada *RM* does not specify 
> nonsense, meaning that if some reading of the RM seems to be nonsense, it 
> is the wrong reading.

That's correct.

> I don't remember Dewar saying anything similar about compilers.

Me either. Compilers do nonsense all of the time (usually not on purpose, 
though).

                           Randy.

Re: Examining individual bytes of an integer

[AdaMagica]

Am Montag, 15. Oktober 2018 22:18:16 UTC+2 schrieb Randy Brukardt:
> "Niklas Holsti" <niklas.holsti@tidorum.invalid> wrote in message 
> news:g2k5fbF9thnU1@mid.individual.net...
> > On 18-10-15 13:18 , AdaMagica wrote:
> >> There is the old Dewar rule that no compiler is doing nonsense. It
> >> would be a very unfriendly compiler to not follow this advice.
> >
> > As I remember it, Dewar's rule says that the Ada *RM* does not specify 
> > nonsense, meaning that if some reading of the RM seems to be nonsense, it 
> > is the wrong reading.
> 
> That's correct.
> 
> > I don't remember Dewar saying anything similar about compilers.
> 
> Me either. Compilers do nonsense all of the time (usually not on purpose, 
> though).

OK, I stay corrected. But I still claim that it would be extremely unfair of a compiler not to follow this advice (A'Address = A (A'First)'Address with increasing addresses for further components).

I'm apt to try a wager that Randy agrees. Praps I'm gonna lose again... Tant pis!

Re: Examining individual bytes of an integer

[Dmitry A. Kazakov]

On 2018-10-16 12:18, AdaMagica wrote:

> OK, I stay corrected. But I still claim that it would be extremely unfair of a compiler not to follow this advice (A'Address = A (A'First)'Address with increasing addresses for further components).

It is a real pain when writing storage pools that must do some 
bookkeeping around allocated objects. There should really be 
X'Allocation_Address there in addition.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: Examining individual bytes of an integer

[Randy Brukardt]

"Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> wrote in message 
news:pq4jjh$5ig$1@gioia.aioe.org...
> On 2018-10-16 12:18, AdaMagica wrote:
>
>> OK, I stay corrected. But I still claim that it would be extremely unfair 
>> of a compiler not to follow this advice (A'Address = A (A'First)'Address 
>> with increasing addresses for further components).
>
> It is a real pain when writing storage pools that must do some bookkeeping 
> around allocated objects. There should really be X'Allocation_Address 
> there in addition.

Meaning what? In Janus/Ada, the allocation address is always what 'Address 
returns. But since objects may be in multiple parts, that isn't necessarily 
useful for anything.

In any case, X(X'First)'Address always works to get the address of the first 
element of an array. If that's what you want, say so and there isn't going 
to be an issue.

                                        Randy.

Re: Examining individual bytes of an integer

[Dmitry A. Kazakov]

On 2018-10-17 00:35, Randy Brukardt wrote:
> "Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> wrote in message
> news:pq4jjh$5ig$1@gioia.aioe.org...
>> On 2018-10-16 12:18, AdaMagica wrote:
>>
>>> OK, I stay corrected. But I still claim that it would be extremely unfair
>>> of a compiler not to follow this advice (A'Address = A (A'First)'Address
>>> with increasing addresses for further components).
>>
>> It is a real pain when writing storage pools that must do some bookkeeping
>> around allocated objects. There should really be X'Allocation_Address
>> there in addition.
> 
> Meaning what?

Meaning the first address of the object, e.g. of the dope of an 
indefinite array as it was returned by the allocator:

    Allocation_Address := Pointer_To_Object.all'Address - Offset;

Presently the offset to the object beginning must be estimated during 
run-time.

Another, probably more ugly method is create a fake storage pool with 
Deallocate storing its address argument without doing actual 
deallocation. The object's access type is cast to the fake-pool-specific 
access type. Unchecked_Deallocation called. The stored value is the true 
address.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: Examining individual bytes of an integer

[briot.emmanuel]

> Meaning the first address of the object, e.g. of the dope of an 
> indefinite array as it was returned by the allocator:
> 
>     Allocation_Address := Pointer_To_Object.all'Address - Offset;
> 
> Presently the offset to the object beginning must be estimated during 
> run-time.

GNATCOLL has a package GNATCOLL.Storage_Pools.Headers which does some similar
low-level tricks.
When you use this storage pool, it will allocate extra bytes (for a header type that you
provide), before the object'Address.

GNATCOLL uses it for instance to store reference counting information without impacting
the declaration of objects, or changing what their 'Address means.

Emmanuel

Re: Examining individual bytes of an integer

[Dmitry A. Kazakov]

On 2018-10-17 11:47, briot.emmanuel@gmail.com wrote:
>> Meaning the first address of the object, e.g. of the dope of an
>> indefinite array as it was returned by the allocator:
>>
>>      Allocation_Address := Pointer_To_Object.all'Address - Offset;
>>
>> Presently the offset to the object beginning must be estimated during
>> run-time.
> 
> GNATCOLL has a package GNATCOLL.Storage_Pools.Headers which does some similar
> low-level tricks.
> When you use this storage pool, it will allocate extra bytes (for a header type that you
> provide), before the object'Address.

Right, this is the major use case: to allocate something in front of 
each object.

> GNATCOLL uses it for instance to store reference counting information without impacting
> the declaration of objects, or changing what their 'Address means.

I have an implementation of the linked list with links allocated in pool 
so that the list nodes have the type the original object as-is. Same 
approach.

P.S. It is kind of multiple inheritance for poor. Reference counts or 
links are "inherited" into the object's representation in the pool.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: Examining individual bytes of an integer

[Randy Brukardt]

"Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> wrote in message 
news:pq6qp0$1nbs$1@gioia.aioe.org...
> On 2018-10-17 00:35, Randy Brukardt wrote:
>> "Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> wrote in message
>> news:pq4jjh$5ig$1@gioia.aioe.org...
>>> On 2018-10-16 12:18, AdaMagica wrote:
>>>
>>>> OK, I stay corrected. But I still claim that it would be extremely 
>>>> unfair
>>>> of a compiler not to follow this advice (A'Address = A 
>>>> (A'First)'Address
>>>> with increasing addresses for further components).
>>>
>>> It is a real pain when writing storage pools that must do some 
>>> bookkeeping
>>> around allocated objects. There should really be X'Allocation_Address
>>> there in addition.
>>
>> Meaning what?
>
> Meaning the first address of the object, e.g. of the dope of an indefinite 
> array as it was returned by the allocator:
>
>    Allocation_Address := Pointer_To_Object.all'Address - Offset;
>
> Presently the offset to the object beginning must be estimated during 
> run-time.

I see. Janus/Ada allocates descriptors separately from the rest of the 
object (there will be two calls to Allocate in such a case), and the 
descriptor contains a pointer at the data. We did that in large part because 
it's necessary to support slice actuals: in that case, the bounds and data 
location can be different than those for the original object. (And we didn't 
want a lot of different representations, it's hard enough when there are 
just a handful.)

> Another, probably more ugly method is create a fake storage pool with 
> Deallocate storing its address argument without doing actual deallocation. 
> The object's access type is cast to the fake-pool-specific access type. 
> Unchecked_Deallocation called. The stored value is the true address.

Yuk. This is why 'Address returns the descriptor in Janus/Ada. You can 
always get to the data with indexing.

                                    Randy.

Re: Examining individual bytes of an integer

[Randy Brukardt]

"AdaMagica" <christ-usch.grein@t-online.de> wrote in message 
news:35f53cd9-4979-49b8-a5df-2c1cf0764507@googlegroups.com...
> Am Montag, 15. Oktober 2018 22:18:16 UTC+2 schrieb Randy Brukardt:
>> "Niklas Holsti" <niklas.holsti@tidorum.invalid> wrote in message
>> news:g2k5fbF9thnU1@mid.individual.net...
>> > On 18-10-15 13:18 , AdaMagica wrote:
>> >> There is the old Dewar rule that no compiler is doing nonsense. It
>> >> would be a very unfriendly compiler to not follow this advice.
>> >
>> > As I remember it, Dewar's rule says that the Ada *RM* does not specify
>> > nonsense, meaning that if some reading of the RM seems to be nonsense, 
>> > it
>> > is the wrong reading.
>>
>> That's correct.
>>
>> > I don't remember Dewar saying anything similar about compilers.
>>
>> Me either. Compilers do nonsense all of the time (usually not on purpose,
>> though).
>
> OK, I stay corrected. But I still claim that it would be extremely unfair 
> of a compiler not to follow this advice (A'Address = A (A'First)'Address 
> with increasing addresses for further components).
>
> I'm apt to try a wager that Randy agrees. Praps I'm gonna lose again... 
> Tant pis!

You would. For Janus/Ada, that's true only for statically constrained 
arrays. For an unconstrained array, the 'Address points at the array 
descriptor, and for dynamically constrained array, it points at the indirect 
address. That's the model for the entire compiler: all objects are 
statically sized, and if needed, the actual data is found by indirection.

Because of the Implementation Advice. I've considered changing 'Address to 
point at the first data item. But that then does not work for address 
clauses for unconstrained array types (those have to point at the 
descriptor). Nor does it work for code that needs to look at the descriptor 
(not much of that, of course). Given that pretty much no one should be using 
'Address anyway, this hasn't been an issue in practice, so it's not a high 
priority to deal with.

                                   Randy.

Re: Examining individual bytes of an integer

[Niklas Holsti]

On 18-10-17 01:33 , Randy Brukardt wrote:

> ... Given that pretty much no one should be using 'Address anyway,

Anyone doing HW interfacing with DMA I/O has to use 'Address, because 
that's what is programmed into the DMA channel. And that is quite common 
in embedded Ada programs that do not use an operating system for I/O.

I guess current users of Janus/Ada on MS Windows don't make such programs.

-- 
Niklas Holsti
Tidorum Ltd
niklas holsti tidorum fi
       .      @       .

Re: Examining individual bytes of an integer

[Randy Brukardt]

"Niklas Holsti" <niklas.holsti@tidorum.invalid> wrote in message 
news:g2o4gbF44ejU1@mid.individual.net...
> On 18-10-17 01:33 , Randy Brukardt wrote:
>
>> ... Given that pretty much no one should be using 'Address anyway,
>
> Anyone doing HW interfacing with DMA I/O has to use 'Address, because 
> that's what is programmed into the DMA channel. And that is quite common 
> in embedded Ada programs that do not use an operating system for I/O.
>
> I guess current users of Janus/Ada on MS Windows don't make such programs.

The *reason* to use 'Address and address clauses is to interface with 
hardware. That's fine, but there aren't that many people writing such 
interfaces. "No one" is a bit much, perhaps, but it should be way below 5%. 
(If it isn't, that says Ada is way under-utilized.)

In any case, indexing always gets the right answer with the possible 
exception of packed arrays. So there's not a lot of need or value to 
following that Implementation Advice.

                                                   Randy.

Re: Examining individual bytes of an integer

[Dmitry A. Kazakov]

On 2018-10-17 00:33, Randy Brukardt wrote:

> That's the model for the entire compiler: all objects are
> statically sized, and if needed, the actual data is found by indirection.

That makes me missing RM rule (not advice): for any instance of

    new X

Allocate of the corresponding storage pool is called exactly once.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: Examining individual bytes of an integer

[Randy Brukardt]

"Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> wrote in message 
news:pq6r7i$1o47$1@gioia.aioe.org...
> On 2018-10-17 00:33, Randy Brukardt wrote:
>
>> That's the model for the entire compiler: all objects are
>> statically sized, and if needed, the actual data is found by indirection.
>
> That makes me missing RM rule (not advice): for any instance of
>
>    new X
>
> Allocate of the corresponding storage pool is called exactly once.
>
> -- 
> Regards,
> Dmitry A. Kazakov
> http://www.dmitry-kazakov.de 

Re: Examining individual bytes of an integer

[Randy Brukardt]

"Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> wrote in message 
news:pq6r7i$1o47$1@gioia.aioe.org...
> On 2018-10-17 00:33, Randy Brukardt wrote:
>
>> That's the model for the entire compiler: all objects are
>> statically sized, and if needed, the actual data is found by indirection.
>
> That makes me missing RM rule (not advice): for any instance of
>
>    new X
>
> Allocate of the corresponding storage pool is called exactly once.

I would fight that at every level, since it would make a useful 
implementation of discriminated records impossible. Ada has always been very 
clear that non-contiguous representations are allowed.

It might make sense to introduce some sort of aspect to require a contigious 
representation for an array or record, so long as that's only required to be 
supported on sensible types.

In particular, this type should work with raising Storage_Error or some 
other exception on all Ada compilers:

     type Rec (D : Natural) is record
         Arr : String (1 .. D);
     end record;

     C0Rec : constant Rec(0) := (D => 0, Arr => "");
     C1Rec : constant Rec(1) := (D => 1, Arr => "R");
     A_Rec : Rec := C0Rec;

     A_Rec := C1Rec;
     A_Rec := (D => 3, Arr => "RLB");

A type like this pretty much requires discontiguous representations and 
reallocation on assignment.

                                          Randy.

Re: Examining individual bytes of an integer

[Dmitry A. Kazakov]

On 2018-10-18 00:10, Randy Brukardt wrote:
> "Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> wrote in message
> news:pq6r7i$1o47$1@gioia.aioe.org...
>> On 2018-10-17 00:33, Randy Brukardt wrote:
>>
>>> That's the model for the entire compiler: all objects are
>>> statically sized, and if needed, the actual data is found by indirection.
>>
>> That makes me missing RM rule (not advice): for any instance of
>>
>>     new X
>>
>> Allocate of the corresponding storage pool is called exactly once.
> 
> I would fight that at every level, since it would make a useful
> implementation of discriminated records impossible. Ada has always been very
> clear that non-contiguous representations are allowed.

I know, but on the other hand it would make a lot of useful applications 
non-portable.

> It might make sense to introduce some sort of aspect to require a contigious
> representation for an array or record, so long as that's only required to be
> supported on sensible types.

I would rather consider a general method for handling all sorts of type 
constraints in a regular manner available for the programmer.

This should include stripping constraints when statically known when the 
object is passed as a parameter and adding constraints when a statically 
constrained object gets dynamically constrained (e.g. slices, conversion 
to T'Class). We could use hidden parameters for this (such objects 
cannot be used for interfacing with other languages anyway, if that is a 
concern).

Another point is calculation of such constraints in a separate body 
(possibly static) when doing assignment or allocation. Presently 
reasonable object initialization is impossible when constraints are 
dynamic. The same case: static checks for dimensioned arithmetic are 
impossible because the constraint (dimension) must be calculated in the 
same body as the operation itself, but we want it calculated and checked 
at compile time instead.

> In particular, this type should work with raising Storage_Error or some
> other exception on all Ada compilers:
> 
>       type Rec (D : Natural) is record
>           Arr : String (1 .. D);
>       end record;
> 
>       C0Rec : constant Rec(0) := (D => 0, Arr => "");
>       C1Rec : constant Rec(1) := (D => 1, Arr => "R");
>       A_Rec : Rec := C0Rec;
> 
>       A_Rec := C1Rec;
>       A_Rec := (D => 3, Arr => "RLB");
> 
> A type like this pretty much requires discontiguous representations and
> reallocation on assignment.

It is no loss, Storage_Error is OK to me. IMO no built-in type shall use 
hidden dynamic allocation, except when required by the target OS (e.g. 
for tasks). Anything that is impossible to implement otherwise must be 
moved to the library (like Unbounded_String).

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: Examining individual bytes of an integer

[AdaMagica]

Am Mittwoch, 17. Oktober 2018 00:33:03 UTC+2 schrieb Randy Brukardt:
> You would.
Thanx for the detailed explanation. I don't know anything about compiler writing, I just try to understand what the RM says. And normally (OK, what's normal) ImplAdv is followed. But I know one has to check the compiler doc to see whether a certain implementation follows the advice.

You never stop learning.

So this was for A'Address = A (A'First)'Address, which is not necessarily true.

There remains the other statement:
"By reasonable induction  :-)  the second, third, etc. array elements
follow in increasing address order. But not _guaranteed_, I agree."

I still claim that this has too be true (of course with a grain of salt for packead arrays).

Re: Examining individual bytes of an integer

[Randy Brukardt]

"AdaMagica" <christ-usch.grein@t-online.de> wrote in message 
news:39be8a25-cc06-4db4-9481-7f484077522d@googlegroups.com...
...
>There remains the other statement:
>"By reasonable induction  :-)  the second, third, etc. array elements
>follow in increasing address order. But not _guaranteed_, I agree."
>
>I still claim that this has too be true (of course with a grain of salt for 
>packead arrays).

I agree, it would make indexing calculations really expensive if it isn't 
true. And I don't see a compiler author doing that on purpose. Packed arrays 
are a different kettle of fish, of course.

                                         Randy.

Re: Examining individual bytes of an integer

[Niklas Holsti]

On 18-10-19 00:18 , Randy Brukardt wrote:
> "AdaMagica" <christ-usch.grein@t-online.de> wrote in message
> news:39be8a25-cc06-4db4-9481-7f484077522d@googlegroups.com...
> ...
>> There remains the other statement:
>> "By reasonable induction  :-)  the second, third, etc. array elements
>> follow in increasing address order. But not _guaranteed_, I agree."
>>
>> I still claim that this has too be true (of course with a grain of salt for
>> packead arrays).
>
> I agree, it would make indexing calculations really expensive if it isn't
> true. And I don't see a compiler author doing that on purpose.

Oh, I'm not so sure... Storing by *reverse* index order should not be 
much more expensive, if at all. I seem to remember hearing about a 
Pascal compiler in which record components were stored in reverse 
lexical declaration order, just because of the way the compiler turned 
out, or so I understood.

And the people working on probabilistic WCET analysis want to randomize 
lots of things, such as cache address mappings, and are creating HW 
processors with such features. Perhaps they would like to randomize also 
array component order...

-- 
Niklas Holsti
Tidorum Ltd
niklas holsti tidorum fi
       .      @       .

Re: Examining individual bytes of an integer

[AdaMagica]

Am Freitag, 19. Oktober 2018 08:27:19 UTC+2 schrieb Niklas Holsti:
> On 18-10-19 00:18 , Randy Brukardt wrote:
> > "AdaMagica" wrote in message
> > news:39be8a25-cc06-4db4-9481-7f484077522d@googlegroups.com...
> > ...
> >> There remains the other statement:
> >> "By reasonable induction  :-)  the second, third, etc. array elements
> >> follow in increasing address order. But not _guaranteed_, I agree."
> >>
> >> I still claim that this has too be true (of course with a grain of salt for
> >> packead arrays).
> >
> > I agree, it would make indexing calculations really expensive if it isn't
> > true. And I don't see a compiler author doing that on purpose.
> 
> Oh, I'm not so sure... Storing by *reverse* index order should not be 
> much more expensive, if at all.

Hm, but what would be the purpose or gain? Just for the fun of it?

> I seem to remember hearing about a 
> Pascal compiler in which record components were stored in reverse 
> lexical declaration order, just because of the way the compiler turned 
> out, or so I understood.

But records are still another kettle of fish (echoing Randy). Ada does not prescribe how the components are allocated - they may be rearranged in any order.

Re: Examining individual bytes of an integer

[Niklas Holsti]

On 18-10-19 11:28 , AdaMagica wrote:
> Am Freitag, 19. Oktober 2018 08:27:19 UTC+2 schrieb Niklas Holsti:
>> On 18-10-19 00:18 , Randy Brukardt wrote:
>>> "AdaMagica" wrote in message
>>> news:39be8a25-cc06-4db4-9481-7f484077522d@googlegroups.com...
>>> ...
>>>> There remains the other statement:
>>>> "By reasonable induction  :-)  the second, third, etc. array elements
>>>> follow in increasing address order. But not _guaranteed_, I agree."
>>>>
>>>> I still claim that this has too be true (of course with a grain of salt for
>>>> packead arrays).
>>>
>>> I agree, it would make indexing calculations really expensive if it isn't
>>> true. And I don't see a compiler author doing that on purpose.
>>
>> Oh, I'm not so sure... Storing by *reverse* index order should not be
>> much more expensive, if at all.
>
> Hm, but what would be the purpose or gain? Just for the fun of it?

Who knows? Perhaps the target machine has some funny instructions that 
make it easier to check index ranges for reverse indexing? The Pascal 
compiler had no particular reason for placing the components in reverse 
order, and for sure it surprised the users who noticed it (I don't 
recall why they noticed it; perhaps they tried to interface with C code 
or with HW).

Or maybe the compiler sees that the array is always traversed from high 
indices to low, and the machine has some memory prefetching that works 
better when data is accessed in increasing address order?

-- 
Niklas Holsti
Tidorum Ltd
niklas holsti tidorum fi
       .      @       .

Re: Examining individual bytes of an integer

[Niklas Holsti]

On 18-10-14 22:15 , Henrik Härkönen wrote:
> I'd like to examine individual bytes of larger (in bit size)
> integer type. In C would probably move byte pointer etc. Also
> I'd need to have a way to construct the integer from individual bytes.
>
> Would this be "the Ada-way", the Unchecked_Conversion?

It depends... for this particular case, I would say "no", for the 
endianness reasons that Jeffrey described.

I would use the unsigned (modular) integer types in the predefined 
package Interfaces, such as Interfaces.Unsigned_32, which come with 
shift and rotate operations.

For example, to extract the most significant octet of a 32-bit unsigned 
integer with value 435343, I would do (assuming "use Interfaces"):

    Z : Unsigned_32 := 435343;

    MS_Octet : constant Unsigned_32 := Shift_Right (Z, 24);

To compose an unsigned 32-bit integer from the octets 77, 88, 99, and 
22, listed in big-endian order, I would do:

    Z : Unsigned_32 :=
          Shift_Left (77, 24)
       or Shift_Left (88, 16)
       or Shift_Left (99,  8)
       or             22;

(If you need lots of such expressions, you could define an "&" operator 
to make that look like 77 & 88 & 99 & 22.)

Using a record type with named octet components (and a Bit_Order clause) 
is ok, too, but then one needs some Unchecked_Conversions.

If you really want to inspect the structure of integers in memory, 
address by address, then I think the only way is the same as in C: make 
an octet pointer to the memory representation and read octet by octet. 
(But note that the Storage_Unit for some strange machines may not be 8 
bits.)

(I have become very sensitive to endianness problems because my daily 
work involves writing Ada programs targeted to big-endian SPARC 
machines, but tested initially on little-endian PCs.)

> ---8<---
>
> procedure Main is
>
>    type Byte is mod 256;
>    type Word is mod 2 ** 32;
>
>    type E2 is array(1 .. 4) of Byte;
>
>    Z: Word;
>    Y: E2;
>
>    function Convert_To_E2 is new Ada.Unchecked_Conversion (Source => Word,
>                                                            Target => E2);
>
>    function Convert_To_Word is new Ada.Unchecked_Conversion (Source => E2,
>                                                              Target => Word);
>
> begin
>    Y := Convert_To_E2(Word(435343));
>    Z := Convert_To_Word(Y);
> end Main;
>
> --->8---
>
> At least that looks nice and neat, I could also use record with
> assigned names to bytes if necessary.
>
> -Henrik
>

-- 
Niklas Holsti
Tidorum Ltd
niklas holsti tidorum fi
       .      @       .

Re: Examining individual bytes of an integer

[Niklas Holsti]

On 18-10-15 00:04 , Niklas Holsti wrote:
> On 18-10-14 22:15 , Henrik Härkönen wrote:
>> I'd like to examine individual bytes of larger (in bit size)
>> integer type. In C would probably move byte pointer etc. Also
>> I'd need to have a way to construct the integer from individual bytes.

Adding a little to my earlier answer:

> If you really want to inspect the structure of integers in memory,
> address by address, then I think the only way is the same as in C: make
> an octet pointer to the memory representation and read octet by octet.
> (But note that the Storage_Unit for some strange machines may not be 8
> bits.)

If you want to use the above method, the tools are in 
System.Storage_Elements (address arithmetic) and 
System.Address_To_Access_Conversions (convert an address into an access 
value).

-- 
Niklas Holsti
Tidorum Ltd
niklas holsti tidorum fi
       .      @       .

Re: Examining individual bytes of an integer

[Simon Wright]

Niklas Holsti <niklas.holsti@tidorum.invalid> writes:

> (I have become very sensitive to endianness problems because my daily
> work involves writing Ada programs targeted to big-endian SPARC
> machines, but tested initially on little-endian PCs.)

My favourite was the BE system with a BE interface card to an LE network
protocol .. the interface card was only interested in the 16-bit message
header components, so it converted *the whole packet* as if it only
contained 16-bit values. Which it didn't.

Re: Examining individual bytes of an integer

[Björn Lundin]

On 2018-10-14 23:04, Niklas Holsti wrote:
> 
> (I have become very sensitive to endianness problems because my daily
> work involves writing Ada programs targeted to big-endian SPARC
> machines, but tested initially on little-endian PCs.)


I usually decide how to work with the bytes (little- or bigEndian)
and implement something like this


  type Byte is range 0..255;
  for  Byte'Size use 8;

  type Byte_Array is array (Positive range <>) of Byte;

  subtype Byte_Array_2 is Byte_Array(1..2);
  subtype Byte_Array_4 is Byte_Array(1..4);


 procedure Swap4 (Bytes: in out Byte_Array_4) is
    Result : Byte_Array_4;
  begin
    Result := (1 => Bytes(4), 2 => Bytes(3),
               3 => Bytes(2), 4 => Bytes(1));
    Bytes := Result;
  end Swap4;
  -------------------------------------------------

 procedure Swap2 (Bytes: in out Byte_Array_2) is
    Result : Byte_Array_2;
  begin
    Result := (1 => Bytes(2), 2 => Bytes(1));
    Bytes := Result;
  end Swap2;
  -------------------------------------------------
  function Mirror (Data : Byte_Array) return Byte_Array is
  begin
    case Data'Length is
    when 2 =>
       declare
         Tmp : Byte_Array_2 := Data;
       begin
         case System.Default_Bit_Order is
           when System.High_Order_First => null;       --ppc
           when System. Low_Order_First => Swap2(Tmp); --x86
         end case;
         return Tmp;
       end;
    when 4 =>
       declare
         Tmp : Byte_Array_4 := Data;
       begin
         case System.Default_Bit_Order is
           when System.High_Order_First => null;       --ppc
           when System. Low_Order_First => Swap4(Tmp); --x86
         end case;
         return Tmp;
       end;
    when others => raise Constraint_Error with
           "not implemented lenght" & Data'Length'img;
    end case;
  end Mirror;
  -------------------------------------------------


  Then I call Mirror once (at input),
  and have the bytes in the same order no matter endianess.
  The I can modify the bytes as I please.

  I also call Mirror again at output, so I can write the
  modifications in a machine friendly way.


  (code is taken from project, but altered here without compilation,
   especially the 4byte part)

-- 
--
Björn

Re: Examining individual bytes of an integer

[Matt Borchers]

Does anybody ever do this kind of thing?

procedure Main is

   type Byte is mod 256;
   type Word is mod 2 ** 32;

   type E2 is array(1 .. 4) of Byte;
   
   Z: Word;
   Y: E2;
   for Y'Address use Z'Address;

begin
   --here you can reference Z or Y directly.
   --two types overlaid on the same memory location.
   --of course if it matters, you still have to know the endian to know if Y(1) or Y(4) is the low-order byte of Z.
end Main; 

Re: Examining individual bytes of an integer

[Matt Borchers]

This may require two additional lines after the type declarations:
1) for Byte'Size use 8;
2) for Word'Size use 32;

I'm not sure if those would be necessary, but I think so.

Re: Examining individual bytes of an integer

[Simon Wright]

Matt Borchers <mattborchers@gmail.com> writes:

> This may require two additional lines after the type declarations:
> 1) for Byte'Size use 8;
> 2) for Word'Size use 32;
>
> I'm not sure if those would be necessary, but I think so.

Or you could use the types in Interfaces, where this is done
already. Those types are intended for exactly this job.

Re: Examining individual bytes of an integer

[Dmitry A. Kazakov]

On 2018-10-15 01:42, Matt Borchers wrote:
> Does anybody ever do this kind of thing?

Sure

> procedure Main is
> 
>     type Byte is mod 256;
>     type Word is mod 2 ** 32;
> 
>     type E2 is array(1 .. 4) of Byte;
>     
>     Z: Word;
>     Y: E2;

       pragma Import (Ada, Y); -- Kill initialization, if any

>     for Y'Address use Z'Address;

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: Examining individual bytes of an integer

[Simon Wright]

Matt Borchers <mattborchers@gmail.com> writes:

> of course if it matters, you still have to know the endian to
> know if Y(1) or Y(4) is the low-order byte of Z.

   case System.Bit_Order is
      when System.High_Order_First =>
         ...
      when System.Low_Order_First =>
         ...
   end case;

Re: Examining individual bytes of an integer

[Simon Wright]

Simon Wright <simon@pushface.org> writes:

> Matt Borchers <mattborchers@gmail.com> writes:
>
>> of course if it matters, you still have to know the endian to
>> know if Y(1) or Y(4) is the low-order byte of Z.
>
>    case System.Bit_Order is

System.Default_Bit_Order

>       when System.High_Order_First =>
>          ...
>       when System.Low_Order_First =>
>          ...
>    end case;

Re: Examining individual bytes of an integer

[Randy Brukardt]

"Matt Borchers" <mattborchers@gmail.com> wrote in message 
news:4f2fb3ba-9cca-46ad-b970-ccbcb7d08a2c@googlegroups.com...
> Does anybody ever do this kind of thing?

Yes, and they should be drawn-and-quartered. :-)

> procedure Main is
>
>   type Byte is mod 256;
>   type Word is mod 2 ** 32;
>
>   type E2 is array(1 .. 4) of Byte;
>
>   Z: Word;
>   Y: E2;
>   for Y'Address use Z'Address;

This sort of thing either destroys optimization (if it choses to pay 
attention to it) or destroys your code (if it doesn't). In particular, 
Z'Address is not required to be meaningful for objects that aren't aliased 
or by-reference (which Z is not above). (See 13.3(16).) Z might get placed 
into a register or optimized away completely, in which case Z'Address 
references nothing.

You could of course declare Z as aliased:

    Z : aliased Word;

but then you're guarenteeing worse code than you would get from one of the 
other methods. (Address clauses are almost always implemented with a level 
of indirection, because of their potentially dynamic nature.)]

                                     Randy.

Re: Examining individual bytes of an integer

[Henrik Härkönen]

Thanks everyone for the very insightful answers! Learned a bunch again. :)

In this particular case I don't need to worry about the endianness, it's more of an "artistic" approach to see the individual bytes. :) But I can certainly see Niklas' point with different systems.

I was also reading about the memory overlays, those that Matt described, but it was mentioned to be a bit dangerous, if one makes writes with it and is not careful about the correct sizes of source and target. 

https://en.wikibooks.org/wiki/Ada_Programming/Type_System#Overlays

Re: Examining individual bytes of an integer

[Dmitry A. Kazakov]

On 2018-10-14 21:15, Henrik Härkönen wrote:
> I'd like to examine individual bytes of larger (in bit size) integer type. In C would probably move byte pointer etc. Also I'd need to have a way to construct the integer from individual bytes.
> 
> Would this be "the Ada-way", the Unchecked_Conversion?

I would say no. I am customary using arithmetic operations in order to 
serialize and de-serialize integers to/from bit, octet, whatever medium 
stream. It is the only way which is 100% safe and portable.

As an example, here is stream I/O of signed integers using chain code 
representation:

    http://www.dmitry-kazakov.de/ada/strings_edit.htm#10.1

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de