Endianness independance

Endianness independance

[Lionel.DRAGHI]

| -----Message d'origine-----
| De: Marin David Condic [mailto:mcondic.auntie.spam@acm.org]
...
| prone because of common data representation - then Ada is an abysmal
| failure. The Ada standard has almost nothing at all to say about the
| representation of data as it gets out of the program and it 
| has gaping holes
...
One big step towards interoperability and portability could be
Endian-Portable record representation.

Does someone knows if Norman H. Cohen proposal for Endian-Portable record
representation clauses is examined within language revision?

Lionel Draghi.

Re: Endianness independance

[Stephen Leake]

Lionel.DRAGHI@fr.thalesgroup.com writes:

> One big step towards interoperability and portability could be
> Endian-Portable record representation.
> 
> Does someone knows if Norman H. Cohen proposal for Endian-Portable record
> representation clauses is examined within language revision?

I was not aware that he had made a proposal that required language
revision. 

I use a variant of his technique now, with good results (once I
actually _know_ what the hardware endianness is :).

-- 
-- Stephe

RE: Endianness independance

[Lionel.DRAGHI]

| -----Message d'origine-----
| De: Stephen Leake [mailto:Stephen.A.Leake@nasa.gov]
..
| I was not aware that he had made a proposal that required language
| revision. 
| 
| I use a variant of his technique now, with good results (once I
| actually _know_ what the hardware endianness is :).
| 
You are probably speaking of his technique to get Bit_Order attribute effect
with Ada 83 code, presented in the "Endian-Independent Record Representation
Clauses" paper?

I was speaking of "A Proposal for Endian-Portable Record Representation
Clauses", which suggest to get non default bit order support mandatory, and
the byte offset (the number between "at" and "range" within representation
clauses) related to machine scalar instead of byte.

As i understand it, this proposal cause no upward compatibility problem, but
i have no idea of the rework it could cause within compilers. 

-- 
Lionel Draghi.

Re: Endianness independance

[Marin David Condic]

This isn't really the proper domain for a programming language. There are
too many machine architectures, communication mechanisms and data
representations out there for a language to try to standardize them all. The
language wants to be targetable to a multitude of hardware, so it has to be
abstracted from underlying representation. By picking one form of endianness
over another or one form of floating point representation over another or
whatever other possible representations one might select, it would be ruling
out use of the language on a whole range of equipment. This isn't really the
place one tries to create interoperability.

Realistically, the answer is to assess the communications mechanisms needed,
establish a clearly defined protocol & representation and then build your
software to deal with that. Ada can be very good at handling the
representation selected, but it really can't define the representation for
you.

MDC
--
======================================================================
Marin David Condic
I work for: http://www.belcan.com/
My project is: http://www.jsf.mil/

Send Replies To: m c o n d i c @ a c m . o r g

    "Going cold turkey isn't as delicious as it sounds."
        -- H. Simpson
======================================================================

<Lionel.DRAGHI@fr.thalesgroup.com> wrote in message
news:mailman.6.1046445067.1343.comp.lang.ada@ada.eu.org...
> ...
> One big step towards interoperability and portability could be
> Endian-Portable record representation.
>
> Does someone knows if Norman H. Cohen proposal for Endian-Portable record
> representation clauses is examined within language revision?
>
> Lionel Draghi.

Re: Endianness independance

[Randy Brukardt]

Lionel.DRAGHI@fr.thalesgroup.com wrote in message ...
>I was speaking of "A Proposal for Endian-Portable Record Representation
>Clauses", which suggest to get non default bit order support mandatory,
and
>the byte offset (the number between "at" and "range" within
representation
>clauses) related to machine scalar instead of byte.
>
>As i understand it, this proposal cause no upward compatibility
problem, but
>i have no idea of the rework it could cause within compilers.


His proposal was turned into AI-133.

Unfortunately, he stopped attending meetings about that time. When we
(the ARG) considered his proposal, we couldn't understand the the
details (which were never written up in the standard form), and
essentially assigned someone to make a complete proposal. It's still on
someone's action list, but it seems likely that other priorities (such
as coming to a consensus on a solution to the mutually dependent types
problem) will keep it from being addressed again.

                    Randy Brukardt
                    ARG Editor

Re: Endianness independance

[Amir Yantimirov]

"Marin David Condic" <mcondic.auntie.spam@acm.org> wrote in message news:<b3oqhj$h83$1@slb9.atl.mindspring.net>...
> This isn't really the proper domain for a programming language. There are
> too many machine architectures, communication mechanisms and data
> representations out there for a language to try to standardize them all. The
> language wants to be targetable to a multitude of hardware, so it has to be
> abstracted from underlying representation. By picking one form of endianness
> over another or one form of floating point representation over another or
> whatever other possible representations one might select, it would be ruling
> out use of the language on a whole range of equipment. This isn't really the
> place one tries to create interoperability.

I think this sentence isn't true at least for 5 years already. 
Interoperability is so crucial that today where is no excuse for
existance of  hardware what don't support common data representation.

As you pointed in other thread this wasn't a goal when Ada was
created. Contrary, such implicit "genericity" is a common trait of all
older languages. And opposite, a newer languages as Java and C#
exactly specify representation of integer and floating point types.
This is a direct answer to common demand of developers.

So we lost seductive ability to write single code for various
representation of data, do we? The obvious answer is that genericity
should be explicit only.

> <Lionel.DRAGHI@fr.thalesgroup.com> wrote in message
> news:mailman.6.1046445067.1343.comp.lang.ada@ada.eu.org...
> > ...
> > One big step towards interoperability and portability could be
> > Endian-Portable record representation.
> >
> > Does someone knows if for Endian-Portable record
> > representation clauses is examined within language revision?
> >
> > Lionel Draghi.

I don't know Norman H. Cohen proposal but that hou I see it can be
done.

Primitive types are defined as bitstrings of various length. Endianess
is a way to store primitives to memory/stream. Only small/big
endianess are supported by compiler, all other should be implemented
by hands. Any type/variable has one of endianess setting: neutral
(assumes native), small-endian, big-endian.

For example:
type TGAFileHeader = record <endianess=small, alignment=1>
     (all staff)
end record;

You works with record as usual and toss it as bytes sequence to file
and this works right on any system.

Amir Yantimirov
http://www174.pair.com/yamir/programming/

Re: Endianness independance

[Simon Wright]

amir@iae.nsk.su (Amir Yantimirov) writes:

> I think this sentence isn't true at least for 5 years already.
> Interoperability is so crucial that today where is no excuse for
> existance of hardware what don't support common data representation.

On the one side you have Intel. On the other SPARC and PowerPC (well,
as usually configured on eg iMacs). There isn't a common
representation; it's up to software to make sure of interoperability
when you need it.

> As you pointed in other thread this wasn't a goal when Ada was
> created. Contrary, such implicit "genericity" is a common trait of
> all older languages. And opposite, a newer languages as Java and C#
> exactly specify representation of integer and floating point types.
> This is a direct answer to common demand of developers.

I don't know about C#, but it's not obvious that its creators would
have been bothered about execution on non-Intel hardware.

It is not at all likely that a Java int has the same representation at
runtime on a PC as on an iBook. It would be painfully slow on one of
them (probably the PC!).

What you do get is support from the networking infrastructure (RMI?)
to convert the data to a common format "on the wire" and back to the
native format at the other end.

Where is "bit 0"? I always thought it was the least significant bit
(binary 1), but (depending on the programming/hardware conventions)
you may find it's the _most_ significant bit.


I do agree that, most of the time, what you need is for all this cruft
to be hidden. Every so often, though, it will creep out and bite you.

Re: Endianness independance

[Marin David Condic]

Well, but if hardware is not standardized, then why would you expect
languages to be standardized? (With respect to data representations.)
Language X can say "All integers of any kind will be 32-bit,
twos-compliment, network byte order....." That's nice for platforms where
that is always available and the only integer type available. That means you
can't use Language X on a lot of hardware platforms that don't support that
type. Or if the hardware has other types available, you can't use those.
Where does that get you?

And even if you went so far as to thoroughly dictate the exact precise
representation of all data within the program - right down to the number of
electrons it takes to make a bit equal to one :-) - how is that going to
guarantee interoperability? Program X built by manufacturer X running on box
X thinks the data going down the wire looks like this.... Program Y built by
manufacturer Y running on box Y thinks the data looks like that..... Both
have the same kinds of integers and floats and strings, but they're all in a
different order. How does the language/compiler solve that?

All other things being equal, it would be nice for a language like Ada to
have some reassurances about data representations - but that's what
representation clauses and data types can do already. You *still* have to
define clearly what the communication traffic is going to look like and then
make sure your Ada code lines up with that. The fact that Ada does give you
rep clauses helps you insure you've got the right representation of data for
the communication traffic defined, but it can't guarantee anything. Once the
ones and zeros are outside of the language's control, all bets are off. So
adding things to the standard isn't really going to help. And that was my
original point about how Ada isn't going to eliminate or reduce the chances
that some data sent by one program is going to be misinterpreted by another
program. It can't check what the other program's understanding of things is.

MDC
--
======================================================================
Marin David Condic
I work for: http://www.belcan.com/
My project is: http://www.jsf.mil/

Send Replies To: m c o n d i c @ a c m . o r g

    "Going cold turkey isn't as delicious as it sounds."
        -- H. Simpson
======================================================================

Amir Yantimirov <amir@iae.nsk.su> wrote in message
news:5115eb96.0303010248.1b2b8d37@posting.google.com...
>
> I think this sentence isn't true at least for 5 years already.
> Interoperability is so crucial that today where is no excuse for
> existance of  hardware what don't support common data representation.
>
> As you pointed in other thread this wasn't a goal when Ada was
> created. Contrary, such implicit "genericity" is a common trait of all
> older languages. And opposite, a newer languages as Java and C#
> exactly specify representation of integer and floating point types.
> This is a direct answer to common demand of developers.
>
> So we lost seductive ability to write single code for various
> representation of data, do we? The obvious answer is that genericity
> should be explicit only.
>

Re: Endianness independance

[Jeffrey Creem]

"Simon Wright" <simon@pushface.org> wrote in message
news:x7vheanwb32.fsf@smaug.pushface.org...
> amir@iae.nsk.su (Amir Yantimirov) writes:
>
>
> I don't know about C#, but it's not obvious that its creators would
> have been bothered about execution on non-Intel hardware.
>
> It is not at all likely that a Java int has the same representation at
> runtime on a PC as on an iBook. It would be painfully slow on one of
> them (probably the PC!).
>

Incorrect. Java int is Big endian no matter which platform you are on....And
for me at least, it remains painfully slow (though I have a much lower
threshold of pain than most people for this sort of thing)...

Re: Endianness independance

[Simon Wright]

"Jeffrey Creem" <jeff@thecreems.com> writes:

> "Simon Wright" <simon@pushface.org> wrote in message
> news:x7vheanwb32.fsf@smaug.pushface.org...
> > amir@iae.nsk.su (Amir Yantimirov) writes:
> >
> >
> > I don't know about C#, but it's not obvious that its creators would
> > have been bothered about execution on non-Intel hardware.
> >
> > It is not at all likely that a Java int has the same representation at
> > runtime on a PC as on an iBook. It would be painfully slow on one of
> > them (probably the PC!).
> >
> 
> Incorrect. Java int is Big endian no matter which platform you are on....And
> for me at least, it remains painfully slow (though I have a much lower
> threshold of pain than most people for this sort of thing)...

Good grief.

Still, I was right about which was likely to be worst ..

I wonder what GCJ does? .. but not a lot.

Re: Endianness independance

[Amir Yantimirov]

"Marin David Condic" <mcondic.auntie.spam@acm.org> wrote in message news:<b3qa6q$thh$1@slb0.atl.mindspring.net>...
> Well, but if hardware is not standardized, then why would you expect
> languages to be standardized? (With respect to data representations.)
> Language X can say "All integers of any kind will be 32-bit,
> twos-compliment, network byte order....." That's nice for platforms where
> that is always available and the only integer type available. That means you
> can't use Language X on a lot of hardware platforms that don't support that
> type. Or if the hardware has other types available, you can't use those.
> Where does that get you?

First, hardware in a large degree IS follow de facto standards, except
some marginal examples. There isn't any gain to be different and
uncompatible. And I wonder what reasons were for hardware be so
diverse in the past. I think, twos-compliment, 8 bit per byte, little-
and big-endian crowds pretty much covers together 99.9% of all
systems.

Second, data representation is only model. Processor with 36 bit word
perfectly capable to operate with 8, 16, 32, 64 bits integers. The
only difference is performance. And for most cases interoperability is
far far more important. Same for hypothetical future processors with
5-state elements.

> And even if you went so far as to thoroughly dictate the exact precise
> representation of all data within the program - right down to the number of
> electrons it takes to make a bit equal to one :-) - how is that going to
> guarantee interoperability? Program X built by manufacturer X running on box
> X thinks the data going down the wire looks like this.... Program Y built by
> manufacturer Y running on box Y thinks the data looks like that..... Both
> have the same kinds of integers and floats and strings, but they're all in a
> different order. How does the language/compiler solve that?

We solves that by hand without much brain efforts so the task to
automate it is not difficult by itself. Ideally we should only mark
sertain types and interfaces that deals with communications as having
particular endianess and thats all. But including notion of endianess
(as kind of storage specifier) in existent Ada type system seems
impossible to me.

By the way, chips of different manufactures with different programs
already swarms on any PC. One my colleague feeds NVidia GeForce chip
with data, other struggles to share some algorithm between central
processor and custom multimedia chip. Guess, that language they use.
;)

Amir Yantimirov
http://www174.pair.com/yamir/programming/

Re: Endianness independance

[Marin David Condic]

I agree - solve it "by hand". That is to say, one establishes the
communications protocol and tests to make sure the programs involved work
with that protocol. But remember that the original notion was that somehow
Ada was invented to make sure that when a new device was plugged into some
big military system, you wouldn't have a case where it would misinterpret
the data and cause a failure. My contention was that A) it wasn't invented
for that purpose and B) the language can't really solve that problem. The
language can't check the representations of things that exist outside of the
language. If you have thousands of messages flying around through a system
you are always going to have some unusual cases generated by unusual needs
and it is not possible to anticipate all those circumstances from within a
language standard. You have to manually determine what the messages are
supposed to look like and test to be sure the software understands them.
Often there are corner cases or things that are difficult to test or unusual
enough that they get missed. Hence, the possibility of errors.

A language might *help* minimizing representation problems because if
everyone is using the same thing they are at least starting from a common
base. But the representation of data outside the language isn't defined by
the language and attempting to do so is only going to hamstring the language
& keep it from being usable in a large variety of circumstances. For
example, Ada's selection of ASCII for character representation pretty much
insured that it would not become a popular language on IBM mainframes where
EBCDIC ruled the roost. Had the standard gone further and tried to define
external representations for more things, they would have just kept ruling
out more and more architectures on which it would be difficult or impossible
to meet the standard. And none of this would result in more
interoperatbility because the problems tend to come up from inadequate
understanding on the part of the programmers about the data being dealt with
rather than some intrinsic data representations that the language can't
check anyway.

MDC
--
======================================================================
Marin David Condic
I work for: http://www.belcan.com/
My project is: http://www.jsf.mil/

Send Replies To: m c o n d i c @ a c m . o r g

    "Going cold turkey isn't as delicious as it sounds."
        -- H. Simpson
======================================================================

Amir Yantimirov <amir@iae.nsk.su> wrote in message
news:5115eb96.0303020149.4d438e40@posting.google.com...
>
> We solves that by hand without much brain efforts so the task to
> automate it is not difficult by itself. Ideally we should only mark
> sertain types and interfaces that deals with communications as having
> particular endianess and thats all. But including notion of endianess
> (as kind of storage specifier) in existent Ada type system seems
> impossible to me.
>

RE: Endianness independance

[Lionel.DRAGHI]

| -----Message d'origine-----
| De: Marin David Condic [mailto:mcondic.auntie.spam@acm.org]
...
| 
| Realistically, the answer is to assess the communications 
| mechanisms needed,
| establish a clearly defined protocol & representation and 
| then build your
| software to deal with that. Ada can be very good at handling the
| representation selected, but it really can't define the 
| representation for
| you.

I agree with this. 
What i am complaining about, is that you can't just rely on representation
clauses to have interoperability between the same code compiled on two
different platform, and exchanging data through a stream socket for exemple.
I think that the representation/size/alignment clauses could give enough
informations to the compiler to ensure interoperability (provided RM
modifications). 

Am i wrong? As said Amir Yantimirov:
"But including notion of endianess (as kind of storage specifier) 
in existent Ada type system seems impossible to me."

Note that Norman Cohen's paper do not address the whole interoperability
problem, he just propose a way to portably specify "a data layout". However,
this is, i think, the most difficult part of the job.

Lionel Draghi 

Re: Endianness independance

[Stephen Leake]

"Marin David Condic" <mcondic.auntie.spam@acm.org> writes:

> Well, but if hardware is not standardized, then why would you expect
> languages to be standardized? (With respect to data representations.)

The point is that Ada provides a _standard_, _single_ representation
clause that works for both kinds of hardware:

--  Abstract :
--
--  Define constants to reflect hardware bit and word endianness in
--  record representation clauses. Obviously, this file is highly
--  system-dependent.
--
with System;
package SAL.Endianness is
   pragma Pure;

   --  This is for gnat on a Mongoose processor
   System_Name : constant System.Name := System.SYSTEM_NAME_GNAT;

   type Byte_Order_Type is (Big_Endian, Little_Endian);
   Byte_Order : constant Byte_Order_Type := Big_Endian;

   --  The Mongoose documentation numbers bits with LSB = 0. However,
   --  the byte order is Big_Endian, so the Ada compiler assumes bits
   --  are numbered with MSB = 0, since that allows representation
   --  clauses with fields that cross natural boundaries.

   Bit_Order      : constant := -1; -- 1 or -1
   High_Bit_First : constant :=  1; -- 0 or  1
   Low_Bit_First  : constant :=  0; -- 1 or  0

   --  Use one of these, corresponding to the natural size of the
   --  record you are laying out.
   LSBit_8        : constant :=  7; -- 0 or  7
   LSBit_16       : constant := 15; -- 0 or 15
   LSBit_32       : constant := 31; -- 0 or 31
   LSBit_56       : constant := 55; -- 0 or 55

end SAL.Endianness;

package Foo is
   type B_Field_Control_Type is record
      --  [1] Table 4.2.8
      Spare_1 : Interfaces_More.Unsigned_29;
      Data    : Interfaces_More.Unsigned_3;
   end record;

   for B_Field_Control_Type use record
      Spare_1 at 0 range
         Low_Bit_First  * (LSBit_32 + Bit_Order *  3) + High_Bit_First * (LSBit_32 + Bit_Order * 31) ..
         High_Bit_First * (LSBit_32 + Bit_Order *  3) + Low_Bit_First  * (LSBit_32 + Bit_Order * 31);

      Data at 0 range
         Low_Bit_First  * (LSBit_32 + Bit_Order *  0) + High_Bit_First * (LSBit_32 + Bit_Order *  2) ..
         High_Bit_First * (LSBit_32 + Bit_Order *  0) + Low_Bit_First  * (LSBit_32 + Bit_Order *  2);

   end record;
   for B_Field_Control_Type'Size use 32;
end foo;

In C, you have to use the preprocessor to get this.

In Ada, it would be nice if you could say:

for B_Field_Control_Type'Bit_Order use Little_Bit_First;

But you can't, because the compiler would have to know whether the
data is "naturally" 32, 16, or 8 bit (that's what the "LSBit_32" is
for above).

-- 
-- Stephe

Re: Endianness independance

[Stephen Leake]

Lionel.DRAGHI@fr.thalesgroup.com writes:

> | -----Message d'origine-----
> | De: Marin David Condic [mailto:mcondic.auntie.spam@acm.org]
> ...
> | 
> | Realistically, the answer is to assess the communications 
> | mechanisms needed,
> | establish a clearly defined protocol & representation and 
> | then build your
> | software to deal with that. Ada can be very good at handling the
> | representation selected, but it really can't define the 
> | representation for
> | you.
> 
> I agree with this. 
> What i am complaining about, is that you can't just rely on representation
> clauses to have interoperability between the same code compiled on two
> different platform, and exchanging data through a stream socket for exemple.
> I think that the representation/size/alignment clauses could give enough
> informations to the compiler to ensure interoperability (provided RM
> modifications). 

It is not possible to get compatible stream representations via
representation clauses.

You must override 'Write and 'Read to get that. 

The reason is that representation clauses can't swap bytes within
scalars (integers and floats), as is required by endian-independant
stream representations.

> Am i wrong? 

No.

> As said Amir Yantimirov: "But including notion of endianess (as kind
> of storage specifier) in existent Ada type system seems impossible
> to me."

The problem is there are _two_ requirements for endianness;

1) match the native hardware

2) match the stream representation

The compiler matches the native hardware for scalars, and rep clauses
allow you to match the native hardware for records.

Stream attributes allow you to match the stream representation. This
allows for more complex things than just endianness.

> Note that Norman Cohen's paper do not address the whole
> interoperability problem, he just propose a way to portably specify
> "a data layout". 

Correct. He addresses 1) above, but not 2).

> However, this is, i think, the most difficult part of the job.

The most difficult part is the one you haven't figured out yet;
everything else is easy :).

-- 
-- Stephe

Re: Endianness independance

[Marin David Condic]

Well, yes, of course. Ada has excellent facilities for dealing with
representation and I did mention something about this elsewhere. My point is
that if System X programmed with Compiler X, Version X to target hardware X
thinks it is supposed to be sending a 16 bit integer to System Y programmed
with Compiler Y, Version Y to target hardware Y and Y thinks its supposed to
be receiving a 32 bit integer, you'll have a problem. Compiler X can't check
Source Code Y to make sure it got it right. The language could forbid the
very existence of 32 bit integers and solve the problem - except that Y
would then get programmed in something else. The only way to deal with the
issue is with some kind of verification external to the language. The
language definition can't fix this - except possibly by eliminating possible
representations and that only makes it useless in some contexts.

MDC
--
======================================================================
Marin David Condic
I work for: http://www.belcan.com/
My project is: http://www.jsf.mil/

Send Replies To: m c o n d i c @ a c m . o r g

    "Going cold turkey isn't as delicious as it sounds."
        -- H. Simpson
======================================================================

Stephen Leake <Stephen.A.Leake@nasa.gov> wrote in message
news:uu1ekigg5.fsf@nasa.gov...
>
> The point is that Ada provides a _standard_, _single_ representation
> clause that works for both kinds of hardware:
>

RE: Endianness independance

[Lionel.DRAGHI]

| -----Message d'origine-----
| De: Stephen Leake [mailto:Stephen.A.Leake@nasa.gov]
...
| It is not possible to get compatible stream representations via
| representation clauses.
| 
| You must override 'Write and 'Read to get that. 
| 
| The reason is that representation clauses can't swap bytes within
| scalars (integers and floats), as is required by endian-independant
| stream representations.
| 
This is what i called the "easy" part of the job. 
But, obviously i suppose the compiler will do the job. If i must override
Stream attributes, i won't consider it so easy :-)

I can't believe it's impossible, after all, this is actually done by Glade,
isn't it?

Lionel Draghi.

Re: Endianness independance

[David C. Hoos]

----- Original Message -----
From: <Lionel.DRAGHI@fr.thalesgroup.com>
To: <comp.lang.ada@ada.eu.org>
Sent: Monday, March 03, 2003 11:52 AM
Subject: RE: Endianness independance


> | -----Message d'origine-----
> | De: Stephen Leake [mailto:Stephen.A.Leake@nasa.gov]
> ...
> | It is not possible to get compatible stream representations via
> | representation clauses.
> |
> | You must override 'Write and 'Read to get that.
> |
> | The reason is that representation clauses can't swap bytes within
> | scalars (integers and floats), as is required by endian-independant
> | stream representations.
> |
> This is what i called the "easy" part of the job.
> But, obviously i suppose the compiler will do the job. If i must override
> Stream attributes, i won't consider it so easy :-)
>
> I can't believe it's impossible, after all, this is actually done by
Glade,
> isn't it?
Yes, Glade does it, but the only thing Glade has to worry about is that
the stream representation is independent of hardware platform and OS.

Glade does this by providing a replacement body for the
System.Stream_Attributes package.

However, I have done a lot of endianess-independent work where a
specific network representation is required -- i.e., where an Ada
program has to communicate with a program writen in a language
not necessarily known to me.

My personal preference has been to care not at all about memory
representation of records, but to override the 'Write and 'Read
attributes of record components as required to produce/interpret the
required network representation.

In the case of memory-mapped IO interfaces, I read/write the raw data
from/to a "memory stream," using my overriding stream attributes.

Overriding stream attibutes is not all that difficult -- since it is done at
the record component level.  It is only necessary that the components
for which the default stream attributes are overridden begin and end
on octet boundaries.

When this is not the case, I define a record type to aggregate the
components that do not both begin and end on octet boundaries into
a type that does, and then use that type as the component of the main
record.

The actual implementation of the stream attributes frequently makes
use of a combination of unchecked conversion and instantiations of
generic procedures that manipulate arrays of bits.

>
> Lionel Draghi.
>
> _______________________________________________
> comp.lang.ada mailing list
> comp.lang.ada@ada.eu.org
> http://ada.eu.org/mailman/listinfo/comp.lang.ada
>

Re: Endianness independance

[Pascal Obry]

Lionel.DRAGHI@fr.thalesgroup.com writes:

> I can't believe it's impossible, after all, this is actually done by Glade,
> isn't it?

Yes it is, using XDR encoding.

Pascal.

-- 

--|------------------------------------------------------
--| Pascal Obry                           Team-Ada Member
--| 45, rue Gabriel Peri - 78114 Magny Les Hameaux FRANCE
--|------------------------------------------------------
--|         http://perso.wanadoo.fr/pascal.obry
--| "The best way to travel is by means of imagination"
--|
--| gpg --keyserver wwwkeys.pgp.net --recv-key C1082595

Re: Endianness independance

[Jeffrey Carter]

Stephen Leake wrote:
> "Marin David Condic" <mcondic.auntie.spam@acm.org> writes:
> 
> 
>>Well, but if hardware is not standardized, then why would you expect
>>languages to be standardized? (With respect to data representations.)
> 
> 
> The point is that Ada provides a _standard_, _single_ representation
> clause that works for both kinds of hardware:
> 
> --  Abstract :
> --
> --  Define constants to reflect hardware bit and word endianness in
> --  record representation clauses. Obviously, this file is highly
> --  system-dependent.
> --
> with System;
> package SAL.Endianness is
>    pragma Pure;
> 
>    --  This is for gnat on a Mongoose processor
>    System_Name : constant System.Name := System.SYSTEM_NAME_GNAT;
> 
>    type Byte_Order_Type is (Big_Endian, Little_Endian);
>    Byte_Order : constant Byte_Order_Type := Big_Endian;
> 
>    --  The Mongoose documentation numbers bits with LSB = 0. However,
>    --  the byte order is Big_Endian, so the Ada compiler assumes bits
>    --  are numbered with MSB = 0, since that allows representation
>    --  clauses with fields that cross natural boundaries.
> 
>    Bit_Order      : constant := -1; -- 1 or -1
>    High_Bit_First : constant :=  1; -- 0 or  1
>    Low_Bit_First  : constant :=  0; -- 1 or  0
> 
>    --  Use one of these, corresponding to the natural size of the
>    --  record you are laying out.
>    LSBit_8        : constant :=  7; -- 0 or  7
>    LSBit_16       : constant := 15; -- 0 or 15
>    LSBit_32       : constant := 31; -- 0 or 31
>    LSBit_56       : constant := 55; -- 0 or 55
> 
> end SAL.Endianness;

You can make this system independent by using System.Default_Bit_Order, 
which, thanks to an AI (I don't remember which, but you can look it up) 
is static.

-- 
Jeff Carter
"If I could find a sheriff who so offends the citizens of Rock
Ridge that his very appearance would drive them out of town ...
but where would I find such a man? Why am I asking you?"
Blazing Saddles

RE: Endianness independance

[Lionel.DRAGHI]

| -----Message d'origine-----
| De: David C. Hoos [mailto:david.c.hoos.sr@ada95.com]
...
| Yes, Glade does it, but the only thing Glade has to worry 
| about is that
| the stream representation is independent of hardware platform and OS.
| 
| Glade does this by providing a replacement body for the
| System.Stream_Attributes package.
| 
| However, I have done a lot of endianess-independent work where a
| specific network representation is required -- i.e., where an Ada
| program has to communicate with a program writen in a language
| not necessarily known to me.
| 
OK, but i don't pretend to get this general case taken into account within
the language scope, altrough some 
for My_Stream'representation use XDR; 
could be great! 

My point is just about interoperability between the same Ada code compiled
on two platform. I want both exe to put the same thing in the stream.
It's not the case even with a representation clause, and i feel this is
abnormal. 
I know it's far from easy to achieve, but i thought Norman's proposal was
helping a lot...

Lionel

Re: Endianness independance

[David C. Hoos, Sr.]

----- Original Message ----- 
From: <Lionel.DRAGHI@fr.thalesgroup.com>
To: <comp.lang.ada@ada.eu.org>
Sent: March 04, 2003 5:34 AM
Subject: RE: Endianness independance


> 
> 
> | -----Message d'origine-----
> | De: David C. Hoos [mailto:david.c.hoos.sr@ada95.com]
> ...
> | Yes, Glade does it, but the only thing Glade has to worry 
> | about is that
> | the stream representation is independent of hardware platform and OS.
> | 
> | Glade does this by providing a replacement body for the
> | System.Stream_Attributes package.
> | 
> | However, I have done a lot of endianess-independent work where a
> | specific network representation is required -- i.e., where an Ada
> | program has to communicate with a program writen in a language
> | not necessarily known to me.
> | 
> OK, but i don't pretend to get this general case taken into account within
> the language scope, altrough some 
> for My_Stream'representation use XDR; 
> could be great! 
> 
> My point is just about interoperability between the same Ada code compiled
> on two platform. I want both exe to put the same thing in the stream.
> It's not the case even with a representation clause, and i feel this is
> abnormal. 
> I know it's far from easy to achieve, but i thought Norman's proposal was
> helping a lot...
> 
Representation clauses are neither the issue, nor the answer to the problem.

If all you want is platform-independence, then replace the file s-stratt.adb
in your standard gnat distribution with the one from the same version of
glade, and re-compile and rebuild the library files (.a), and you have
what you want.  Now, as long as your types are declared the same on both
platforms (with no representation clauses) you will have platform-
independence.

I'm not sure why GNAT didn't do this from the beginning -- perhaps because
they didn't provide an implementation of the Distributed Systems Annex at
the beginning.  Also, there is a small performance penalty paid for
shuffling bits around to achieve platform independence.
 

Re: Endianness independance

[John Harbaugh]

If you are looking for an implementation that is platform and compiler
independent, take a look at the December 1999 Ada Letters.  Mike Mardis
wrote a very good article titled "Endian-Safe Record Representation
Clauses for Ada Programs."

Cheers,

	- John Harbaugh

"David C. Hoos, Sr." wrote:
> 
> ----- Original Message -----
> From: <Lionel.DRAGHI@fr.thalesgroup.com>
> To: <comp.lang.ada@ada.eu.org>
> Sent: March 04, 2003 5:34 AM
> Subject: RE: Endianness independance
> 
> >
> >
> > | -----Message d'origine-----
> > | De: David C. Hoos [mailto:david.c.hoos.sr@ada95.com]
> > ...
> > | Yes, Glade does it, but the only thing Glade has to worry
> > | about is that
> > | the stream representation is independent of hardware platform and OS.
> > |
> > | Glade does this by providing a replacement body for the
> > | System.Stream_Attributes package.
> > |
> > | However, I have done a lot of endianess-independent work where a
> > | specific network representation is required -- i.e., where an Ada
> > | program has to communicate with a program writen in a language
> > | not necessarily known to me.
> > |
> > OK, but i don't pretend to get this general case taken into account within
> > the language scope, altrough some
> > for My_Stream'representation use XDR;
> > could be great!
> >
> > My point is just about interoperability between the same Ada code compiled
> > on two platform. I want both exe to put the same thing in the stream.
> > It's not the case even with a representation clause, and i feel this is
> > abnormal.
> > I know it's far from easy to achieve, but i thought Norman's proposal was
> > helping a lot...
> >
> Representation clauses are neither the issue, nor the answer to the problem.
> 
> If all you want is platform-independence, then replace the file s-stratt.adb
> in your standard gnat distribution with the one from the same version of
> glade, and re-compile and rebuild the library files (.a), and you have
> what you want.  Now, as long as your types are declared the same on both
> platforms (with no representation clauses) you will have platform-
> independence.
> 
> I'm not sure why GNAT didn't do this from the beginning -- perhaps because
> they didn't provide an implementation of the Distributed Systems Annex at
> the beginning.  Also, there is a small performance penalty paid for
> shuffling bits around to achieve platform independence.
>

Re: Endianness independance

[Stephen Leake]

Jeffrey Carter <jrcarter@acm.org> writes:

> You can make this system independent by using
> System.Default_Bit_Order, which, thanks to an AI (I don't remember
> which, but you can look it up) is static.

What do you mean by "system independent"? This code is already
portable without change across targets that differ in endianness;
that's whay I mean by "system independent".

Exactly how would I use System.Default_Bit_Order for this purpose?

I can check that System.Default_Bit_Order has a particular value, and
only provide a rep clause that is correct for that value. But I don't
see how to use System.Default_Bit_Order in the rep clause; it does not
provide enough information.

-- 
-- Stephe

Re: Endianness independance

[Simon Wright]

"David C. Hoos, Sr." <david.c.hoos.sr@ada95.com> writes:

> If all you want is platform-independence, then replace the file
> s-stratt.adb in your standard gnat distribution with the one from
> the same version of glade, and re-compile and rebuild the library
> files (.a), and you have what you want.  Now, as long as your types
> are declared the same on both platforms (with no representation
> clauses) you will have platform- independence.

You can certainly use the 3.14 GLADE file with 3.15 (and I think
GCC-3, too). Not a huge amount of change.

I suspect you'd be well advised to have the same release at both ends,
even so!

Re: Endianness independance

[Jeffrey Carter]

Stephen Leake wrote:
> Jeffrey Carter <jrcarter@acm.org> writes:
> 
>>You can make this system independent by using
>>System.Default_Bit_Order, which, thanks to an AI (I don't remember
>>which, but you can look it up) is static.
> 
> What do you mean by "system independent"? This code is already
> portable without change across targets that differ in endianness;
> that's whay I mean by "system independent".
> 
> Exactly how would I use System.Default_Bit_Order for this purpose?
> 
> I can check that System.Default_Bit_Order has a particular value, and
> only provide a rep clause that is correct for that value. But I don't
> see how to use System.Default_Bit_Order in the rep clause; it does not
> provide enough information.

You said your package was "system dependent" because you have to change 
it for different platforms. Using Default_Bit_Order you can make it work 
without change on all platforms.

You can use 'Pos of Default_Bit_Order to calculate all of the named 
numbers in your package. You can then continue to use your package as 
before, but without changing it when you change platforms.

-- 
Jeff Carter
"Perfidious English mouse-dropping hoarders."
Monty Python & the Holy Grail

Re: Endianness independance

[Warren W. Gay VE3WWG]

Simon Wright wrote:
> "David C. Hoos, Sr." <david.c.hoos.sr@ada95.com> writes:
> 
>>If all you want is platform-independence, then replace the file
>>s-stratt.adb in your standard gnat distribution with the one from
>>the same version of glade, and re-compile and rebuild the library
>>files (.a), and you have what you want.  Now, as long as your types
>>are declared the same on both platforms (with no representation
>>clauses) you will have platform- independence.
> 
> You can certainly use the 3.14 GLADE file with 3.15 (and I think
> GCC-3, too). Not a huge amount of change.
> 
> I suspect you'd be well advised to have the same release at both ends,
> even so!

If you look carefully at GLADE, you will notice that all that is
being done there is "byte swapping". There is more to independent
network formats than that (it just happens to work with _most_
modern platforms at the moment).

The problem that XDR solved was that they specified a network
independent format for floating point types. If you try to use
GLADE on hosts that use different floating point formats, then
no amount of byte swapping is going to make them compatible!

I would have preferred to see GLADE make use of the XDR facilities
that are available on any Linux/UNIX platform, and I am certain
that XDR exists in some form on any Windows platform that
supports DCOM.  XDR would give you true compatibility at the
floating point type level, in addition to the less difficult
types like integers.

Perhaps someone can take on that XDR project for the NG Glade? ;-)

-- 
Warren W. Gay VE3WWG
http://home.cogeco.ca/~ve3wwg

Re: Endianness independance

[Stephen Leake]

Jeffrey Carter <jrcarter@acm.org> writes:

> Stephen Leake wrote:
> > Jeffrey Carter <jrcarter@acm.org> writes:
> >
> >>You can make this system independent by using
> >>System.Default_Bit_Order, which, thanks to an AI (I don't remember
> >>which, but you can look it up) is static.
> > What do you mean by "system independent"? This code is already
> > portable without change across targets that differ in endianness;
> > that's whay I mean by "system independent".
> > Exactly how would I use System.Default_Bit_Order for this purpose?
> > I can check that System.Default_Bit_Order has a particular value, and
> > only provide a rep clause that is correct for that value. But I don't
> > see how to use System.Default_Bit_Order in the rep clause; it does not
> > provide enough information.
> 
> You said your package was "system dependent" because you have to
> change it for different platforms. 

Hmm. Maybe you mean the SAL.Endianness package (I posted two packages;
SAL.Endianness and Foo - please be specific!). Yes, that has to be
different for different platforms. But, you only have to write it once
for each platform, and I already have. So I don't have to edit code
when I change platforms; I just have to change which code is included
in the compilation.

> Using Default_Bit_Order you can make it work without change on all
> platforms. You can use 'Pos of Default_Bit_Order to calculate all of
> the named numbers in your package. You can then continue to use your
> package as before, but without changing it when you change
> platforms.

Well, maybe. I'm comfortable with the package as it is. If you'd like
to contribute, feel free to post your version :).

-- 
-- Stephe

Re: Endianness independance

[David C. Hoos]

----- Original Message ----- 
From: "Warren W. Gay VE3WWG" <ve3wwg@cogeco.ca>
Newsgroups: comp.lang.ada
To: <comp.lang.ada@ada.eu.org>
Sent: Wednesday, March 05, 2003 11:28 AM
Subject: Re: Endianness independance
<snip>
> If you look carefully at GLADE, you will notice that all that is
> being done there is "byte swapping". There is more to independent
> network formats than that (it just happens to work with _most_
> modern platforms at the moment).
> 
> The problem that XDR solved was that they specified a network
> independent format for floating point types. If you try to use
> GLADE on hosts that use different floating point formats, then
> no amount of byte swapping is going to make them compatible!
> 
Agreed.  However, at the end of the day, there is no solution at all
for communication between machines with differing ranges for the
exponent and mantissa, so the notion of platform independence
must have certain caveats about some details of the platforms which
must at least support some level of commonality.

Re: Endianness independance

[Simon Wright]

"Warren W. Gay VE3WWG" <ve3wwg@cogeco.ca> writes:

> If you look carefully at GLADE, you will notice that all that is
> being done there is "byte swapping". There is more to independent
> network formats than that (it just happens to work with _most_
> modern platforms at the moment).

I don't know what version of GLADE you are looking at, but the code
for 3.15p does a lot more than this. I am no expert as to whether it
achieves the purpose or not on eg a VAX (but I don't believe that GNAT
supports VAXes anyway).

This is reading a single-length Float from the net:

   function I_F (Stream : access RST) return Float is
      I       : constant Precision := Single;
      E_Size  : Integer  renames Fields (I).E_Size;
      E_Bias  : Integer  renames Fields (I).E_Bias;
      E_Last  : Integer  renames Fields (I).E_Last;
      F_Size  : Integer  renames Fields (I).F_Size;
      F_Mask  : SE       renames Fields (I).F_Mask;
      E_Bytes : SEO      renames Fields (I).E_Bytes;
      F_Bytes : SEO      renames Fields (I).F_Bytes;

      E : Unsigned;
      P : Boolean;
      X : Float;
      S : SEA (1 .. F_L);
      L : SEO;

   begin
      Ada.Streams.Read (Stream.all, S, L);

      if L /= S'Last then
         raise Data_Error;
      end if;

      --  Extract Fraction, Exponent and Sign.
      X := Float (S (F_L + 1 - F_Bytes) and F_Mask);
      for N in F_L + 2 - F_Bytes .. F_L loop
         X := X * FB + Float (S (N));
      end loop;
      X := Scaling (X, -F_Size); --  Float

      if BS <= S (1) then
         P := False;
         E := Unsigned (S (1) - BS);
      else
         P := True;
         E := Unsigned (S (1));
      end if;

      for N in 2 .. E_Bytes loop
         E := E * BB + Unsigned (S (N));
      end loop;
      E := Shift_Right (E, Integer (E_Bytes) * SU - E_Size - 1);

      --  Look for special cases.
      if X = 0.0 then

         --  Signed zeros.
         if E = 0 then
            if P then
               return Float'Copy_Sign (0.0, 1.0);
            else
               return Float'Copy_Sign (0.0, -1.0);
            end if;

         else

            --  Signed infinites.
            if E = Unsigned (E_Last) then
               if P then
                  return Float'Safe_Last;
               else
                  return Float'Safe_First;
               end if;
            end if;
         end if;
      end if;

      --  Denormalized float.
      if E = 0 then

         X := Scaling (X, 1 - E_Bias); --  Flaot

      --  Normalized float.
      else

         X := Scaling (X + 1.0, Integer (E) - E_Bias); --  Float

      end if;

      if P then
         X := Float'Copy_Sign (X, 1.0);
      else
         X := Float'Copy_Sign (X, -1.0);
      end if;

      return X;
   end I_F;

Re: Endianness independance

[Simon Wright]

"David C. Hoos" <david.c.hoos.sr@ada95.com> writes:

> Agreed.  However, at the end of the day, there is no solution at all
> for communication between machines with differing ranges for the
> exponent and mantissa, so the notion of platform independence must
> have certain caveats about some details of the platforms which must
> at least support some level of commonality.

No _general_ solution, but a limited solution is possible surely?

Re: Endianness independance

[Warren W. Gay VE3WWG]

Simon Wright wrote:
> "Warren W. Gay VE3WWG" <ve3wwg@cogeco.ca> writes:
>>If you look carefully at GLADE, you will notice that all that is
>>being done there is "byte swapping". There is more to independent
>>network formats than that (it just happens to work with _most_
>>modern platforms at the moment).
> 
> I don't know what version of GLADE you are looking at, but the code
> for 3.15p does a lot more than this. I am no expert as to whether it
> achieves the purpose or not on eg a VAX (but I don't believe that GNAT
> supports VAXes anyway).
> 
> This is reading a single-length Float from the net:
> 
>    function I_F (Stream : access RST) return Float is
>       I       : constant Precision := Single;
>       E_Size  : Integer  renames Fields (I).E_Size;
>       E_Bias  : Integer  renames Fields (I).E_Bias;
>       E_Last  : Integer  renames Fields (I).E_Last;
>       F_Size  : Integer  renames Fields (I).F_Size;
>       F_Mask  : SE       renames Fields (I).F_Mask;
>       E_Bytes : SEO      renames Fields (I).E_Bytes;
>       F_Bytes : SEO      renames Fields (I).F_Bytes;
> 
>       E : Unsigned;
>       P : Boolean;
>       X : Float;
>       S : SEA (1 .. F_L);
>       L : SEO;
> 
>    begin
>       Ada.Streams.Read (Stream.all, S, L);
> 
>       if L /= S'Last then
>          raise Data_Error;
>       end if;
> 
>       --  Extract Fraction, Exponent and Sign.
>       X := Float (S (F_L + 1 - F_Bytes) and F_Mask);
>       for N in F_L + 2 - F_Bytes .. F_L loop
>          X := X * FB + Float (S (N));
>       end loop;
>       X := Scaling (X, -F_Size); --  Float
...
>    end I_F;

Well Simon, I have to admit this sure looks a lot different
than the code I remember looking at. I havn't reviewed this
in great detail, but obviously there is more than byte
swapping here. I am sure that the last time I looked was
pre 3.15p.

I am actually rather happy to see this level of support.

It would be best if the format supported was XDR, but without
more investigation, I cannot say here and now if this is
it (XDR support would make Ada more easily able to speak
on equal network terms with existing RPC C code). However,
simply guaranteeing that the format is compatible
between two GLADE sessions is good enough for most purposes
that I would personally have in mind.

Thanks for pointing this out. Warren.
-- 
Warren W. Gay VE3WWG
http://home.cogeco.ca/~ve3wwg

Re: Endianness independance

[Warren W. Gay VE3WWG]

Simon Wright wrote:
> "David C. Hoos" <david.c.hoos.sr@ada95.com> writes:
>>Agreed.  However, at the end of the day, there is no solution at all
>>for communication between machines with differing ranges for the
>>exponent and mantissa, so the notion of platform independence must
>>have certain caveats about some details of the platforms which must
>>at least support some level of commonality.
> 
> No _general_ solution, but a limited solution is possible surely?

Certainly. At worst comes to worst, you can convert a value to
an exponential string format and exchange that. The XDR network format
was designed to be a bit more efficient than that however,
which is the only reason it is perhaps superior than string form.

-- 
Warren W. Gay VE3WWG
http://home.cogeco.ca/~ve3wwg