Endian and Ada

Endian and Ada

[Michael Anthony Porcelli]

I'm taking a computer architecture class and my eyes have been opened to the
incredible lack of progress that computer science has made in the area of
architecture-independant programming.  One of the main problems that my
professor likes to point out the endian problem and the fact that so much
software is *not* endian-independant (due mainly to the widespread use of C
and C++).  However, he is not familiar enough with Ada (nor am I) to know if
Ada is endian-independant.  I'm almost positive that it's *possible* to make
endian-dependant code using unchecked programming (necessary for systems
writing).  I was wondering, however, if the day-to-day Ada software out
there is written endian-independant (i.e. the language facilities used in
most day-to-day programming don't depend on whether your architecture is
"big" endian or "little" endian.)

Thanks,

-Mike

Re: Endian and Ada

[Robert Dewar]

Mike asks

"I'm taking a computer architecture class and my eyes have been opened to the
incredible lack of progress that computer science has made in the area of
architecture-independant programming.  One of the main problems that my
professor likes to point out the endian problem and the fact that so much
software is *not* endian-independant (due mainly to the widespread use of C
and C++).  However, he is not familiar enough with Ada (nor am I) to know if
Ada is endian-independant.  I'm almost positive that it's *possible* to make
endian-dependant code using unchecked programming (necessary for systems
writing).  I was wondering, however, if the day-to-day Ada software out
there is written endian-independant (i.e. the language facilities used in
most day-to-day programming don't depend on whether your architecture is
"big" endian or "little" endian.)"


To a first approximation, NONE of the semantics of Ada depend on the
endianness, you would have to work hard to make an Ada program that
worked BE and did not work LE, or vice versa.

The issue arises when transporting data between targets of different
endianness.

One nice approach is to use the stream facilities of Ada in conjuncction
with a 
target independent representation such as XDR. A future version of GNAT
will provide the option of using XDR for stream representation to
facilitiate this approach (you could program it yourself, but it would
be nice to have it built in).

Re: Endian and Ada

[Mike Young]

Michael Anthony Porcelli wrote:
> 
> I'm taking a computer architecture class and my eyes have been opened to the
> incredible lack of progress that computer science has made in the area of
> architecture-independant programming.  One of the main problems that my
> professor likes to point out the endian problem and the fact that so much
> software is *not* endian-independant (due mainly to the widespread use of C
> and C++).  However, he is not familiar enough with Ada (nor am I) to know if
> Ada is endian-independant.  I'm almost positive that it's *possible* to make
> endian-dependant code using unchecked programming (necessary for systems
> writing).  I was wondering, however, if the day-to-day Ada software out
> there is written endian-independant (i.e. the language facilities used in
> most day-to-day programming don't depend on whether your architecture is
> "big" endian or "little" endian.)
> 
> Thanks,
> 
> -Mike

===========
Byte ordering becomes an issue when communicating across different 
architectures, but is not normally a concern otherwise. I can think of 
few languages, with assembler being a *possible* exception, where this 
is an issue worthy of conscious thought. The question you might now ask 
yourself (or your professor) is why byte-ordering should be a concern 
for programmers not involved in cross-platform programming issues, 
regardless of programming language.

Mike.

Re: Endian and Ada

[Robert A Duff]

In article <4kamb9$om2@flute.aix.calpoly.edu>,
Michael Anthony Porcelli <mporcell@flute.aix.calpoly.edu> wrote:
>...One of the main problems that my
>professor likes to point out the endian problem and the fact that so much
>software is *not* endian-independant (due mainly to the widespread use of C
>and C++).  However, he is not familiar enough with Ada (nor am I) to know if
>Ada is endian-independant.

What, exactly, do you mean by "endian-independent" code?  There's a big
difference between (1) you can compile a fairly self-contained program
on both sorts of machines, and it will work, versus (2) the program is
distributed across a heterogeneous network, and wants to pass
endian-independent data from one part to another.

- Bob

Re: Endian and Ada

[Theodore E. Dennison]

Michael Anthony Porcelli wrote:
> 
> I'm taking a computer architecture class and my eyes have been opened to the
> incredible lack of progress that computer science has made in the area of
> architecture-independant programming.  One of the main problems that my
> professor likes to point out the endian problem and the fact that so much
> software is *not* endian-independant (due mainly to the widespread use of C
> and C++).  However, he is not familiar enough with Ada (nor am I) to know if
> Ada is endian-independant.  ...

It is not.

> writing).  I was wondering, however, if the day-to-day Ada software out
> there is written endian-independant (i.e. the language facilities used in
> most day-to-day programming don't depend on whether your architecture is
> "big" endian or "little" endian.)

In my experience, the answer is no. Typically the last application written
"loses" and has to swap bytes, unless a endian-aware communications protocol
(such as ASN.1) is used.

-- 
T.E.D.          
                |  Work - mailto:dennison@escmail.orl.mmc.com  |
                |  Home - mailto:dennison@iag.net              |
                |  URL  - http://www.iag.net/~dennison         |

Re: Endian and Ada

[Joseph Wisniewski]

Just an FYI. I'll check this in my stack of magazines, but I think I 
remember an article in Embedded Systems Programming about 2 years ago,
that had _an_ example of how to rep-spec for "endian-independence".

Joe

Mike Young <mikey@mcs.com> wrote:
: Michael Anthony Porcelli wrote:
: > 
: > I'm taking a computer architecture class and my eyes have been opened to the
: > incredible lack of progress that computer science has made in the area of
: > architecture-independant programming.  One of the main problems that my
: > professor likes to point out the endian problem and the fact that so much
: > software is *not* endian-independant (due mainly to the widespread use of C
: > and C++).  However, he is not familiar enough with Ada (nor am I) to know if
: > Ada is endian-independant.  I'm almost positive that it's *possible* to make
: > endian-dependant code using unchecked programming (necessary for systems
: > writing).  I was wondering, however, if the day-to-day Ada software out
: > there is written endian-independant (i.e. the language facilities used in
: > most day-to-day programming don't depend on whether your architecture is
: > "big" endian or "little" endian.)
: > 
: > Thanks,
: > 
: > -Mike

: ===========
: Byte ordering becomes an issue when communicating across different 
: architectures, but is not normally a concern otherwise. I can think of 
: few languages, with assembler being a *possible* exception, where this 
: is an issue worthy of conscious thought. The question you might now ask 
: yourself (or your professor) is why byte-ordering should be a concern 
: for programmers not involved in cross-platform programming issues, 
: regardless of programming language.

: Mike.

--
Joe Wisniewski	
  Commercial Software Solutions, Ltd. -- Embedded/RT SW Consulting
  Co-Author: Program Smarter, Not Harder - Get Mission Critical Projects
	Right the First Time ISBN 0-07-021232-5

  wisniew@primenet.com   --- The "Baltimore Browns"?     Pazhaloosta!
  Ada95 --> She'll take you to places you never thought possible!
  

Re: Endian and Ada

[Mike Young]

Kelly Grant wrote:
>
> the Win32 environment (winsock specifically) we have methods like NtoHS (Network
> to Host System) that will know whether to swap bytes or not).  I think the
> network byte ordering is big endian, but I don't know for certain.

=======
Not just winsock, but all Berkeley sockets, meaning almost all unices. 
(You might also want to note that some languages are case sensitive. 
That's ntohs, thank you. :) Network byte order is commonly accepted (if 
not actually specified somewhere) to mean big-endian.

Re: Endian and Ada

[Theodore E. Dennison]

Dale Pontius wrote:
> 
> In article <dewar.828991983@schonberg>,
>         dewar@cs.nyu.edu (Robert Dewar) writes:
> >
> >target independent representation such as XDR. A future version of GNAT
> >
> What is XDR?

According to FOLDOC ( http://wfn-shop.Princeton.EDU/foldoc/ ), it is-

    eXternal Data Representation

(XDR) A standard for machine independent data structures developed by
Sun Microsystems for use in remote procedure call systems. It is defined
in RFC 1014 and is similar to ASN.1. 


-- 
T.E.D.          
                |  Work - mailto:dennison@escmail.orl.mmc.com  |
                |  Home - mailto:dennison@iag.net              |
                |  URL  - http://www.iag.net/~dennison         |

Re: Endian and Ada

[Thomas Koenig]

In comp.lang.ada, pontius@twonky.btv.ibm.com (Dale Pontius) wrote:
>What is XDR?

XDR (External Data Representation) is defined in RFC 1014.  It is the
basis of Sun's RPC (or ONC/RPC, as it is called in these 'open' days
;-), defined in RFC 1057, which in turn is the basis for such protocols
as NFS (RFC 1094) or NIS.

Re: Endian and Ada

[Kelly Grant]

mporcell@flute.aix.calpoly.edu (Michael Anthony Porcelli) wrote:
>I'm taking a computer architecture class and my eyes have been opened to the
>incredible lack of progress that computer science has made in the area of
>architecture-independant programming.  One of the main problems that my
>professor likes to point out the endian problem and the fact that so much
>software is *not* endian-independant (due mainly to the widespread use of C
>and C++).  However, he is not familiar enough with Ada (nor am I) to know if
>Ada is endian-independant.  I'm almost positive that it's *possible* to make
>endian-dependant code using unchecked programming (necessary for systems
>writing).  I was wondering, however, if the day-to-day Ada software out
>there is written endian-independant (i.e. the language facilities used in
>most day-to-day programming don't depend on whether your architecture is
>"big" endian or "little" endian.)
>

We are currently facing this issue.  We are working on control software on Intel 
platforms communicating with Motorola (and other) hardware on remote platforms.  
We are passing data across radio modems, and byte ordering is a concern.  We are 
going to use "Network Byte Ordering" (I don't know whose standard this is, but in 
the Win32 environment (winsock specifically) we have methods like NtoHS (Network 
to Host System) that will know whether to swap bytes or not).  I think the 
network byte ordering is big endian, but I don't know for certain.

Of course you can try to implement certain tricks.  For instance, you could 
define a standard that the first 4 byte value read is '1', and determine the 
endian-ness of the data by examining that known value.  Perhaps the first entry 
in a file, or the first data packet shared between cooperating computers...

It is *not* possible for a language to be endian-independant. Assumptions must be 
made and it is not possible to know which endian a particular file or packet is 
using just by looking at the data.  The use of C or C++ is not an issue. When an 
application writes an multi-byte integer to a file, it will commit itself to big 
endian or little endian as soon as it is written, generally based on the base 
architecture.

Kelly

Re: Endian and Ada

[Michael Anthony Porcelli]

Re: Endian and Ada

[Michael Anthony Porcelli]

Newsgroups: comp.lang.ada
Subject: Re: Endian and Ada
Summary: 
Expires: 
References: <4kamb9$om2@flute.aix.calpoly.edu>
Sender: 
Followup-To: 
Distribution: world
Organization: California Polytechnic State University, San Luis Obispo
Keywords: 
Cc: 

After reading your posts and thinking about this issue more I think that
perhaps I've come to a better understanding of this issue and, I have come
up with more questions (clearer and more specific ones).  I'm wondering if
you all could check my understanding and let me know if it's correct or not.

1.  Endian becomes an issue when cross platform data distribution/transfer
is involved regardless of what programming language is used.

2.  It *is* possible to write "endian free" code in any language (possibly
not assembly).  I'm not saying, however, that it's possible to write *all* 
software "endian free" but that it's possible to write *some* software "endian 
free".  This is done fairly simply by just not writing code that depends on 
byte-ordering.  What I mean by "endian free code" here is that if the code
was compiled and run on a big endian computer, that the same code would
compile and run successfuly (and identically) on a little endian computer 
(assuming all other non-endian cross-platform problems are non-existant).

3.  It is possible to write non-"endian free" code in any language that
allows direct byte manipulation.  C and C++ definitely allow this.  I do
believe that Ada allows this (unchecked programming).  Am I correct here?

4.  Whenever possible, write "endian free" code.  This seems to me to be
simply good software-engineering practice.  It also seems to me that most,
if not all software applications (with the possible exception of systems
software) can be written "endian free".  Perhaps efficiency might be
sacrificed from time to time, but the portability is worth it.

5.  This is kind-of my big question here.  Can systems software be written
without depending on byte-ordering?  I have basically no experience writing
systems software so I have no idea here.  It seems that if the answer is
yes, then it's only a matter of bad software engineering that byte-order
dependant code is still written (except for software that manipulates
cross-platform data transfer, #1 above).  If the answer is no, then porting
systems software becomes a royal pain (according to my understanding, it
is as a direct result of the endian problem.)

6.  The Java solution.  I guess my question is whether or not that Java
"platform" is a solution to this.  In case you haven't heard, Sun is making
processors that run Java byte-code directly.  My guess would be that the
choice of "big" or "little" endian is part of the Java platform architecture
standard (I don't know this for sure).  If, in fact, this is true, *and* if
the whole world follows suit (making Java-native processors) then it seems
that the endian problem would "go away" (eventually).  What I'm not talking
about here is the Java language itself.  There doesn't seem to be a need to.
According to my understanding it is completely "safe", i.e. allows no
facilities at all for writing unchecked byte-manipulating code.

7.  Processors that switch from big to little.  I've only heard of these and
I don't know if they are actually out on the market yet.  I do believe that
HP (and others) are working on making architectures that can run in either
big or little endian mode.  One of the main reasons that this is being done
is the possibility that Windows is going to take over the world.  It seems
that the powers-that-be feel that making a switchable architecture is easier
than re-writing windows to be big endian.  I'd appreciate hearing any more
on this from someone who has worked with these type of processors or has any
more information regarding this trend.  Personally, I'd hate to see
Microsoft completely take over the systems software market, but it seems
that is the way things are going.

Again, thanks for your responses to my original post.  Any comments or
corrections to my understanding would be greatly apreciated (that's mainly
why I'm posting).  

Laters,

-Mike

Re: Endian and Ada

[Dale Pontius]

In article <dewar.828991983@schonberg>,
        dewar@cs.nyu.edu (Robert Dewar) writes:
>
>target independent representation such as XDR. A future version of GNAT
>
What is XDR?

Thanks,
Dale Pontius
(NOT speaking for IBM)

Re: Endian and Ada

[Robert A Duff]

In article <4kevdl$254a@flute.aix.calpoly.edu>,
Michael Anthony Porcelli <mporcell@flute.aix.calpoly.edu> wrote:
>1.  Endian becomes an issue when cross platform data distribution/transfer
>is involved regardless of what programming language is used.

Correct.  This is the "hard problem".

>2.  It *is* possible to write "endian free" code in any language (possibly
>not assembly).  I'm not saying, however, that it's possible to write *all* 
>software "endian free" but that it's possible to write *some* software "endian 
>free".  This is done fairly simply by just not writing code that depends on 
>byte-ordering.  What I mean by "endian free code" here is that if the code
>was compiled and run on a big endian computer, that the same code would
>compile and run successfuly (and identically) on a little endian computer 
>(assuming all other non-endian cross-platform problems are non-existant).

Correct.  It's pretty easy to do this in Ada or C or C++.

>3.  It is possible to write non-"endian free" code in any language that
>allows direct byte manipulation.  C and C++ definitely allow this.  I do
>believe that Ada allows this (unchecked programming).  Am I correct here?

Correct.  The nice thing about Ada is that when you write
system-dependent code, you normally say so explicitly in the source
code.  In any case, it's certainly possible to write such code in Ada.

>4.  Whenever possible, write "endian free" code.  This seems to me to be
>simply good software-engineering practice.  It also seems to me that most,
>if not all software applications (with the possible exception of systems
>software) can be written "endian free".  Perhaps efficiency might be
>sacrificed from time to time, but the portability is worth it.

Right.  I don't think efficiency is even a big issue here.  (Assuming
you're not passing data to a different-endian machine.)

>5.  This is kind-of my big question here.  Can systems software be written
>without depending on byte-ordering?  I have basically no experience writing
>systems software so I have no idea here.  It seems that if the answer is
>yes, then it's only a matter of bad software engineering that byte-order
>dependant code is still written (except for software that manipulates
>cross-platform data transfer, #1 above).  If the answer is no, then porting
>systems software becomes a royal pain (according to my understanding, it
>is as a direct result of the endian problem.)

Yes, even system software can, and should, be written to work properly
on either endian machines.

>6.  The Java solution.  I guess my question is whether or not that Java
>"platform" is a solution to this.  In case you haven't heard, Sun is making
>processors that run Java byte-code directly.  My guess would be that the
>choice of "big" or "little" endian is part of the Java platform architecture
>standard (I don't know this for sure).  If, in fact, this is true, *and* if
>the whole world follows suit (making Java-native processors) then it seems
>that the endian problem would "go away" (eventually).  What I'm not talking
>about here is the Java language itself.  There doesn't seem to be a need to.
>According to my understanding it is completely "safe", i.e. allows no
>facilities at all for writing unchecked byte-manipulating code.

I don't know enough about Java to say anything here.

>7.  Processors that switch from big to little.  I've only heard of these and
>I don't know if they are actually out on the market yet.  I do believe that
>HP (and others) are working on making architectures that can run in either
>big or little endian mode.  One of the main reasons that this is being done
>is the possibility that Windows is going to take over the world.  It seems
>that the powers-that-be feel that making a switchable architecture is easier
>than re-writing windows to be big endian.  I'd appreciate hearing any more
>on this from someone who has worked with these type of processors or has any
>more information regarding this trend.  Personally, I'd hate to see
>Microsoft completely take over the systems software market, but it seems
>that is the way things are going.

Yes, processors that switch endian-ness exist, and are on the market.
However, the O.S. usually sets the processor to one particular
endian-ness, and assumes that forever.  It's easy to write an OS that
works properly with either endian-ness, assuming you recompile the OS
for whichever one; it's not so easy to write an OS that can switch back
and forth.

And yes, Microsoft will probably take over the world.

- Bob

Re: Endian and Ada

[Larry Kilgallen]

In article <4kevdl$254a@flute.aix.calpoly.edu>, mporcell@flute.aix.calpoly.edu (Michael Anthony Porcelli) writes:

> 7.  Processors that switch from big to little.  I've only heard of these and
> I don't know if they are actually out on the market yet.  I do believe that
> HP (and others) are working on making architectures that can run in either
> big or little endian mode.

The Alpha AXP 21064 processor introduced by DEC in 1992 is reasonably
endian-neutral since it leaves byte manipulation to the compilers.
A typical instruction sequence might be slightly biased toward being
little-endian due to an extra instruction being used for computing
offsets, but it is a close match.

It happens that DEC's compilers are little-endian, but the one you
write could be big-endian.

Larry Kilgallen
LJK Software

Re: Endian and Ada

[LJMetzger]

On 9 Apr 1996 17:29:41 -0700 Mike Porcelli wrote:
>After reading your posts and thinking about this issue more I think that
>perhaps I've come to a better understanding of this issue and, I have
come
>up with more questions (clearer and more specific ones).  I'm wondering
if
>you all could check my understanding and let me know if it's correct or
not.
>
>1.  Endian becomes an issue when cross platform data
distribution/transfer
>is involved regardless of what programming language is used.

Any time you deal with target hardware that is not identical to the host
hardware, not only do you have to deal with the Endian problem,
but also the "which bit is lsb problem too".

e.g. I use Verdix Ada 6.2.0 (e) on a Sun (SunOs 4.1.x ) to create 
a prom image for an AMD29050 embedded processor.  The 29050
communicates with hardware (e.g. A/D converters) from different 
vendors, and on some of the hardware bit 0 is lsb, and on others
bit 0 is msb, hence the need for "Mr. Bitswap".



Lewis Metzger          LJMetzger@aol.com
Fair Lawn, NJ

Re: Endian and Ada

[phil]

ljmetzger@aol.com (LJMetzger) wrote:

> Any time you deal with target hardware that is not identical to the host
> hardware, not only do you have to deal with the Endian problem,
> but also the "which bit is lsb problem too".
> 
> e.g. I use Verdix Ada 6.2.0 (e) on a Sun (SunOs 4.1.x ) to create 
> a prom image for an AMD29050 embedded processor.  The 29050
> communicates with hardware (e.g. A/D converters) from different 
> vendors, and on some of the hardware bit 0 is lsb, and on others
> bit 0 is msb, hence the need for "Mr. Bitswap".

This is getting at the crux of the problem.

I hope the following is not too trivial an exposition for this group, but one
must be clear of the hardware situation to avoid 'endian', or more general
Data Representation pitfalls.
 
1) Hardware designers produced central processing units (CPUs) that work in
binary (there are a few exceptions). You may only work, at the machine level,
with a pool of Bits that may be either 'set' or 'not set' (e.g 1 or 0).

2) Storage elements have for a long time been available in packages organised
in 8-bit Bytes. That is to say, one cannot access single bits, only one or
more bytes.

3) The CPU can read bytes from memory and internally select individual bits
from them or combine bytes to make a Word. 

4) CPUs may be wired to read or write several bytes at a time (typically 4,
the so-called 32-bit CPUs), and work on them internally as a single entity.

5) These same CPUs have instructions which allow them to take 1, 2 or 4 bytes
at a time from memory, being termed a Byte, Word or Double-Word operation.  

6) In order to work with data containing a wide range of values, one must
define a Data Representation whereby the data is mapped onto a number of bits.


7) The Data Representation used by computer programmers involves terms such as
Floating Point, Integer (both Signed and Unsigned), Character, etc. The
compiler and operating system determines how these are mapped onto the target
machines Byte, Word or Double-Word.

8) If we work on only one target machine type we may store and correctly read
back each data type without knowledge of how the data is mapped to the
hardware.

9) Suppose we write a program to store our data, say an Integer type, on a
floppy disk. Suppose our system opts to store it in 2 consecutive bytes with
the most significant bits in the first and the least significant bits in the
second byte. The first byte would have 2^15..2^8 and the second 2^7..2^0. This
machine would probably adopt the convention of presenting these data
respectively to the 'Data0'..'Data7' pins of the storage device.

10) Then suppose this floppy disk is moved to a machine of different design,
and/or running a different operating system, and possibly with a compiler of
different source. Without knowledge of the Data Representation used we cannot
know whether our Integer will be reassembled from the 2 bytes in the correct
or reversed order.

11) Unfortunately the specifications of the data representation becomes rather
opaque, and gets more so as it passes up  the hierarchy through storage
hardware spec, to Basic Input/Output System (BIOS) spec, Operating System,
Compiler and finally Application Program.

12) Anyone dealing in data transfer between systems (and that includes just
carrying a floppy disk from a Sun to a PC) must check and allow for the data
representations on each. We all know that Unix uses just line-feed to
terminate a line of text, and this is virtually illegible on a PC that expects
line-feed *and* carriage-return. Conversion programs exist to correct for
this.

13) If the data representations is NOT identical then a conversion method must
be established on one or both machines. 

14) Hence Ada, nor any other compiler alone has control of the 'endian-ness'
of the data.

15) I am not familiar with XDR (eXternal Data Representation) referred to
earlier in the thread, but if it exists in the form of an Input/Output system
call on each machine one could ask for an Integer (say) to be written in a
standard way, which when read by the complementary call on a different machine
would guarantee to return the same value. In other words each machine would
have a library to convert 'standard' external data format to its internal
format (byte reversed or whatever).

16) Maybe Ada should insist on the availability of such a library and *force*
(at least by default) Input/Output to be written to such a spec. Or is this
achieved by running Ada on a machine with a  standardised operating system?

17) Vendors who make hardware that is internally like the standard XDR will
have a null conversion in their library and will gain a performance advantage.

Phil Addison
--
-- Phil Addison.	            ------ <phil@severn.demon.co.uk>

Re: Endian and Ada

[Bob Crispen]

phil@SEVERN.DEMON.CO.UK wrote a splendid 17-point post explaining
why bit-order is one way for one CPU and another way for another, with
some suggestions to the CPU makers.

He forgot but one point: in Ada, we set the bit order to match the
machine so that the code can match the manual.  We simply remember the
times we sat there late at night with the sample driver code in one
hand and the manual in the other and got so mad at the inconsiderate
&%$#*&^s who wrote them that we threw the both at the wall on the
other side of the cube and left for the night, muttering "Catch me
buying a board again from *them*!"
-----
Someone flogging a commercial product on an inappropriate newsgroup asked:

>Is anyone interested in an excellent memory source?

What was that again?

Bob Crispen
revbob@eight-ball.hv.boeing.com
Speaking for myself, not my company