Building portable data structure (NOT a newbie question?)

Building portable data structure (NOT a newbie question?)

[afniii]

From the title this sounds like a newbie question, but I don't think
it is. Just for background I have 5 or so years of experience with Ada
and the last year plus with Ada 95. To build structures which are 
portable between platforms has always required this nasty package in
Ada 83 in which all of the predefined types get redefined and constrained
to some specific value. An endian changer was need after that along
with a float representation converter. Well I'm back at it again with
Ada 95. I really dislike the Ada 83 method and I think it would be
even more difficult to do with the tagged types and classes.

The problem:

There are a pile load of classes and records which are shared among
machines. They are recorded to a movable medium (such as disk) and 
moved to the other machine for access in some cases. In others they
are sent via a TCP/IP socket. In all cases the machine distribion is
heterogenous (sorry, but this is the part that makes it a challenge).
So I need a way to convert the lowest primitives (ie: discrete types
and floating point types) into a platform indepenent form. Still need
to minimize memory and disk allocation (cpu resources are cheap and
abundant) so converting everything to a string is out (but has been
considered).

Oh yeah. I almost forgot. No CORBA, IDL's or such either. I can't
afford them and I want the compiler to catch most of my errors for
me. If I didn't want that, I would program in C. :)


Options:

Convert all discrete values to a fixed endian (big or little endian)
before sending or recording. This is actually what I would like to
have done using streams (see attempts below).

Note however that GNAT does not yet support the 'BIT_ORDER attribute
except for the System.Default_Bit_Order. This would have worked out
well for the discrete value case. If you are reading this Robert Dewar,
any comment on when that 'BIT_ORDER attribute may be ready?

For floats I was considering converting everything to IEEE float.
Then I read up on all of the new floating point attributes and 
decided that there must be a better way. I could use the exponent and
fraction attributes and do something. Well that fizzled pretty quick.


By now you should get the idea of what I am trying to do. If not let
me know and I will elaborate some more.


Attempts:

To let you know what roads I have already walked:

Using streams seems to be the solution. Maybe this is a red herring and
that is why I have not found a good solution yet, but I think not. 
Unfortunately, I can find no way to override the predefined types 'READ
'WRITE, "INPUT, and 'OUTPUT attributes. This would really solve it. Of
course I could write one of those hated units with new definitions of
integer, natural, float, etc in them, but I hated that in Ada 83 so I am
sure I will hate it in Ada 95. Once you make on of those things you have
to always be dragging it around with a with clause then a use clause for
the operators and stuff. Heck, if I want to do that I would with and use
the predefined unit.

So after much horsing around, I found that using streams was really easy
and things worked well. Except the files and such were not portable --
obviously. That is when I tried the 'BIT_ORDER attribute with no success.

Then I realized that the output from the streams units may not, and in
fact are not, be protable because the type element defined in streams is 
platform or implementation independent. But, if I can get everything 
uniform in a stream, I can convert the stream to a fixed stream 
(ie: a byte stream where the element is always defined as mod 8). 
Then I would need to write a simple convert utility on each platform to 
go back and forth and IT would be solved.

So after much more horsing around, I became a believer of the Ada 95 LRM
and that I could not redefine the 'READ etc attributes for the predefined
type without first deriving a new type from them. I don't like that
solution at all.


So whose got a better idea?
I would like the output to be at stream from the Ada package streams.




(The return address on this post is probably wrong. Please use the
email address given below if you feel you want to respond that way)


Al Niessner III
email: afniii@hades.arl.psu.edu

Re: Building portable data structure (NOT a newbie question?)

[Larry Kilgallen]

In article <afniiik9q3c42e.fsf@Elysium.i-have-a-misconfigured-system-so-shoot-me>, afniii@hades.arl.psu.edu writes:

> There are a pile load of classes and records which are shared among
> machines. They are recorded to a movable medium (such as disk) and 
> moved to the other machine for access in some cases. In others they
> are sent via a TCP/IP socket. In all cases the machine distribion is
> heterogenous (sorry, but this is the part that makes it a challenge).
> So I need a way to convert the lowest primitives (ie: discrete types
> and floating point types) into a platform indepenent form. Still need
> to minimize memory and disk allocation (cpu resources are cheap and
> abundant) so converting everything to a string is out (but has been
> considered).
> 
> Oh yeah. I almost forgot. No CORBA, IDL's or such either. I can't
> afford them and I want the compiler to catch most of my errors for
> me. If I didn't want that, I would program in C. :)

If you eventually give up and convert everything to a string, I
would recommend the ASN.1 format as it has enough self-checking
to make afficianados of C run in terror.

Larry Kilgallen

Re: Building portable data structure (NOT a newbie question?)

[Robert A Duff]

In article <afniiik9q3c42e.fsf@Elysium.i-have-a-misconfigured-system-so-shoot-me>,
 <afniii@hades.arl.psu.edu> wrote:
>Using streams seems to be the solution.

Yes, probably.

>... Maybe this is a red herring and
>that is why I have not found a good solution yet, but I think not. 
>Unfortunately, I can find no way to override the predefined types 'READ
>'WRITE, "INPUT, and 'OUTPUT attributes. 

Right, you're not allowed to define Integer'Read and whatnot.

>...This would really solve it. Of
>course I could write one of those hated units with new definitions of
>integer, natural, float, etc in them, but I hated that in Ada 83 so I am
>sure I will hate it in Ada 95. Once you make on of those things you have
>to always be dragging it around with a with clause then a use clause for
>the operators and stuff. Heck, if I want to do that I would with and use
>the predefined unit.

One thing you can do is put a whole lot of code in child units of a
given unit.  E.g. you might have:
    
    package Basic_Types is
      ... -- defines your version of Integer and so forth
    end;
    
    with Basic_Types; use Basic_Types;
    package My_Application is end; -- empty package

Then put all the rest of your code under My_Application.  Then,
My_Application.This_Package and My_Application.That_Package can all see
the stuff in Basic_Types without explicitly having to "with" and "use"
it.

Bit_Order specifications are not required on most machines, by the way.
Only on word-addressable machines.  This is because on byte-addressable
machines, they can cause gaps within components, which is a pain for the
compiler.  GNAT might implement them someday, though -- ask ACT.  A
sensible thing to might be to implement the non-default Bit_Order, but
only in the cases where there are no gaps (which is probably your case).

>Then I realized that the output from the streams units may not, and in
>fact are not, be protable because the type element defined in streams is 
>platform or implementation independent. But, if I can get everything 
>uniform in a stream, I can convert the stream to a fixed stream 
>(ie: a byte stream where the element is always defined as mod 8). 

I suspect that Stream_Elements will always be 8 bits, except on unusual
systems.

- Bob

Re: Building portable data structure (NOT a newbie question?)

[Mark Eichin]

sounds like a stream that used XDR or ASN.1 for it's over-the-wire
encoding would be useful; I recall seeing at least examples of both of
these mentioned here, though I don't recally anything quite that
complete. 

Re: Building portable data structure (NOT a newbie question?)

[Norman H. Cohen]

Robert A Duff wrote:

> Bit_Order specifications are not required on most machines, by the way.
> Only on word-addressable machines.  This is because on byte-addressable
> machines, they can cause gaps within components, which is a pain for the
> compiler.  

The existence of these nasty gaps depends on the interpretation of bit
offsets in a nondefault-endian component clause, something that is not
specified by the RM.  If you insist that

   at B range 10 .. 12

be synonymous, assuming 8-bit bytes, with

   at B+1 range 2 .. 4

then you do indeed get noncontiguous bit fields.  But another
interpretation is to look at the highest bit number, say b, specified
with a given byte offset, and to view all component clauses with that
byte offset as specifying a contiguous range of bits within the
smallest  "loadable storage unit" (byte, halfword, word, or doubleword
on a typical 32-bit or 64-bit machine) having at least b+1 bits.  For
example, given the component clauses

   A at 0 range 0 .. 5;
   B at 0 range 6 .. 11;
   C at 0 range 12 .. 15;

and no other "at 0" component clauses, we would assume that A, B, and C
reside within a 16-bit loadable storage unit at offset 0, and that when
this 16-bit unit is loaded into a register, B is a contiguous field of
bits somewhere in the middle of this 16-bit unit.  Whether A is at the
high-order end or the low-order end of the 16-bit unit depends on which
bit ordering applies to the record type, but either way, there is a
sensible interpretation with B specifying a field that is contiguous
when loaded.

Given this interpretation of bit offsets, it is practical to require all
compilers to support nondefault-endian record-representation
clauses.  Such a requirement would allow a programmer to portably
specify a layout in an arbitrary bit order.  A compiler that is
big-endian by default and a compiler that is little-endian by default
would interpret the record-representation clause the same way, i.e., on
both a big-endian machine and a little-endian machine, the layout would
be divided into "loadable storage units" with the same sizes and byte
offsets, and the left-to-right bit position of each record component
within corresponding loadable storage units would be the same.

-- 
Norman H. Cohen
mailto:ncohen@watson.ibm.com
http://www.research.ibm.com/people/n/ncohen

Re: Building portable data structure (NOT a newbie question?)

[Robert I. Eachus]

In article <afniiik9q3c42e.fsf@Elysium.i-have-a-misconfigured-system-so-shoot-me> afniii@hades.arl.psu.edu writes:

  > Using streams seems to be the solution. Maybe this is a red herring and
  > that is why I have not found a good solution yet, but I think not. 
  > Unfortunately, I can find no way to override the predefined types 'READ
  > 'WRITE, "INPUT, and 'OUTPUT attributes. This would really solve it.

  Not forbidden, but very compiler specific.  However, I think that
GNAT is headed in the direction you want.  (Canonical stream
representation for scalar types independent of hardware.)

  > Of course I could write one of those hated units with new
  > definitions of integer, natural, float, etc in them, but I hated
  > that in Ada 83 so I am sure I will hate it in Ada 95. Once you
  > make on of those things you have to always be dragging it around
  > with a with clause then a use clause for the operators and
  > stuff. Heck, if I want to do that I would with and use the
  > predefined unit.

  You can clean it up a lot in Ada 95...  Redefine the operations in a
package, and make everything child units of this package.  Now the
predefined types don't keep popping back up.  Works well when you are
creating a subsystem, where you want this structure anyway.

  > Then I realized that the output from the streams units may not,
  > and in fact are not, be protable because the type element defined
  > in streams is platform or implementation independent. But, if I
  > can get everything uniform in a stream, I can convert the stream
  > to a fixed stream (ie: a byte stream where the element is always
  > defined as mod 8).  Then I would need to write a simple convert
  > utility on each platform to go back and forth and IT would be
  > solved.

  > So after much more horsing around, I became a believer of the Ada
  > 95 LRM and that I could not redefine the 'READ etc attributes for
  > the predefined type without first deriving a new type from them. I
  > don't like that solution at all.

  No really easy solution.  Best is if someone provides a version of
GNAT (and you can be that someone) that uses a canonical
representation and bit order for all scalars.  Then your high-level
code just works with standard streams.  Or you could implement some
other canonical format, such as XDR.


--

					Robert I. Eachus

with Standard_Disclaimer;
use  Standard_Disclaimer;
function Message (Text: in Clever_Ideas) return Better_Ideas is...