Ada95 Streams Question

Ada95 Streams Question

[Chris.Morgan]

I have a question about Ada95 stream implementations. I mentioned this work a
little while back in response to a request for Ada95 success stories, but now
I'm feeling a little unsettled by accusations of being a hacker!

I'm principally aiming this at comp.lang.ada but cc:ing to report@gnat.com in
case someone there who's given up on the newsgroup cares to comment.

Background
==========

I have connected an Ada95 executable to a C++ executable using streams. The Ada
is the brains of the application, and the C++ is the Motif pretty face. On both
sides I have defined a set of portable types which application types are
converted to for transmission and converted from on receipt.

On the C++ side I have specialised the iostream operators in our C++ GUI
library for each low-level type (e.g. 1,2 and 4-byte signed integers, 4-byte
float, 8-byte float and so on) so that binary reads are performed, not ASCII.
This is simply a case of casting the object to a char* of the correct size
before doing the read or write (>> or <<).

On the Ada side I have declared types that correspond to these low-level types,
deriving portable integer types from Interfaces.Integer_n and unsigned integers
from Interfaces.Unsigned_n types. The only special thing I have done is the C++
library stil expects a linefeed after each object (it normally reads files) so
I add that on after calling the 'write attribute for each object
(character'write(ASCII.LF)).

This is all working fine and I get good performance with a named pipe created
in /tmp which is a form of shared memory. In fact I have quite a nice scheme
going where I have defined a root tagged type which defines the header for any
message to the C++, and then any real message is derived from the root. To
write any particular derived type is simply matter of calling
type'write(object) and away it goes. The only bit not tested yet is floats
which aren't very important in this app.

Question
========

How likely is it that the GNAT (or some other Ada95 compiler) implementation of
Ada stream_io would arbitrarily break this arrangement in future? I am told
that the in-stream representation is only recommended to conform to the
in-memory, but what are the actual chances of some implementation not following
this advice?

Am I really relying entirely on a pure coincidence that the C++ and the Ada95
definitions can work together, or is it in fact the only sensible scheme?

Some of my more senior colleagues think that Ada95 stream_io is broken for
Ada/C++ interworking, but the fact that it works exactly as I expected in my
setup is either a) pure coincidence or b) the result of Ada streams (at least
with GNAT) being implemented precisely to allow this. I know which I suspect!

Of course I am talking about the simple case where no tags are written (I'm not
using any dispatching) and no doping information, so I am suggesting that two
objects which are bitwise compatible in memory can naturally be passed forward
and back down streams between Ada95 and C++ and I can't see why this should
ever break.

(Note : I'm using GNAT3.05 on Sun-SPARC-Solaris2.5. Previous versions of GNAT
had a stream_io bug which held me up but which is now fixed. The C++ compiler
is the SunSoft SPARCCompilerC++ from SunSoft Visual Workshop for C++. Naturally
if the GUI worked with g++ I would have tried direct interworking.)

Thoughts?

Chris

--
--chris.morgan@baesema.co.uk
--Team Ada

Re: Ada95 Streams Question

[Robert I. Eachus]

In article <009A400CB5CAE623.78BE@smcsr3.smcs.se.baesema.co.uk> Chris.Morgan@BAESEMA.CO.UK writes:

   Chris.Morgan@BAESEMA.CO.UK asked:

  > I have a question about Ada95 stream implementations. I mentioned
  > this work a little while back in response to a request for Ada95
  > success stories, but now I'm feeling a little unsettled by
  > accusations of being a hacker!...

  > How likely is it that the GNAT (or some other Ada95 compiler)
  > implementation of Ada stream_io would arbitrarily break this
  > arrangement in future? I am told that the in-stream representation
  > is only recommended to conform to the in-memory, but what are the
  > actual chances of some implementation not following this advice?

    I think that in your situation, the chances are good that it will
always work, but you are taking a chance, and it is probably the first
place to look for trouble if you install a new compiler version, etc.

  > Am I really relying entirely on a pure coincidence that the C++
  > and the Ada95 definitions can work together, or is it in fact the
  > only sensible scheme?

    If the Ada and C++ are on the same processor architecture, you are
pretty safe.  If they can be on different processors, especially
processors with different bit or byte orientations, all bets are off.
In those cases I have used RPC routines to maintain architecture
independence.

--

					Robert I. Eachus

with Standard_Disclaimer;
use  Standard_Disclaimer;
function Message (Text: in Clever_Ideas) return Better_Ideas is...

Re: Ada95 Streams Question

[Samuel Tardieu]

>>>>> "Chris" == Chris Morgan <Chris.Morgan@BAESEMA.CO.UK> writes:

Chris> How likely is it that the GNAT (or some other Ada95 compiler)
Chris> implementation of Ada stream_io would arbitrarily break this
Chris> arrangement in future? I am told that the in-stream
Chris> representation is only recommended to conform to the in-memory,
Chris> but what are the actual chances of some implementation not
Chris> following this advice?

To be general, Ada compilers are free to use any representation of
streams. One may well choose raw conversions (as GNAT does) while
another one will adopt XDR encoding.

Chris> Am I really relying entirely on a pure coincidence that the C++
Chris> and the Ada95 definitions can work together, or is it in fact
Chris> the only sensible scheme?

Streams are not portable between Ada and another language. Of
course, an implementation is free to document how it will encode data
so you can get it from another language (and vice-versa).

Chris> Of course I am talking about the simple case where no tags are
Chris> written (I'm not using any dispatching) and no doping
Chris> information, so I am suggesting that two objects which are
Chris> bitwise compatible in memory can naturally be passed forward
Chris> and back down streams between Ada95 and C++ and I can't see why
Chris> this should ever break.

See above.

  Sam
-- 
  Samuel Tardieu -- sam@ada.eu.org

Re: Ada95 Streams Question

[Robert Dewar]

Sam said

"To be general, Ada compilers are free to use any representation of
streams. One may well choose raw conversions (as GNAT does) while
another one will adopt XDR encoding."

That's correct, as we see from 13.13.2

9   For elementary types, the representation in terms of stream elements is
implementation defined.  For composite types, the Write or Read attribute for
each component is called in a canonical order.  The canonical order of
components is last dimension varying fastest for an array, and positional
aggregate order for a record.  Bounds are not included in the stream if T is
an array type.  If T is a discriminated type, discriminants are included only
if they have defaults.  If T is a tagged type, the tag is not included.


GNAT by default follows the implementation advice in paragaph 17

17   If a stream element is the same size as a storage element, then the
normal in-memory representation should be used by Read and Write for scalar
objects.  Otherwise, Read and Write should use the smallest number of stream
elements needed to represent all values in the base range of the scalar type.


However, this is easily changed, the library package System.Stream_Attributes
(in files s-stratt.ads/adb) comletely define the implementation choice for
representation of elementary types, and this unit is easily replaced, e.g.
by one that uses XDR or some other canonical representation.

Re: Ada95 Streams Question

[Robert Dewar]

"I have connected an Ada95 executable to a C++ executable using streams. The Ada
is the brains of the application, and the C++ is the Motif pretty face. On both
sides I have defined a set of portable types which application types are
converted to for transmission and converted from on receipt."

This should work reasonably reliably with GNAT, cannot say for other
Ada 95 compilers which might do something completely different for
stream rerpresentations of primitive data.

It is a bit of a puzzle to us why you don't just use a procedural
interface. It would be MUCH more portable, and MUCH more efficient

Re: Ada95 Streams Question

[Laurent Pautet]

<michael@ifr.luftfahrt.uni-stuttgart.de> wrote:

| If you just look at the wording of this annex the big picture is a
| huge monolithic program that is now broken into pieces (partitions)
| and executed as a whole on a homogenous network of computers.

This wording is the *spirit* of the annex. And not a
restriction. Everything is there to do whatever you want. Especially
multiprogramming in an heterogenous network.

| That is definitely not what I understand by distributed computing.
| Although this could certainly be interpreted in a more open way, this
| is exactly what is currently implemented for GNAT.

GNAT implements what has been defined in the LRM in order to
validate. The annex has been implemented in order to work in the
context you mentionned above. But it is also working in other
situations. "monolithic application + homogenous network" is a
minimum, but nothing prevents GNAT distributed annex from having a
design that allows you to develop a "multiprogram application + an
heterogenous network". There are already examples of multiprograms
communicating by using distributed objects. Moreover, nothing prevents
GNAT from adding new features as long as the default behaviour follows
the LRM.

| This is not meant as a criticism of the GNAT team. They do a marvelous
| job and just implemented what the standard suggests. I just want the
| Ada community to reconsider the real "openness" of Ada with respect to
| heterogeneous computer environments and client server computing.

I agree that some implementation restrictions are too strong if we
consider what is outside the Ada community and we can argue on
them. Once again, the default compilor behaviour has to follow these
restrictions, but nothing prevents a compilor vendor to bypass these
implementation restrictions in a second step.
-- 
-- Laurent

Re: Ada95 Streams Question

[Chris.Morgan]

Samuel wrote :

>To be general, Ada compilers are free to use any representation of
>streams. One may well choose raw conversions (as GNAT does) while
>another one will adopt XDR encoding.
>

(Please note. My comments below apply to my particular case - using streams
between different processes on the same machine running Solaris.)

This is a good point. However if a compiler took the trouble of offering XDR,
it would presumably document this, and perhaps offer raw streams as well (maybe
with a configuration pragma since they seem, to me, the easiest to support).
Either way, the C++ can again be connected up quite straightforwardly (since
XDR is a natural format for network transfer, I'm sure C++ can be made to read
that in preference to raw data).

It seems to me that yes, in principle what I'm doing is not portable, but it is
unlikely to break when connecting Ada to C++ on comparable platforms, whatever
the version of GNAT. For another compiler it would either work the same, or
some other easily understood way. What I would be surprised by is if an Ada
stream were some new entity with unknown internal structure (neither ASCII, nor
raw binary, nor XDR etc). Why would anyone bother when the Unix (Solaris)
streams facility is already a large superset of the whole Ada Streams concept?

That I think is the root of my question. To me it would be as bad as if the
compiler implementation team redesigned arithmetic on the machine and used
different float layouts than the C compiler (or ABI) of the hardware vendor. On
special machines, yes you may have to. On machines with a proper operating
system, please don't bother!!

Greetings from London (warm and hazy).

Chris Morgan

Re: Ada95 Streams Question

[Robert Dewar]

Michael says:

The first problem arises when you try to exchange stream data files between
computers of different architectures. This is just not foreseen by the Ada 95
standard (see implementation advice in RM and current practice of GNAT).

  The *default* implementation of streams in GNAT indeed follows the
  IA in the RM. However GNAT is set up to make it very easy to choose
  some other implementation, and we have heterogenous distribution very
  much in mind.

  All primitive data is converted to and from stream format using a simple
  procedural interface (System.Stream_Attributes in files s-stratt.ad?) All
  you need to do to get an architecture independent view of streams is to
  replace this one unit (which has a very simple structure) with a different
  one using, e.g. XDR, for data representation.

  We intend to provide an XDR replacement for s-stratt.ad? in the near
  future with some kind of configuratoin pragma to select which one you
  want.

For the described setup it would be even more elegant to utilize an
implementation of the distributed system annex but as this builds on the
stream facility as well you face the same problem as mentioned above.
If you just look at the wording of this annex the big picture is a huge
monolithic program that is now broken into pieces (partitions)

  That's not what I see reading the wording in the annex! There is no
  huge monolithic program. What is called here a partition is what would
  normally be called a program, and a distributed Ada "program" is in fact
  a collection of partitions. Note in particular that with GNAT's source
  based approach, you do not even need to compile these programs on the
  same machine or in the same "library", consistency can be maintained
  purely at the source level.

  You are reading the annex far too narrowly!

and executed as a whole on a homogenous network of computers. That is
definitely not what I understand by distributed computing.

  Well distribution may be homogenous or heterogenous. The Ada model is
  certainly aimed at homogenous distribution, but extends fine to the
  hetergenous case (we first demonstrated hetergenous distribution using
  this annex nearly two years ago at Tri-Ada, running on Sun's and SGI's)

Although this could certainly be interpreted in a more open way, this is
exactly what is currently implemented for GNAT.

  We do interprete it very openly, look more closely. Have a look at
  s-stratt. Even if you can't replace this yourself (which is really
  easy to do), other people including us will provide replacements
  soon.

Re: Ada95 Streams Question

[michael]

Robert Dewar <dewar@GNAT.COM> wrote:
> "I have connected an Ada95 executable to a C++ executable using streams. The Ada
> is the brains of the application, and the C++ is the Motif pretty face. On both
> sides I have defined a set of portable types which application types are
> converted to for transmission and converted from on receipt."
> 
> This should work reasonably reliably with GNAT, cannot say for other
> Ada 95 compilers which might do something completely different for
> stream rerpresentations of primitive data.
> 
> It is a bit of a puzzle to us why you don't just use a procedural
> interface. It would be MUCH more portable, and MUCH more efficient

Of course I can only speculate why such an approach might make sense for the
original poster of this question (I don't know who that was) but I can say
that I can think of a lot of reasons to try this because we are facing
exactly the same problem.

We are in the busyness of vehicle trajectory optimization. The software package
we have developed for this purpose manly consists of two parts. One part, which
the original poster called the "pretty face", is the graphical user interface
which controls the whole operation of the software and the second part is
"the brains of the application" which does all the numerics and contains
the user provided procedures to describe the vehicle model. When you are
working with a model this "brain" has to be recompiled and relinked several
times and this process is controlled by the "pretty face". According to
this setup it obviously makes sense to keep these two parts separate and not
link them into one big executable. But, when the "brain" is working it has to
closely interact with the "pretty face" which is, e.g. showing the progress of the
optimization process.

The two processes are connected with each other via files and pipes and it
would now be very elegant to use streams to read and write the data. I always
thought that this is exactly what streams are made for.

A further advantage of this separation is that you can run the "brain" on
another computer, maybe a high performance server. This kind of distribution
seems to be quite natural to me but I sometimes have the feeling that the Ada
community has not yet understood the benefits of distribution and client server
computing.

The first problem arises when you try to exchange stream data files between
computers of different architectures. This is just not foreseen by the Ada 95
standard (see implementation advice in RM and current practice of GNAT).

For the described setup it would be even more elegant to utilize an implementation
of the distributed system annex but as this builds on the stream facility as well
you face the same problem as mentioned above. If you just look at the wording
of this annex the big picture is a huge monolithic program that is now broken
into pieces (partitions) and executed as a whole on a homogenous network of
computers. That is definitely not what I understand by distributed computing.
Although this could certainly be interpreted in a more open way, this is
exactly what is currently implemented for GNAT.

This is not meant as a criticism of the GNAT team. They do a marvelous job and
just implemented what the standard suggests. I just want the Ada community to
reconsider the real "openness" of Ada with respect to heterogeneous computer
environments and client server computing.

Michael

-- 
------------------------------------------------------------------------
--Dipl.-Ing. Michael Paus   (Member: Team Ada)
--University of Stuttgart, Inst. of Flight Mechanics and Flight Control
--Forststrasse 86, 70176 Stuttgart, Germany
--Phone: (+49) 711-121-1434  FAX: (+49) 711-634856
--Email: Michael.Paus@ifr.luftfahrt.uni-stuttgart.de (NeXT-Mail welcome)

Re: Ada95 Streams Question

[Theodore E. Dennison]

michael@ifr.luftfahrt.uni-stuttgart.de wrote:
> 
> the user provided procedures to describe the vehicle model. When you are
> working with a model this "brain" has to be recompiled and relinked several
> times and this process is controlled by the "pretty face". According to
> this setup it obviously makes sense to keep these two parts separate and not
> link them into one big executable. But, when the "brain" is working it has to

This screams UIL (Motif) to me.

> The two processes are connected with each other via files and pipes and it
> would now be very elegant to use streams to read and write the data. I always
> thought that this is exactly what streams are made for.
> 
> A further advantage of this separation is that you can run the "brain" on
> another computer, maybe a high performance server. This kind of distribution
> seems to be quite natural to me but I sometimes have the feeling that the Ada
> community has not yet understood the benefits of distribution and client server
> computing.

A X-windows server does exactly this (in a more portable manner).

All of this sounds to me like you are using a lot of work to re-invent 
X-Windows and Motif. If you have a UNIX box, these both probably came with
it. If you don't have a UNIX platform, I'll bet there is a commercially
available version of X and Motif for your platform.

-- 
T.E.D.          
                |  Work - mailto:dennison@escmail.orl.mmc.com  |
                |  Home - mailto:dennison@iag.net              |
                |  URL  - http://www.iag.net/~dennison         |

Re: Ada95 Streams Question

[michael]

dewar@cs.nyu.edu (Robert Dewar) wrote:
> [...]
>   All primitive data is converted to and from stream format using a simple
>   procedural interface (System.Stream_Attributes in files s-stratt.ad?) All
>   you need to do to get an architecture independent view of streams is to
>   replace this one unit (which has a very simple structure) with a different
>   one using, e.g. XDR, for data representation.

I already had a look at that and it indeed appears to be quite straight forward
to make the necessay changes. What I am not sure about is whether I can just
substitute s-stratt.adb with a new XDR version or whether I will have to rebuild
the compiler too. I just did not yet have the time to check that out (I admit
that a look into the makefile should make that clear).

>   We intend to provide an XDR replacement for s-stratt.ad? in the near
>   future with some kind of configuratoin pragma to select which one you
>   want.

Controlling the behaviour of the compiler with a configuration pragma is probably
the best solution for this problem.

>   Well distribution may be homogenous or heterogenous. The Ada model is
>   certainly aimed at homogenous distribution, but extends fine to the
>   hetergenous case (we first demonstrated hetergenous distribution using
>   this annex nearly two years ago at Tri-Ada, running on Sun's and SGI's)

That's interesting, because the current version of the distributed system
annex does not work at all on an SGI (at least not on my one). Although
it compiles and links out of the box I have not yet succeeded to get any
of the examples working. On the Sun these examples do work without problems
and so do my own ones.

-- 
------------------------------------------------------------------------
--Dipl.-Ing. Michael Paus   (Member: Team Ada)
--University of Stuttgart, Inst. of Flight Mechanics and Flight Control
--Forststrasse 86, 70176 Stuttgart, Germany
--Phone: (+49) 711-121-1434  FAX: (+49) 711-634856
--Email: Michael.Paus@ifr.luftfahrt.uni-stuttgart.de (NeXT-Mail welcome)

Re: Ada95 Streams Question

[Robert Dewar]

Michael asked

"I already had a look at that and it indeed appears to be quite straight forward
to make the necessay changes. What I am not sure about is whether I can just
substitute s-stratt.adb with a new XDR version or whether I will have to rebuild
the compiler too. I just did not yet have the time to check that out (I admit
that a look into the makefile should make that clear)."

There is no need to rebuild the compiler, this is strictly a runtime
routine. Of course, as usual, you should compile with -gnatn to avoid
overheads from extra calls to these small routines in the runtime library.

Re: Ada95 Streams Question

[Robert A Duff]

In article <dewar.835633090@schonberg>, Robert Dewar <dewar@cs.nyu.edu> wrote:
>Michael says:
>For the described setup it would be even more elegant to utilize an
>implementation of the distributed system annex but as this builds on the
>stream facility as well you face the same problem as mentioned above.
>If you just look at the wording of this annex the big picture is a huge
>monolithic program that is now broken into pieces (partitions)
>
>  That's not what I see reading the wording in the annex! There is no
>  huge monolithic program. What is called here a partition is what would
>  normally be called a program, and a distributed Ada "program" is in fact
>  a collection of partitions.

And the Distributed Systems annex (and chapter 10) says that you can
start and stop partitions independently of one another, while the
distributed program as a whole is running.  This doesn't sound
"monolithic" to me.  On the contrary, a distributed program is a rather
loosely connected set of relatively independent partitions.

For example, suppose you find a bug in one partition.  You can stop it,
fix the bug, recompile, and run the new version of that partition, while
the program as a whole continues to run.  (Assuming, of course, that
you've designed the other partitions to work properly in that scenario.)

>and executed as a whole on a homogenous network of computers. That is
>definitely not what I understand by distributed computing.
>
>  Well distribution may be homogenous or heterogenous. The Ada model is
>  certainly aimed at homogenous distribution, but extends fine to the
>  hetergenous case (we first demonstrated hetergenous distribution using
>  this annex nearly two years ago at Tri-Ada, running on Sun's and SGI's)

I believe the attitude of the designers of the Distributed Systems annex
was:

    1. Provide support for homogeneous distributed systems.

    2. Solving the problem of heterogeneous is beyond the scope of the
       annex, but we don't want to get in the way of supporting
       heterogeneous systems.  We hope compiler writers will address
       those issues, and who knows, maybe that support will become
       standard someday.  Thus, the DS annex should not do anything that
       *prevents* its use in heterogeneous systems.

- Bob

Re: Ada95 Streams Question

[Robert Dewar]

Bob Duff said

"    2. Solving the problem of heterogeneous is beyond the scope of the
       annex, but we don't want to get in the way of supporting
       heterogeneous systems.  We hope compiler writers will address
       those issues, and who knows, maybe that support will become
       standard someday.  Thus, the DS annex should not do anything that
       *prevents* its use in heterogeneous systems."

In fact there is nothing in the annex that is in any way relevant to the
homgenous vs hetergenous distribution issue, except for the recommendation
that the default stream representation of primitive data types correspond
to their memory representation, and that is only a recommendation, which
is simply clearly inappropriate for a hetergenous implementation (the
whole point of implementation advice is that it is advice, you ignore it
if it is inappropriate to your implementation).

Sure there are additional implementation problems to be solved in the
hetergenous case, but these are implementation issues, not language issues.

Re: Ada95 Streams Question

[Laurent Pautet]

Robert Dewar <dewar@cs.nyu.edu> wrote:
>
> There is no need to rebuild the compiler, this is strictly a runtime
> routine. Of course, as usual, you should compile with -gnatn to avoid
> overheads from extra calls to these small routines in the runtime library.

Are you sure ? I think that for instance, arrays are always marshalled
into a stream array rounded up to 4 bytes. For instance, a 13 bytes
array is mapped onto a 16 bytes stream array. Therefore, you need to
modify the compiler.


-- 
-- Laurent

Re: Ada95 Streams Question

[Robert Dewar]

Laurent said

"Are you sure ? I think that for instance, arrays are always marshalled
into a stream array rounded up to 4 bytes. For instance, a 13 bytes
array is mapped onto a 16 bytes stream array. Therefore, you need to
modify the compiler."

this is wrong, a 13 byte array causes the bytes of the array to be written
out as individual stream elements, and the resulting stream will be 13
bytes long. I really don't know what Laurent has in mind, but probably
something significant, since he knows this code pretty well! But right
now I don't see it. The unit s-stratt among other things dicates the
length in bytes used to represent each individual primitive data type.

Re: Ada95 Streams Question

[sam]

>>>>> "Robert" == Robert Dewar <dewar@cs.nyu.edu> writes:

Robert> this is wrong, a 13 byte array causes the bytes of the array
Robert> to be written out as individual stream elements, and the
Robert> resulting stream will be 13 bytes long. I really don't know
Robert> what Laurent has in mind, but probably something significant,
Robert> since he knows this code pretty well!

Laurent's remark was concerning XDR IMHO, not the way it's done in
GNAT.

  Sam
-- 
  Samuel Tardieu -- sam@ada.eu.org