2dsa | !2dsa ?

2dsa | !2dsa ?

[Rod Kay]

Hello all,

    I've heard that the Distributed Systems Annex (DSA) may be dropped from the Ada standard soon. Can anyone confirm this ?

    I've been using the Polyorb implementation of DSA for some time and find it very useful. The way in which it abstracts away socket 'plumbing' details makes for very simple/understandable  comms.

Re: 2dsa | !2dsa ?

[Randy Brukardt]

"Rod Kay" <rodakay5@gmail.com> wrote in message 
news:d3f3b7fb-e051-428b-a947-32310a64933en@googlegroups.com...
> Hello all,
>
>    I've heard that the Distributed Systems Annex (DSA) may be dropped from
> the Ada standard soon. Can anyone confirm this ?

Annex E remains in the proposed Ada 202x standard.

Compiler support, of course, is up to vendors. Dunno if anyone is still 
supporting it.

>    I've been using the Polyorb implementation of DSA for some time and 
> find
> it very useful. The way in which it abstracts away socket 'plumbing' 
> details makes
> for very simple/understandable  comms.

That was the promise, not sure it ever really was realized. Since the Annex 
was weakened enough that third-party support isn't really possible anymore 
(necessary to allow it to be used with current middleware), it's really a 
vendor-specific thing these days.

                            Randy.

Re: 2dsa | !2dsa ?

[Dmitry A. Kazakov]

On 2020-12-23 02:32, Randy Brukardt wrote:
> "Rod Kay" <rodakay5@gmail.com> wrote in message
> news:d3f3b7fb-e051-428b-a947-32310a64933en@googlegroups.com...
>> Hello all,
>>
>>     I've heard that the Distributed Systems Annex (DSA) may be dropped from
>> the Ada standard soon. Can anyone confirm this ?
> 
> Annex E remains in the proposed Ada 202x standard.
> 
> Compiler support, of course, is up to vendors. Dunno if anyone is still
> supporting it.

It should be moved to the user level. As specified in the Annex there 
seems no obvious way to provide a user-defined transport for DSA, and 
there seems no way to have different implementations of DSA in the same 
program.

>>     I've been using the Polyorb implementation of DSA for some time and
>> find
>> it very useful. The way in which it abstracts away socket 'plumbing'
>> details makes
>> for very simple/understandable  comms.

Yes, but too simple to be universally useful.

> That was the promise, not sure it ever really was realized. Since the Annex
> was weakened enough that third-party support isn't really possible anymore
> (necessary to allow it to be used with current middleware), it's really a
> vendor-specific thing these days.

Yes, I always wished to include DSA support based on various 
communication protocols I have implemented in Ada, rather than plain 
sockets. E.g. I have a ready-to-go DSA implementation for interprocess 
communication over shared memory, but no idea how to make GNAT aware of 
it. Or AQMP and ASN.1 look like a straightforward candidate as a DSA 
transport as they have detailed type description systems to map Ada objects.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: 2dsa | !2dsa ?

[Maxim Reznik]

I forked a older (Garlic) GNAT DSA implementation and found it quite hackable. :)
My idea is to implement a WebSocket/WebRTC transport and compile it by GNAT-LLVM to WebAssembly to have distributed Ada applications in a browser. I have a working proof of concept demo already :)

https://github.com/reznikmm/garlic/tree/web_socket

Re: 2dsa | !2dsa ?

[Dmitry A. Kazakov]

On 2020-12-24 13:02, Maxim Reznik wrote:
> I forked a older (Garlic) GNAT DSA implementation and found it quite hackable. :)
> My idea is to implement a WebSocket/WebRTC transport and compile it by GNAT-LLVM to WebAssembly to have distributed Ada applications in a browser. I have a working proof of concept demo already :)
> 
> https://github.com/reznikmm/garlic/tree/web_socket

My question is how to proceed without GLADE/Garlic etc. I have DSA 
implemented, e.g. System.RPC. I need GNAT to accept it as such.

In a more distant perspective I need a work-around of stream attributes. 
They are non-portable, so there must an option to replace them for DSA 
and/or provide a non-stream parameter marshaling when the transport is a 
higher-level protocol, e.g. CANopen, EtherCAT, ASN.1, AMQP etc. For 
these you must map Ada types into the types defined by the protocol.

Without this DSA is pretty much pointless.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: 2dsa | !2dsa ?

[Rod Kay]

On Friday, 25 December 2020 at 00:30:59 UTC+11, Dmitry A. Kazakov wrote:
> On 2020-12-24 13:02, Maxim Reznik wrote: 
> > I forked a older (Garlic) GNAT DSA implementation and found it quite hackable. :) 
> > My idea is to implement a WebSocket/WebRTC transport and compile it by GNAT-LLVM to WebAssembly to have distributed Ada applications in a browser. I have a working proof of concept demo already :) 
> > 
> > https://github.com/reznikmm/garlic/tree/web_socket
> My question is how to proceed without GLADE/Garlic etc. I have DSA 
> implemented, e.g. System.RPC. I need GNAT to accept it as such. 
> 
> In a more distant perspective I need a work-around of stream attributes. 
> They are non-portable, so there must an option to replace them for DSA 
> and/or provide a non-stream parameter marshaling when the transport is a 
> higher-level protocol, e.g. CANopen, EtherCAT, ASN.1, AMQP etc. For 
> these you must map Ada types into the types defined by the protocol. 
> 
> Without this DSA is pretty much pointless.
> -- 
> Regards, 
> Dmitry A. Kazakov 
> http://www.dmitry-kazakov.de

Re: 2dsa | !2dsa ?

[Rod Kay]

Apologies for the prior empty post.

Is it likely that the ARG might address the aformentioned issues ?

Re: 2dsa | !2dsa ?

[Randy Brukardt]

"Rod Kay" <rodakay5@gmail.com> wrote in message 
news:30f2c0d8-58c2-4ca9-bfd6-2aa38309d27cn@googlegroups.com...
>Apologies for the prior empty post.
>
>Is it likely that the ARG might address the aformentioned issues ?

As of now, it doesn't appear that there would be any point. Annex E is an 
optional annex, and so far as we're aware, no compiler vendor has any plans 
for increasing support for that annex. So the ARG could change the annex but 
it seems unlikely that any changes would make it into implementations. 
(We've been told not to expect even the implementation of bugs fixes 
included in Ada 202x, even from the vendor that originally requested the bug 
fixes.)

                                      Randy.

Re: 2dsa | !2dsa ?

[Dmitry A. Kazakov]

On 2020-12-29 00:41, Randy Brukardt wrote:
> "Rod Kay" <rodakay5@gmail.com> wrote in message
> news:30f2c0d8-58c2-4ca9-bfd6-2aa38309d27cn@googlegroups.com...
>> Apologies for the prior empty post.
>>
>> Is it likely that the ARG might address the aformentioned issues ?
> 
> As of now, it doesn't appear that there would be any point. Annex E is an
> optional annex, and so far as we're aware, no compiler vendor has any plans
> for increasing support for that annex. So the ARG could change the annex but
> it seems unlikely that any changes would make it into implementations.
> (We've been told not to expect even the implementation of bugs fixes
> included in Ada 202x, even from the vendor that originally requested the bug
> fixes.)

Why there should be any vendor support in the first place? Why not to 
redefine it as a set of abstract tagged types allowing custom user 
implementations like storage pools and streams do?

The idea of having an IDL, statically assigned partitions, linking 
everything together before start is not the way the distributed systems 
are designed and work today. CORBA died for a reason.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: 2dsa | !2dsa ?

[Luke A. Guest]

On 29/12/2020 14:56, Dmitry A. Kazakov wrote:

>> for increasing support for that annex. So the ARG could change the 
>> annex but
>> it seems unlikely that any changes would make it into implementations.
>> (We've been told not to expect even the implementation of bugs fixes
>> included in Ada 202x, even from the vendor that originally requested 
>> the bug
>> fixes.)
> 
> Why there should be any vendor support in the first place? Why not to 
> redefine it as a set of abstract tagged types allowing custom user 
> implementations like storage pools and streams do?

Would the compiler still need any support for this or would it just be a 
set of interfaces at library level?

Re: 2dsa | !2dsa ?

[Dmitry A. Kazakov]

On 2020-12-29 16:14, Luke A. Guest wrote:
> On 29/12/2020 14:56, Dmitry A. Kazakov wrote:
> 
>>> for increasing support for that annex. So the ARG could change the 
>>> annex but
>>> it seems unlikely that any changes would make it into implementations.
>>> (We've been told not to expect even the implementation of bugs fixes
>>> included in Ada 202x, even from the vendor that originally requested 
>>> the bug
>>> fixes.)
>>
>> Why there should be any vendor support in the first place? Why not to 
>> redefine it as a set of abstract tagged types allowing custom user 
>> implementations like storage pools and streams do?
> 
> Would the compiler still need any support for this or would it just be a 
> set of interfaces at library level?

Yes, because the idea is to have remote objects and remote calls looking 
exactly same as local objects and local calls.

So the compiler must translate a call to an RPC to a call to some user 
primitive operation like System.RPC does. The operation would have a 
controlling parameter "connection" or "remote partition". The actual 
input values of the original call must be marshaled, e.g. as an output 
stream. The output values and the result will be returned via an input 
stream and deserialized from there into the actual parameters/result or 
else into a remote exception occurrence to re-raise locally if that was 
the outcome.

Here lie a lot of problems. First is non-portability of stream 
attributes. Second is lack of support for bulky transfers and 
multiplexing. It is highly desirable that the output stream could be 
written in chunks as well as reading the input stream. E.g. if you pass 
large objects or if you want to multiplex RPCs made from different tasks 
rather than interlock them (which for synchronous RPC would result in 
catastrophic performance).

The current Annex E is very crude to allow efficient, low-latency, 
real-time implementations.

P.S. If Ada supported delegation, introspection and getter/setter 
interface, then, probably, all remote call/objects stuff could be made 
at the library level. But for now, compiler magic is needed.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: 2dsa | !2dsa ?

[Randy Brukardt]

"Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> wrote in message 
news:rsfg32$1etf$1@gioia.aioe.org...
> On 2020-12-29 00:41, Randy Brukardt wrote:
>> "Rod Kay" <rodakay5@gmail.com> wrote in message
>> news:30f2c0d8-58c2-4ca9-bfd6-2aa38309d27cn@googlegroups.com...
>>> Apologies for the prior empty post.
>>>
>>> Is it likely that the ARG might address the aformentioned issues ?
>>
>> As of now, it doesn't appear that there would be any point. Annex E is an
>> optional annex, and so far as we're aware, no compiler vendor has any 
>> plans
>> for increasing support for that annex. So the ARG could change the annex 
>> but
>> it seems unlikely that any changes would make it into implementations.
>> (We've been told not to expect even the implementation of bugs fixes
>> included in Ada 202x, even from the vendor that originally requested the 
>> bug
>> fixes.)
>
> Why there should be any vendor support in the first place? Why not to 
> redefine it as a set of abstract tagged types allowing custom user 
> implementations like storage pools and streams do?

Marshalling/unmarshalling surely require vendor support, and there has to be 
a standard interface for the marshalling stuff to talk to. That to me was 
always the value of Annex E. My understanding is that there is not much 
interest in doing any work at all, even to correct the mistakes in the 
existing definitions.

In any case, Storage_Pools and Streams are some of the most expensive 
features of Ada to support. That's not a model for "lightweight" support of 
anything.

My advice would be to talk to your vendor if you feel strongly about this 
sort of support.

                            Randy.

Re: 2dsa | !2dsa ?

[Rod Kay]

Thanks to all for the info. Not quite the news I'd hoped for :) . I'll probably still persist with polyorb dsa for a while, since I've invested a goodish amount of time (and code) there.

Regards.

Re: 2dsa | !2dsa ?

[Shark8]

On Thursday, December 24, 2020 at 6:30:59 AM UTC-7, Dmitry A. Kazakov wrote:
> On 2020-12-24 13:02, Maxim Reznik wrote: 
> > I forked a older (Garlic) GNAT DSA implementation and found it quite hackable. :) 
> > My idea is to implement a WebSocket/WebRTC transport and compile it by GNAT-LLVM to WebAssembly to have distributed Ada applications in a browser. I have a working proof of concept demo already :) 
> > 
> > https://github.com/reznikmm/garlic/tree/web_socket
> My question is how to proceed without GLADE/Garlic etc. I have DSA 
> implemented, e.g. System.RPC. I need GNAT to accept it as such. 
What do you mean by this?
How is GNAT "not accepting" it?
More, you mentioned the possibility of using a set of tagged-types, "like Storage_Pool", elsethread; is this how you implemented them, or a suggestion for a new standard interface?

> 
> In a more distant perspective I need a work-around of stream attributes. 
> They are non-portable, so there must an option to replace them for DSA 
> and/or provide a non-stream parameter marshaling when the transport is a 
> higher-level protocol, e.g. CANopen, EtherCAT, ASN.1, AMQP etc. For 
> these you must map Ada types into the types defined by the protocol. 
I have some ideas on how this could be achieved.
I'll make a note for my To-Do list on Byron, but I was thinking that a good method to achieve this for an OS (WRT ASN.1) would be to have at the base a SOM-like system with the base meta-object having a serialize/deserialize pair of methods (with one parameter being the encoding scheme)... I don't see why that [the idea, in principle] couldn't be 'lifted' up to the DSA level and integrated into it [PolyOrb or other DSA implementation].

Re: 2dsa | !2dsa ?

[Dmitry A. Kazakov]

On 2021-01-11 15:59, Shark8 wrote:
> On Thursday, December 24, 2020 at 6:30:59 AM UTC-7, Dmitry A. Kazakov wrote:
>> On 2020-12-24 13:02, Maxim Reznik wrote:
>>> I forked a older (Garlic) GNAT DSA implementation and found it quite hackable. :)
>>> My idea is to implement a WebSocket/WebRTC transport and compile it by GNAT-LLVM to WebAssembly to have distributed Ada applications in a browser. I have a working proof of concept demo already :)
>>>
>>> https://github.com/reznikmm/garlic/tree/web_socket
>> My question is how to proceed without GLADE/Garlic etc. I have DSA
>> implemented, e.g. System.RPC. I need GNAT to accept it as such.
> What do you mean by this?
> How is GNAT "not accepting" it?

It does not invoke the provided DSA implementation for remote calls.

> More, you mentioned the possibility of using a set of tagged-types, "like Storage_Pool", elsethread; is this how you implemented them, or a suggestion for a new standard interface?

It was a suggestion. The current way is System.RPC see ARM E.5. It seems 
that merely replacing it is not enough for GNAT.

>> In a more distant perspective I need a work-around of stream attributes.
>> They are non-portable, so there must an option to replace them for DSA
>> and/or provide a non-stream parameter marshaling when the transport is a
>> higher-level protocol, e.g. CANopen, EtherCAT, ASN.1, AMQP etc. For
>> these you must map Ada types into the types defined by the protocol.
> I have some ideas on how this could be achieved.
> I'll make a note for my To-Do list on Byron, but I was thinking that a good method to achieve this for an OS (WRT ASN.1) would be to have at the base a SOM-like system with the base meta-object having a serialize/deserialize pair of methods (with one parameter being the encoding scheme)...

The problem is that

1. They must doubly dispatch on the object type and on the container 
type, since there could be any number of containers = encoding/decoding 
schema.

2. They must support chunking. E.g. deserialize cannot be a function. 
This will get you a whole bunch of nasty problems with partial 
serialization and indefinite types. [I do not see these resolved without 
properly done constructors and co-routines.]

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: 2dsa | !2dsa ?

[Shark8]

On Monday, January 11, 2021 at 8:32:25 AM UTC-7, Dmitry A. Kazakov wrote:
> On 2021-01-11 15:59, Shark8 wrote: 
> > On Thursday, December 24, 2020 at 6:30:59 AM UTC-7, Dmitry A. Kazakov wrote: 
> >> On 2020-12-24 13:02, Maxim Reznik wrote: 
> >>> I forked a older (Garlic) GNAT DSA implementation and found it quite hackable. :) 
> >>> My idea is to implement a WebSocket/WebRTC transport and compile it by GNAT-LLVM to WebAssembly to have distributed Ada applications in a browser. I have a working proof of concept demo already :) 
> >>> 
> >>> https://github.com/reznikmm/garlic/tree/web_socket 
> >> My question is how to proceed without GLADE/Garlic etc. I have DSA 
> >> implemented, e.g. System.RPC. I need GNAT to accept it as such. 
> > What do you mean by this? 
> > How is GNAT "not accepting" it?
> It does not invoke the provided DSA implementation for remote calls.
This is a dumb question, but did you remember to recompile the run-time library?
https://www.cs.fsu.edu/~baker/ada/gnat/html/gnat_ugn_20.html#SEC230
(I'm pretty sure you'd have to, to get a change into DSA behavior.)

> > More, you mentioned the possibility of using a set of tagged-types, "like Storage_Pool", elsethread; is this how you implemented them, or a suggestion for a new standard interface?
> It was a suggestion. The current way is System.RPC see ARM E.5. It seems 
> that merely replacing it is not enough for GNAT.
Given "The package System.RPC is a language-defined interface to the PCS" it seems like that there are two readings:
(1) System.RPC defines the forms with which remote-calls are done, and/or
(2) implements the actual communications.
As it is written, I would think that #1 is the proper interpretation, if this is the case than recompiling System.RPC is "mucking about with the interface" and not much else.

The item "An object of the type Params_Stream_Type is used for identifying the particular remote subprogram that is being called, as well as marshalling and unmarshalling the parameters or result of a remote subprogram call, as part of sending them between partition" seems to indicate that #2 is the proper reading though.

> >> In a more distant perspective I need a work-around of stream attributes. 
> >> They are non-portable, so there must an option to replace them for DSA 
> >> and/or provide a non-stream parameter marshaling when the transport is a 
> >> higher-level protocol, e.g. CANopen, EtherCAT, ASN.1, AMQP etc. For 
> >> these you must map Ada types into the types defined by the protocol. 
> > I have some ideas on how this could be achieved. 
> > I'll make a note for my To-Do list on Byron, but I was thinking that a good method to achieve this for an OS (WRT ASN.1) would be to have at the base a SOM-like system with the base meta-object having a serialize/deserialize pair of methods (with one parameter being the encoding scheme)...
> The problem is that 
> 
> 1. They must doubly dispatch on the object type and on the container 
> type, since there could be any number of containers = encoding/decoding 
> schema. 
Why?
You could have the encoding/decoding method as a parameter.


> 2. They must support chunking. E.g. deserialize cannot be a function.
Why?
I can't think of anything off the top of my head that directly prohibits us from making it a function; at least X'Stream'Input-wise.

> This will get you a whole bunch of nasty problems with partial 
> serialization and indefinite types. [I do not see these resolved without 
> properly done constructors and co-routines.]
I don't see why the partial-serialization is a problem: if you have some type's value to serialize, you're already constrained.
If you're reading from a stream to initialize/reconstruct a type-value, then you're doing X'Stream'Input, right?

Re: 2dsa | !2dsa ?

[Dmitry A. Kazakov]

On 2021-01-11 21:35, Shark8 wrote:
> On Monday, January 11, 2021 at 8:32:25 AM UTC-7, Dmitry A. Kazakov wrote:
>> On 2021-01-11 15:59, Shark8 wrote:
>>> On Thursday, December 24, 2020 at 6:30:59 AM UTC-7, Dmitry A. Kazakov wrote:
>>>> On 2020-12-24 13:02, Maxim Reznik wrote:
>>>>> I forked a older (Garlic) GNAT DSA implementation and found it quite hackable. :)
>>>>> My idea is to implement a WebSocket/WebRTC transport and compile it by GNAT-LLVM to WebAssembly to have distributed Ada applications in a browser. I have a working proof of concept demo already :)
>>>>>
>>>>> https://github.com/reznikmm/garlic/tree/web_socket
>>>> My question is how to proceed without GLADE/Garlic etc. I have DSA
>>>> implemented, e.g. System.RPC. I need GNAT to accept it as such.
>>> What do you mean by this?
>>> How is GNAT "not accepting" it?
>> It does not invoke the provided DSA implementation for remote calls.
> This is a dumb question, but did you remember to recompile the run-time library?

No. Why should I?

> https://www.cs.fsu.edu/~baker/ada/gnat/html/gnat_ugn_20.html#SEC230
> (I'm pretty sure you'd have to, to get a change into DSA behavior.)

Do you mean that other implementations rebuild all RTL? That is a mess. 
Is it possible to fool GNAT using linker instead of breaking everything?

>>> More, you mentioned the possibility of using a set of tagged-types, "like Storage_Pool", elsethread; is this how you implemented them, or a suggestion for a new standard interface?
>> It was a suggestion. The current way is System.RPC see ARM E.5. It seems
>> that merely replacing it is not enough for GNAT.
> Given "The package System.RPC is a language-defined interface to the PCS" it seems like that there are two readings:
> (1) System.RPC defines the forms with which remote-calls are done, and/or
> (2) implements the actual communications.
> As it is written, I would think that #1 is the proper interpretation, if this is the case than recompiling System.RPC is "mucking about with the interface" and not much else.

My understanding is that the Do_RPC gets called with the parameters of 
the call written into the Params stream including the target function 
name. Then you do communication with the remote host associated with the 
Partition parameter. The communication basically reads the Params stream 
and sends read stuff to the other side. Then you sit and wait for the 
remote host to respond. If the response was success, you write its 
payload into the Result stream. If the response indicates remote 
exception you re-raise it locally.

BTW, there is a lot of overhead there. The design support streams being 
actual communication streams. But for that, the partition would be set 
into the stream prior calling to the first Write. Furthermore, it is to 
tightly coupled. There is no indications where parameters start and end 
in the stream. The remote host needs to be able to say "cut that, I do 
not receive this call anyway", "I cannot parse parameters for this 
function" etc.

>>>> In a more distant perspective I need a work-around of stream attributes.
>>>> They are non-portable, so there must an option to replace them for DSA
>>>> and/or provide a non-stream parameter marshaling when the transport is a
>>>> higher-level protocol, e.g. CANopen, EtherCAT, ASN.1, AMQP etc. For
>>>> these you must map Ada types into the types defined by the protocol.
>>> I have some ideas on how this could be achieved.
>>> I'll make a note for my To-Do list on Byron, but I was thinking that a good method to achieve this for an OS (WRT ASN.1) would be to have at the base a SOM-like system with the base meta-object having a serialize/deserialize pair of methods (with one parameter being the encoding scheme)...
>> The problem is that
>>
>> 1. They must doubly dispatch on the object type and on the container
>> type, since there could be any number of containers = encoding/decoding
>> schema.
> Why?
> You could have the encoding/decoding method as a parameter.

A scalar value is not enough. For one you need to enumerate all possible 
methods to build a case statement on them, which you cannot do as 
implementation must be located in different libraries.

Then you need some context to encode/decode referential types. Imagine a 
parameter containing an access type, e.g. a node of a linked list. You 
cannot deep-copy everything with an infinite recursion on top.

>> 2. They must support chunking. E.g. deserialize cannot be a function.
> Why?

Because it will block waiting all data read. So you will need an extra 
task for each transfer in order to accumulate all chuncks.

>> This will get you a whole bunch of nasty problems with partial
>> serialization and indefinite types. [I do not see these resolved without
>> properly done constructors and co-routines.]
> I don't see why the partial-serialization is a problem: if you have some type's value to serialize, you're already constrained.
> If you're reading from a stream to initialize/reconstruct a type-value, then you're doing X'Stream'Input, right?

Remember, it does not block. So when you call X'Stream'Input you may get 
0 stream elements read at any point. E.g. you read a 4-octet Integer. 
You get only two octets, because the other two are not here. You cannot 
return Integer, it is not yet constructed. You cannot wait, because you 
are in the callback from the reader task. You must remember the state of 
partial deserialization and pass control back to the reader. This is 
what a co-routines are.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de