On intended use cases of the distributed annex

On intended use cases of the distributed annex

[Alejandro R. Mosteo]

Hello people,

I am working nowadays in some heavy simulations that could benefit form parallelizing batches of work. Currently I either manually perform the launches of different lots, or leverage a cluster managed with Condor [1] at my institution, but using C code.

I was thinking about the possibility of requesting the installation of an Ada compiler in such cluster, and also if the DSA can fulfill my needs. I guess I could dig into the docs but probably some of you know some pitfalls that could elude me.

My question is: is the DSA well-suited for load balancing distribution, even if it is of simple nature. My impression from hearsay is that the DSA allows one to define a structure of nodes, and that you program as if this was a single entity, and the remote interactions are done for you. What I'm not sure of is if it supports, for example, is things like "allocate as many nodes as online machines", node migrations and things like that. Or, things like resilience through redundant nodes.

In other words, does the DSA support reconfiguration (even if it is at launch time) of nodes without editing a config file, or even online reconfiguration without manual intervention? 

Thank you in advance, 
Alex.

[1] http://research.cs.wisc.edu/htcondor/description.html

Re: On intended use cases of the distributed annex

[tonyg]

I used the DSA quite regularly and recently, but the job was not as complex as this. However I do know of a guy working at the french nuclear research institute who made a stack of PC's do some heavy computation using the DSA to break apart the computation job and put it back together using asynchronous messaging. It was quite a feat!

On Wednesday, 24 September 2014 10:37:49 UTC+1, Alejandro R. Mosteo  wrote:
> Hello people,
> 
> 
> 
> I am working nowadays in some heavy simulations that could benefit form parallelizing batches of work. Currently I either manually perform the launches of different lots, or leverage a cluster managed with Condor [1] at my institution, but using C code.
> 
> 
> 
> I was thinking about the possibility of requesting the installation of an Ada compiler in such cluster, and also if the DSA can fulfill my needs. I guess I could dig into the docs but probably some of you know some pitfalls that could elude me.
> 
> 
> 
> My question is: is the DSA well-suited for load balancing distribution, even if it is of simple nature. My impression from hearsay is that the DSA allows one to define a structure of nodes, and that you program as if this was a single entity, and the remote interactions are done for you. What I'm not sure of is if it supports, for example, is things like "allocate as many nodes as online machines", node migrations and things like that. Or, things like resilience through redundant nodes.
> 
> 
> 
> In other words, does the DSA support reconfiguration (even if it is at launch time) of nodes without editing a config file, or even online reconfiguration without manual intervention? 
> 
> 
> 
> Thank you in advance, 
> 
> Alex.
> 
> 
> 
> [1] http://research.cs.wisc.edu/htcondor/description.html

Re: On intended use cases of the distributed annex

[tonyg]

On Wednesday, 24 September 2014 11:08:42 UTC+1, tonyg  wrote:
> I used the DSA quite regularly and recently, but the job was not as complex as this. However I do know of a guy working at the french nuclear research institute who made a stack of PC's do some heavy computation using the DSA to break apart the computation job and put it back together using asynchronous messaging. It was quite a feat!
> 
> 
> 
> On Wednesday, 24 September 2014 10:37:49 UTC+1, Alejandro R. Mosteo  wrote:
> 
> > Hello people,
> 
> > 
> 
> > 
> 
> > 
> 
> > I am working nowadays in some heavy simulations that could benefit form parallelizing batches of work. Currently I either manually perform the launches of different lots, or leverage a cluster managed with Condor [1] at my institution, but using C code.
> 
> > 
> 
> > 
> 
> > 
> 
> > I was thinking about the possibility of requesting the installation of an Ada compiler in such cluster, and also if the DSA can fulfill my needs. I guess I could dig into the docs but probably some of you know some pitfalls that could elude me.
> 
> > 
> 
> > 
> 
> > 
> 
> > My question is: is the DSA well-suited for load balancing distribution, even if it is of simple nature. My impression from hearsay is that the DSA allows one to define a structure of nodes, and that you program as if this was a single entity, and the remote interactions are done for you. What I'm not sure of is if it supports, for example, is things like "allocate as many nodes as online machines", node migrations and things like that. Or, things like resilience through redundant nodes.
> 
> > 
> 
> > 
> 
> > 
> 
> > In other words, does the DSA support reconfiguration (even if it is at launch time) of nodes without editing a config file, or even online reconfiguration without manual intervention? 
> 
> > 
> 
> > 
> 
> > 
> 
> > Thank you in advance, 
> 
> > 
> 
> > Alex.
> 
> > 
> 
> > 
> 
> > 
> 
> > [1] http://research.cs.wisc.edu/htcondor/description.html

Oh sorry the answer is yes, but you may have to operate a store of types of processing nodes in a passive partition.

Re: On intended use cases of the distributed annex

[Pascal Obry]

Alejandro,

> In other words, does the DSA support reconfiguration (even if it is at
> launch time) of nodes without editing a config file, or even online
> reconfiguration without manual intervention? 

Yes. A Remote_Types partition can be launched at runtime as many time as
needed. For load balancing I've been using a very simple protocol:

1. some partitions are producers of works items

2. the work-items are stored into a queue

3. many partitions are consumers of those work-item, you can start one
on each node if needed.

4. consumers take a job into the queue, do whatever is needed and post
the results possibly in another queue for other consumers to work.

All this works fine and is a great way to achieve load balancing. In
fact nothing new there, you'll do that for a parallel applications using
tasks and protected-object (for the queue). The nice thing with the
distributed annex is that it there is nothing new, just a prama. The
queue of work-items I've described above will be protected with
protected object as many partitions will access it.

-- 
  Pascal Obry /  Magny Les Hameaux (78)

  The best way to travel is by means of imagination

  http://v2p.fr.eu.org
  http://www.obry.net

  gpg --keyserver keys.gnupg.net --recv-key F949BD3B

Re: On intended use cases of the distributed annex

[Alejandro R. Mosteo]

Thanks, Pascal and Tony. That sounds very encouraging, I will then certainly dig in into the details. That producer-consumer setup is like what I had in mind.

I guess the specific details of launching a set of processes in multiple machines will be compiler-dependent. In my case, I have (or could obtain) a list of IPs of machines I can access through ssh. If you have experience with gnat, could you briefly comment on this? Does one need to launch the process in every machine, or will a single command trigger a number of connections/executions?

Thanks again,
Álex.

Re: On intended use cases of the distributed annex

[Alejandro R. Mosteo]

> I guess the specific details of launching a set of processes in multiple machines will be compiler-dependent. In my case, I have (or could obtain) a list of IPs of machines I can access through ssh. If you have experience with gnat, could you briefly comment on this? Does one need to launch the process in every machine, or will a single command trigger a number of connections/executions?

Don't worry, I have just started to read about this. 

Cheers.

Re: On intended use cases of the distributed annex

[Maciej Sobczak]

W dniu środa, 24 września 2014 11:37:49 UTC+2 użytkownik Alejandro R. Mosteo napisał:

> I was thinking about the possibility of requesting the installation of an Ada compiler in such cluster, and also if the DSA can fulfill my needs.

You can make it work, but be aware that DSA was not invented with heterogeneous systems in mind. That is, the annex does not define the wire-level protocol nor even the data type mappings, so depending on the particular implementation you might be forced to use the same compiler version for all nodes and obviously all of the nodes will have to be written in Ada. This might sound like heaven for some, but in a distributed system it might as well become a maintenance and integration nightmare and if at the end you will have to use additional protocols for integration with existing software or for adding software written by others or for other functions like remote monitoring or configuration scripts, etc., then it will become a bloody mess, which kind of contradicts the intentions of using Ada in the first place. For this reason I would not use DSA at all and instead try some technology-neutral communication solution.

I don't claim that the above considerations are bound to be problems, but certainly they are something to think about before you start.

-- 
Maciej Sobczak * http://www.msobczak.com * http://www.inspirel.com

Re: On intended use cases of the distributed annex

[J-P. Rosen]

Le 25/09/2014 10:30, Maciej Sobczak a écrit :
> You can make it work, but be aware that DSA was not invented with
> heterogeneous systems in mind. That is, the annex does not define the
> wire-level protocol nor even the data type mappings, so depending on
> the particular implementation you might be forced to use the same
> compiler version for all nodes and obviously all of the nodes will
> have to be written in Ada. This might sound like heaven for some, but
> in a distributed system it might as well become a maintenance and
> integration nightmare and if at the end you will have to use
> additional protocols for integration with existing software or for
> adding software written by others or for other functions like remote
> monitoring or configuration scripts, etc., then it will become a
> bloody mess, which kind of contradicts the intentions of using Ada in
> the first place. For this reason I would not use DSA at all and
> instead try some technology-neutral communication solution.
> 
> I don't claim that the above considerations are bound to be problems,
> but certainly they are something to think about before you start.
> 
Right, but with PolyOrb you may have the same objects viewed as
distributed Ada objects from Ada, and Corba objects from the rest of the
world...

-- 
J-P. Rosen
Adalog
2 rue du Docteur Lombard, 92441 Issy-les-Moulineaux CEDEX
Tel: +33 1 45 29 21 52, Fax: +33 1 45 29 25 00
http://www.adalog.fr

Re: On intended use cases of the distributed annex

[Maciej Sobczak]

W dniu czwartek, 25 września 2014 11:26:38 UTC+2 użytkownik J-P. Rosen napisał:

> Right, but with PolyOrb you may have the same objects viewed as
> distributed Ada objects from Ada, and Corba objects from the rest of the
> world...

Except that CORBA is a technology that has lost its "cool factor" some decade or two ago and you might find it difficult to convince anybody in your team to write software using it. If Ada is *perceived* as an old technology that is no longer relevant in modern IT, then introducing CORBA into the picture will not help make Ada look any more attractive - now you have not one, but two old and irrelevant technologies to convince your colleagues to work with.

-- 
Maciej Sobczak * http://www.msobczak.com * http://www.inspirel.com

Re: On intended use cases of the distributed annex

[Brad Moore]

On 14-09-25 02:30 AM, Maciej Sobczak wrote:
> W dniu środa, 24 września 2014 11:37:49 UTC+2 użytkownik Alejandro R. Mosteo napisał:
>
>> I was thinking about the possibility of requesting the installation of an Ada compiler in such cluster, and also if the DSA can fulfill my needs.
>
> You can make it work, but be aware that DSA was not invented with heterogeneous systems in mind. That is, the annex does not define the wire-level protocol nor even the data type mappings, so depending on the particular implementation you might be forced to use the same compiler version for all nodes and obviously all of the nodes will have to be written in Ada. This might sound like heaven for some, but in a distributed system it might as well become a maintenance and integration nightmare and if at the end you will have to use additional protocols for integration with existing software or for adding software written by others or for other functions like remote monitoring or configuration scripts, etc., then it will become a bloody mess, which kind of contradicts the intentions of using Ada in the first place. For this reason I would not use DSA at all and instead try some technology-neutral communication solution.
>
> I don't claim that the above considerations are bound to be problems, but certainly they are something to think about before you start.
>

Not quite true that all the nodes need to be written in Ada.  I have 
used DSA to send C++ objects to C++ applications (or at least objects of 
C++ classes that are derived from Ada types), and as well had C++ 
applications remotely interacting with each other C++ and other Ada 
applications using the DSA.
I also appreciate that the DSA is more closely integrated with the 
compiler, allowing the compiler to check and catch problems that a 
library only approach does not provide.

Re: On intended use cases of the distributed annex

[Maciej Sobczak]

W dniu czwartek, 25 września 2014 17:02:07 UTC+2 użytkownik Brad Moore napisał:

> Not quite true that all the nodes need to be written in Ada.  I have 
> used DSA to send C++ objects to C++ applications (or at least objects of 
> C++ classes that are derived from Ada types), and as well had C++ 
> applications remotely interacting with each other C++ and other Ada 
> applications using the DSA.

This sounds interesting. Can you give a bit more details on this? Was it related to the DSA implementation that actually used CORBA under the hood? Was it necessary to extract some interface definition files from Ada specs?
Or did you just reverse-engineered the DSA wire protocol that was in use?
Was your solution immune to compiler upgrades?
And so on - lot's of questions. :-)

> I also appreciate that the DSA is more closely integrated with the 
> compiler, allowing the compiler to check and catch problems that a 
> library only approach does not provide.

Was it also closely integrated with the C++ nodes? ;-)

The "close integration with the compiler" makes most sense when there is a single build process for all the nodes. Otherwise there is no guarantee that individual nodes are compiled with the same specs and the whole notion of static type safety goes down the drain. This is particularly relevant in bigger systems where distinct nodes are developed by different teams and/or are deployed at different time schedules. More on this here:

http://www.inspirel.com/articles/What_Is_Wrong_With_IDL.html

Still, I'm very much interested in your Ada/C++ integration over DSA.

-- 
Maciej Sobczak * http://www.msobczak.com * http://www.inspirel.com

Re: On intended use cases of the distributed annex

[Brad Moore]

On 14-09-26 01:36 AM, Maciej Sobczak wrote:
> W dniu czwartek, 25 września 2014 17:02:07 UTC+2 użytkownik Brad Moore napisał:
>
>> Not quite true that all the nodes need to be written in Ada.  I have
>> used DSA to send C++ objects to C++ applications (or at least objects of
>> C++ classes that are derived from Ada types), and as well had C++
>> applications remotely interacting with each other C++ and other Ada
>> applications using the DSA.
>
> This sounds interesting. Can you give a bit more details on this?

Sure, I wrote a paper back in 2008 describing this, which you can 
download here.

http://sourceforge.net/projects/dequesterity/files/RemoteBuffers.pdf/download

The paper grew out of the buffer work that eventually became the 
dequesterity project, which is the underlying buffer abstraction I was 
working on.

I still have the source code for the distributed demo system, but that 
is not currently available on any public site, since it is too limited 
and specific of an application to be generally useful, other than 
perhaps providing a base for new design.

The distributed monitoring application demo evolved out of my interest 
in distributed systems.

I built a working demo of the concept, where one could monitor and 
control any number of remote nodes.  I did have some nodes compiled 
under Linux, and others compiled under Windows, and the nodes could 
communicate with each other.

Was it related to the DSA implementation that actually used CORBA under 
the hood?

The original version used the DSA implementation called Glade I think 
which did not involve CORBA. Later, I integrated the PolyORB version, 
which was pretty new at the time, and utilized CORBA.


Was it necessary to extract some interface definition files from Ada specs?

I basically defined C++ classes, and mirroring Ada specs as Tagged 
types. The C++ code used the C++ classes, but made calls to Ada servers 
with exported C/C++ function calls, which knew how to marshal the C++ 
classes. The code involved a message factory which could create objects 
of different types in the class hierarchy, which could be passed into a 
C/C++ application code. Although I did not try this, Java supposedly 
also could be used since the GCC compiler uses the same binary API for 
the different languages.


> Or did you just reverse-engineered the DSA wire protocol that was in use?
I didn't look at the DSA wire protocol. One thing I wanted to do however 
was distribute the nodes to each other without there having to be a 
central server. My initial implementation had a central server, which 
was an RCI Ada package, but I moved to a peer to peer type system.
I distributed the Remote Access to Class Wide Type objects to other 
peers using my own multicast messaging approach. Once the nodes had 
discovered each other, they could make calls to the RACW remote pointers 
directly, which was pretty cool.

That was the one major limitation I found with the DSA, was that it 
didn't directly support multicast communications, however I was able to 
work around this. It think that could be an area of improvement for DSA, 
to provide some sort of group broadcast/discovery.


> Was your solution immune to compiler upgrades?

I'd say not completely immune, but I did switch from Glade to PolyORB 
relatively painlessly, which was a completely different implementation. 
I don't think I needed to make any changes to my code. I only needed to 
configure the underlying DSA differently.

The idea of the DSA is that the Ada code should be decoupled from the 
underlying DSA implementation.


> And so on - lot's of questions. :-)
>
>> I also appreciate that the DSA is more closely integrated with the
>> compiler, allowing the compiler to check and catch problems that a
>> library only approach does not provide.
>
> Was it also closely integrated with the C++ nodes? ;-)

I'm not sure I understand your question. I think the C++ code was not as 
closely integrated, to be expected, since C++ generally does not provide 
as much checking as an Ada compiler, and the C++ compiler does not have 
the same semantic knowledge of the Ada compiler and vice versa.

But the C++ classes worked pretty much as expected for the C++ code.

>
> The "close integration with the compiler" makes most sense when there is a single build process for all the nodes.

True. All the applications that talk to each other are compiled at the 
same time in my case.  Although, as I mentioned, I had nodes 
communicating between OS'es, (Windows and Linux), so for that scenario, 
the compilations would have been separate.

I think separate compilation could work in other cases, but if any of 
the distributed object classes are modified, I think all the 
applications that pass there objects around would need to be recompiled.

Otherwise there is no guarantee that individual nodes are compiled with 
the same specs and the whole notion of static type safety goes down the 
drain.

With the DSA there are mechanism's to at least check the consistency of 
the system. Eg. P'Version, and P'Body_Version in RM E.3, although I 
never experimented with these.

I think it would be difficult to change the object structures of 
distributed classes, and have out of date nodes be expected to still 
work. Probably someone might be able to devise some sort of 
introspection mechanism, but that's far beyond what I was trying to do. 
Some sort of dynamic typing system such as the one described in your 
link below could be helpful here.


  This is particularly relevant in bigger systems where distinct nodes 
are developed by different teams and/or are deployed at different time 
schedules. More on this here:

I can see how large systems and teams can make changes difficult or 
messy. I've seen large deployments that have tried different approaches.
    1) Replace everything at the same time. Can be very difficult, 
because the upgrade can take a significant amount of time, which might 
be too long to go without a working system. This is likely not an option 
for many large systems.
    2) Replace parts of the system. In this case, backwards 
compatibility is very important. Often its better to create a new API 
for new stuff and keep the old one in place until it can be fully 
deprecated, if ever.

The example of HTML is a good one here, which basically follows case 2. 
New features can be added to HTML and newer versions of HTML, but they 
had better not break the support for earlier versions of HTML.
Another example is SNMP. Once an SNMP interface is defined, and 
distributed for use in a large deployment. It usually means the existing 
API's cannot be changed, and any new changes are typically brought in, 
in the form of new API's.

>
> http://www.inspirel.com/articles/What_Is_Wrong_With_IDL.html
>
> Still, I'm very much interested in your Ada/C++ integration over DSA.
>

I'd be happy to answer any other questions you might have.

For future work, I was always interested in RM E.5 (24.1-24.2) and other 
paragraphs in the same section that suggest that the user could supply 
their own PCS system to replace the compilers version.

I don't know how easy this would be to do, and I am not aware of anyone 
who has tried to do this, but I recognize that different applications 
may have needs for specific protocols for specific networks that may be 
outside the ones provided by the implementation.


Brad