Ada Networking (General/Design)

Ada Networking (General/Design)

[Shark8]

In a related, though only tangential, vein to my other Ada-networking thread, I thought it would be good to ask what [you find] the best way to handle network-communication is.

Is it using Streams? Wrapping up IP-ports and packets in their own interface-packages and using those? Something I'm not even thinking of? (In short, I'm curious as to how other Ada programmers approach it.)

Re: Ada Networking (General/Design)

[Shark8]

On Sunday, August 12, 2012 2:26:23 PM UTC-6, Shark8 wrote:
> In a related, though only tangential, vein to my other Ada-networking thread, I thought it would be good to ask what [you find] the best way to handle network-communication is.
> 
> 
> 
> Is it using Streams? Wrapping up IP-ports and packets in their own interface-packages and using those? Something I'm not even thinking of? (In short, I'm curious as to how other Ada programmers approach it.)

*It should probably read sockets instead of IP-ports and packets. I'm very new to implementing things for networks, so please forgive my misunderstandings.

Re: Ada Networking (General/Design)

[sbelmont700]

On Sunday, August 12, 2012 4:26:23 PM UTC-4, Shark8 wrote:
> In a related, though only tangential, vein to my other Ada-networking thread, I thought it would be good to ask what [you find] the best way to handle network-communication is.
> 
> 
> 
> Is it using Streams? Wrapping up IP-ports and packets in their own interface-packages and using those? Something I'm not even thinking of? (In short, I'm curious as to how other Ada programmers approach it.)

Depending on its applicability to your situation, the DSA is perhaps the most elegant solution; simply take the whole thing to the next level higher, and avoid network programming altogether.

Re: Ada Networking (General/Design)

[Shark8]

On Sunday, August 12, 2012 6:10:22 PM UTC-6, (unknown) wrote:
> On Sunday, August 12, 2012 4:26:23 PM UTC-4, Shark8 wrote:
> 
> > In a related, though only tangential, vein to my other Ada-networking thread, I thought it would be good to ask what [you find] the best way to handle network-communication is.
> 
> > 
> 
> > 
> 
> > 
> 
> > Is it using Streams? Wrapping up IP-ports and packets in their own interface-packages and using those? Something I'm not even thinking of? (In short, I'm curious as to how other Ada programmers approach it.)
> 
> 
> 
> Depending on its applicability to your situation, the DSA is perhaps the most elegant solution; simply take the whole thing to the next level higher, and avoid network programming altogether.

LOL -- True; I *really* like the underlying ideas for the DSA. (That doesn't mean that I really understand the bits-n-pieces/nitty-gritty... but you have to start somewhere.)

Though there's a lot of programs (or it would be more correct to say 'networks') that aren't DSA but might be useful. Say something like SETI@Home, or EDonkey/Gnutilla/BitTorrent --I know for certain there are Bittorrents being used to distribute people's software, probably most prominently certain linux flavors-- and they are likely going to need some interfacing support... I really don't see how it would be possible to "overlay the DSA" such that it enables you to treat those as part of your local system.

Re: Ada Networking (General/Design)

[Dmitry A. Kazakov]

On Sun, 12 Aug 2012 13:26:23 -0700 (PDT), Shark8 wrote:

> In a related, though only tangential, vein to my other Ada-networking
> thread, I thought it would be good to ask what [you find] the best way to
> handle network-communication is.

Depends on the protocol, load and QoS requirements. E.g. in one case we
must handle 200K+ connections, in another 80�s is the response time limit.

> Is it using Streams?

Rarely. Most network protocols are packet-oriented.

> Wrapping up IP-ports and packets in their own
> interface-packages and using those?

Certainly. There will be many packages reflecting at least the protocol
layers, encapsulating OS-dependent stuff etc.

> Something I'm not even thinking of?
> (In short, I'm curious as to how other Ada programmers approach it.)

See above, it depends. 

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: Ada Networking (General/Design)

[Patrick]

I  am glad that shark8 posted this, I want to learn about this too.

Does anyone have a favourite approach to TCP/IP ?

Re: Ada Networking (General/Design)

[Dmitry A. Kazakov]

On Mon, 13 Aug 2012 05:33:28 -0700 (PDT), Patrick wrote:

> Does anyone have a favourite approach to TCP/IP ?

TCP vs. UDP changes little (when point to point). You still have packet
even when sent over TCP stream. It is difficult to outline a universal
solution for all case.

There are a reader and a writer tasks encapsulated into I/O objects. There
are packages implementing protocol layers. Each layer object derives from
or otherwise uses the lower level object in a mix-in. Lower layers provide
operations for the higher levels and define abstract operations or callback
to override. Two tasks have the advantage of having it full-duplex
(performance, deadlock prevention upon protocol errors), being able to use
blocking calls. It may impose some difficulties as you would likely have to
route actions from the writer to the reader task.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: Ada Networking (General/Design)

[Marc C]

On Monday, August 13, 2012 7:33:28 AM UTC-5, Patrick wrote:

> Does anyone have a favourite approach to TCP/IP ?

After years of working with TCP/IP sockets, my favorite approach now is...not to. Not directly anyway.

There are higher-level messaging protocols that handle most, if not all, of the socket management details for you.

Among them are ZeroMQ (http://www.zeromq.org), along with its Ada binding (http://www.zeromq.org/bindings:ada), and YAMI4 (http://www.inspirel.com/yami4).

I collected ZeroMQ, AMQP, and STOMP together into a set of text-oriented messaging interfaces called TOMI_4_Ada (http://sourceforge.net/projects/tomi4ada/). 

While you still have to be cognizant of things like host names and port numbers, by and large that's about the extent of what you need to be aware of when using these high-level protocols.

If I never have to write another setsockopt(), c_select(), and accept() again, I'll be very happy :-)

Marc A. Criley

Re: Ada Networking (General/Design)

[Dmitry A. Kazakov]

On Mon, 13 Aug 2012 06:12:23 -0700 (PDT), Marc C wrote:

> On Monday, August 13, 2012 7:33:28 AM UTC-5, Patrick wrote:
> 
>> Does anyone have a favourite approach to TCP/IP ?
> 
> After years of working with TCP/IP sockets, my favorite approach now is...not to. Not directly anyway.
> 
> There are higher-level messaging protocols that handle most, if not all,
> of the socket management details for you.

Hmm, there is not that many things you needed to manage sockets. Setting or
clearing TCP_NO_DELAY is not a huge problem, or?

> Among them are ZeroMQ (http://www.zeromq.org),

Does this really implement protocols, e.g. DLMS, ModBus etc? I didn't read
the documentation, but it looks rather like some text messaging or
middleware stuff on top of some transport like TCP, than a protocol
implementation generator tool (if that were possible).

> If I never have to write another setsockopt(), c_select(), and accept() again, I'll be very happy :-)

If Ada provided higher level socket library with an integrated support of
protected objects and tasks...

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: Ada Networking (General/Design)

[tmoran]

> If Ada provided higher level socket library with an integrated support of
> protected objects and tasks...
  What do you mean?

Re: Ada Networking (General/Design)

[Dmitry A. Kazakov]

On Mon, 13 Aug 2012 15:51:31 +0000 (UTC), tmoran@acm.org wrote:

>> If Ada provided higher level socket library with an integrated support of
>> protected objects and tasks...
>   What do you mean?

The most difficult and unavoidable part about socket programming is
tasking. We need reader and writer tasks or else socket-select-driven
co-routines. The reader and writer parts have to communicate each other in
some intricate way because it is two tasks but one protocol state machine,
or one task and many state machines in the case of socket select. There is
the issue of blocking socket I/O non-abortable by Ada means. All this
incredibly complicates design.

If Ada ever have to support sockets it should be a high level Ada
tasking-friendly abstraction, which I am not ready to outline.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: Ada Networking (General/Design)

[Patrick]

Hi Dmitry

I want to write applications to communicate with scientific instruments. Some use RS232, some GPIB and some TCP/IP. 

Your feedback is helpful but a little scary too.

Could you recommend a software project that gets socket programming done right?

Re: Ada Networking (General/Design)

[Patrick]

Sorry for "cc"ing you Dmitry

I wish Google would not set this by default, it seems a bit like nagging people, they can answer when they have time to return to the online discussion.

Re: Ada Networking (General/Design)

[Marc C]

On Monday, August 13, 2012 8:31:31 AM UTC-5, Dmitry A. Kazakov wrote:
> On Mon, 13 Aug 2012 06:12:23 -0700 (PDT), Marc C wrote:
> 
> > After years of working with TCP/IP sockets, my favorite approach now is...not to. Not directly anyway.

<snip>
>
> Hmm, there is not that many things you needed to manage sockets. Setting or
> clearing TCP_NO_DELAY is not a huge problem, or?

While I'd extract the socket code from an earlier project for reuse, it always needed to be modified a bit to get integrated into the new one. It just gets tedious after awhile. So adopting a more abstract, pre-existing transport protocol just eliminated the need to deal with that.

> > Among them are ZeroMQ (http://www.zeromq.org),
> Does this really implement protocols, e.g. DLMS, ModBus etc?

Nope, ZeroMQ is essentially just a transport layer, which is all the functionality I wanted of it, so that I could stop having to mess around with sockets.

TOMI_4_Ada essentially arose out of my desire to have a simple, consistent Ada library for text-oriented message interfaces. My projects rarely have any need for extensive or high performance protocols, so a nice clean set of client/server and topic-supportive publish/subscribe services meets my needs.

Marc

Re: Ada Networking (General/Design)

[Dmitry A. Kazakov]

On Mon, 13 Aug 2012 09:30:18 -0700 (PDT), Patrick wrote:

> I want to write applications to communicate with scientific instruments.
> Some use RS232, some GPIB and some TCP/IP. 

We are using a middleware for that. It abstracts industrial devices and
their protocols away, so that the application(s) would not care where the
process variables read and written come from. But it is a commercial
product.

And of course RS232, GPIB, TCP/IP is as much telling as ISA, PCI, PCIe. The
proper description should include the application level and everything down
to the transport, e.g. Steale autopilot, AK (list of commands, their
semantics), over TCP/IP.

If you want to communicate industrial devices, TCP/IP would be your least
problem. You need a middleware to decouple device-specific stuff from the
application logic.

> Could you recommend a software project that gets socket programming done
> right?

I guess there exit some open source examples, e.g. AWS (though I did see
its sources.)

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: Ada Networking (General/Design)

[Dmitry A. Kazakov]

On Mon, 13 Aug 2012 10:54:35 -0700 (PDT), Marc C wrote:

> On Monday, August 13, 2012 8:31:31 AM UTC-5, Dmitry A. Kazakov wrote:
>> On Mon, 13 Aug 2012 06:12:23 -0700 (PDT), Marc C wrote:
>> 
>>> Among them are ZeroMQ (http://www.zeromq.org),
>> Does this really implement protocols, e.g. DLMS, ModBus etc?
> 
> Nope, ZeroMQ is essentially just a transport layer,

I see. Then it is probably not what the OP wanted.

BTW, did you evaluate the distributed systems Annex vs. this stuff? I am
interested in an opinion, because we are using other technology, so that I
had no opportunity to evaluate the Annex. Was it too heavy-weight for you?

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: Ada Networking (General/Design)

[Marc C]

On Monday, August 13, 2012 1:28:22 PM UTC-5, Dmitry A. Kazakov wrote:

> BTW, did you evaluate the distributed systems Annex vs. this stuff? I am
> interested in an opinion, because we are using other technology, so that I
> had no opportunity to evaluate the Annex. Was it too heavy-weight for you?

Essentially the DSA was ruled out because the systems with which I would be communicating are very unlikely to be done in Ada. And, while I could supply an Ada-implemented "client" with Languages-Other-Than-Ada bindings to it, by offering a "native" client that sends, say JSON across ZeroMQ, it removes a source of hesitancy. In fact, just saying "here's a ZeroMQ port, send me a JSON object with these fields (or an XML doc passing this schema)" and let them go off and build it themselves makes the developers happy.  They don't have to depend, then, on *my* API implementation working correctly.

In addition, by employing a standard-ish transport medium--ZeroMQ, AMQP for now--it's much easier to tie in other systems. So long as my system is network-accessible, anyone can get to it--they don't need any client code from me.  This therefore enhances interoperability since I supply the only platform-dependent piece, and whoever talks to me can be running whatever OS/HW platform they want.

Marc

Re: Ada Networking (General/Design)

[tmoran]

>>> If Ada provided higher level socket library with an integrated support of
>>> protected objects and tasks...
>>   What do you mean?
>
>The most difficult and unavoidable part about socket programming is
>tasking. We need reader and writer tasks or else socket-select-driven
>co-routines. The reader and writer parts have to communicate each other in
>some intricate way because it is two tasks but one protocol state machine,
>or one task and many state machines in the case of socket select. There is
>the issue of blocking socket I/O non-abortable by Ada means. All this

The Claw.Sockets package has a simple Socket_Type with
  procedure Open(Socket     : in out Socket_Type;  -- or Server_Type
                 Domain_Name: in     String;  -- or Network_Address_Type
                 Port       : in     Port_Type;
                 Timeout    : in     Duration := 30.0);
and Text_IO style Get/Put for Strings, Input/Output for streams, etc.
It also has a non-blocking Async_Socket_Type with overridable
When_Connect, When_Readable, etc routines, but I find using blocking
sockets with tasks whose structure encodes the state machine is much
simpler.  All the blocking socket routines allow a timeout parameter,
so they in effect poll for data - or a Please_Quit request.  (The
non-blocking routines are of course also polling, via the Windows message
loop.) There are also ftp, pop3, smtp, http, etc packages built on top.

Re: Ada Networking (General/Design)

[Dmitry A. Kazakov]

On Mon, 13 Aug 2012 11:59:42 -0700 (PDT), Marc C wrote:

> This therefore enhances interoperability since I supply the only
> platform-dependent piece, and whoever talks to me can be running whatever
> OS/HW platform they want.

In automation it is different. Customers are absolutely reluctant to take
any changes in their software, even if that were possible. So the
responsibility to adapt is all mine.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: Ada Networking (General/Design)

[Dmitry A. Kazakov]

On Mon, 13 Aug 2012 19:11:30 +0000 (UTC), tmoran@acm.org wrote:

>>>> If Ada provided higher level socket library with an integrated support of
>>>> protected objects and tasks...
>>>   What do you mean?
>>
>>The most difficult and unavoidable part about socket programming is
>>tasking. We need reader and writer tasks or else socket-select-driven
>>co-routines. The reader and writer parts have to communicate each other in
>>some intricate way because it is two tasks but one protocol state machine,
>>or one task and many state machines in the case of socket select. There is
>>the issue of blocking socket I/O non-abortable by Ada means. All this
> 
> The Claw.Sockets package has a simple Socket_Type with
>   procedure Open(Socket     : in out Socket_Type;  -- or Server_Type
>                  Domain_Name: in     String;  -- or Network_Address_Type
>                  Port       : in     Port_Type;
>                  Timeout    : in     Duration := 30.0);
> and Text_IO style Get/Put for Strings, Input/Output for streams, etc.
> It also has a non-blocking Async_Socket_Type with overridable
> When_Connect, When_Readable, etc routines, but I find using blocking
> sockets with tasks whose structure encodes the state machine is much
> simpler.  All the blocking socket routines allow a timeout parameter,

In an Ada friendly way they should be entry calls, with the semantics of
canceling the request when a timed entry call gets timed out. That is when
the socket is blocking:

    select
        Socket.Receive (Packet);
    or delay 10.0;  -- Failed to read
    or terminate; -- Yes I know, it is illegal to have terminate + delay
    end select;

Or maybe other way round, they should be "entry points" to which socket I/O
could call to.

For non-blocking I/O there should be a way to have a pseudo-task ran by the
events on the socket, rather than scheduled.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: Ada Networking (General/Design)

[tmoran]

> In an Ada friendly way they should be entry calls, with the semantics of
> canceling the request when a timed entry call gets timed out. That is when
> the socket is blocking:
>
>     select
>         Socket.Receive (Packet);
>     or delay 10.0;  -- Failed to read
>     or terminate; -- Yes I know, it is illegal to have terminate + delay
>     end select;
>
> Or maybe other way round, they should be "entry points" to which socket I/O
> could call to.
>
  Get(Socket, Timeout => 10.0,
      Item => Buffer,
      Last => Last);
  where Last = Buffer'first-1 on timeout, seems simpler to me.

> For non-blocking I/O there should be a way to have a pseudo-task ran by the
> events on the socket, rather than scheduled.
  That's what overidable procedures When_Connect, When_Readable,
When_Disconnect, etc are for.  They are called when the event occurs.

Re: Ada Networking (General/Design)

[anon]

In <25fb2cbf-be17-44dd-9d95-22e9866be5a9@googlegroups.com>, Patrick <patrick@spellingbeewinnars.org> writes:
>Hi Dmitry
>
>I want to write applications to communicate with scientific instruments. 
>Some use RS232, some GPIB and some TCP/IP. 
>
>Your feedback is helpful but a little scary too.
>
>Could you recommend a software project that gets socket programming 
>done right?

To see an Ada example you just need to look at the file "g-socket.ads" 
aka "GNAT.Sockects" and instead of coping and editing the listed program 
"pingpong.adb". You can just download the text file from:

http://www.rtems.info/ftp/pub/rtems/4.8.2/ada-examples-4.8.2/pingpong/pingpong.adb


Then edit the line
       Group : constant String := "239.255.128.128";
to
       Group : constant String := "127.0.0.1";

then compile and run, if the 127.0.0.1 is setup as a local loopback.
If you examine the code you will see a few logical errors in the 
listing.


As for UART there was and might still be a UART I/O ada package 
on net or it might listed under Serial. Or you could use the Ada/C 
Serial_Port_driver in MaRTE_1.9.

Then use the serial driver to build a GPIB interface packages.

Re: Ada Networking (General/Design)

[Patrick]

Thanks for the great tips, this will help a lot in communicating with instrumentation.

Re: Ada Networking (General/Design)

[Maciej Sobczak]

W dniu poniedziałek, 13 sierpnia 2012 15:31:31 UTC+2 użytkownik Dmitry A. Kazakov napisał:

> Hmm, there is not that many things you needed to manage sockets. Setting or
> clearing TCP_NO_DELAY is not a huge problem, or?

Or managing multiple transmissions to several targets concurrently? Or having it done in background? Or dealing with serialization across different hardware platforms? Or having the possibility to intermix high-priority traffic with low-priority one? Or having it all integrated with the language so that you can use standard tasking features to wait, time out, etc. on your communication activities? Or raising the conceptual level to data-centric communication where data providers only care about providing data instead of messing with irrelevant low-level stuff? Or...

There's a lot more to networking than sockets and this is where higher-level solutions can be very helpful.

> If Ada provided higher level socket library with an integrated support of
> protected objects and tasks...

YAMI4 does provide this level of language integration.

-- 
Maciej Sobczak * http://www.inspirel.com/

Re: Ada Networking (General/Design)

[Dmitry A. Kazakov]

On Mon, 13 Aug 2012 20:28:58 +0000 (UTC), tmoran@acm.org wrote:

>> For non-blocking I/O there should be a way to have a pseudo-task ran by the
>> events on the socket, rather than scheduled.
>   That's what overidable procedures When_Connect, When_Readable,
> When_Disconnect, etc are for.  They are called when the event occurs.

No, that is upside-down. The point is to write I/O code as if it were
synchronous and half-duplex:

   send
   receive
   process
   send
   ...

Compare it with Ada task. You write a task as if it didn't share the
processor, you don't hook at timer interrupts.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: Ada Networking (General/Design)

[tmoran]

>>> For non-blocking I/O there should be a way to have a pseudo-task ran by the
>>> events on the socket, rather than scheduled.
>>   That's what overidable procedures When_Connect, When_Readable,
>> When_Disconnect, etc are for.  They are called when the event occurs.
>
>No, that is upside-down. The point is to write I/O code as if it were
>synchronous and half-duplex:
>
>   send
>   receive
>   process
>   send
>   ...
>
>Compare it with Ada task. You write a task as if it didn't share the
>processor, you don't hook at timer interrupts.
  Sorry, I misunderstood "pseudo-task ran by the events on the socket"
to mean roughly "interrupt-driven".  Claw.Sockets has "Async_Socket_Type
and procedures When_xxx for that.  I agree that it's usually easier
to "write I/O code as if it were synchronous and half-duplex:" with
the simple Socket_Type and Put, Get, Process, Put...