Help writing first daemon

Help writing first daemon

[Patrick]

Hi Everyone

I am setting out to write my first daemon. There seems to be a lot of options for inter-process communication on posix systems. I am planning on designing a scientific instrument control server that will control ports, like the serial port, based on signals it is sent. It will also collect data from instruments and store the data(I was thinking in a postgresql database).

I just need signals passed on one machine so yaml4 seems like overkill. I was thinking that I could write many little commands, with each command specific to an instrument it is designed to control and the daemon as more general infrastructure(basically middleware), so it wouldn't really be a parent/child relationship. I'm guessing plain old pipes are out.

Does FIFO/names pipes sound reasonable for this sort of thing? I am concerned that with many commands acting on the daemon there will be too much overhead with these commands creating file handlers.

I am also going to control instruments over Ethernet anyways so would investing time in socket programming solve two problems by making it a socket based server? Once the socket is set up by the daemon, the smaller "satellite" commands would not need as much overhead to connect to a port as they would to create a file handler, would they?

Lastly, this will be done in Ada where ever possible, is there an Ada oriented way to do this sort of thing?

Thanks for reading-Patrick

Re: Help writing first daemon

[Dmitry A. Kazakov]

On Fri, 7 Sep 2012 15:05:52 -0700 (PDT), Patrick wrote:

> I am setting out to write my first daemon. There seems to be a lot of
> options for inter-process communication on posix systems. I am planning on
> designing a scientific instrument control server that will control ports,
> like the serial port, based on signals it is sent. It will also collect
> data from instruments and store the data(I was thinking in a postgresql
> database).

Process automation middleware is much work.
 
> I just need signals passed on one machine so yaml4 seems like overkill. I
> was thinking that I could write many little commands, with each command
> specific to an instrument it is designed to control and the daemon as more
> general infrastructure(basically middleware), so it wouldn't really be a
> parent/child relationship. I'm guessing plain old pipes are out.

Pipes could serve as a transport for the middleware, which could run its
publisher/subscriber services on top of them.

But, if you consider running the middleware as a system service (daemon),
the main reason for that would be the architecture of shared memory. Shared
memory / single instance middleware has its advantages and disadvantages.
Anyway, the point is that you don't not need pipes if the memory is shared.
I.e. pipes and daemon are mutually exclusive.
 
> I am also going to control instruments over Ethernet anyways so would
> investing time in socket programming solve two problems by making it a
> socket based server? Once the socket is set up by the daemon, the smaller
> "satellite" commands would not need as much overhead to connect to a port
> as they would to create a file handler, would they?

But instruments do not run middleware. The middleware has two interfaces.
The interface for applications and the interface for end devices.
Instruments of same type are handled by a middleware device
driver/interface designed for them. Distribution layer should consider
TCP/IP- or pipes-based protocols as a mere device.
 
> Lastly, this will be done in Ada where ever possible, is there an Ada
> oriented way to do this sort of thing?

I designed process automation middleware both in C++ and Ada 2005. Ada
offers many possibilities, e.g tasking which significantly changes the way
of implementing middleware services. It may also mean more work on the
architecture, because you have to weigh multitude of possible solutions.
There are also many difficulties, e.g. lack of full MI sometimes extremely
complicates middleware design, worse, it cripples the API.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: Help writing first daemon

[björn lundin]

Den lördagen den 8:e september 2012 kl. 00:05:52 UTC+2 skrev Patrick:
> Hi Everyone
> 
> I am setting out to write my first daemon. There seems to be a lot of options for inter-process communication on posix systems. I am planning on designing a scientific instrument control server that will control ports, like the serial port, based on signals it is sent. It will also collect data from instruments and store the data(I was thinking in a postgresql database).
> 
This fits the description of the system I work on nicely,
but it is delaing with material handling equipment though.
But 

middleware <-> I/O process <-> device 

is the same. 

> Does FIFO/names pipes sound reasonable for this sort of thing? I am concerned that with many commands acting on the daemon there will be too much overhead with these commands creating file handlers.
> 

we use named pipes on Linux and shared memory/semaphores on AIX
Windows has something like memory mapped files.

I did the pipes implementation, and it actually works on posix ystem, and with small
adjustemens on windows too.
Every process/daemon started has an enviremonet variable PROCESS_NAME - 
withch also is the name of the pipe. This makes it easy for the sending process
to find the pipe it wants


> I am also going to control instruments over Ethernet anyways so would investing time in socket >programming solve two problems by making it a socket based server? Once the socket is set up by the >daemon, the smaller "satellite" commands would not need as much overhead to connect to a port as >they would to create a file handler, would they?

Sockets works basically the same for posix/windows.
There is a socket binding bundled with gnat. 
There is AdaSockets.

This is an eays way, and since you already know that you will use sockets,
the choise would be easy.
Let the children (if manny) connect to the parent, and there is only 1 port to remember.

> Lastly, this will be done in Ada where ever possible, 
>is there an Ada oriented way to do this sort of thing?

Perhaps the distributed annex?
But I would  find it easier to use socket directly

--
Björn Lundin

Re: Help writing first daemon

[Patrick]

Hi Dmitry, Hi Björn 

Thanks for your great feedback. Since it sounds sensible to do so, I think I will look into building everything on top of sockets as I have to use it anyways.

Have a great day!-Patrick

Re: Help writing first daemon Heres a Server

[anon]

--
--  TCP/IP Echo Service Daemon, based on RFC 342 and RFC 862
--  Normally, a Sender program that accepts a word or quoted 
--  sentence from command line is transmitted to the server and 
--  the return message is displayed. 
--
--  This Daemon was tested using "Ada Dual TCP/IP Stacks" using
--  IPv6 and Linux IPv4,
--
--
--  Protocol: TCP
--  Protocol: UDP not setup at this time.
--
--  Usage:    Listener
--
--  Tester:   Telnet <hostname>  -port 7
--  Example:  Telnet 127.0.0.1 7
--
--  While telnet operational every character including control 
--  characters will be echoed
--

with Ada.Characters.Latin_9 ;
with Ada.Exceptions ;
with Ada.Text_IO ;
with GNAT.Sockets ;   -- GNAT to reg OS 
-- with Sockets ;     -- Ada TCP/IP Stacks

use Ada.Characters.Latin_9 ;
use Ada.Exceptions ;
use Ada.Text_IO ;
use GNAT.Sockets ;
-- use Sockets ;     -- Ada TCP/IP Stacks

procedure Listener is

  --
  --  Operational options
  --
  --  Set to True to use IPv6, But some GNAT.Socket packages, are
  --  not setup to handle IPv6 yet. One way to check is to look at 
  --  the body for the "Bind" function. Because the "Bind" function
  --  raises a exception, if IPv6 is use, for those packages that 
  --  are IPv4 only.
  --
  IP_V6   : constant Boolean := False ;
  --
  -- Displays a logging message if True 
  --
  LOGGING : constant Boolean := False ;



  task type Echo is
    entry Start ( Incoming : Socket_Type ) ;
  end Echo ;

  type Echo_Access is access Echo ;
  type Sock_Addr_Access is access all Sock_Addr_Type ;

  --
  -- Echo -- the main processthat preforms the echo operation
  --      -- one problem is there is no recovery of memory use 
  --         by this task once the task ends.
  --

  task body Echo is
      Data     : Character     ;

      Channel  : Stream_Access ;
      Socket   : Socket_Type   ;

    begin
      accept Start ( Incoming : Socket_Type ) do
          Socket := Incoming ;
      end Start ;

      Channel := Stream ( Socket ) ;
      loop
        Data := Character ' Input ( Channel ) ;
        exit when Data = ASCII.Nul ;
        Character ' Output ( Channel, Data ) ;
      end loop ;
      Close_Socket ( Socket ) ;
    exception
      when Socket_Error =>
          Put_Line ( "Connection closed" ) ;
          Close_Socket ( Socket ) ;
    end Echo ;

  --
  Accepting_Socket : Socket_Type ;
  Incoming_Socket  : Socket_Type ;
  Address          : Sock_Addr_Access ;

  Dummy            : Echo_Access ;

  TCP_Error        : exception ;

begin
  --  
  --  Create Socket and sets stacks. With error checking to insure 
  --  stacks is valid because some system only have IPv4 and other 
  --  have remove IPv4. If both stacks are installed and GNAT 
  --  allows both, then use IPv6.
  --
  if IP_V6 then
    begin
      --
      -- set IPv6
      --
      Create_Socket ( Accepting_Socket, Family_Inet6, Socket_Stream ) ;
      Address := new Sock_Addr_Type ( Family_Inet6 );
    exception
      when Socket_Error =>
        Put_Line ( "Error: IP version 6 is not supported" ) ;
        raise TCP_Error ;
    end ;
  else
    begin
      --
      -- set Default IPv4
      --
      Create_Socket ( Accepting_Socket ) ;
      Address := new Sock_Addr_Type ;
    exception
      when Socket_Error =>
        Put_Line ( "Error: IP version 4 is not supported" ) ;
        raise TCP_Error ;
    end ;
  end if ;
  --
  --  Address.Addr is current host can be localhost
  --  Address.Port is 7 based on RFC 342 update RFC 862 
  --
  Address.all.Addr := Addresses ( Get_Host_By_Name ( Host_Name ), 1 ) ;
  Address.all.Port := 7 ;
  --
  --  Bind Address to socket
  --
  Bind_Socket ( Accepting_Socket, Address.all ) ;
  --
  --  Set stacks to receive connect events
  --
  Listen_Socket ( Accepting_Socket ) ;
  --
  --  Handle connections
  --
  loop
    --
    --  Wait until client request connect then accept connection
    --
    Accept_Socket ( Accepting_Socket, 
                    Incoming_Socket, Address.all ) ;
    --
    --  Log message, if required
    --
    if LOGGING then
      Put ( "Received From: " ) ;
      Put ( Image ( Address.all ) ) ;
    end if ;
    --
    --  Create a single usage task to handle daemon process
    --  task will die once connection is ended.  In this design 
    --  there is a possible memory leak because once the task
    --  dies there is no memory recover of the dead task.
    --
    Dummy := new Echo ;
    Dummy.Start ( Incoming_Socket ) ;
  end loop ;

exception
  when TCP_Error => 
      Put_Line ( "Error: Server was not initialized" ) ;

  when others => 
      Put_Line ( "Error: Server is beening shutdown" ) ;
      Shutdown_Socket ( Accepting_Socket, Shut_Read_Write ) ;
end Listener ;



In <3b3a796d-50e0-4304-9f8d-295fb2ed0e82@googlegroups.com>, Patrick <patrick@spellingbeewinnars.org> writes:
>
>Hi Everyone
>
>I am setting out to write my first daemon. There seems to be a lot of optio=
>ns for inter-process communication on posix systems. I am planning on desig=
>ning a scientific instrument control server that will control ports, like t=
>he serial port, based on signals it is sent. It will also collect data from=
> instruments and store the data(I was thinking in a postgresql database).
>
>I just need signals passed on one machine so yaml4 seems like overkill. I w=
>as thinking that I could write many little commands, with each command spec=
>ific to an instrument it is designed to control and the daemon as more gene=
>ral infrastructure(basically middleware), so it wouldn't really be a parent=
>/child relationship. I'm guessing plain old pipes are out.
>
>Does FIFO/names pipes sound reasonable for this sort of thing? I am concern=
>ed that with many commands acting on the daemon there will be too much over=
>head with these commands creating file handlers.
>
>I am also going to control instruments over Ethernet anyways so would inves=
>ting time in socket programming solve two problems by making it a socket ba=
>sed server? Once the socket is set up by the daemon, the smaller "satellite=
>" commands would not need as much overhead to connect to a port as they wou=
>ld to create a file handler, would they?
>
>Lastly, this will be done in Ada where ever possible, is there an Ada orien=
>ted way to do this sort of thing?
>
>Thanks for reading-Patrick

Re: Help writing first daemon

[Patrick]

Thanks for the example :)

Re: Help writing first daemon

[Julian Leyh]

Am Samstag, 8. September 2012 00:05:52 UTC+2 schrieb Patrick:
> Lastly, this will be done in Ada where ever possible, is there an Ada oriented way to do this sort of thing?

Annex E, Distributed Systems. For example PolyORB.

Re: Help writing first daemon

[Maciej Sobczak]

W dniu sobota, 8 września 2012 10:06:28 UTC+2 użytkownik Dmitry A. Kazakov napisał:

> But, if you consider running the middleware as a system service (daemon),
> the main reason for that would be the architecture of shared memory.

No, there is no connection between these two concepts.
System services are created so they can be started and run unattended even after system restarts. A free-standing web server is a possible example here.

They way other components will communicate with such a service is a consequence of other design choices, like whether they are on the same machine or not. If they are on different machines, then shared memory is out of question (for nit-pickers: reflective memory does not count).

Again, web browsers communicating via TCP with a web server running as a system service is a prime example here.

> I.e. pipes and daemon are mutually exclusive.

Not at all. These concepts solve different problems and have no implication on each other.

-- 
Maciej Sobczak * http://www.msobczak.com * http://www.inspirel.com

Re: Help writing first daemon

[Dmitry A. Kazakov]

On Tue, 11 Sep 2012 07:38:45 -0700 (PDT), Maciej Sobczak wrote:

> W dniu sobota, 8 wrze�nia 2012 10:06:28 UTC+2 u�ytkownik Dmitry A. Kazakov napisa�:
> 
>> But, if you consider running the middleware as a system service (daemon),
>> the main reason for that would be the architecture of shared memory.
> 
> No, there is no connection between these two concepts.

The concepts are single vs. multiple instances middleware.

> System services are created so they can be started and run unattended even
> after system restarts. A free-standing web server is a possible example here.

And there is no reason for a web server to be a system service. I see web
server as an application.

>> I.e. pipes and daemon are mutually exclusive.
>
> Not at all. These concepts solve different problems and have no
> implication on each other.

A dedicated system service and pipes are in the same league. Pipes
themselves is a service. If the middleware is considered service, then in
place of pipes. An conversely, if pipes to be used in the middleware, then
each application would have a more or less complete instance of the
middleware of its own.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: Help writing first daemon

[Maciej Sobczak]

W dniu wtorek, 11 września 2012 17:13:06 UTC+2 użytkownik Dmitry A. Kazakov napisał:

> >> But, if you consider running the middleware as a system service (daemon),
> >> the main reason for that would be the architecture of shared memory.
> > 
> > No, there is no connection between these two concepts.
> 
> The concepts are single vs. multiple instances middleware.

No. System services and shared memory are low-level building blocks that can be used to construct software that actually does something useful. Whether that software exists in a single vs. multiple instances is a matter of terminology that is defined on a much higher level and has nothing to do with how many processes there are or how they communicate.
For example, a well known database server is said to exist in a single instance, while being physically composed of many processes (presumably system services) communicating with each other. These concepts exist on different levels.

> > System services are created so they can be started and run unattended even
> > after system restarts. A free-standing web server is a possible example here.
> 
> And there is no reason for a web server to be a system service.

I will repeat, then: *unattended*. This usually means that the software can start automatically and does not rely on any interactions via terminal or GUI.

> I see web
> server as an application.

It's possible to use it this way. It is just not how everybody does it.

(Note: we didn't define "application" here and I'm not willing to spend my time on terminology wars.)


> A dedicated system service and pipes are in the same league. Pipes
> themselves is a service.

Pipes are a system-level service, just like dynamic memory. You can use them to build other, high-level services like web servers or database servers, but web server is certainly not in this league.

But I promised not to jump into terminology wars, so I quit.

-- 
Maciej Sobczak * http://www.msobczak.com * http://www.inspirel.com

Re: Help writing first daemon

[Dmitry A. Kazakov]

On Tue, 11 Sep 2012 13:50:27 -0700 (PDT), Maciej Sobczak wrote:

> W dniu wtorek, 11 wrze�nia 2012 17:13:06 UTC+2 u�ytkownik Dmitry A. Kazakov napisa�:
> 
>>>> But, if you consider running the middleware as a system service (daemon),
>>>> the main reason for that would be the architecture of shared memory.
>>> 
>>> No, there is no connection between these two concepts.
>> 
>> The concepts are single vs. multiple instances middleware.
> 
> No. System services and shared memory are low-level building blocks that
> can be used to construct software that actually does something useful.
> Whether that software exists in a single vs. multiple instances is a
> matter of terminology that is defined on a much higher level and has
> nothing to do with how many processes there are or how they communicate.

Not the software, but states (and thus data). In a single instance
middleware objects maintained by it are physically shared by all
applications. That precludes I/O, like pipes. In a multiple instances
middleware objects are replicated and some I/O is necessary to maintain
consistency of a logical object, physically distributed across
applications.

> For example, a well known database server is said to exist in a single
> instance, while being physically composed of many processes (presumably
> system services) communicating with each other. These concepts exist on
> different levels.

E.g. DB vs. DB replica.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de