Tasking and timing out

Tasking and timing out

[Thomas Løcke]

Hey all,

I need to be able to be abandon a socket connection in a task and let
the task complete if:

a. the socket connect takes too long to complete

or

b. the Start task entry isn't called within a set timeframe

My current solution appears to do the trick just fine, but I'm not sure
if it's a "good" solution, specifically the nested select block pains
me to no end - I can't explain why though.   :)

Here's what I've got:

https://gist.github.com/3754529

Or without the fancy syntax highlighting from GitHub:


task body AMI_Action
is
begin
    select
       accept Start do
          select
             delay 2.0;
             raise AMI_Action_Error with Connect_Timed_Out_Message;
          then abort
             AWS.Net.Std.Connect (Socket => Action_Socket,
                                  Host => Config.Get (PBX_Host),
                                  Port => Config.Get (PBX_Port));
          end select;
       end Start;
    or
       delay 3.0;
       raise AMI_Action_Error with No_Call_On_Start_Received;
    end select;

    loop
       -- Mad socket action!
    end loop;
exception
    --  Catch exceptions, log issue and complete.
end AMI_Action;


The general idea is that if the call to AWS.Net.Std.Connect takes more
than 2 seconds to complete, then I raise an exception or if the Start
entry isn't called before 3 seconds have passed, then I also raise an
exception. In both cases I just want the task to log the issue and
complete.

It works - at least I haven't been able to make it fail in my tests,
it's just that I'm not sure if this is the right way to live up to the
requirements - maybe there's a better way?

Advice and/or suggestions are more than welcome.

-- 
Thomas L�cke | thomas@12boo.net | http://12boo.net

Re: Tasking and timing out

[Dmitry A. Kazakov]

On Thu, 20 Sep 2012 10:17:22 +0200, Thomas L�cke wrote:

> I need to be able to be abandon a socket connection in a task and let
> the task complete if:

ATC is almost never a good idea. I wonder how it worked in your case. Maybe
because the default socket timeout was incidentally close to 2 sec?

In any case you should not expect Ada run-time aborting outstanding OS I/O
requests. In some better world, but not under Windows or Linux.

A proper solution for blocking sockets is to close the socket from an
independent task and catching socket error when it propagates or else to
manipulate the socket timeout before starting any blocking operation.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: Tasking and timing out

[Thomas Løcke]

On 09/20/2012 10:53 AM, Dmitry A. Kazakov wrote:
> ATC is almost never a good idea. I wonder how it worked in your case. Maybe
> because the default socket timeout was incidentally close to 2 sec?


I tested this with a call to a dummy procedure that did nothing but hang
for X seconds, so the fact that it works have nothing to do with the
default socket timeout. When X > 2 the exception was raised, and when
X < 2 then the task proceeded to enter the loop.

ATC might almost never be a good idea, but that "almost" surely means
that in some cases it can be a good idea?  :)

As it is said in the John Barnes book "Programming in Ada 2005":

- The general effect can of course be programmed by the introduction of
- an agent task and the use of the abort statement but this is a heavy
- solution not at all appropriate for most applications needing a mode
- change.

The "then abort" syntax seems appropriate for this specific case, where
all I want is a completed task in case things aren't happening within a
set time limit.


> In any case you should not expect Ada run-time aborting outstanding OS I/O
> requests. In some better world, but not under Windows or Linux.


I had not thought about that. What you're saying is that despite the
"then abort" syntax I can not safely rely on the termination of the
ongoing AWS.Net.Std.Connect procedure? In that case then this is indeed
a rather nasty problem. Again from John Barnes:

- The general idea is that if the statements between "then abort" and
- "end select" do not complete before the expiry of the delay then they
- are abandoned and the statements following the delay are executed
- instead.

If I cannot rely on the runtime to actually abandon the call to
AWS.Net.Std.Connect, then I wonder what circumstances John Barnes is
referring to?


> A proper solution for blocking sockets is to close the socket from an
> independent task and catching socket error when it propagates or else to
> manipulate the socket timeout before starting any blocking operation.


I did think about having an agent task to manage this, but it seemed
messy and heavy compared to the extreme simplicity of this solution.
Also I found it hard to meet my two requirements.

I felt relatively safe using "then abort" because John Barnes
specifically mentions this form of select statement as a solution to the
problem of quitting action A and do B instead if A doesn't complete
within a specified duration.

I've got something to think about now. Thanks.

-- 
Thomas L�cke | thomas@12boo.net | http://12boo.net

Re: Tasking and timing out

[Dmitry A. Kazakov]

On Thu, 20 Sep 2012 11:36:38 +0200, Thomas L�cke wrote:

> ATC might almost never be a good idea, but that "almost" surely means
> that in some cases it can be a good idea?  :)

ARM 9.8 provides the guideline.

Specifically, you should consider:

1. Operations which can be prematurely aborted (no side effects)

2. Operations which cannot be aborted:
   2.1 Ones for which 9.8 provides safety against abort:
      2.1.1. Ones which are time bounded
      2.1.2. Ones which could deadlock when abort attempted (difficult to
         construct, but I would not exclude a possibility of)
   2.2 Ones for which 9.8 does not provide cover:
      2.2.1. Ones for which abort would cause malfunction (memory, resource
         leaks, corruption etc)
      2.2.2. Ones of which premature termination is impossible, e.g. system
         calls, most I/O, waitable system objects etc.
 
> As it is said in the John Barnes book "Programming in Ada 2005":
> 
> - The general effect can of course be programmed by the introduction of
> - an agent task and the use of the abort statement but this is a heavy
> - solution not at all appropriate for most applications needing a mode
> - change.

There is not much difference between ATC and explicit aborting tasks. Both
represent the case when the task control flow is preempted. The perils are
basically same.

>> In any case you should not expect Ada run-time aborting outstanding OS I/O
>> requests. In some better world, but not under Windows or Linux.
> 
> I had not thought about that. What you're saying is that despite the
> "then abort" syntax I can not safely rely on the termination of the
> ongoing AWS.Net.Std.Connect procedure?

Not if that is 1) an OS-native thread, 2) blocked by the OS, e.g. upon
waiting for an outstanding I/O request.

A good example is using ATC when reading the keyboard input using Get_Line.
Depending on the implementation of Get_Line it may work or may not work. 80
to 20 for the latter.

> If I cannot rely on the runtime to actually abandon the call to
> AWS.Net.Std.Connect, then I wonder what circumstances John Barnes is
> referring to?

#1 + #2.1.1 above. E.g. some computations, even with controlled types
involved. Anything else requires *careful* analysis and thus is necessarily
very fragile. Ergo, for the software design POV, it is better to forget
about ATC, too problematic.

>> A proper solution for blocking sockets is to close the socket from an
>> independent task and catching socket error when it propagates or else to
>> manipulate the socket timeout before starting any blocking operation.
> 
> I did think about having an agent task to manage this, but it seemed
> messy and heavy compared to the extreme simplicity of this solution.

Not really. Usually you have some worker task anyway due to the
asynchronous nature of I/O and because you liked to share the connection
between several tasks.

The clients are separate tasks asking the worker for data ready. They would
do that by a timed entry call to the worker's entry or else to a protected
object's entry. Upon timeout of this call you would call some Cancel
operation on the communication object which internally would close the
socket at some stage. That in turn will cause an error in the worker task,
which would drop the connection and then would try to reconnect etc.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: Tasking and timing out

[Adam Beneschan]

First of all, doesn't AWS.Net.Std.Connect have its own facility for
handling timeouts?  I'm looking at the source, and it looks like it
does, but I don't know if I'm looking at the most recent version.  If
it does, you should definitely use that, and not rely on ATC.  

Anyway ...

On Thursday, September 20, 2012 2:36:39 AM UTC-7, Thomas Løcke wrote:
> On 09/20/2012 10:53 AM, Dmitry A. Kazakov wrote:

> I had not thought about that. What you're saying is that despite the
> "then abort" syntax I can not safely rely on the termination of the
> ongoing AWS.Net.Std.Connect procedure? 

That's correct.  Here are my thoughts on this, to add to what Dmitry
said:

If your 2-second period times out, the call to AWS.Net.Std.Connect is
*aborted*.  RM 9.8 defines exactly what that means.  The important
thing is that it defines certain points called "abort completion
points".  They include things like the beginning or end of an ACCEPT
statement, the beginning of end of a DELAY statement, entry calls, and
a number of other cases.  When the code being aborted gets to one of
those points, then it will be aborted.  However, the language doesn't
define whether the code could be aborted *before* one of those points.
In particular, a blocking I/O operation (such as Ada.Text_IO.Get_Line
from a console, or a socket connect) is *not* an abort completion point.
It's up to an Ada implementation to decide where things could be
aborted.  In your case, since you're calling a procedure in another
package, you'd have to peek at the source of that package to find out
where abort completion points might be happening.

If the code in Connect is executing some OS operation that's
doing the connect, then aborting the code could have several different
effects, depending on the Ada implementation:

(1) The OS operation is not affected, since it's not an abort
    completion point.  The program still waits for the OS call to
    return; and then, later, when some other abort completion point is
    reached, the abort will finally take place.
    
(2) The expiration of the "delay 2.0" causes some sort of interrupt
    handler to be executed.  This may or may not disrupt the OS call.
    The interrupt handler then manipulates the program stack to cause
    it to continue executing at the statement after the "delay 2.0".
    Whether this actually works or not is highly OS- and
    implementation-dependent, and it's entirely possible that on some
    OS's it could do damage.

(3) The Ada implementation provides facilities for abortable I/O
    operations, and integrates this with its runtime so that an abort
    will work properly.  Of course, this means that
    AWS.Net.Std.Connect would have to use those facilities, rather
    than calling the OS operation directly.  Also, a facility like
    this would have to be written separately for each kind of I/O
    operation.  

Whether a facility *could* be written to make a particular type of I/O
abortable depends on the OS, and I don't think it can always be done.
On Linux, there's a connect() call to connect to a socket.  But I
don't think there's a cancel_connect() call that would let you
terminate a connect() call that had started (perhaps in a different
thread).  This means that if you want a timeout, you probably have to
know *ahead* of time, before you call connect(), whether the connect()
will have an expiration period and how long it is.  (And I'm not even
100% sure that you can do a "timed connect" in Linux even if you do
know.)

Hope this helps,

                         -- Adam

Re: Tasking and timing out

[Thomas Løcke]

On 09/20/2012 12:21 PM, Dmitry A. Kazakov wrote:
> #1 + #2.1.1 above. E.g. some computations, even with controlled types
> involved. Anything else requires *careful* analysis and thus is necessarily
> very fragile. Ergo, for the software design POV, it is better to forget
> about ATC, too problematic.


I've taken your advice to heart Dmitry, and re-coded the routine without
ATC, and it is much better for it.

Thank you for your help and for explaining exactly why my solution was
bad.

:)

-- 
Thomas L�cke | thomas@12boo.net | http://12boo.net

Re: Tasking and timing out

[Thomas Løcke]

On 09/20/2012 08:33 PM, Adam Beneschan wrote:
> First of all, doesn't AWS.Net.Std.Connect have its own facility for
> handling timeouts?  I'm looking at the source, and it looks like it
> does, but I don't know if I'm looking at the most recent version.  If
> it does, you should definitely use that, and not rely on ATC.
>
> Anyway ...
> [snip lots of good stuff]


Awesome explanation Adam! Your post really drove the point Dmitry made
home.

And yes, AWS does have some facilities for this, and yes I'm using those
in my new non-ATC solution.

"then abort" is out the window!

:)

-- 
Thomas L�cke | thomas@12boo.net | http://12boo.net

Re: Tasking and timing out

[sbelmont700]

On Thursday, September 20, 2012 2:33:46 PM UTC-4, Adam Beneschan wrote:
> 
> (3) The Ada implementation provides facilities for abortable I/O
>     operations, and integrates this with its runtime so that an abort
>     will work properly.  Of course, this means that
>     AWS.Net.Std.Connect would have to use those facilities, rather
>     than calling the OS operation directly.  Also, a facility like
>     this would have to be written separately for each kind of I/O
>     operation.  
> 

Can you (or someone) elaborate on this?  Do these currently exist in any fashion in an implementation, or are they standarized in any way by the LRM?  Or is this purely hypothetical?  My curiosity has been piqued.

Thank you

-sb