Dynamic allocation of tasks

Dynamic allocation of tasks

[Florian Weimer]

Dynamically allocating tasks isn't really a problem, but deallocating
them is. :-/

I found no way to complete a task before its allocator completes.
Most of the time, this is a good thing.  But if you are allocating
tasks using an allocator for a library-level access type, you'll almost
certainly run into problems, because these tasks won't complete until the
program terminates.  There are workarounds, of course, and I'll describe
two of them below.  If you know different ones, I'd be glad if you could
share them; I'm going to add a summary to the FAQ at www.adapower.com
(if David Botton agrees).


First Solution: Static allocation

This solution uses an approach comparable to a typical implementation
of Ada.Strings.Bounded.  Of course, this simple solution isn't possible
in all cases.  Code example:

   task type Some_Task;
   type Task_Array is array (Positive range <>) of Some_Task;

   type Dynamic_Tasks_Type (Number_Of_Tasks : Natural) is record
      Tasks : Task_Array (1 .. Number_Of_Tasks);
      Last_Allocated_Task : Natural := 0; -- an index into Tasks
   end record;

Some_Task typically has got some kind of start entry, so that superfluous
elements in Tasks which aren't really used cause no harm (besides
unnecessary resource consumption).

This approach gets more and more clumsy if you've got more than one
task type because you have to specify each task count individually.


Second Solution: Localize the access type used to allocate tasks

The idea behind this is that an access type which is declared locally
doesn't cause the problems of a library-level one.  This is not always
directly possible, e.g. in the following pseudocode (which uses a
hypothetical package Dynamic_Tasks which allocates tasks dynamically):

   Dynamic_Tasks.Allocates_Tasks_As_Side_Effect;
   Dynamic_Tasks.Allocates_More_Tasks_As_Side_Effect;

   Dynamic_Tasks.Do_Something_With_These_Tasks;
   Dynamic_Tasks.Do_Something_Else;

   Dynamic_Tasks.Wait_For_These_Tasks;

All these functions must access the allocated tasks, and they have to
use an access of a type declared in the enclosing package for that.

Usually, this access type is located at the library level, but if you
make the package a generic one, this is no longer the case.  The drawback
of this solution: It might result in severe code bloat if the compiler
doesn't support shared generics.

In order to avoid this code bloat, one could declare in a generic
subprogram the access type whose allocator is used to allocate the tasks.
The operations listed in the example would be placed into a local
subprogram, and this subprogram would be passed as actual parameter to
the generic subprogram instantiation.

Re: Dynamic allocation of tasks

[Robert Dewar]

In article <87r9d3gvix.fsf@deneb.cygnus.argh.org>,
  Florian Weimer <fw-usenet@deneb.cygnus.argh.org> wrote:
> Dynamically allocating tasks isn't really a problem, but
deallocating
> them is. :-/
>
> I found no way to complete a task before its allocator
completes.

terminate task (e.g. abort it), then use unchecked
deallocation (see RM D.12(4)).


Sent via Deja.com http://www.deja.com/
Before you buy.

Re: Dynamic allocation of tasks

[Florian Weimer]

Robert Dewar <robert_dewar@my-deja.com> writes:

> > I found no way to complete a task before its allocator
> completes.
> 
> terminate task (e.g. abort it), then use unchecked
> deallocation (see RM D.12(4)).

I tried this (with GNAT 3.12p), and it didn't work (there was a resource
leak, of both memory and kernel threads IIRC).  After reading the RM,
I was convinced that I couldn't expect an implementation to support
this, but a more careful study revealed that it's probably reasonable
to expect this behavior (abort of task causes abnormal completion of
the task body, but it *does* cause completion, which means the task
body is finalized next, and, after that, it is terminated).

What's the effect of an abort statement if the task in question is
blocked in a syscall (such as read(2))?

Re: Dynamic allocation of tasks

[Lutz Donnerhacke]

* Florian Weimer wrote:
>What's the effect of an abort statement if the task in question is
>blocked in a syscall (such as read(2))?

OS depended: May block forever.

Re: Dynamic allocation of tasks

[Florian Weimer]

lutz@iks-jena.de (Lutz Donnerhacke) writes:

> * Florian Weimer wrote:
> >What's the effect of an abort statement if the task in question is
> >blocked in a syscall (such as read(2))?
> 
> OS depended: May block forever.

Yes, but we can assume an OS where unkillable processes stuck in the
kernel are considered buggy kernel behavior (e.g., Linux).

In my last posting, I have failed to mention a piece of information which
might be important: after an "abort Some_Task;", "Some_Task'Terminated'
never became true (even if I waited three or five seconds before
examining the attribute).  In fact, this motivated my first RM
interpretation that an abnormal task is not a terminated task.  (No,
compiler behavior does not determine my RM interpretations, but it
suggest the direction in which I search for evidence.  The Ada tasking
stuff is rather new to me, and I don't see the whole picture yet.)

Re: Dynamic allocation of tasks

[Robert A Duff]

Florian Weimer <fw@deneb.cygnus.argh.org> writes:

> In my last posting, I have failed to mention a piece of information which
> might be important: after an "abort Some_Task;", "Some_Task'Terminated'
> never became true (even if I waited three or five seconds before
> examining the attribute).  In fact, this motivated my first RM
> interpretation that an abnormal task is not a terminated task.

That's correct -- abnormal is not the same thing as terminated.  An
"abort T;" says, more or less, "Dear T, please start aborting yourself
when you get around to it."  When the abort statement is over, the
aborting task doesn't know much of anything about the status of T.

I didn't see your exact program, so I don't know what's going on.
Perhaps you have a higher priority task running, which prevents the
abortee from running, so it never gets around to terminating itself?
Maybe the higher priority task was the one doing the abort?

In Ada, especially Ada 95, the way the run-time system normally does
aborts is that the abortee does the work.  The aborter sends some sort
of message to the abortee, and the abortee then does something that's a
lot like raising an exception.  This causes local variables to be
finalized, and eventually terminates the task.  But the task has to run
for that to happen, which means that it has to be the highest priority
task at some point (assuming a single CPU).

Another point is that if the abortee doesn't do certain things (such as
entry calls), then it might *never* notice that it is abnormal.  Unless
the implementation supports the real-time annex, where the rules are
more stringent.

>...  (No,
> compiler behavior does not determine my RM interpretations, but it
> suggest the direction in which I search for evidence.

;-)

- Bob

Re: Dynamic allocation of tasks

[Florian Weimer]

Robert A Duff <bobduff@world.std.com> writes:

> I didn't see your exact program, so I don't know what's going on.

As I wrote earlier, I think I've identified the problem (it's not
about task not reacting to abort, but about terminated tasks not being
deallocated properly).  Robert Dewar made a cryptic comment which I
didn't understand, and perhaps it's really an Ada issue (and not a
GNAT one; it shows up on Solaris as well, BTW).  The following program
allocates more and more memory:

procedure Test_Leak is

   procedure Use_Tasks is
      task type Some_Task;
      task body Some_Task is
      begin
         null;
      end;

      type Task_Array is array (1 .. 1) of Some_Task;
      TA : Task_Array;
   begin
      null;
   end;

begin
   loop
      Use_Tasks;
   end loop;
end;

Each elaboration of the declaration of TA allocates a few storage
elements which are never freed.  The problem disappears if the tasks
is not wrapped within an array (which is probably the reason why no
one has noticed it before).

Of course, this behavior is fully conforming with the letters of the
RM (simply because the RM doesn't talk about storage leaks, except
that an instance of Ada.Unchecked_Deallocation "should actually
reclaim storage" ;), but I wonder whether it violates the spirit of
the RM...

Re: Dynamic allocation of tasks

[Tucker Taft]

Florian Weimer wrote:
> ...  The following program
> allocates more and more memory:
> 
> procedure Test_Leak is
> 
>    procedure Use_Tasks is
>       task type Some_Task;
>       task body Some_Task is
>       begin
>          null;
>       end;
> 
>       type Task_Array is array (1 .. 1) of Some_Task;
>       TA : Task_Array;
>    begin
>       null;
>    end;
> 
> begin
>    loop
>       Use_Tasks;
>    end loop;
> end;
> 
> Each elaboration of the declaration of TA allocates a few storage
> elements which are never freed.  The problem disappears if the tasks
> is not wrapped within an array (which is probably the reason why no
> one has noticed it before).
> 
> Of course, this behavior is fully conforming with the letters of the
> RM (simply because the RM doesn't talk about storage leaks, except
> that an instance of Ada.Unchecked_Deallocation "should actually
> reclaim storage" ;), but I wonder whether it violates the spirit of
> the RM...

This seems like it is definitely a bug, whether or not it violates the 
spirit of the RM.  Unfortunately, compilers and run-time systems are complex
beasts, and it is impossible to test all combinations of features.
Storage leaks are particularly difficult to test for, since writing
a self-checking test for storage leaks is a pain in the neck.

I heartily suggest you report the bug to the appropriate authorities ;-).

-- 
-Tucker Taft   stt@averstar.com   http://www.averstar.com/~stt/
Technical Director, Distributed IT Solutions  (www.averstar.com/tools)
AverStar (formerly Intermetrics, Inc.)   Burlington, MA  USA

Re: Dynamic allocation of tasks

[Robert Dewar]

In article <87ya7aewb8.fsf@deneb.cygnus.argh.org>,
  Florian Weimer <fw-usenet@deneb.cygnus.argh.org> wrote:
> Robert Dewar <robert_dewar@my-deja.com> writes:
>
> > > I found no way to complete a task before its allocator
> > completes.
> >
> > terminate task (e.g. abort it), then use unchecked
> > deallocation (see RM D.12(4)).
>
> I tried this (with GNAT 3.12p), and it didn't work (there was
a resource
> leak, of both memory and kernel threads IIRC).

As always you should post your code. For example, we often
see people doing an abort immediately followed by an
unchecked deallocation, and you can't rely on that to work!


Sent via Deja.com http://www.deja.com/
Before you buy.

Re: Dynamic allocation of tasks

[Florian Weimer]

Robert Dewar <robert_dewar@my-deja.com> writes:

> > I tried this (with GNAT 3.12p), and it didn't work (there was a
> > resource leak, of both memory and kernel threads IIRC).
> 
> As always you should post your code. 

Well, until recently, I was unable to reproduce the behavior in a
sufficiently small test program.  This test program indicates that the
resource leak is a GNAT specific issue, and I'm going to start a thread
on GNAT chat for that.

> For example, we often see people doing an abort immediately followed
> by an unchecked deallocation, and you can't rely on that to work!

Is it save to wait (using delay) until T'Terminated becomes true,
and deallocate after that?