Re: GNAT and large number of threads

[Gene <gene.ressler@gmail.com>]

On Apr 26, 6:50 pm, Wiktor Moskwa <wiktorDOTmos...@gmail.com> wrote:
> Hi,
>
> I'm using GNAT GPL 2006 on Linux x86.
> Recently I had a problem with creating large number of tasks(*) in my
> Ada program. I create tasks in a loop and when I reach the limit of
> stack address space the main thread freezes - no exceptions, nothing.
>
> Using ltrace tool reveals:
> ...
> pthread_create(0x9656e98, 0xbffca3d0, 0x80c3280, 0x9656e58, 10256) = 0
> pthread_create(0x965a670, 0xbffca3d0, 0x80c3280, 0x965a630, 10256) = 0
> pthread_create(0xbfd86048, 0xbffca3d0, 0x80c3280, 0xbfd86008, 10256)= 12
> --- SIGSEGV (Segmentation fault) ---
>
> Error 12 is ENOMEM (the system lacked the necessary resources to
> create another thread). GNAT seems to completely ignore this error.
> Quick search in GNAT GPL sources shows that only EAGAIN error is taken
> into account, others are silently ignored.
> =======================================================================
> [file: s-taprop-linux.adb; procedure: Create_Task]
> Result := pthread_create
>   (T.Common.LL.Thread'Access,
>    Attributes'Access,
>    Thread_Body_Access (Wrapper),
>    To_Address (T));
> pragma Assert (Result = 0 or else Result = EAGAIN);
>
> Succeeded := Result = 0;
>
> Result := pthread_attr_destroy (Attributes'Access);
> pragma Assert (Result = 0);
>
> Set_Priority (T, Priority);
> =======================================================================
> Succeeded (out paramter) is set to False in my case but I guess that
> pthread_attr_destroy or Set_Priority cannot be invoked after failure
> of pthread_create. I'm not sure because I don't know neither pthreads
> nor GNAT compiler.
> Nevertheless it looks like a GNAT bug to me, what do you think?
>
> My question is how can my program predict/calculate (approximetly) the
> maximum number of tasks that can be created with known maximum stack
> size per task (specified with pragma Storage_Size)?
>
> [*] I'm writing a simulator of a distributed protocol wiht one task per
> node so I really need large number of threads to simulate many nodes.
>
> Thanks,
> Wiktor
>
> --
> Wiktor Moskwa

I can't answer your question directly, but the approach of using tasks
as abstract data structures is asking for trouble.

Tasks are appropriate when you _need_ independent computation threads:
I/O with multiple devices, editing a file while a copy is being
written to disk, you get the idea.  Protocol simulation doesn't fit
this bill.  Nodes are essentially data. If you have only a single
processor, you should run in the environment task.  If you need to
exploit an MP environment, then use a task pool of N tasks for N
processors.

In other words, represent data with data structures and computation
threads with tasks.  For your problem, you might want to represent
each node as a set of input and output queues.  Apply one (or more)
computation thread(s) to identify an appropriate node, remove inputs,
create and add outputs to respective queues, then move on to another
node. A heap keyed on number of inputs left to define will always have
a node with all inputs defined at the top. Multiple threads from a
pool work exactly the same way except the queues are protected for
synchronization.