Launching background job from Ada.Real_Time.Timing_Events

Launching background job from Ada.Real_Time.Timing_Events

[Alejandro R. Mosteo]

Hello all,

I'm stuck with something not that strange, so I guess I need another 
viewpoint on it.

I want to trigger, in the future, a possibly long computation in the 
background. For that I had thought of using the built-in Real_Time events.

My idea was to queue the job when the event is triggered, and to have a 
worker thread waiting for jobs in the queue. However, I've just realized 
that the event handler is itself a protected action, so I can't call 
another potentially blocking enqueue from there.

Any ideas? Basically, how to trigger another task (without creating it) 
from within a protected handler?

Thanks,
Alex.

Re: Launching background job from Ada.Real_Time.Timing_Events

[Mark Lorenzen]

On Tuesday, May 24, 2016 at 4:22:36 PM UTC+2, Alejandro R. Mosteo wrote:
> Hello all,
> 
> I'm stuck with something not that strange, so I guess I need another 
> viewpoint on it.
> 
> I want to trigger, in the future, a possibly long computation in the 
> background. For that I had thought of using the built-in Real_Time events.
> 
> My idea was to queue the job when the event is triggered, and to have a 
> worker thread waiting for jobs in the queue. However, I've just realized 
> that the event handler is itself a protected action, so I can't call 
> another potentially blocking enqueue from there.
> 
> Any ideas? Basically, how to trigger another task (without creating it) 
> from within a protected handler?
> 
> Thanks,
> Alex.

You can let the PO implementing the job queue be the same as the PO implementing the timing event handler.

If you want to detach the high-priority PO implementing the timer event handler from the PO implementing the job queue, then let a task block on an entry in the first PO and put the job into the second PO. The task is then the active object moving data between the POs and thus detaching them.

Regards,

Mark L

Re: Launching background job from Ada.Real_Time.Timing_Events

[Alejandro R. Mosteo]

On 24/05/16 16:39, Mark Lorenzen wrote:
> On Tuesday, May 24, 2016 at 4:22:36 PM UTC+2, Alejandro R. Mosteo wrote:
>> Hello all,
>>
>> I'm stuck with something not that strange, so I guess I need another
>> viewpoint on it.
>>
>> I want to trigger, in the future, a possibly long computation in the
>> background. For that I had thought of using the built-in Real_Time events.
>>
>> My idea was to queue the job when the event is triggered, and to have a
>> worker thread waiting for jobs in the queue. However, I've just realized
>> that the event handler is itself a protected action, so I can't call
>> another potentially blocking enqueue from there.
>>
>> Any ideas? Basically, how to trigger another task (without creating it)
>> from within a protected handler?
>>
>> Thanks,
>> Alex.
>
> You can let the PO implementing the job queue be the same as the PO implementing the timing event handler.
>
> If you want to detach the high-priority PO implementing the timer event handler from the PO implementing the job queue, then let a task block on an entry in the first PO and put the job into the second PO. The task is then the active object moving data between the POs and thus detaching them.

Thank you! I don't know what I was thinking...

>
> Regards,
>
> Mark L

Re: Launching background job from Ada.Real_Time.Timing_Events

[Jeffrey R. Carter]

On 05/24/2016 07:39 AM, Mark Lorenzen wrote:
>
> If you want to detach the high-priority PO implementing the timer event
> handler from the PO implementing the job queue, then let a task block on an
> entry in the first PO and put the job into the second PO. The task is then
> the active object moving data between the POs and thus detaching them.

This is known as a forwarder. There's a generic forwarder in the PragmAda 
Reusable Components. The main problem with forwarders is that there's no easy 
way to terminate them.

The PragmARCs are at

pragmada.x10hosting.com

and

github.com/jrcarter

-- 
Jeff Carter
"This scene's supposed to be in a saloon, but
the censor cut it out. It'll play just as well
this way." [in a soda fountain]
Never Give a Sucker an Even Break
113

Re: Launching background job from Ada.Real_Time.Timing_Events

[Jeffrey R. Carter]

On 05/24/2016 07:22 AM, Alejandro R. Mosteo wrote:
>
> My idea was to queue the job when the event is triggered, and to have a worker
> thread waiting for jobs in the queue. However, I've just realized that the event
> handler is itself a protected action, so I can't call another potentially
> blocking enqueue from there.

I can see how this would work, with the queue in the same PO as the handler as 
Lorenzen suggested, but it seems messy to me. You need to store the 
Timing_Events somewhere until they're handled, and since they're limited that 
will require using access types. That in turn requires doing memory management, 
and that will probably need another task to periodically look at all the events 
and free those that have been handled. This has lots of opportunities for error.

A simpler approach would be to package up a Time and your job info in a record 
you can put on a priority queue in Time order, and have the worker task do a 
delay until the Time. The only problem with this is that you need to have enough 
tasks to ensure all jobs are started at the appropriate time, or a willingness 
to create a new task if an event is queued when there are no workers available. 
I have used the latter approach and it works fine if dynamically created tasks 
are acceptable.

-- 
Jeff Carter
"This scene's supposed to be in a saloon, but
the censor cut it out. It'll play just as well
this way." [in a soda fountain]
Never Give a Sucker an Even Break
113

Re: Launching background job from Ada.Real_Time.Timing_Events

[Dmitry A. Kazakov]

On 24/05/2016 16:22, Alejandro R. Mosteo wrote:

> I'm stuck with something not that strange, so I guess I need another
> viewpoint on it.
>
> I want to trigger, in the future, a possibly long computation in the
> background. For that I had thought of using the built-in Real_Time events.
>
> My idea was to queue the job when the event is triggered, and to have a
> worker thread waiting for jobs in the queue. However, I've just realized
> that the event handler is itself a protected action, so I can't call
> another potentially blocking enqueue from there.

Neither is protected action. To enqueue anything into FIFO you need no 
locking. This is lock-free. To dequeue, provided single consumer there 
is no need to lock anything either. If there are several consumers, you 
use a protected object to dequeue only.

1-to-n: The dequeue is a waitable entry. Once you have got the packet 
out of the queue you process it outside any protected actions.

1-to-1: You use a reset-event implemented using protected object to wake 
up the consumer, which dequeues the packet. Reset event is 
race-condition free to query if the queue is empty and if so, to wait 
for the event.

> Any ideas? Basically, how to trigger another task (without creating it)
> from within a protected handler?

Always by waiting for an entry. E.g. as described above.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: Launching background job from Ada.Real_Time.Timing_Events

[Alejandro R. Mosteo]

On 25/05/16 00:21, Jeffrey R. Carter wrote:
> On 05/24/2016 07:39 AM, Mark Lorenzen wrote:
>>
>> If you want to detach the high-priority PO implementing the timer event
>> handler from the PO implementing the job queue, then let a task block
>> on an
>> entry in the first PO and put the job into the second PO. The task is
>> then
>> the active object moving data between the POs and thus detaching them.
>
> This is known as a forwarder. There's a generic forwarder in the
> PragmAda Reusable Components. The main problem with forwarders is that
> there's no easy way to terminate them.

Indeed I was also thinking about this. I can't think of anything besides 
periodical checks to some bail-out flag.

> The PragmARCs are at
>
> pragmada.x10hosting.com
>
> and
>
> github.com/jrcarter

Thanks!

Re: Launching background job from Ada.Real_Time.Timing_Events

[Alejandro R. Mosteo]

On 25/05/16 01:52, Jeffrey R. Carter wrote:
> On 05/24/2016 07:22 AM, Alejandro R. Mosteo wrote:
>>
>> My idea was to queue the job when the event is triggered, and to have
>> a worker
>> thread waiting for jobs in the queue. However, I've just realized that
>> the event
>> handler is itself a protected action, so I can't call another potentially
>> blocking enqueue from there.
>
> I can see how this would work, with the queue in the same PO as the
> handler as Lorenzen suggested, but it seems messy to me. You need to
> store the Timing_Events somewhere until they're handled, and since
> they're limited that will require using access types. That in turn
> requires doing memory management, and that will probably need another
> task to periodically look at all the events and free those that have
> been handled. This has lots of opportunities for error.
> A simpler approach would be to package up a Time and your job info in a
> record you can put on a priority queue in Time order, and have the
> worker task do a delay until the Time. The only problem with this is
> that you need to have enough tasks to ensure all jobs are started at the
> appropriate time, or a willingness to create a new task if an event is
> queued when there are no workers available. I have used the latter
> approach and it works fine if dynamically created tasks are acceptable.

I'm in no need for strict real time requirements so I can do with this 
softer approach. Thanks for the suggestion.

Re: Launching background job from Ada.Real_Time.Timing_Events

[Alejandro R. Mosteo]

On 25/05/16 09:23, Dmitry A. Kazakov wrote:
> On 24/05/2016 16:22, Alejandro R. Mosteo wrote:
>
>> I'm stuck with something not that strange, so I guess I need another
>> viewpoint on it.
>>
>> I want to trigger, in the future, a possibly long computation in the
>> background. For that I had thought of using the built-in Real_Time
>> events.
>>
>> My idea was to queue the job when the event is triggered, and to have a
>> worker thread waiting for jobs in the queue. However, I've just realized
>> that the event handler is itself a protected action, so I can't call
>> another potentially blocking enqueue from there.
>
> Neither is protected action. To enqueue anything into FIFO you need no
> locking. This is lock-free. To dequeue, provided single consumer there
> is no need to lock anything either. If there are several consumers, you
> use a protected object to dequeue only.
> 1-to-n: The dequeue is a waitable entry. Once you have got the packet
> out of the queue you process it outside any protected actions.
>
> 1-to-1: You use a reset-event implemented using protected object to wake
> up the consumer, which dequeues the packet. Reset event is
> race-condition free to query if the queue is empty and if so, to wait
> for the event.

I'm sorry but I don't follow your explanation. To make things simpler, 
let's consider only the 1-to-1 case. How's this protected used without a 
blocking operation?

Thanks.

>> Any ideas? Basically, how to trigger another task (without creating it)
>> from within a protected handler?
>
> Always by waiting for an entry. E.g. as described above.
>

Re: Launching background job from Ada.Real_Time.Timing_Events

[Jeffrey R. Carter]

On 06/02/2016 02:13 PM, Alejandro R. Mosteo wrote:
> On 25/05/16 00:21, Jeffrey R. Carter wrote:
>>
>> This is known as a forwarder. There's a generic forwarder in the
>> PragmAda Reusable Components. The main problem with forwarders is that
>> there's no easy way to terminate them.
> 
> Indeed I was also thinking about this. I can't think of anything besides
> periodical checks to some bail-out flag.

One could complicate the generic significantly to allow this. Custom forwarders
can do it easily, but then you have code duplication.

-- 
Jeff Carter
"It has been my great privilege, many years ago,
whilst traveling through the mountains of Paraguay,
to find the Yack'Wee Indians drinking the juice of
the cacti."
The Old Fashioned Way
152

Re: Launching background job from Ada.Real_Time.Timing_Events

[Dmitry A. Kazakov]

On 02/06/2016 23:25, Alejandro R. Mosteo wrote:

> I'm sorry but I don't follow your explanation. To make things simpler,
> let's consider only the 1-to-1 case. How's this protected used without a
> blocking operation?

If you have FIFO, a lock-free implementation is a buffer with one read 
and one write index (mod buffer length). The read index never advances 
beyond the write index. Reader updates the read index and looks at the 
write index, and conversely the writer. Indices must have pragma Atomic, 
and the buffer not cached. For example:

http://www.dmitry-kazakov.de/ada/components.htm#10.1.1

Of course if you want to wait for a FIFO state, e.g. 'not full' or 'not 
empty' you will need an event (a protected object). But you don't need 
to have the FIFO a protected object.

It is an important point not to have containers protected. Operations on 
container might be quite expensive depending on the element types and 
the container structure. As a rule you should not maintain containers on 
the context of a protected action. E.g. if you have to do a binary 
search to get an element, that is not to do in a protected action.

>>> Any ideas? Basically, how to trigger another task (without creating it)
>>> from within a protected handler?

By changing an entry barrier, of course.

The task to trigger waits for a protected entry to open, it is released 
after leaving the entry.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: Launching background job from Ada.Real_Time.Timing_Events

[Alejandro R. Mosteo]

On 03/06/16 09:26, Dmitry A. Kazakov wrote:
> On 02/06/2016 23:25, Alejandro R. Mosteo wrote:
>
>> I'm sorry but I don't follow your explanation. To make things simpler,
>> let's consider only the 1-to-1 case. How's this protected used without a
>> blocking operation?
>
> If you have FIFO, a lock-free implementation is a buffer with one read
> and one write index (mod buffer length). The read index never advances
> beyond the write index. Reader updates the read index and looks at the
> write index, and conversely the writer. Indices must have pragma Atomic,
> and the buffer not cached. For example:
>
> http://www.dmitry-kazakov.de/ada/components.htm#10.1.1

Thanks, I was unsure if you referred to using a lock-free data type, or 
if it could be trivially done with the standard Ada containers with some 
well-known pattern I was ignorant of.

I'll take a look at your library, I'm very interested in lock-free 
containers.

A friend working on hardware architectures recently told me that a 
widely-used lock-free library (in the c++ world) had been proven with 
SPARK-like methods to be buggy. Alas, I can't remember names/dates.

Thanks,
Alex.

>
> Of course if you want to wait for a FIFO state, e.g. 'not full' or 'not
> empty' you will need an event (a protected object). But you don't need
> to have the FIFO a protected object.
>
> It is an important point not to have containers protected. Operations on
> container might be quite expensive depending on the element types and
> the container structure. As a rule you should not maintain containers on
> the context of a protected action. E.g. if you have to do a binary
> search to get an element, that is not to do in a protected action.
>
>>>> Any ideas? Basically, how to trigger another task (without creating it)
>>>> from within a protected handler?
>
> By changing an entry barrier, of course.
>
> The task to trigger waits for a protected entry to open, it is released
> after leaving the entry.
>

Re: Launching background job from Ada.Real_Time.Timing_Events

[Dmitry A. Kazakov]

On 03/06/2016 12:03, Alejandro R. Mosteo wrote:
> On 03/06/16 09:26, Dmitry A. Kazakov wrote:
>> On 02/06/2016 23:25, Alejandro R. Mosteo wrote:
>>
>>> I'm sorry but I don't follow your explanation. To make things simpler,
>>> let's consider only the 1-to-1 case. How's this protected used without a
>>> blocking operation?
>>
>> If you have FIFO, a lock-free implementation is a buffer with one read
>> and one write index (mod buffer length). The read index never advances
>> beyond the write index. Reader updates the read index and looks at the
>> write index, and conversely the writer. Indices must have pragma Atomic,
>> and the buffer not cached. For example:
>>
>> http://www.dmitry-kazakov.de/ada/components.htm#10.1.1
>
> Thanks, I was unsure if you referred to using a lock-free data type, or
> if it could be trivially done with the standard Ada containers with some
> well-known pattern I was ignorant of.

Yes, but the larger point was that using a standard container protected 
by either a monitor task or else by a protected interlocking object is 
IMO a better design that a specialized container in a protected object.

> I'll take a look at your library, I'm very interested in lock-free
> containers.
>
> A friend working on hardware architectures recently told me that a
> widely-used lock-free library (in the c++ world) had been proven with
> SPARK-like methods to be buggy. Alas, I can't remember names/dates.

That is quite possible, however I don't know what technique they used 
for making proofs. It is much about the semantics of the language 
constructs in presence of concurrent access. I am not sure about so many 
things even in Ada. I can imagine proving an implementation wrong, but 
the reverse might be exceptionally tricky. I remember reading some 
papers about this stuff, that used formal mathematical proofs, it was 
very complicated. I believe one could use a brute-force enumeration of 
possible accesses in some simulated time for making the proof in SPARK 
environment.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de