Fun with Tasking

Fun with Tasking

[mhamel_98]

Below is sample code I made representing something I wanted to do with
a program, of course it doesn't work quite the way I would like.
Hopefully this sample doesn't cause eye (or brain) injury, it is more
symbolic than useful - the procedure "Exec" is, of course, really a
package (rather, several similar packages).

The problem is, how to "transport" a task?  As seen in the sample
below, I would like to use just a single address in the Control_Block
type, but it's never that simple, is it?  In actuality, how I get this
program in its full incarnation to work is to define the "test_task"s
in the acc_add_test part of the program.  I really don't like doing
this, as I mentioned, "exec" is representative of a family of several
different packages.  I would like the task definition to be local to
the "exec" pacakage(s), not all kludged together in acc_add_test,
which then requires all sorts of (ideally private) types being moved
out of the local "exec" packages into acc_add_test and then the
Control_Block record now has nearly a dozen different task pointers
stuffed into it, only one of which is going to be used at any given
time.

One last caveat, the Acc_Add_Test program should have *no* visibility
into the "exec" procedure/package.  The "exec" package(s), in fact,
"push" their procedures into the main program.  So is there something
simple I'm having a brain fart over in missing here, or is a complete
re-architecture required here?

with Text_Io;
with System;
with System.Address_To_Access_Conversions;

procedure Acc_Add_test is

  Exec_Count : constant Natural := 10;

  type Control_Block is
    record
      Task_Addr : System.Address;
    end record;

  type Mode_Type is (preprocess, process);

  procedure Exec (Mode : in     Mode_Type;
                  Cntl : in out Control_Block) is

    task type Test_Task is
      entry Start (Addr  : System.Address);
      entry Complete;
    end Test_Task;

    package Convert is new System.Address_To_Access_Conversions
(Test_Task);
    use Convert;

    task body Test_Task is
      Task_Ptr : Convert.Object_Pointer := null;
      Count    : Natural := 0;
    begin
      loop
        begin
          select
            accept Start (Addr : System.Address) do
              Task_Ptr := Convert.To_Pointer (Addr);
            end Start;

          or
            accept Complete do
              Count := Count + 1;
            end Complete;
            if Count = Exec_Count then
              Text_Io.Put_Line ("Completed All Tasks");
              abort Task_Ptr.all;
            end if;
          end select;
        end;
      end loop;
    end Test_Task;

  begin
    case Mode is
      when Preprocess =>
        declare
          Task_Ptr : Convert.Object_Pointer;
          Address  : System.Address;
        begin
          Task_Ptr := new Test_Task;
          Address := Convert.To_Address (Task_Ptr);
          Task_Ptr.Start (Address);
          Cntl.Task_Addr := Address;
        end;

      when Process =>
        declare
          Task_Ptr : Convert.Object_Pointer;
        begin
          Task_Ptr := Convert.To_Pointer (Cntl.Task_Addr);
          Task_Ptr.Complete;
        end;
    end case;
  end Exec;

  Controls : Control_Block;

begin
  Exec (Preprocess, Controls);
  for I in 1 .. Exec_Count loop
    Exec (Process, Controls);
  end loop;
end Acc_Add_Test;

Thanks for any input!

Re: Fun with Tasking

[Adam Beneschan]

On Mar 28, 8:53 am, mhamel...@yahoo.com wrote:

> Below is sample code I made representing something I wanted to do with
> a program, of course it doesn't work quite the way I would like.
> Hopefully this sample doesn't cause eye (or brain) injury, it is more
> symbolic than useful - the procedure "Exec" is, of course, really a
> package (rather, several similar packages).
>
> The problem is, how to "transport" a task?  As seen in the sample
> below, I would like to use just a single address in the Control_Block
> type, but it's never that simple, is it?  In actuality, how I get this
> program in its full incarnation to work is to define the "test_task"s
> in the acc_add_test part of the program.  I really don't like doing
> this, as I mentioned, "exec" is representative of a family of several
> different packages.  I would like the task definition to be local to
> the "exec" pacakage(s), not all kludged together in acc_add_test,
> which then requires all sorts of (ideally private) types being moved
> out of the local "exec" packages into acc_add_test and then the
> Control_Block record now has nearly a dozen different task pointers
> stuffed into it, only one of which is going to be used at any given
> time.
>
> One last caveat, the Acc_Add_Test program should have *no* visibility
> into the "exec" procedure/package.  The "exec" package(s), in fact,
> "push" their procedures into the main program.  So is there something
> simple I'm having a brain fart over in missing here, or is a complete
> re-architecture required here?
>
> with Text_Io;
> with System;
> with System.Address_To_Access_Conversions;
>
> procedure Acc_Add_test is
>
>   Exec_Count : constant Natural := 10;
>
>   type Control_Block is
>     record
>       Task_Addr : System.Address;
>     end record;
>
>   type Mode_Type is (preprocess, process);
>
>   procedure Exec (Mode : in     Mode_Type;
>                   Cntl : in out Control_Block) is
>
>     task type Test_Task is
>       entry Start (Addr  : System.Address);
>       entry Complete;
>     end Test_Task;
>
>     package Convert is new System.Address_To_Access_Conversions
> (Test_Task);
>     use Convert;
>
>     task body Test_Task is
>       Task_Ptr : Convert.Object_Pointer := null;
>       Count    : Natural := 0;
>     begin
>       loop
>         begin
>           select
>             accept Start (Addr : System.Address) do
>               Task_Ptr := Convert.To_Pointer (Addr);
>             end Start;
>
>           or
>             accept Complete do
>               Count := Count + 1;
>             end Complete;
>             if Count = Exec_Count then
>               Text_Io.Put_Line ("Completed All Tasks");
>               abort Task_Ptr.all;
>             end if;
>           end select;
>         end;
>       end loop;
>     end Test_Task;
>
>   begin
>     case Mode is
>       when Preprocess =>
>         declare
>           Task_Ptr : Convert.Object_Pointer;
>           Address  : System.Address;
>         begin
>           Task_Ptr := new Test_Task;
>           Address := Convert.To_Address (Task_Ptr);
>           Task_Ptr.Start (Address);
>           Cntl.Task_Addr := Address;
>         end;
>
>       when Process =>
>         declare
>           Task_Ptr : Convert.Object_Pointer;
>         begin
>           Task_Ptr := Convert.To_Pointer (Cntl.Task_Addr);
>           Task_Ptr.Complete;
>         end;
>     end case;
>   end Exec;
>
>   Controls : Control_Block;
>
> begin
>   Exec (Preprocess, Controls);
>   for I in 1 .. Exec_Count loop
>     Exec (Process, Controls);
>   end loop;
> end Acc_Add_Test;
>
> Thanks for any input!


Your main problem is that your tasks aren't going to live long
enough.  When a task is created by an allocator, it depends on the
"master" that declares the access type (or if the access type is a
derived type, which isn't the case here, on the "ultimate ancestor" of
the access type).  See RM section 9.3.  In this case, the master that
declares the access type (Convert.Object_Pointer) is the Exec
procedure.  This means that any task that you create using this access
type depends on Exec, and Exec will not finish until any such tasks
are completed.  It looks to me that you're trying to be able to call
Exec multiple times, and have it start a task the first time and then
use the same task the next time Exec is called.  This won't work if
the task type is declared inside Exec, because any such tasks will not
be allowed to keep running if Exec isn't running.  You'll need to move
the task types (and the access type that points to the task type)
outside Exec.

This is the only thing that makes sense, anyway.  If your task type is
declared inside Exec, the task body will have access to Exec's
parameters and local variables; how then could it be possible for the
task body to stay running while Exec isn't?

I'm not real clear on what you're trying to do.  Your sample has
"Exec" as a procedure, which makes it a master, but your discussion
talks about "Exec" *packages*---and a package is not a master.  If you
wanted an Exec *package* that declares its own task type and its own
procedure to do the processing, but the task type isn't inside the
procedure, then I suspect that you may be able to accomplish what
you're trying to do; and there should be a way to keep certain things
private without having to resort to hokeyness like System.Address or
Address_To_Access_Conversions.  But I'm not up to trying to guess what
you're trying to do; I'd recommend that you redo your example a bit to
avoid the task dependence problem and then ask again if you still
aren't sure how to do what you need to do.

                                 -- Adam

Re: Fun with Tasking

[mhamel_98]

On Mar 28, 12:55 pm, "Adam Beneschan" <a...@irvine.com> wrote:
> On Mar 28, 8:53 am, mhamel...@yahoo.com wrote:
>
>
>
>
>
> > Below is sample code I made representing something I wanted to do with
> > a program, of course it doesn't work quite the way I would like.
> > Hopefully this sample doesn't cause eye (or brain) injury, it is more
> > symbolic than useful - the procedure "Exec" is, of course, really a
> > package (rather, several similar packages).
>
> > The problem is, how to "transport" a task?  As seen in the sample
> > below, I would like to use just a single address in the Control_Block
> > type, but it's never that simple, is it?  In actuality, how I get this
> > program in its full incarnation to work is to define the "test_task"s
> > in the acc_add_test part of the program.  I really don't like doing
> > this, as I mentioned, "exec" is representative of a family of several
> > different packages.  I would like the task definition to be local to
> > the "exec" pacakage(s), not all kludged together in acc_add_test,
> > which then requires all sorts of (ideally private) types being moved
> > out of the local "exec" packages into acc_add_test and then the
> > Control_Block record now has nearly a dozen different task pointers
> > stuffed into it, only one of which is going to be used at any given
> > time.
>
> > One last caveat, the Acc_Add_Test program should have *no* visibility
> > into the "exec" procedure/package.  The "exec" package(s), in fact,
> > "push" their procedures into the main program.  So is there something
> > simple I'm having a brain fart over in missing here, or is a complete
> > re-architecture required here?
>
> > with Text_Io;
> > with System;
> > with System.Address_To_Access_Conversions;
>
> > procedure Acc_Add_test is
>
> >   Exec_Count : constant Natural := 10;
>
> >   type Control_Block is
> >     record
> >       Task_Addr : System.Address;
> >     end record;
>
> >   type Mode_Type is (preprocess, process);
>
> >   procedure Exec (Mode : in     Mode_Type;
> >                   Cntl : in out Control_Block) is
>
> >     task type Test_Task is
> >       entry Start (Addr  : System.Address);
> >       entry Complete;
> >     end Test_Task;
>
> >     package Convert is new System.Address_To_Access_Conversions
> > (Test_Task);
> >     use Convert;
>
> >     task body Test_Task is
> >       Task_Ptr : Convert.Object_Pointer := null;
> >       Count    : Natural := 0;
> >     begin
> >       loop
> >         begin
> >           select
> >             accept Start (Addr : System.Address) do
> >               Task_Ptr := Convert.To_Pointer (Addr);
> >             end Start;
>
> >           or
> >             accept Complete do
> >               Count := Count + 1;
> >             end Complete;
> >             if Count = Exec_Count then
> >               Text_Io.Put_Line ("Completed All Tasks");
> >               abort Task_Ptr.all;
> >             end if;
> >           end select;
> >         end;
> >       end loop;
> >     end Test_Task;
>
> >   begin
> >     case Mode is
> >       when Preprocess =>
> >         declare
> >           Task_Ptr : Convert.Object_Pointer;
> >           Address  : System.Address;
> >         begin
> >           Task_Ptr := new Test_Task;
> >           Address := Convert.To_Address (Task_Ptr);
> >           Task_Ptr.Start (Address);
> >           Cntl.Task_Addr := Address;
> >         end;
>
> >       when Process =>
> >         declare
> >           Task_Ptr : Convert.Object_Pointer;
> >         begin
> >           Task_Ptr := Convert.To_Pointer (Cntl.Task_Addr);
> >           Task_Ptr.Complete;
> >         end;
> >     end case;
> >   end Exec;
>
> >   Controls : Control_Block;
>
> > begin
> >   Exec (Preprocess, Controls);
> >   for I in 1 .. Exec_Count loop
> >     Exec (Process, Controls);
> >   end loop;
> > end Acc_Add_Test;
>
> > Thanks for any input!
>
> Your main problem is that your tasks aren't going to live long
> enough.  When a task is created by an allocator, it depends on the
> "master" that declares the access type (or if the access type is a
> derived type, which isn't the case here, on the "ultimate ancestor" of
> the access type).  See RM section 9.3.  In this case, the master that
> declares the access type (Convert.Object_Pointer) is the Exec
> procedure.  This means that any task that you create using this access
> type depends on Exec, and Exec will not finish until any such tasks
> are completed.  It looks to me that you're trying to be able to call
> Exec multiple times, and have it start a task the first time and then
> use the same task the next time Exec is called.  This won't work if
> the task type is declared inside Exec, because any such tasks will not
> be allowed to keep running if Exec isn't running.  You'll need to move
> the task types (and the access type that points to the task type)
> outside Exec.
>
> This is the only thing that makes sense, anyway.  If your task type is
> declared inside Exec, the task body will have access to Exec's
> parameters and local variables; how then could it be possible for the
> task body to stay running while Exec isn't?
>
> I'm not real clear on what you're trying to do.  Your sample has
> "Exec" as a procedure, which makes it a master, but your discussion
> talks about "Exec" *packages*---and a package is not a master.  If you
> wanted an Exec *package* that declares its own task type and its own
> procedure to do the processing, but the task type isn't inside the
> procedure, then I suspect that you may be able to accomplish what
> you're trying to do; and there should be a way to keep certain things
> private without having to resort to hokeyness like System.Address or
> Address_To_Access_Conversions.  But I'm not up to trying to guess what
> you're trying to do; I'd recommend that you redo your example a bit to
> avoid the task dependence problem and then ask again if you still
> aren't sure how to do what you need to do.
>
>                                  -- Adam- Hide quoted text -
>
> - Show quoted text -

Thank you Adam.  While making this sample program a sneaking suspicion
started to creep up that I really didn't know what I was doing, and
you helped verify that suspicion.  What's really going on is I have a
number of tasks I'm going to run, and while themselves completely
independent of one another, they send updates to a single listening,
or collating, task.  That is the task at the heart of my initial
post.  My approach was to start up this collating task and give it's
pointer (or address) to the processing tasks.  This works fine if the
collating task definition is in the acc_add_test procedure and the
Control_Block has a pointer to that task within it.  However, I would
like the collating task definition to reside with the "exec" procedure/
package as the sample has it.  How would one declare and start up that
task, without any visibility into the package, just a procedure
pointer with a mode argument?  My (terribly flawed) idea was the
PreProcess element of the procedure would set that up, but as you
pointed out, the procedure would never exit while the task is
running.  So how to define the task and put it on "hold" until the
processing tasks can utilize it?  I don't expect it's possible without
incurring as much ugliness as the program has already, but I thought
I'd give it a try.  Thanks again =)

Re: Fun with Tasking

[Ed Falis]

mhamel_98@yahoo.com wrote:

> Thank you Adam.  While making this sample program a sneaking suspicion
> started to creep up that I really didn't know what I was doing, and
> you helped verify that suspicion.

You sound like a fine candidate for a copy of "Concurrency in Ada" by
Burns and Welling.  I did one of the first Ada tasking implementations,
and still learned a lot from this book when I read it a couple of years
ago.

- Ed

Re: Fun with Tasking

[Randy Brukardt]

<mhamel_98@yahoo.com> wrote in message
news:1175104396.231171.125360@e65g2000hsc.googlegroups.com...
...
> Thank you Adam.  While making this sample program a sneaking suspicion
> started to creep up that I really didn't know what I was doing, and
> you helped verify that suspicion.  What's really going on is I have a
> number of tasks I'm going to run, and while themselves completely
> independent of one another, they send updates to a single listening,
> or collating, task.  That is the task at the heart of my initial
> post.  My approach was to start up this collating task and give it's
> pointer (or address) to the processing tasks.  This works fine if the
> collating task definition is in the acc_add_test procedure and the
> Control_Block has a pointer to that task within it.  However, I would
> like the collating task definition to reside with the "exec" procedure/
> package as the sample has it.  How would one declare and start up that
> task, without any visibility into the package, just a procedure
> pointer with a mode argument?  My (terribly flawed) idea was the
> PreProcess element of the procedure would set that up, but as you
> pointed out, the procedure would never exit while the task is
> running.  So how to define the task and put it on "hold" until the
> processing tasks can utilize it?  I don't expect it's possible without
> incurring as much ugliness as the program has already, but I thought
> I'd give it a try.  Thanks again =)

I'm not going to try to figure out precisely what it is that you are trying
to do (a listening task doesn't require anything special to listen!), but a
couple of thoughts about trying to manage unrelated tasks with another task.

(1) You really don't want to try to manage unrelated anything in Ada (Ada is
strongly typed, after all). If you are using a very new compiler, probably
the best way to relate some tasks that would otherwise be unrelated is to
define them all to be derived from the same task interface. The idea is that
you would put the control operations into the task interface, and then the
manager would use access-to-interface'class to keep track of the tasks and
call operations on them as needed. But you'll need a complete and non-buggy
implementation of the entire Ada language, as amended; so far as I know only
GNAT Pro even claims to support this. So you'd be better off trying to do
this next year...

Note that you have to start each task individually. The usual trick is to
use a constructor function for that purpose.

(2) Otherwise, I'd say that it can't be done reasonably. You could use
Task-Ids to track the status of the tasks,  but that won't help starting
them.

(3) Probably the best plan is to forget about having the management task
interoperate directly with the other tasks. Rather, define an abstract
tagged type (or interface, if you prefer) Processor, and have each package
define any instance of that tagged type. It would have the needed management
operations ("Start", "Status", "Stop", etc.), and again you would create an
appropriate instance of each one (probably with the main program), and have
the management task save a list of these as access-to-Processor'class, and
manage them appropriately.

The advantage of this technique is that you can wrap anything you need in
these operations: if you need a conditional entry call, for instance, you
would just implement one of those operations that way. (That's precisely how
task interfaces get implemented anyway; you're just doing some of the work
that the compiler would done.)

Hope this helps.

                                      Randy.

Re: Fun with Tasking

[Jeffrey R. Carter]

mhamel_98@yahoo.com wrote:
> 
> Thank you Adam.  While making this sample program a sneaking suspicion
> started to creep up that I really didn't know what I was doing, and
> you helped verify that suspicion.  What's really going on is I have a
> number of tasks I'm going to run, and while themselves completely
> independent of one another, they send updates to a single listening,
> or collating, task.  That is the task at the heart of my initial
> post.  My approach was to start up this collating task and give it's
> pointer (or address) to the processing tasks.  This works fine if the
> collating task definition is in the acc_add_test procedure and the
> Control_Block has a pointer to that task within it.  However, I would
> like the collating task definition to reside with the "exec" procedure/
> package as the sample has it.  How would one declare and start up that
> task, without any visibility into the package, just a procedure
> pointer with a mode argument?  My (terribly flawed) idea was the
> PreProcess element of the procedure would set that up, but as you
> pointed out, the procedure would never exit while the task is
> running.  So how to define the task and put it on "hold" until the
> processing tasks can utilize it?  I don't expect it's possible without
> incurring as much ugliness as the program has already, but I thought
> I'd give it a try.  Thanks again =)

Tasking /is/ fun. I seem to be unusual in that I don't find it 
difficult. I'd like to be able to show off and solve all your problems, 
but I still have no idea what you're trying to achieve. I've got a lot 
of information about how you're trying to implement it, but no idea what 
"it" is. If you can describe the problem, there's probably an elegant 
way to implement a solution in Ada.

-- 
Jeff Carter
"All citizens will be required to change their underwear
every half hour. Underwear will be worn on the outside,
so we can check."
Bananas
29

Re: Fun with Tasking

[mhamel_98]

On Mar 29, 12:59 am, "Jeffrey R. Carter" <jrcar...@acm.org> wrote:
> mhamel...@yahoo.com wrote:
>
> > Thank you Adam.  While making this sample program a sneaking suspicion
> > started to creep up that I really didn't know what I was doing, and
> > you helped verify that suspicion.  What's really going on is I have a
> > number of tasks I'm going to run, and while themselves completely
> > independent of one another, they send updates to a single listening,
> > or collating, task.  That is the task at the heart of my initial
> > post.  My approach was to start up this collating task and give it's
> > pointer (or address) to the processing tasks.  This works fine if the
> > collating task definition is in the acc_add_test procedure and the
> > Control_Block has a pointer to that task within it.  However, I would
> > like the collating task definition to reside with the "exec" procedure/
> > package as the sample has it.  How would one declare and start up that
> > task, without any visibility into the package, just a procedure
> > pointer with a mode argument?  My (terribly flawed) idea was the
> > PreProcess element of the procedure would set that up, but as you
> > pointed out, the procedure would never exit while the task is
> > running.  So how to define the task and put it on "hold" until the
> > processing tasks can utilize it?  I don't expect it's possible without
> > incurring as much ugliness as the program has already, but I thought
> > I'd give it a try.  Thanks again =)
>
> Tasking /is/ fun. I seem to be unusual in that I don't find it
> difficult. I'd like to be able to show off and solve all your problems,
> but I still have no idea what you're trying to achieve. I've got a lot
> of information about how you're trying to implement it, but no idea what
> "it" is. If you can describe the problem, there's probably an elegant
> way to implement a solution in Ada.
>
> --
> Jeff Carter
> "All citizens will be required to change their underwear
> every half hour. Underwear will be worn on the outside,
> so we can check."
> Bananas
> 29- Hide quoted text -
>
> - Show quoted text -

Great help and advice all around, thanks much c.l.a!

Anyhoo Jeff, I'll try to explain in more detail what my program is
doing and hopefully you can see where I'm going wrong.  The program
has a number of packages, each handling what I call a subsystem, to be
verbose: ephemeris, residual, vector, covariance and a few others
(this sort of equates to the "exec" procedure in my sample).  Each of
these systems has a number of procedures of interest, like a
differencing of quantities, a statistical analysis, etc.  Now each of
these procedures of each of these subsystems can have up to 60 (and
more as we go on) objects - each one I handle as it's own task.  I've
found that throwing 60 threads at the OS (W2K in this case) is, while
better then serially and single-threaded on multi-cpu's, not terribly
efficient.  For this application, I've found most procedures operate
best if N+1 threads are running, N being the processors available.
The output is all .csv files.  For awhile, to show an ensemble of
data, I would do a lot of cut and pasting between .csv's, one day, I
thought, why not have the program do this?  So I implemented these
"collating" tasks that all 60 or so threads would send some
information to for some sort of unified processing.  Each procedure of
each subsystem/package is slightly different either in the data types
being handled or the manner in which the data is processed so a
"generic" collating task isn't really feasible.

That's what's happening, now how do I handle it?  As the program
starts, I have each subsystem loads its procedures (via procedure
pointers) to a task manager.  A command is recieved via CLI and the
task manager then inserts, let's say, 60 copies of a specified
procedure into a list which are doled out to worker threads so that
only N+1 are active at any time.  The issue I'm having is, how do you
have a unique collating task created for those 60 processes to send
updates to?  As it is now, I have all of these (nearly a dozen) unique
collating tasks defined in the task manager.  I would rather have them
defined in the subsystem package to which they really belong.  As the
subsystem packages "push" their procedures to the task manager, the
task manager has no visibility into the subsystem.  My aim was to
define an area (a System.Address for example) in a control block
record that the task manager would have space for "a task".  What this
task was, what it did, where it came from the task manager would not
know nor care, but it would be the owner, or master, in some sense so
the "preprocess" or initialization component could exit.  I hope this
makes a little sense.

Re: Fun with Tasking

[Steve]

<mhamel_98@yahoo.com> wrote in message 
news:1175181454.290582.90280@b75g2000hsg.googlegroups.com...
[snip]
>
> Great help and advice all around, thanks much c.l.a!
>
> Anyhoo Jeff, I'll try to explain in more detail what my program is
> doing and hopefully you can see where I'm going wrong.  The program
> has a number of packages, each handling what I call a subsystem, to be
> verbose: ephemeris, residual, vector, covariance and a few others
> (this sort of equates to the "exec" procedure in my sample).  Each of
> these systems has a number of procedures of interest, like a
> differencing of quantities, a statistical analysis, etc.  Now each of
> these procedures of each of these subsystems can have up to 60 (and
> more as we go on) objects - each one I handle as it's own task.  I've
> found that throwing 60 threads at the OS (W2K in this case) is, while
> better then serially and single-threaded on multi-cpu's, not terribly
> efficient.  For this application, I've found most procedures operate
> best if N+1 threads are running, N being the processors available.
> The output is all .csv files.  For awhile, to show an ensemble of
> data, I would do a lot of cut and pasting between .csv's, one day, I
> thought, why not have the program do this?  So I implemented these
> "collating" tasks that all 60 or so threads would send some
> information to for some sort of unified processing.  Each procedure of
> each subsystem/package is slightly different either in the data types
> being handled or the manner in which the data is processed so a
> "generic" collating task isn't really feasible.
>

You are still rather heavily interlacing your implementation with what you 
are looking for, but I think I have at least a partial understanding.

Here are some suggestions:
  Don't use function pointers.
  Don't use System.Address
  In the context of your problem description, these are tools that a C 
programmer uses when writing a C program in Ada.

  Create an abstract base class (tagged type), and subclass for the specific 
functionality of the subsystems.  Note: Some Ada programmers may get their 
knickers in a bind by my use of terms like "class" and "method" since these 
are not Ada keywords and they are keywords in other programming languages, I 
am not describing syntax, but structure.

  For example a package defining a base class might be something like:
------------------------------
package Data_Processing is

  type Data_Processing_Type is abstract tagged null record;

  type Data_Processing_Acc is access all Data_Processing_Type'class;

  procedure Process_Data( Data_Set : Data_Processing_Type ) is abstract;

end Data_Processing;
--------------------------------
  Derive a specfic processing type for one of the types of processing that 
needs to be done, hiding any data to be processed in the private part of the 
package spec:

with Data_Processing;
 use Data_Processing;

package Ephemeris_Processing is

  type Ephemeris_Processing_Type is new Data_Processing_Type with private;

  procedure Process_Data( Data_Set : Ephemeris_Processing_Type );

  function Create_Ephemeris_Processor( data_value : Natural ) return 
Data_Processing_Acc;

private
  type Ephemeris_Processing_Type is new Data_Processing_Type with
    record
      data_value : Natural;
    end record;

  type Ephemeris_Processing_Acc is access all Ephemeris_Processing_Type;

end Ephemeris_Processing;
--------------------------------
with Ada.Text_IO;
with Ada.Integer_Text_IO;

package body Ephemeris_Processing is

  procedure Process_Data( Data_Set : Ephemeris_Processing_Type ) is
  begin
    Ada.Text_IO.Put( "Value is:" );
    Ada.Integer_Text_IO.Put( Data_Set.data_value );
    Ada.Text_IO.New_Line;
  end Process_Data;

  function Create_Ephemeris_Processor( data_value : Natural ) return 
Data_Processing_Acc is
    ret : Ephemeris_Processing_Acc;
  begin
    ret := new Ephemeris_Processing_Type;
    ret.data_value := data_value;
    return Data_Processing_Acc(ret);
  end Create_Ephemeris_Processor;

end Ephemeris_Processing;
--------------------------------
And then use dynamic dispatching to invoke the processing of the data for 
the specific processing to be done.

with Data_Processing;
 use Data_Processing;
with Ephemeris_Processing;
 use Ephemeris_Processing;

procedure Dispatch_Demo is

  work : Data_Processing_Acc;

begin
  work := Create_Ephemeris_Processor( 42 );
  Process_Data( work.ALL );
end Dispatch_Demo;
--------------------------------

You can compile and test the snippets of code above to see the dispatching 
work.
This doesn't really describe how to handle the tasking, but a better way to 
break down the problem.

Derive different classes from Data_Processing_Type for each of the different 
types of processing you need.  Information about each of the derived classes 
is encapsulated in a separate package for each type of processing.

You can then create list of processing to be done and send elements of the 
list to a task.  The task receives an access to a data processing type and 
calls the Process_Data function.

One possible implementation is to put the list of operations to be done in a 
protected object.  Create as many threads as you want and have each thread 
get a data processing object from the list, process the data, and then go 
back for more work to do.  This approach scales well.

I hope this helps,
Steve
(The Duck) 

Re: Fun with Tasking

[Jeffrey R. Carter]

mhamel_98@yahoo.com wrote:
> 
> Anyhoo Jeff, I'll try to explain in more detail what my program is
> doing and hopefully you can see where I'm going wrong.  The program
> has a number of packages, each handling what I call a subsystem, to be
> verbose: ephemeris, residual, vector, covariance and a few others
> (this sort of equates to the "exec" procedure in my sample).  Each of
> these systems has a number of procedures of interest, like a
> differencing of quantities, a statistical analysis, etc.  Now each of
> these procedures of each of these subsystems can have up to 60 (and
> more as we go on) objects - each one I handle as it's own task.  I've
> found that throwing 60 threads at the OS (W2K in this case) is, while
> better then serially and single-threaded on multi-cpu's, not terribly
> efficient.  For this application, I've found most procedures operate
> best if N+1 threads are running, N being the processors available.
> The output is all .csv files.  For awhile, to show an ensemble of
> data, I would do a lot of cut and pasting between .csv's, one day, I
> thought, why not have the program do this?  So I implemented these
> "collating" tasks that all 60 or so threads would send some
> information to for some sort of unified processing.  Each procedure of
> each subsystem/package is slightly different either in the data types
> being handled or the manner in which the data is processed so a
> "generic" collating task isn't really feasible.
> 
> That's what's happening, now how do I handle it?  As the program
> starts, I have each subsystem loads its procedures (via procedure
> pointers) to a task manager.  A command is recieved via CLI and the
> task manager then inserts, let's say, 60 copies of a specified
> procedure into a list which are doled out to worker threads so that
> only N+1 are active at any time.  The issue I'm having is, how do you
> have a unique collating task created for those 60 processes to send
> updates to?  As it is now, I have all of these (nearly a dozen) unique
> collating tasks defined in the task manager.  I would rather have them
> defined in the subsystem package to which they really belong.  As the
> subsystem packages "push" their procedures to the task manager, the
> task manager has no visibility into the subsystem.  My aim was to
> define an area (a System.Address for example) in a control block
> record that the task manager would have space for "a task".  What this
> task was, what it did, where it came from the task manager would not
> know nor care, but it would be the owner, or master, in some sense so
> the "preprocess" or initialization component could exit.  I hope this
> makes a little sense.

There's still a lot of implementation in here. Talking about procedure 
pointers and addresses is not only implementation, it's quite low level. 
One of Ada's great strengths is its ability to express, implement, and 
use high-level abstractions. The low-level stuff is still available, but 
generally best hidden well down in the implementation of such an 
abstraction.

What I get from your description is: You have classes of computations 
("subsystems"); each class can have numerous computations carried on at 
the same time; each computation produces output; you want some of the 
output from all the computations of the same class to be subjected to 
collating.

Everything else seems like implementation detail that may be blocking 
your view of a solution.

Steve has given you some ideas about how to abstract the idea of 
computations and controlling their scheduling. He didn't have get into 
the resulting output or how to subject some of it, within a class of 
computations, to collating.

I would probably take a different approach than Steve, with different 
subsystems more independent, in order to ease the collating within the 
subsystems, but that may be more a matter of style than substance.

If you can produce high-level package specifications that only address 
the high-level concepts of your problem, without worrying about 
implementation (private parts, bodies), with a road map of how they work 
together, you'll probably find that you can then work out a way to 
implement them.

-- 
Jeff Carter
"Don't knock masturbation. It's sex with someone I love."
Annie Hall
45

Re: Fun with Tasking

[Stephen Leake]

mhamel_98@yahoo.com writes:

> Anyhoo Jeff, I'll try to explain in more detail what my program is
> doing and hopefully you can see where I'm going wrong.  The program
> has a number of packages, each handling what I call a subsystem, to be
> verbose: ephemeris, residual, vector, covariance and a few others
> (this sort of equates to the "exec" procedure in my sample).  Each of
> these systems has a number of procedures of interest, like a
> differencing of quantities, a statistical analysis, etc.  Now each of
> these procedures of each of these subsystems can have up to 60 (and
> more as we go on) objects - each one I handle as it's own task.  

There's no reason for each of these things to be a _task_.

They should each be in separate _packages_, and be separate _objects_,
but not _tasks_.

Tasks are only needed for controlling timing, and then you need more
than one task only when there are truly asynchronous events. Or if you
want to gain speed by using parallel processors.

In the system you describe, there are bunch of calculations, that can
be done correctly if they are done serially.

Since you talk about multiple processors, you should have one task per
processor, and portion the computations among the tasks. 

-- 
-- Stephe