Re: Fun with Tasking

["Steve" <nospam_steved94@comcast.net>]

<mhamel_98@yahoo.com> wrote in message 
news:1175181454.290582.90280@b75g2000hsg.googlegroups.com...
[snip]
>
> Great help and advice all around, thanks much c.l.a!
>
> Anyhoo Jeff, I'll try to explain in more detail what my program is
> doing and hopefully you can see where I'm going wrong.  The program
> has a number of packages, each handling what I call a subsystem, to be
> verbose: ephemeris, residual, vector, covariance and a few others
> (this sort of equates to the "exec" procedure in my sample).  Each of
> these systems has a number of procedures of interest, like a
> differencing of quantities, a statistical analysis, etc.  Now each of
> these procedures of each of these subsystems can have up to 60 (and
> more as we go on) objects - each one I handle as it's own task.  I've
> found that throwing 60 threads at the OS (W2K in this case) is, while
> better then serially and single-threaded on multi-cpu's, not terribly
> efficient.  For this application, I've found most procedures operate
> best if N+1 threads are running, N being the processors available.
> The output is all .csv files.  For awhile, to show an ensemble of
> data, I would do a lot of cut and pasting between .csv's, one day, I
> thought, why not have the program do this?  So I implemented these
> "collating" tasks that all 60 or so threads would send some
> information to for some sort of unified processing.  Each procedure of
> each subsystem/package is slightly different either in the data types
> being handled or the manner in which the data is processed so a
> "generic" collating task isn't really feasible.
>

You are still rather heavily interlacing your implementation with what you 
are looking for, but I think I have at least a partial understanding.

Here are some suggestions:
  Don't use function pointers.
  Don't use System.Address
  In the context of your problem description, these are tools that a C 
programmer uses when writing a C program in Ada.

  Create an abstract base class (tagged type), and subclass for the specific 
functionality of the subsystems.  Note: Some Ada programmers may get their 
knickers in a bind by my use of terms like "class" and "method" since these 
are not Ada keywords and they are keywords in other programming languages, I 
am not describing syntax, but structure.

  For example a package defining a base class might be something like:
------------------------------
package Data_Processing is

  type Data_Processing_Type is abstract tagged null record;

  type Data_Processing_Acc is access all Data_Processing_Type'class;

  procedure Process_Data( Data_Set : Data_Processing_Type ) is abstract;

end Data_Processing;
--------------------------------
  Derive a specfic processing type for one of the types of processing that 
needs to be done, hiding any data to be processed in the private part of the 
package spec:

with Data_Processing;
 use Data_Processing;

package Ephemeris_Processing is

  type Ephemeris_Processing_Type is new Data_Processing_Type with private;

  procedure Process_Data( Data_Set : Ephemeris_Processing_Type );

  function Create_Ephemeris_Processor( data_value : Natural ) return 
Data_Processing_Acc;

private
  type Ephemeris_Processing_Type is new Data_Processing_Type with
    record
      data_value : Natural;
    end record;

  type Ephemeris_Processing_Acc is access all Ephemeris_Processing_Type;

end Ephemeris_Processing;
--------------------------------
with Ada.Text_IO;
with Ada.Integer_Text_IO;

package body Ephemeris_Processing is

  procedure Process_Data( Data_Set : Ephemeris_Processing_Type ) is
  begin
    Ada.Text_IO.Put( "Value is:" );
    Ada.Integer_Text_IO.Put( Data_Set.data_value );
    Ada.Text_IO.New_Line;
  end Process_Data;

  function Create_Ephemeris_Processor( data_value : Natural ) return 
Data_Processing_Acc is
    ret : Ephemeris_Processing_Acc;
  begin
    ret := new Ephemeris_Processing_Type;
    ret.data_value := data_value;
    return Data_Processing_Acc(ret);
  end Create_Ephemeris_Processor;

end Ephemeris_Processing;
--------------------------------
And then use dynamic dispatching to invoke the processing of the data for 
the specific processing to be done.

with Data_Processing;
 use Data_Processing;
with Ephemeris_Processing;
 use Ephemeris_Processing;

procedure Dispatch_Demo is

  work : Data_Processing_Acc;

begin
  work := Create_Ephemeris_Processor( 42 );
  Process_Data( work.ALL );
end Dispatch_Demo;
--------------------------------

You can compile and test the snippets of code above to see the dispatching 
work.
This doesn't really describe how to handle the tasking, but a better way to 
break down the problem.

Derive different classes from Data_Processing_Type for each of the different 
types of processing you need.  Information about each of the derived classes 
is encapsulated in a separate package for each type of processing.

You can then create list of processing to be done and send elements of the 
list to a task.  The task receives an access to a data processing type and 
calls the Process_Data function.

One possible implementation is to put the list of operations to be done in a 
protected object.  Create as many threads as you want and have each thread 
get a data processing object from the list, process the data, and then go 
back for more work to do.  This approach scales well.

I hope this helps,
Steve
(The Duck)