* Fun with Tasking @ 2007-03-28 15:53 mhamel_98 2007-03-28 16:55 ` Adam Beneschan 0 siblings, 1 reply; 10+ messages in thread From: mhamel_98 @ 2007-03-28 15:53 UTC (permalink / raw) Below is sample code I made representing something I wanted to do with a program, of course it doesn't work quite the way I would like. Hopefully this sample doesn't cause eye (or brain) injury, it is more symbolic than useful - the procedure "Exec" is, of course, really a package (rather, several similar packages). The problem is, how to "transport" a task? As seen in the sample below, I would like to use just a single address in the Control_Block type, but it's never that simple, is it? In actuality, how I get this program in its full incarnation to work is to define the "test_task"s in the acc_add_test part of the program. I really don't like doing this, as I mentioned, "exec" is representative of a family of several different packages. I would like the task definition to be local to the "exec" pacakage(s), not all kludged together in acc_add_test, which then requires all sorts of (ideally private) types being moved out of the local "exec" packages into acc_add_test and then the Control_Block record now has nearly a dozen different task pointers stuffed into it, only one of which is going to be used at any given time. One last caveat, the Acc_Add_Test program should have *no* visibility into the "exec" procedure/package. The "exec" package(s), in fact, "push" their procedures into the main program. So is there something simple I'm having a brain fart over in missing here, or is a complete re-architecture required here? with Text_Io; with System; with System.Address_To_Access_Conversions; procedure Acc_Add_test is Exec_Count : constant Natural := 10; type Control_Block is record Task_Addr : System.Address; end record; type Mode_Type is (preprocess, process); procedure Exec (Mode : in Mode_Type; Cntl : in out Control_Block) is task type Test_Task is entry Start (Addr : System.Address); entry Complete; end Test_Task; package Convert is new System.Address_To_Access_Conversions (Test_Task); use Convert; task body Test_Task is Task_Ptr : Convert.Object_Pointer := null; Count : Natural := 0; begin loop begin select accept Start (Addr : System.Address) do Task_Ptr := Convert.To_Pointer (Addr); end Start; or accept Complete do Count := Count + 1; end Complete; if Count = Exec_Count then Text_Io.Put_Line ("Completed All Tasks"); abort Task_Ptr.all; end if; end select; end; end loop; end Test_Task; begin case Mode is when Preprocess => declare Task_Ptr : Convert.Object_Pointer; Address : System.Address; begin Task_Ptr := new Test_Task; Address := Convert.To_Address (Task_Ptr); Task_Ptr.Start (Address); Cntl.Task_Addr := Address; end; when Process => declare Task_Ptr : Convert.Object_Pointer; begin Task_Ptr := Convert.To_Pointer (Cntl.Task_Addr); Task_Ptr.Complete; end; end case; end Exec; Controls : Control_Block; begin Exec (Preprocess, Controls); for I in 1 .. Exec_Count loop Exec (Process, Controls); end loop; end Acc_Add_Test; Thanks for any input! ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Fun with Tasking 2007-03-28 15:53 Fun with Tasking mhamel_98 @ 2007-03-28 16:55 ` Adam Beneschan 2007-03-28 17:53 ` mhamel_98 0 siblings, 1 reply; 10+ messages in thread From: Adam Beneschan @ 2007-03-28 16:55 UTC (permalink / raw) On Mar 28, 8:53 am, mhamel...@yahoo.com wrote: > Below is sample code I made representing something I wanted to do with > a program, of course it doesn't work quite the way I would like. > Hopefully this sample doesn't cause eye (or brain) injury, it is more > symbolic than useful - the procedure "Exec" is, of course, really a > package (rather, several similar packages). > > The problem is, how to "transport" a task? As seen in the sample > below, I would like to use just a single address in the Control_Block > type, but it's never that simple, is it? In actuality, how I get this > program in its full incarnation to work is to define the "test_task"s > in the acc_add_test part of the program. I really don't like doing > this, as I mentioned, "exec" is representative of a family of several > different packages. I would like the task definition to be local to > the "exec" pacakage(s), not all kludged together in acc_add_test, > which then requires all sorts of (ideally private) types being moved > out of the local "exec" packages into acc_add_test and then the > Control_Block record now has nearly a dozen different task pointers > stuffed into it, only one of which is going to be used at any given > time. > > One last caveat, the Acc_Add_Test program should have *no* visibility > into the "exec" procedure/package. The "exec" package(s), in fact, > "push" their procedures into the main program. So is there something > simple I'm having a brain fart over in missing here, or is a complete > re-architecture required here? > > with Text_Io; > with System; > with System.Address_To_Access_Conversions; > > procedure Acc_Add_test is > > Exec_Count : constant Natural := 10; > > type Control_Block is > record > Task_Addr : System.Address; > end record; > > type Mode_Type is (preprocess, process); > > procedure Exec (Mode : in Mode_Type; > Cntl : in out Control_Block) is > > task type Test_Task is > entry Start (Addr : System.Address); > entry Complete; > end Test_Task; > > package Convert is new System.Address_To_Access_Conversions > (Test_Task); > use Convert; > > task body Test_Task is > Task_Ptr : Convert.Object_Pointer := null; > Count : Natural := 0; > begin > loop > begin > select > accept Start (Addr : System.Address) do > Task_Ptr := Convert.To_Pointer (Addr); > end Start; > > or > accept Complete do > Count := Count + 1; > end Complete; > if Count = Exec_Count then > Text_Io.Put_Line ("Completed All Tasks"); > abort Task_Ptr.all; > end if; > end select; > end; > end loop; > end Test_Task; > > begin > case Mode is > when Preprocess => > declare > Task_Ptr : Convert.Object_Pointer; > Address : System.Address; > begin > Task_Ptr := new Test_Task; > Address := Convert.To_Address (Task_Ptr); > Task_Ptr.Start (Address); > Cntl.Task_Addr := Address; > end; > > when Process => > declare > Task_Ptr : Convert.Object_Pointer; > begin > Task_Ptr := Convert.To_Pointer (Cntl.Task_Addr); > Task_Ptr.Complete; > end; > end case; > end Exec; > > Controls : Control_Block; > > begin > Exec (Preprocess, Controls); > for I in 1 .. Exec_Count loop > Exec (Process, Controls); > end loop; > end Acc_Add_Test; > > Thanks for any input! Your main problem is that your tasks aren't going to live long enough. When a task is created by an allocator, it depends on the "master" that declares the access type (or if the access type is a derived type, which isn't the case here, on the "ultimate ancestor" of the access type). See RM section 9.3. In this case, the master that declares the access type (Convert.Object_Pointer) is the Exec procedure. This means that any task that you create using this access type depends on Exec, and Exec will not finish until any such tasks are completed. It looks to me that you're trying to be able to call Exec multiple times, and have it start a task the first time and then use the same task the next time Exec is called. This won't work if the task type is declared inside Exec, because any such tasks will not be allowed to keep running if Exec isn't running. You'll need to move the task types (and the access type that points to the task type) outside Exec. This is the only thing that makes sense, anyway. If your task type is declared inside Exec, the task body will have access to Exec's parameters and local variables; how then could it be possible for the task body to stay running while Exec isn't? I'm not real clear on what you're trying to do. Your sample has "Exec" as a procedure, which makes it a master, but your discussion talks about "Exec" *packages*---and a package is not a master. If you wanted an Exec *package* that declares its own task type and its own procedure to do the processing, but the task type isn't inside the procedure, then I suspect that you may be able to accomplish what you're trying to do; and there should be a way to keep certain things private without having to resort to hokeyness like System.Address or Address_To_Access_Conversions. But I'm not up to trying to guess what you're trying to do; I'd recommend that you redo your example a bit to avoid the task dependence problem and then ask again if you still aren't sure how to do what you need to do. -- Adam ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Fun with Tasking 2007-03-28 16:55 ` Adam Beneschan @ 2007-03-28 17:53 ` mhamel_98 2007-03-28 20:06 ` Ed Falis ` (2 more replies) 0 siblings, 3 replies; 10+ messages in thread From: mhamel_98 @ 2007-03-28 17:53 UTC (permalink / raw) On Mar 28, 12:55 pm, "Adam Beneschan" <a...@irvine.com> wrote: > On Mar 28, 8:53 am, mhamel...@yahoo.com wrote: > > > > > > > Below is sample code I made representing something I wanted to do with > > a program, of course it doesn't work quite the way I would like. > > Hopefully this sample doesn't cause eye (or brain) injury, it is more > > symbolic than useful - the procedure "Exec" is, of course, really a > > package (rather, several similar packages). > > > The problem is, how to "transport" a task? As seen in the sample > > below, I would like to use just a single address in the Control_Block > > type, but it's never that simple, is it? In actuality, how I get this > > program in its full incarnation to work is to define the "test_task"s > > in the acc_add_test part of the program. I really don't like doing > > this, as I mentioned, "exec" is representative of a family of several > > different packages. I would like the task definition to be local to > > the "exec" pacakage(s), not all kludged together in acc_add_test, > > which then requires all sorts of (ideally private) types being moved > > out of the local "exec" packages into acc_add_test and then the > > Control_Block record now has nearly a dozen different task pointers > > stuffed into it, only one of which is going to be used at any given > > time. > > > One last caveat, the Acc_Add_Test program should have *no* visibility > > into the "exec" procedure/package. The "exec" package(s), in fact, > > "push" their procedures into the main program. So is there something > > simple I'm having a brain fart over in missing here, or is a complete > > re-architecture required here? > > > with Text_Io; > > with System; > > with System.Address_To_Access_Conversions; > > > procedure Acc_Add_test is > > > Exec_Count : constant Natural := 10; > > > type Control_Block is > > record > > Task_Addr : System.Address; > > end record; > > > type Mode_Type is (preprocess, process); > > > procedure Exec (Mode : in Mode_Type; > > Cntl : in out Control_Block) is > > > task type Test_Task is > > entry Start (Addr : System.Address); > > entry Complete; > > end Test_Task; > > > package Convert is new System.Address_To_Access_Conversions > > (Test_Task); > > use Convert; > > > task body Test_Task is > > Task_Ptr : Convert.Object_Pointer := null; > > Count : Natural := 0; > > begin > > loop > > begin > > select > > accept Start (Addr : System.Address) do > > Task_Ptr := Convert.To_Pointer (Addr); > > end Start; > > > or > > accept Complete do > > Count := Count + 1; > > end Complete; > > if Count = Exec_Count then > > Text_Io.Put_Line ("Completed All Tasks"); > > abort Task_Ptr.all; > > end if; > > end select; > > end; > > end loop; > > end Test_Task; > > > begin > > case Mode is > > when Preprocess => > > declare > > Task_Ptr : Convert.Object_Pointer; > > Address : System.Address; > > begin > > Task_Ptr := new Test_Task; > > Address := Convert.To_Address (Task_Ptr); > > Task_Ptr.Start (Address); > > Cntl.Task_Addr := Address; > > end; > > > when Process => > > declare > > Task_Ptr : Convert.Object_Pointer; > > begin > > Task_Ptr := Convert.To_Pointer (Cntl.Task_Addr); > > Task_Ptr.Complete; > > end; > > end case; > > end Exec; > > > Controls : Control_Block; > > > begin > > Exec (Preprocess, Controls); > > for I in 1 .. Exec_Count loop > > Exec (Process, Controls); > > end loop; > > end Acc_Add_Test; > > > Thanks for any input! > > Your main problem is that your tasks aren't going to live long > enough. When a task is created by an allocator, it depends on the > "master" that declares the access type (or if the access type is a > derived type, which isn't the case here, on the "ultimate ancestor" of > the access type). See RM section 9.3. In this case, the master that > declares the access type (Convert.Object_Pointer) is the Exec > procedure. This means that any task that you create using this access > type depends on Exec, and Exec will not finish until any such tasks > are completed. It looks to me that you're trying to be able to call > Exec multiple times, and have it start a task the first time and then > use the same task the next time Exec is called. This won't work if > the task type is declared inside Exec, because any such tasks will not > be allowed to keep running if Exec isn't running. You'll need to move > the task types (and the access type that points to the task type) > outside Exec. > > This is the only thing that makes sense, anyway. If your task type is > declared inside Exec, the task body will have access to Exec's > parameters and local variables; how then could it be possible for the > task body to stay running while Exec isn't? > > I'm not real clear on what you're trying to do. Your sample has > "Exec" as a procedure, which makes it a master, but your discussion > talks about "Exec" *packages*---and a package is not a master. If you > wanted an Exec *package* that declares its own task type and its own > procedure to do the processing, but the task type isn't inside the > procedure, then I suspect that you may be able to accomplish what > you're trying to do; and there should be a way to keep certain things > private without having to resort to hokeyness like System.Address or > Address_To_Access_Conversions. But I'm not up to trying to guess what > you're trying to do; I'd recommend that you redo your example a bit to > avoid the task dependence problem and then ask again if you still > aren't sure how to do what you need to do. > > -- Adam- Hide quoted text - > > - Show quoted text - Thank you Adam. While making this sample program a sneaking suspicion started to creep up that I really didn't know what I was doing, and you helped verify that suspicion. What's really going on is I have a number of tasks I'm going to run, and while themselves completely independent of one another, they send updates to a single listening, or collating, task. That is the task at the heart of my initial post. My approach was to start up this collating task and give it's pointer (or address) to the processing tasks. This works fine if the collating task definition is in the acc_add_test procedure and the Control_Block has a pointer to that task within it. However, I would like the collating task definition to reside with the "exec" procedure/ package as the sample has it. How would one declare and start up that task, without any visibility into the package, just a procedure pointer with a mode argument? My (terribly flawed) idea was the PreProcess element of the procedure would set that up, but as you pointed out, the procedure would never exit while the task is running. So how to define the task and put it on "hold" until the processing tasks can utilize it? I don't expect it's possible without incurring as much ugliness as the program has already, but I thought I'd give it a try. Thanks again =) ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Fun with Tasking 2007-03-28 17:53 ` mhamel_98 @ 2007-03-28 20:06 ` Ed Falis 2007-03-28 22:17 ` Randy Brukardt 2007-03-29 4:59 ` Jeffrey R. Carter 2 siblings, 0 replies; 10+ messages in thread From: Ed Falis @ 2007-03-28 20:06 UTC (permalink / raw) mhamel_98@yahoo.com wrote: > Thank you Adam. While making this sample program a sneaking suspicion > started to creep up that I really didn't know what I was doing, and > you helped verify that suspicion. You sound like a fine candidate for a copy of "Concurrency in Ada" by Burns and Welling. I did one of the first Ada tasking implementations, and still learned a lot from this book when I read it a couple of years ago. - Ed ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Fun with Tasking 2007-03-28 17:53 ` mhamel_98 2007-03-28 20:06 ` Ed Falis @ 2007-03-28 22:17 ` Randy Brukardt 2007-03-29 4:59 ` Jeffrey R. Carter 2 siblings, 0 replies; 10+ messages in thread From: Randy Brukardt @ 2007-03-28 22:17 UTC (permalink / raw) <mhamel_98@yahoo.com> wrote in message news:1175104396.231171.125360@e65g2000hsc.googlegroups.com... ... > Thank you Adam. While making this sample program a sneaking suspicion > started to creep up that I really didn't know what I was doing, and > you helped verify that suspicion. What's really going on is I have a > number of tasks I'm going to run, and while themselves completely > independent of one another, they send updates to a single listening, > or collating, task. That is the task at the heart of my initial > post. My approach was to start up this collating task and give it's > pointer (or address) to the processing tasks. This works fine if the > collating task definition is in the acc_add_test procedure and the > Control_Block has a pointer to that task within it. However, I would > like the collating task definition to reside with the "exec" procedure/ > package as the sample has it. How would one declare and start up that > task, without any visibility into the package, just a procedure > pointer with a mode argument? My (terribly flawed) idea was the > PreProcess element of the procedure would set that up, but as you > pointed out, the procedure would never exit while the task is > running. So how to define the task and put it on "hold" until the > processing tasks can utilize it? I don't expect it's possible without > incurring as much ugliness as the program has already, but I thought > I'd give it a try. Thanks again =) I'm not going to try to figure out precisely what it is that you are trying to do (a listening task doesn't require anything special to listen!), but a couple of thoughts about trying to manage unrelated tasks with another task. (1) You really don't want to try to manage unrelated anything in Ada (Ada is strongly typed, after all). If you are using a very new compiler, probably the best way to relate some tasks that would otherwise be unrelated is to define them all to be derived from the same task interface. The idea is that you would put the control operations into the task interface, and then the manager would use access-to-interface'class to keep track of the tasks and call operations on them as needed. But you'll need a complete and non-buggy implementation of the entire Ada language, as amended; so far as I know only GNAT Pro even claims to support this. So you'd be better off trying to do this next year... Note that you have to start each task individually. The usual trick is to use a constructor function for that purpose. (2) Otherwise, I'd say that it can't be done reasonably. You could use Task-Ids to track the status of the tasks, but that won't help starting them. (3) Probably the best plan is to forget about having the management task interoperate directly with the other tasks. Rather, define an abstract tagged type (or interface, if you prefer) Processor, and have each package define any instance of that tagged type. It would have the needed management operations ("Start", "Status", "Stop", etc.), and again you would create an appropriate instance of each one (probably with the main program), and have the management task save a list of these as access-to-Processor'class, and manage them appropriately. The advantage of this technique is that you can wrap anything you need in these operations: if you need a conditional entry call, for instance, you would just implement one of those operations that way. (That's precisely how task interfaces get implemented anyway; you're just doing some of the work that the compiler would done.) Hope this helps. Randy. ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Fun with Tasking 2007-03-28 17:53 ` mhamel_98 2007-03-28 20:06 ` Ed Falis 2007-03-28 22:17 ` Randy Brukardt @ 2007-03-29 4:59 ` Jeffrey R. Carter 2007-03-29 15:17 ` mhamel_98 2 siblings, 1 reply; 10+ messages in thread From: Jeffrey R. Carter @ 2007-03-29 4:59 UTC (permalink / raw) mhamel_98@yahoo.com wrote: > > Thank you Adam. While making this sample program a sneaking suspicion > started to creep up that I really didn't know what I was doing, and > you helped verify that suspicion. What's really going on is I have a > number of tasks I'm going to run, and while themselves completely > independent of one another, they send updates to a single listening, > or collating, task. That is the task at the heart of my initial > post. My approach was to start up this collating task and give it's > pointer (or address) to the processing tasks. This works fine if the > collating task definition is in the acc_add_test procedure and the > Control_Block has a pointer to that task within it. However, I would > like the collating task definition to reside with the "exec" procedure/ > package as the sample has it. How would one declare and start up that > task, without any visibility into the package, just a procedure > pointer with a mode argument? My (terribly flawed) idea was the > PreProcess element of the procedure would set that up, but as you > pointed out, the procedure would never exit while the task is > running. So how to define the task and put it on "hold" until the > processing tasks can utilize it? I don't expect it's possible without > incurring as much ugliness as the program has already, but I thought > I'd give it a try. Thanks again =) Tasking /is/ fun. I seem to be unusual in that I don't find it difficult. I'd like to be able to show off and solve all your problems, but I still have no idea what you're trying to achieve. I've got a lot of information about how you're trying to implement it, but no idea what "it" is. If you can describe the problem, there's probably an elegant way to implement a solution in Ada. -- Jeff Carter "All citizens will be required to change their underwear every half hour. Underwear will be worn on the outside, so we can check." Bananas 29 ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Fun with Tasking 2007-03-29 4:59 ` Jeffrey R. Carter @ 2007-03-29 15:17 ` mhamel_98 2007-03-30 4:06 ` Steve ` (2 more replies) 0 siblings, 3 replies; 10+ messages in thread From: mhamel_98 @ 2007-03-29 15:17 UTC (permalink / raw) On Mar 29, 12:59 am, "Jeffrey R. Carter" <jrcar...@acm.org> wrote: > mhamel...@yahoo.com wrote: > > > Thank you Adam. While making this sample program a sneaking suspicion > > started to creep up that I really didn't know what I was doing, and > > you helped verify that suspicion. What's really going on is I have a > > number of tasks I'm going to run, and while themselves completely > > independent of one another, they send updates to a single listening, > > or collating, task. That is the task at the heart of my initial > > post. My approach was to start up this collating task and give it's > > pointer (or address) to the processing tasks. This works fine if the > > collating task definition is in the acc_add_test procedure and the > > Control_Block has a pointer to that task within it. However, I would > > like the collating task definition to reside with the "exec" procedure/ > > package as the sample has it. How would one declare and start up that > > task, without any visibility into the package, just a procedure > > pointer with a mode argument? My (terribly flawed) idea was the > > PreProcess element of the procedure would set that up, but as you > > pointed out, the procedure would never exit while the task is > > running. So how to define the task and put it on "hold" until the > > processing tasks can utilize it? I don't expect it's possible without > > incurring as much ugliness as the program has already, but I thought > > I'd give it a try. Thanks again =) > > Tasking /is/ fun. I seem to be unusual in that I don't find it > difficult. I'd like to be able to show off and solve all your problems, > but I still have no idea what you're trying to achieve. I've got a lot > of information about how you're trying to implement it, but no idea what > "it" is. If you can describe the problem, there's probably an elegant > way to implement a solution in Ada. > > -- > Jeff Carter > "All citizens will be required to change their underwear > every half hour. Underwear will be worn on the outside, > so we can check." > Bananas > 29- Hide quoted text - > > - Show quoted text - Great help and advice all around, thanks much c.l.a! Anyhoo Jeff, I'll try to explain in more detail what my program is doing and hopefully you can see where I'm going wrong. The program has a number of packages, each handling what I call a subsystem, to be verbose: ephemeris, residual, vector, covariance and a few others (this sort of equates to the "exec" procedure in my sample). Each of these systems has a number of procedures of interest, like a differencing of quantities, a statistical analysis, etc. Now each of these procedures of each of these subsystems can have up to 60 (and more as we go on) objects - each one I handle as it's own task. I've found that throwing 60 threads at the OS (W2K in this case) is, while better then serially and single-threaded on multi-cpu's, not terribly efficient. For this application, I've found most procedures operate best if N+1 threads are running, N being the processors available. The output is all .csv files. For awhile, to show an ensemble of data, I would do a lot of cut and pasting between .csv's, one day, I thought, why not have the program do this? So I implemented these "collating" tasks that all 60 or so threads would send some information to for some sort of unified processing. Each procedure of each subsystem/package is slightly different either in the data types being handled or the manner in which the data is processed so a "generic" collating task isn't really feasible. That's what's happening, now how do I handle it? As the program starts, I have each subsystem loads its procedures (via procedure pointers) to a task manager. A command is recieved via CLI and the task manager then inserts, let's say, 60 copies of a specified procedure into a list which are doled out to worker threads so that only N+1 are active at any time. The issue I'm having is, how do you have a unique collating task created for those 60 processes to send updates to? As it is now, I have all of these (nearly a dozen) unique collating tasks defined in the task manager. I would rather have them defined in the subsystem package to which they really belong. As the subsystem packages "push" their procedures to the task manager, the task manager has no visibility into the subsystem. My aim was to define an area (a System.Address for example) in a control block record that the task manager would have space for "a task". What this task was, what it did, where it came from the task manager would not know nor care, but it would be the owner, or master, in some sense so the "preprocess" or initialization component could exit. I hope this makes a little sense. ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Fun with Tasking 2007-03-29 15:17 ` mhamel_98 @ 2007-03-30 4:06 ` Steve 2007-03-30 5:46 ` Jeffrey R. Carter 2007-03-31 12:08 ` Stephen Leake 2 siblings, 0 replies; 10+ messages in thread From: Steve @ 2007-03-30 4:06 UTC (permalink / raw) <mhamel_98@yahoo.com> wrote in message news:1175181454.290582.90280@b75g2000hsg.googlegroups.com... [snip] > > Great help and advice all around, thanks much c.l.a! > > Anyhoo Jeff, I'll try to explain in more detail what my program is > doing and hopefully you can see where I'm going wrong. The program > has a number of packages, each handling what I call a subsystem, to be > verbose: ephemeris, residual, vector, covariance and a few others > (this sort of equates to the "exec" procedure in my sample). Each of > these systems has a number of procedures of interest, like a > differencing of quantities, a statistical analysis, etc. Now each of > these procedures of each of these subsystems can have up to 60 (and > more as we go on) objects - each one I handle as it's own task. I've > found that throwing 60 threads at the OS (W2K in this case) is, while > better then serially and single-threaded on multi-cpu's, not terribly > efficient. For this application, I've found most procedures operate > best if N+1 threads are running, N being the processors available. > The output is all .csv files. For awhile, to show an ensemble of > data, I would do a lot of cut and pasting between .csv's, one day, I > thought, why not have the program do this? So I implemented these > "collating" tasks that all 60 or so threads would send some > information to for some sort of unified processing. Each procedure of > each subsystem/package is slightly different either in the data types > being handled or the manner in which the data is processed so a > "generic" collating task isn't really feasible. > You are still rather heavily interlacing your implementation with what you are looking for, but I think I have at least a partial understanding. Here are some suggestions: Don't use function pointers. Don't use System.Address In the context of your problem description, these are tools that a C programmer uses when writing a C program in Ada. Create an abstract base class (tagged type), and subclass for the specific functionality of the subsystems. Note: Some Ada programmers may get their knickers in a bind by my use of terms like "class" and "method" since these are not Ada keywords and they are keywords in other programming languages, I am not describing syntax, but structure. For example a package defining a base class might be something like: ------------------------------ package Data_Processing is type Data_Processing_Type is abstract tagged null record; type Data_Processing_Acc is access all Data_Processing_Type'class; procedure Process_Data( Data_Set : Data_Processing_Type ) is abstract; end Data_Processing; -------------------------------- Derive a specfic processing type for one of the types of processing that needs to be done, hiding any data to be processed in the private part of the package spec: with Data_Processing; use Data_Processing; package Ephemeris_Processing is type Ephemeris_Processing_Type is new Data_Processing_Type with private; procedure Process_Data( Data_Set : Ephemeris_Processing_Type ); function Create_Ephemeris_Processor( data_value : Natural ) return Data_Processing_Acc; private type Ephemeris_Processing_Type is new Data_Processing_Type with record data_value : Natural; end record; type Ephemeris_Processing_Acc is access all Ephemeris_Processing_Type; end Ephemeris_Processing; -------------------------------- with Ada.Text_IO; with Ada.Integer_Text_IO; package body Ephemeris_Processing is procedure Process_Data( Data_Set : Ephemeris_Processing_Type ) is begin Ada.Text_IO.Put( "Value is:" ); Ada.Integer_Text_IO.Put( Data_Set.data_value ); Ada.Text_IO.New_Line; end Process_Data; function Create_Ephemeris_Processor( data_value : Natural ) return Data_Processing_Acc is ret : Ephemeris_Processing_Acc; begin ret := new Ephemeris_Processing_Type; ret.data_value := data_value; return Data_Processing_Acc(ret); end Create_Ephemeris_Processor; end Ephemeris_Processing; -------------------------------- And then use dynamic dispatching to invoke the processing of the data for the specific processing to be done. with Data_Processing; use Data_Processing; with Ephemeris_Processing; use Ephemeris_Processing; procedure Dispatch_Demo is work : Data_Processing_Acc; begin work := Create_Ephemeris_Processor( 42 ); Process_Data( work.ALL ); end Dispatch_Demo; -------------------------------- You can compile and test the snippets of code above to see the dispatching work. This doesn't really describe how to handle the tasking, but a better way to break down the problem. Derive different classes from Data_Processing_Type for each of the different types of processing you need. Information about each of the derived classes is encapsulated in a separate package for each type of processing. You can then create list of processing to be done and send elements of the list to a task. The task receives an access to a data processing type and calls the Process_Data function. One possible implementation is to put the list of operations to be done in a protected object. Create as many threads as you want and have each thread get a data processing object from the list, process the data, and then go back for more work to do. This approach scales well. I hope this helps, Steve (The Duck) ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Fun with Tasking 2007-03-29 15:17 ` mhamel_98 2007-03-30 4:06 ` Steve @ 2007-03-30 5:46 ` Jeffrey R. Carter 2007-03-31 12:08 ` Stephen Leake 2 siblings, 0 replies; 10+ messages in thread From: Jeffrey R. Carter @ 2007-03-30 5:46 UTC (permalink / raw) mhamel_98@yahoo.com wrote: > > Anyhoo Jeff, I'll try to explain in more detail what my program is > doing and hopefully you can see where I'm going wrong. The program > has a number of packages, each handling what I call a subsystem, to be > verbose: ephemeris, residual, vector, covariance and a few others > (this sort of equates to the "exec" procedure in my sample). Each of > these systems has a number of procedures of interest, like a > differencing of quantities, a statistical analysis, etc. Now each of > these procedures of each of these subsystems can have up to 60 (and > more as we go on) objects - each one I handle as it's own task. I've > found that throwing 60 threads at the OS (W2K in this case) is, while > better then serially and single-threaded on multi-cpu's, not terribly > efficient. For this application, I've found most procedures operate > best if N+1 threads are running, N being the processors available. > The output is all .csv files. For awhile, to show an ensemble of > data, I would do a lot of cut and pasting between .csv's, one day, I > thought, why not have the program do this? So I implemented these > "collating" tasks that all 60 or so threads would send some > information to for some sort of unified processing. Each procedure of > each subsystem/package is slightly different either in the data types > being handled or the manner in which the data is processed so a > "generic" collating task isn't really feasible. > > That's what's happening, now how do I handle it? As the program > starts, I have each subsystem loads its procedures (via procedure > pointers) to a task manager. A command is recieved via CLI and the > task manager then inserts, let's say, 60 copies of a specified > procedure into a list which are doled out to worker threads so that > only N+1 are active at any time. The issue I'm having is, how do you > have a unique collating task created for those 60 processes to send > updates to? As it is now, I have all of these (nearly a dozen) unique > collating tasks defined in the task manager. I would rather have them > defined in the subsystem package to which they really belong. As the > subsystem packages "push" their procedures to the task manager, the > task manager has no visibility into the subsystem. My aim was to > define an area (a System.Address for example) in a control block > record that the task manager would have space for "a task". What this > task was, what it did, where it came from the task manager would not > know nor care, but it would be the owner, or master, in some sense so > the "preprocess" or initialization component could exit. I hope this > makes a little sense. There's still a lot of implementation in here. Talking about procedure pointers and addresses is not only implementation, it's quite low level. One of Ada's great strengths is its ability to express, implement, and use high-level abstractions. The low-level stuff is still available, but generally best hidden well down in the implementation of such an abstraction. What I get from your description is: You have classes of computations ("subsystems"); each class can have numerous computations carried on at the same time; each computation produces output; you want some of the output from all the computations of the same class to be subjected to collating. Everything else seems like implementation detail that may be blocking your view of a solution. Steve has given you some ideas about how to abstract the idea of computations and controlling their scheduling. He didn't have get into the resulting output or how to subject some of it, within a class of computations, to collating. I would probably take a different approach than Steve, with different subsystems more independent, in order to ease the collating within the subsystems, but that may be more a matter of style than substance. If you can produce high-level package specifications that only address the high-level concepts of your problem, without worrying about implementation (private parts, bodies), with a road map of how they work together, you'll probably find that you can then work out a way to implement them. -- Jeff Carter "Don't knock masturbation. It's sex with someone I love." Annie Hall 45 ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Fun with Tasking 2007-03-29 15:17 ` mhamel_98 2007-03-30 4:06 ` Steve 2007-03-30 5:46 ` Jeffrey R. Carter @ 2007-03-31 12:08 ` Stephen Leake 2 siblings, 0 replies; 10+ messages in thread From: Stephen Leake @ 2007-03-31 12:08 UTC (permalink / raw) mhamel_98@yahoo.com writes: > Anyhoo Jeff, I'll try to explain in more detail what my program is > doing and hopefully you can see where I'm going wrong. The program > has a number of packages, each handling what I call a subsystem, to be > verbose: ephemeris, residual, vector, covariance and a few others > (this sort of equates to the "exec" procedure in my sample). Each of > these systems has a number of procedures of interest, like a > differencing of quantities, a statistical analysis, etc. Now each of > these procedures of each of these subsystems can have up to 60 (and > more as we go on) objects - each one I handle as it's own task. There's no reason for each of these things to be a _task_. They should each be in separate _packages_, and be separate _objects_, but not _tasks_. Tasks are only needed for controlling timing, and then you need more than one task only when there are truly asynchronous events. Or if you want to gain speed by using parallel processors. In the system you describe, there are bunch of calculations, that can be done correctly if they are done serially. Since you talk about multiple processors, you should have one task per processor, and portion the computations among the tasks. -- -- Stephe ^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2007-03-31 12:08 UTC | newest] Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2007-03-28 15:53 Fun with Tasking mhamel_98 2007-03-28 16:55 ` Adam Beneschan 2007-03-28 17:53 ` mhamel_98 2007-03-28 20:06 ` Ed Falis 2007-03-28 22:17 ` Randy Brukardt 2007-03-29 4:59 ` Jeffrey R. Carter 2007-03-29 15:17 ` mhamel_98 2007-03-30 4:06 ` Steve 2007-03-30 5:46 ` Jeffrey R. Carter 2007-03-31 12:08 ` Stephen Leake
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox