basic question on Ada tasks and running on different cores

basic question on Ada tasks and running on different cores

[Nasser M. Abbasi]

Assume I am using a PC with say 1,000 cores (may be
in few short years).

If I use Ada, and create many, many tasks, will these
tasks automatically be scheduled to run on as many
different cores as possible so to spread the load and
achieve the most parallelism possible?

Is this something that is controlled by Ada run-time
automatically, or is it the OS that that is in charge here
with which task (i.e. thread) runs on which core?

Will the programmer have to do anything other than just
creating the tasks (and ofcourse protect any critical
section as needed), but not worry about the mapping
of tasks to cores and all the other scheduling issues.

Sorry for such a basic question, but I have not looked
at this for long time.

thanks,
--Nasser

Re: basic question on Ada tasks and running on different cores

[Gautier write-only]

On 6 mai, 09:38, "Nasser M. Abbasi" <n...@12000.org> wrote:

> Assume I am using a PC with say 1,000 cores (may be
> in few short years).
>
> If I use Ada, and create many, many tasks, will these
> tasks automatically be scheduled to run on as many
> different cores as possible so to spread the load and
> achieve the most parallelism possible?

Yes. Actually the operating system does it for you, automatically.
The difficulty is for *not* using as many cores or CPUs as available.
_________________________
Gautier's Ada programming
http://freecode.com/users/gdemont

Re: basic question on Ada tasks and running on different cores

[Simon Wright]

"Nasser M. Abbasi" <nma@12000.org> writes:

> If I use Ada, and create many, many tasks, will these tasks
> automatically be scheduled to run on as many different cores as
> possible so to spread the load and achieve the most parallelism
> possible?
>
> Is this something that is controlled by Ada run-time automatically, or
> is it the OS that that is in charge here with which task (i.e. thread)
> runs on which core?
>
> Will the programmer have to do anything other than just creating the
> tasks (and ofcourse protect any critical section as needed), but not
> worry about the mapping of tasks to cores and all the other scheduling
> issues.

One imagines that Ada RTS's will offer the ability to map tasks to
cores, building on the facilities offered by the OS's to specify
processor affinities. Whether Ada 2020 will standardise an interface to
this is to be determined! (ada-auth.org isn't responding at the moment,
so I can't tell whether there's anything in Ada 2012).

Re: basic question on Ada tasks and running on different cores

[Ludovic Brenta]

Simon Wright writes on comp.lang.ada:
> One imagines that Ada RTS's will offer the ability to map tasks to
> cores, building on the facilities offered by the OS's to specify
> processor affinities. Whether Ada 2020 will standardise an interface
> to this is to be determined! (ada-auth.org isn't responding at the
> moment, so I can't tell whether there's anything in Ada 2012).

Ada 2012 has an entire clause, D.16, "Multiprocessor Implementation",
devoted to this.  The salient part is:

7/3 For a task type (including the anonymous type of a
single_task_declaration) or subprogram, the following language-defined
representation aspect may be specified:

8/3 CPU

               The aspect CPU is an expression, which shall be of type
               System.Multiprocessors.CPU_Range.

This allows assigning a single CPU to a task; the problem is that the
value of the aspect_specification must be static.  In subclause D.16.1
"Dispatching Domains" we see:

16/3
The type Dispatching_Domain represents a series of processors on which
a task may execute. Each processor is contained within exactly one
Dispatching_Domain. System_Dispatching_Domain contains the processor or
processors on which the environment task executes. At program start-up
all processors are contained within System_Dispatching_Domain.

17/3
For a task type (including the anonymous type of a
single_task_declaration), the following language-defined representation
aspect may be specified:

18/3
Dispatching_Domain

               The value of aspect Dispatching_Domain is an expression,
               which shall be of type
               Dispatching_Domains.Dispatching_Domain. This aspect is
               the domain to which the task (or all objects of the task
               type) are assigned.

This allows assigning any one of a range of CPUs to a task.

-- 
Ludovic Brenta.
An enhanced, consistent, best practice culturally prioritizes an
emotional intelligence.

Re: basic question on Ada tasks and running on different cores

[Robert A Duff]

"Nasser M. Abbasi" <nma@12000.org> writes:

> If I use Ada, and create many, many tasks, will these
> tasks automatically be scheduled to run on as many
> different cores as possible so to spread the load and
> achieve the most parallelism possible?

Yes.  Unless you have particular requirements, you don't
have to do anything special.

> Is this something that is controlled by Ada run-time
> automatically, or is it the OS that that is in charge here
> with which task (i.e. thread) runs on which core?

The way GNAT works, it's the OS.  But it doesn't have
to be that way.

- Bob

Re: basic question on Ada tasks and running on different cores

[Robert A Duff]

Ludovic Brenta <ludovic@ludovic-brenta.org> writes:

> 8/3 CPU
>
>                The aspect CPU is an expression, which shall be of type
>                System.Multiprocessors.CPU_Range.

If there's more than one of them, how can they be "central"?

- Bob

Re: basic question on Ada tasks and running on different cores

[Vinzent Hoefler]

Robert A Duff wrote:

> Ludovic Brenta <ludovic@ludovic-brenta.org> writes:
>
>> 8/3 CPU
>>
>>                The aspect CPU is an expression, which shall be of type
>>                System.Multiprocessors.CPU_Range.
>
> If there's more than one of them, how can they be "central"?

In sufficiently local scope, they are. Well, "it is". ;)


Vinzent.

-- 
The most likely way for the world to be destroyed, most experts agree,
is by accident. That's where we come in; we're computer professionals.
We cause accidents.
     -- Nathaniel Borenstein

Re: basic question on Ada tasks and running on different cores

[Ludovic Brenta]

Robert A Duff writes:
> Ludovic Brenta <ludovic@ludovic-brenta.org> writes:
>
>> 8/3 CPU
>>
>>                The aspect CPU is an expression, which shall be of type
>>                System.Multiprocessors.CPU_Range.
>
> If there's more than one of them, how can they be "central"?

Mainframes have many "central" processing units and dozens of peripheral
processing units, e.g. embedded in disk and network controllers.  They
also have "peripheral" processors dedicated to system monitoring and
maintenance, i.e. the operator console or its modern equivalent.

-- 
Ludovic Brenta.

Re: basic question on Ada tasks and running on different cores

[Ludovic Brenta]

Dennis Lee Bieber writes:
> On Sun, 06 May 2012 21:43:53 +0200, Ludovic Brenta
> <ludovic@ludovic-brenta.org> declaimed the following in comp.lang.ada:
>
>> Mainframes have many "central" processing units and dozens of peripheral
>> processing units, e.g. embedded in disk and network controllers.  They
>> also have "peripheral" processors dedicated to system monitoring and
>> maintenance, i.e. the operator console or its modern equivalent.
>
> 	They still make "mainframes" <G>

Yes. Mostly to run thousands of Linux virtual machines, and because
their disk subsystems are unmatched in performance and reliability.

-- 
Ludovic Brenta.

Re: basic question on Ada tasks and running on different cores

[anon]

At the movement anyone uses the keywords, "threads", "tasks" or "multiple 
cores" what they are asking for is parallelism system. And at this time 
no standard language come setup for doing parallel.  You have to modify 
a number of compiler sub-systems as well as rewrite the run-time-libraries 
and this includes GCC. The main problem now is that most to all open 
source parallel compilers have disappear off the net since Intel introduce
the Pentium D and Core 2.

An the Ada's run-time-libraries is still concurrent.  Ada 2012, does 
have some intro into CPU control, but when will those features be 
developed and allow Ada's run-time-libraries to be converted to parallel 
is any one guess.


History of parallel Ada.

Back in the 1980s and early 1990s there was the transputers using 
Ada which allowed some parallel coding. To use parallel one or 
more multiple transputers daughter boards had to be added to the 
system design. In this design, the PC system housed the compiler 
and transputer program loader. The while the Ada code was running 
the PC handled all hardware I/O required for the Ada program.

Then in the the early 1990s when dual and quad processor motherboards 
were developed, all most every language except Ada had a parallel 
version. The ARG with the DoD had not fully developed the concept 
for Ada being parallel.  Even though, back in the mid 1990 there was 
a compiler patch and "back end" replacement for GNAT that did allow 
some parallel but due to Adacore's design style of GNAT, the parallel
system was not fully parallel.



In <jo59po$s06$1@speranza.aioe.org>, "Nasser M. Abbasi" <nma@12000.org> writes:
>
>Assume I am using a PC with say 1,000 cores (may be
>in few short years).
>
>If I use Ada, and create many, many tasks, will these
>tasks automatically be scheduled to run on as many
>different cores as possible so to spread the load and
>achieve the most parallelism possible?
>
>Is this something that is controlled by Ada run-time
>automatically, or is it the OS that that is in charge here
>with which task (i.e. thread) runs on which core?
>
>Will the programmer have to do anything other than just
>creating the tasks (and ofcourse protect any critical
>section as needed), but not worry about the mapping
>of tasks to cores and all the other scheduling issues.
>
>Sorry for such a basic question, but I have not looked
>at this for long time.
>
>thanks,
>--Nasser

Re: basic question on Ada tasks and running on different cores

[Maciej Sobczak]

On May 7, 9:36 am, a...@att.net wrote:

> At the movement anyone uses the keywords, "threads", "tasks" or "multiple
> cores" what they are asking for is parallelism system.

No, from the thread (sic!) subject it is pretty clear that the poster
is asking about tasks and cores.

> And at this time
> no standard language come setup for doing parallel.

It is not the first time when you spread misinformation related to
this subject, even though you have been already corrected on this.
Please take into account that publishing incorrect or misleading
information about Ada's ability in this area is a disservice to the
community.

--
Maciej Sobczak * http://www.msobczak.com * http://www.inspirel.com

Re: basic question on Ada tasks and running on different cores

[anon]

Where you proof!!! That I am wrong!!! You have none!!!
The proof must be creditable. That is someone not associate 
with Adacore, ARG. 
Someone like the DoD, IBM, SUN, SGI. And must be posted on 
their web site.

In <822367e8-28c2-4289-a59c-88799d68bc9e@j16g2000vbl.googlegroups.com>, Maciej Sobczak <see.my.homepage@gmail.com> writes:
>On May 7, 9:36=A0am, a...@att.net wrote:
>
>> At the movement anyone uses the keywords, "threads", "tasks" or "multiple
>> cores" what they are asking for is parallelism system.
>
>No, from the thread (sic!) subject it is pretty clear that the poster
>is asking about tasks and cores.
>
>> And at this time
>> no standard language come setup for doing parallel.
>
>It is not the first time when you spread misinformation related to
>this subject, even though you have been already corrected on this.
>Please take into account that publishing incorrect or misleading
>information about Ada's ability in this area is a disservice to the
>community.
>
>--
>Maciej Sobczak * http://www.msobczak.com * http://www.inspirel.com

Re: basic question on Ada tasks and running on different cores

[Ludovic Brenta]

anon wrote on comp.lang.ada:
> Where you proof!!! That I am wrong!!! You have none!!!
> The proof must be creditable. That is someone not associate 
> with Adacore, ARG. 
> Someone like the DoD, IBM, SUN, SGI. And must be posted on 
> their web site.

You must have a case of schizophrenia.  This is not a joke;
I know first-hand what schizophrenia looks like and I know it
when I see it.  Get medical counsel.  If you are mentally ill,
*this is not your fault*.

Symptoms: you live in a fantasy world where your word must be
taken for granted but AdaCore and the ARG are liars.  This
world does not exist. In the real world, *you* must prove that
you are correct; the ARG is the final authority on the
language definition and AdaCore is the final authority on the
implementation of this language in GNAT (and AdaCore is a
member of the ARG, too).  In the real world, the DoD has
nothing to do with Ada anymore except as a customer of AdaCore
and IBM; IBM is, just like AdaCore, the final authority on
their implementations of the language, and SUN and SGI have
both delegated all their Ada activities to... AdaCore.

-- 
Ludovic Brenta.
Communications prioritize transparent matrices.

Re: basic question on Ada tasks and running on different cores

[anon]

I am talking about standard languages aka all languages including Ada.
IBM does maintain a number of languages not just their version of Ada
and all hardware/software companies like IBM, SUN, SGI have been using 
parallel for a number of years. Also IBM, SUN, SGI, and might as well 
include Microsoft are not easily updated like wikipedia is to prove
a point. 

Now, most of the languages that were parallelized in the late 1980s and 
90s disappeared about the time that Intel introduce the Pentium D and Core 
2. And every since then programmers and some professors have been looking 
for one or more parallel languages.

For GCC, the deal is that just downloading and compiling all languages in 
GCC suite will not give you a set of parallel languages. You must modify 
the existing compiler code to add parallel constructs and controls and/or 
link the code to special run-time libraries (compiled for parallel).

Some of these special libraries have links that can either temporary 
lock or replace the OS existing job scheduler, not for normal Home 
version of Windows. 

Then there are three major current design for parallel. First, is 
the OS design, where the OS loads each program but does nothing to 
allow one's program to execute in parallel.  Also the OS is still 
hogging one or more cores with its normal system functions such as 
I/O and video updates as well as the job scheduling. The OS may also 
at times limit the resources a programs needs to complete it job. So, 
to many programmers and algorithms this design is not even parallel.

Secondly, is the run-time design. Which uses library routines to 
allow multiple executions. Now, in GNAT as of GNAT 2011 this is still 
in its initial stages of getting these type of routines working. A 
major problem here is using the standard linker and system job loader
which can kill any performance increase that could be obtain in using 
parallel code. Take a 2 MB program for example if the parallel routine 
to be execute is only 25 KB, then it is very inefficiency to load a 2 MB 
program image for each core as well as reloading for every vs swap. So, 
the answer is to use a special binder and linker as well as a special 
parallel support loader routines to load only the routine 
that needs to be executed. Also, routines like Ada system finalization 
routine needs to be preloaded as part of this parallel support system. 
This would decrease the load time and increase performance. But In the 
case of GNAT, the run-time system requires too many soft_links that 
basically requires the whole program to be loaded each time. Also, for 
the programmer this version is a little too much work because they still
have to make the algorithm parallel and take time to get the code 
efficiency.

Third, is the compiler design.  In this case Ada would either have to 
be expanded to include a pragma like "pragma Parallel ( <routine> ) ;" 
and a modified compiler would then build a parallel version of the 
routine. Or have a "Parallelizer" which world rewrite the source code 
into a more parallel design before using the normal compiler with some 
additional libraries. Most programmer like this design but their 
are major problems like in debugging the algorithm for one. Also, this 
design requires a system utilization to be stable for code efficiency.

In the Run-time and Compiler designs there exist a side-effect of message 
passing which can cause a decrease of performance. And all design can 
have the problem of shared memory, in other words if all core access 
memory at the same time which core has their request answered first or 
last. In the OS design this is not a major problem but in the other 
two design this type of memory bottleneck for numerical calculation may 
result in calculating errors.

And this does not include the parallel paradigms like Data, Message 
Passing, and Shared Memory that must be considered.



Note: An exception to the disappearing parallel languages is the UC 
Berkeley's Unified Parallel C (UPC) with the current version working 
on Microsoft Windows and Apple OSX. It has been tested for others OS 
like Linux but only Windows and Apple are fully supported with binaries. 
The problem with UPC starts when using a simple two statement 
"Hello World" C program is converts to 120 line of source code for 
parallel. To be compile and linked. But if you look at the orginal 
code you can see the compiler should have defaulted to a no parallel 
design.



#include <upc.h>
#include <stdio.h>
#include <stdlib.h>
int main ( ) 
  {
    printf ( "Hello parallel World" ) ;
    return 0;
  }





--  A final note loweing youself to name calling just proves that 
--  you do not know what your talking about concerning "Parallel".



In <30585369.219.1336470732142.JavaMail.geo-discussion-forums@ynbq3>, Ludovic Brenta <ludovic@ludovic-brenta.org> writes:
>anon wrote on comp.lang.ada:
>> Where you proof!!! That I am wrong!!! You have none!!!
>> The proof must be creditable. That is someone not associate 
>> with Adacore, ARG. 
>> Someone like the DoD, IBM, SUN, SGI. And must be posted on 
>> their web site.
>
>You must have a case of schizophrenia.  This is not a joke;
>I know first-hand what schizophrenia looks like and I know it
>when I see it.  Get medical counsel.  If you are mentally ill,
>*this is not your fault*.
>
>Symptoms: you live in a fantasy world where your word must be
>taken for granted but AdaCore and the ARG are liars.  This
>world does not exist. In the real world, *you* must prove that
>you are correct; the ARG is the final authority on the
>language definition and AdaCore is the final authority on the
>implementation of this language in GNAT (and AdaCore is a
>member of the ARG, too).  In the real world, the DoD has
>nothing to do with Ada anymore except as a customer of AdaCore
>and IBM; IBM is, just like AdaCore, the final authority on
>their implementations of the language, and SUN and SGI have
>both delegated all their Ada activities to... AdaCore.
>
>-- 
>Ludovic Brenta.
>Communications prioritize transparent matrices.

Re: basic question on Ada tasks and running on different cores

[Randy Brukardt]

<anon@att.net> wrote in message news:jodnse$c29$1@speranza.aioe.org...
>I am talking about standard languages aka all languages including Ada.

Ada always has supported parallelism.

I think you are confused by various research languages that are trying to 
*automatically* make programs execute in parallel. It's not surprising that 
these disappear, people have been trying to design these since the 1960s 
(some of the research papers I read while in college - in the 1970s - were 
ancient even then).

My personal opinion is that these "holy grail" languages will never really 
work out. If a reasonably sensible solution existed, someone would have 
found it in the last 50 years of looking. I'm sure some progress will be 
made, but only by increasing non-determinism elsewhere in the language --  
which will make debugging (and thus the languages) impractical.

People that are looking for research grants have a lot of advantage to 
spreading FUD about what is and is not possible, so be careful about 
depending on them.

OTOH, *manual* parallelism, as exhibited by Ada's tasks, threads in Windows 
and Linux, and constructs in many other languages, works and has been in use 
for decades. (Ada makes no claim to having invented that.) To claim 
otherwise is just plain wrong.

And you don't have to take my word for it. You can see it yourself. You can 
download the very useful Process Explorer tool for Windows, which lets you 
see what each process on your computer is doing. Install it on any modern 
Windows system. Then compile a program that uses a number of CPU-intensive 
tasks with most Windows Ada compilers (any that claim to use threads, I know 
that GNAT, ObjectAda, and IBM's Ada all do from personal experience). I'm 
sure someone else here could recommend a short Ada program.

If you run such a compiled Ada program and examine the process with Process 
Explorer on a multicore machine, you'll see that several or all of the cores 
will be in use for that single process. (Yes, I've done this on Windows XP. 
Supposely, Windows 7 is better at it but I haven't tested that myself.) 
Typically, more CPU than a single core could use will be consumed by the 
program. That means that the core are running the tasks *in parallel*, 
because they obviously aren't being run sequentially and there is no other 
choice.

I suppose Microsoft could be lying about the parallel execution evidenced by 
this commonly used tool, but I have no idea what gain that would have for 
Microsoft.

Anyway, do the experiment with an appropriate program (just make sure that 
the program doesn't require sequential execution). See if you still think 
that Ada programs can't run parallel threads.

                                                         Randy.

Re: basic question on Ada tasks and running on different cores

[NatarovVI]

> I think you are confused by various research languages that are trying
> to *automatically* make programs execute in parallel. It's not
> surprising that these disappear, people have been trying to design these
> since the 1960s (some of the research papers I read while in college -
> in the 1970s - were ancient even then).
> My personal opinion is that these "holy grail" languages will never
> really work out. If a reasonably sensible solution existed, someone
> would have found it in the last 50 years of looking. I'm sure some
> progress will be made, but only by increasing non-determinism elsewhere
> in the language -- which will make debugging (and thus the languages)
> impractical.

read existentialtypes.wordpress.com, or comments in their translation
to russian. imperative programming and ephemeral data structures born as
just "optimization trick" for executing true (fully parallel functional) 
programs (working with fully parallel persistent data) on cheap 
sequential hardware of the time. trick is do parallel actions sequentially
in time, on one processor. another trick is work with ephemeral copies of 
parts of persistent data, again - sequentially in time.
it's just this two tricks form all modern IT, form many artifacts.

so. sure you can not easily deduce true, parallel, program from it's 
purposely sequential variant. projection on hardware. but if you think 
parallel from start, you CAN get automatic parallelisation (programm will 
work without modification on any hardware, any number of processors).

you just can not do it in imperative, intertwined with dependencies, 
lobotomized manner. in functional languages - you can.

Re: basic question on Ada tasks and running on different cores

[Georg Bauhaus]

On 11.05.12 04:38, NatarovVI wrote:

> you just can not do it in imperative, intertwined with dependencies,
> lobotomized manner. in functional languages - you can.

Do they address predictability of I/O, both local and
communicating,  in reactive systems?

Re: basic question on Ada tasks and running on different cores

[Randy Brukardt]

"NatarovVI" <4KCheshireCat@gmail.com> wrote in message 
news:johu2d$9vj$1@speranza.aioe.org...
...
> you just can not do it in imperative, intertwined with dependencies,
> lobotomized manner. in functional languages - you can.

Sure, but I'm skeptical that the vast majority of programmers can deal with 
any programming language that has the level of non-determinism needed to 
support useful parallelism. Functional programming languages make this 
worse, if anything.

Secondly, I'm skeptical that any language attempting fine-grained 
parallelism can ever perform anywhere near as well as a language using 
coarse parallelism (like Ada) and deterministic sequential semantics for the 
rest. Any parallelism requires scheduling overhead, and that overhead is 
going to be a lot higher for the fine-grained case, simply because there is 
a lot more of it needed (even if it is conceptually simpler).

There are of course special cases where neither issue is a problem or can be 
worked around, but for general purpose programming (especially embedded 
programming, which is pretty much the only *real* programming going on today 
:-), I don't see it happening. I wouldn't mind being wrong (even though it 
would mean the end of my programming career; I already know that I'm not 
productive in non-Ada languages - I cannot tolerate debugging very often).

                                         Randy.

Re: basic question on Ada tasks and running on different cores

[Stephen Leake]

"Randy Brukardt" <randy@rrsoftware.com> writes:

> Secondly, I'm skeptical that any language attempting fine-grained 
> parallelism can ever perform anywhere near as well as a language using 
> coarse parallelism (like Ada) and deterministic sequential semantics for the 
> rest. Any parallelism requires scheduling overhead, and that overhead is 
> going to be a lot higher for the fine-grained case, simply because there is 
> a lot more of it needed (even if it is conceptually simpler).

Have you been following Parasail?
http://parasail-programming-language.blogspot.com/

Not fully implemented yet, but it sounds very interesting.

-- 
-- Stephe

Re: basic question on Ada tasks and running on different cores

[Randy Brukardt]

"Stephen Leake" <stephen_leake@stephe-leake.org> wrote in message 
news:82d369u0nc.fsf@stephe-leake.org...
> "Randy Brukardt" <randy@rrsoftware.com> writes:
>
>> Secondly, I'm skeptical that any language attempting fine-grained
>> parallelism can ever perform anywhere near as well as a language using
>> coarse parallelism (like Ada) and deterministic sequential semantics for 
>> the
>> rest. Any parallelism requires scheduling overhead, and that overhead is
>> going to be a lot higher for the fine-grained case, simply because there 
>> is
>> a lot more of it needed (even if it is conceptually simpler).
>
> Have you been following Parasail?
> http://parasail-programming-language.blogspot.com/
>
> Not fully implemented yet, but it sounds very interesting.

A little bit, but I don't think even Tucker can do scheduling with no cost. 
(So far as I know, there is a significant penalty to running in the Parasail 
virtual machine, and I doubt that he'll be able to get rid of it.) Languages 
like this assume that it is OK to waste some significant fraction of the CPU 
in order to make it up on volume, er parallelism. As I said, I'm skeptical, 
but that doesn't mean that people shouldn't try -- and clearly, there are 
many applications where the performance isn't that critical or even where 
the added cost might actually provide some speed-up.

                                Randy.


                                          Randy.

Re: basic question on Ada tasks and running on different cores

[Shark8]

On Tuesday, May 15, 2012 1:55:01 AM UTC-5, Randy Brukardt wrote:
>
> clearly, there are 
> many applications where the performance isn't that critical or even where 
> the added cost might actually provide some speed-up.

Indeed, though this is really just the recapitulation of the themes "work smarter, not harder". If we might draw a parallel, this is to parallelism what basic-techniques were to data-structures (ie the very early days of CS).

Someday somebody realized, "if I sort things in this list, then finding them will be faster" and so we got all sorts of sort-algorithms and studies on the properties thereof and the like.

And that's where we're at WRT parallelism; we've got people looking into everything from instruction-level to task/program-level and trying to figure out the best way to do things. (But here the problem's more difficult; because, in addition to the amount of parallelization, the different granularities interact with the evaluation-metrics, and we're only just beginning to see to what extent.)

But, in the end, I think you're right: intuitively the fine-grained should have a disadvantage because there will likely be more overhead cost due to the synchronization/data-payload ratio. {We may be wrong though, sometimes intuition leads astray; and even slight improvements in such fields may lead to better compilers and/or OSes.}

Re: basic question on Ada tasks and running on different cores

[NatarovVI]

> Do they address predictability of I/O, both local and communicating,  in
> reactive systems?

again - "parallelism is not concurrency"

parallelism is not about predictability, it's about computational 
efficiency. fast way to get some correct result.
it can be implemented via concurrency, hard way.
but exist other ways also.

programmer writes dependencies, runtime schedules it automatically.
SISAL. NESL. this way.

Re: basic question on Ada tasks and running on different cores

[NatarovVI]

> Sure, but I'm skeptical that the vast majority of programmers can deal
> with any programming language that has the level of non-determinism
> needed to support useful parallelism. Functional programming languages
> make this worse, if anything.

and again - "parallelism is not concurrency"
if first-course CMU students can write correct parallel programs, 
everybody can. right abstractions is the key.
read Robert Harper experience at existentialtypes.wordpress.com

> Secondly, I'm skeptical that any language attempting fine-grained
> parallelism can ever perform anywhere near as well as a language using
> coarse parallelism (like Ada) and deterministic sequential semantics for
> the rest. Any parallelism requires scheduling overhead, and that
> overhead is going to be a lot higher for the fine-grained case, simply
> because there is a lot more of it needed (even if it is conceptually
> simpler).

you really need 100% of performance? maybe you like write in asm?))
seriously, SISAL and NESL can automatically get good part of data 
parallelism, no magic here. and this will be enought for most programmers.
(high order functions will be requirement here, extending data 
parallelism to operations on functions)

p.s. best scheduling is no scheduling...

> going on today :-), I don't see it happening. I wouldn't mind being

it will happen)) and Ada - not only language without need for debugging.
Standard ML also.

Re: basic question on Ada tasks and running on different cores

[Georg Bauhaus]

On 16.05.12 17:40, NatarovVI wrote:
>> Do they address predictability of I/O, both local and communicating,  in
>> reactive systems?
> 
> again - "parallelism is not concurrency"
> 
> parallelism is not about predictability, it's about computational 
> efficiency. fast way to get some correct result.
> it can be implemented via concurrency, hard way.
> but exist other ways also.

What other way that is also efficient?

> programmer writes dependencies, runtime schedules it automatically.
> SISAL. NESL. this way.

OK. I have ranted about this in the other post. Sorry.

Re: basic question on Ada tasks and running on different cores

[Randy Brukardt]

"NatarovVI" <4KCheshireCat@gmail.com> wrote in message 
news:jp0ikh$b7q$1@speranza.aioe.org...
>> Sure, but I'm skeptical that the vast majority of programmers can deal
>> with any programming language that has the level of non-determinism
>> needed to support useful parallelism. Functional programming languages
>> make this worse, if anything.
>
> and again - "parallelism is not concurrency"
> if first-course CMU students can write correct parallel programs,
> everybody can.

Baloney. First-course students don't have to worry about any real-world 
constraints. It's easy to write any kind of program in that environment (it 
was much easier to write programs of all sorts when I was a student).

> right abstractions is the key.
> read Robert Harper experience at existentialtypes.wordpress.com

I'm dubious of the pronouncements of anyone who's income (in this case, 
research grants) depends on people believing those pronouncements. In any 
case, those who ignore history are doomed to repeat it, and there has been 
research into these areas for 50 years. No one yet has found the "holy 
grail" of "free parallelism", and moreover nothing here is new.

Besides, 90% of most applications is I/O. The computation part of most 
programs is quite small. If all you can do is speed those things up, it's 
not going to make a significant difference to most programs. There are, of 
course, a few parts of programs that do extensive computations, but often 
these are in libraries where it makes sense to spend special efforts.

>> Secondly, I'm skeptical that any language attempting fine-grained
>> parallelism can ever perform anywhere near as well as a language using
>> coarse parallelism (like Ada) and deterministic sequential semantics for
>> the rest. Any parallelism requires scheduling overhead, and that
>> overhead is going to be a lot higher for the fine-grained case, simply
>> because there is a lot more of it needed (even if it is conceptually
>> simpler).
>
> you really need 100% of performance?

Of course, for the most critical computations. If you don't need 100% of 
performance, then you surely don't need parallel execution - what could 
possibly be the point?? Using two cores instead of one to do a computation 
that is not critical is insane -- you end up using more power to do the same 
work in the same time -- meaning more heat, lower battery life, etc.

> maybe you like write in asm?))

If necessary. But high-quality compilers can (or should) generate better 
code than you can write by hand, because they can take into account many 
more variables than a hand-programmer can. So the vast majority of the time, 
writing critical code in Ada is enough. (But, yes, Janus/Ada does have a few 
hand-coded ASM lookup loops; those cut compilation time by 25% when they 
were implemented. [No idea if they're needed anymore, of course, the CPU and 
I/O balance has changed a lot in the last 30 years.])

The problem is, if you're trying to implement fine-grained parallelism, you 
have to surround that code with some sort of scheduling mechanism, and that 
overhead means you aren't going to get anywhere near 100% of the CPU at any 
point. That means you'll have to find a

> seriously, SISAL and NESL can automatically get good part of data
> parallelism, no magic here. and this will be enought for most programmers.

Most programmers need no parallelism at all; they just need a good graphics 
library that uses parallelism to do their drawning. (And possibly a few 
other libraries.) The point is, most programmers are writing I/O intensive 
applications that do little computation (think of the apps on your phone, 
for example).

> (high order functions will be requirement here, extending data
> parallelism to operations on functions)

Certain special cases surely exist, and those can be implemented in a 
compiler for almost any language. (At the very least, by well-defined 
libraries.) These things surely could be done in Ada, either via libraries 
(look at Brad Moore's work) or via language-defined primitives. As I've said 
all along, I've very skeptical that anything further will prove practical.

> p.s. best scheduling is no scheduling...

Right. But that's only possible using vector instructions and the like; 
that's a *very* small set of all computing. (I doubt that I have ever 
written a program -- in 30+ years -- that could have been helped by a vector 
instruction.) Any other sort of parallelism will have to be implemented by 
some sort of concurency, and that requires some sort of scheduling.

>> going on today :-), I don't see it happening. I wouldn't mind being
>
> it will happen)) and Ada - not only language without need for debugging.
> Standard ML also.

I can believe that; no one has a clue what an ML program means, so there is 
no need to debug it - discarding makes more sense. ;-) Ada is the only true 
syntax. :-)

                                      Randy.

Re: basic question on Ada tasks and running on different cores

[Jeffrey Carter]

On 05/16/2012 05:11 PM, Randy Brukardt wrote:
>
> The problem is, if you're trying to implement fine-grained parallelism, you
> have to surround that code with some sort of scheduling mechanism, and that
> overhead means you aren't going to get anywhere near 100% of the CPU at any
> point.

The assumption here is that there are more tasks/threads/parallel things than 
there are processors/cores. That's generally true now, but they way things are 
going, it may not be true in the not-too-distant future (1-atom transistors 
could make for a lot of cores). When you have a core per task you no longer need 
scheduling.

-- 
Jeff Carter
"I blow my nose on you."
Monty Python & the Holy Grail
03

--- Posted via news://freenews.netfront.net/ - Complaints to news@netfront.net ---

Re: basic question on Ada tasks and running on different cores

[Dmitry A. Kazakov]

On Wed, 16 May 2012 18:06:49 -0700, Jeffrey Carter wrote:

> On 05/16/2012 05:11 PM, Randy Brukardt wrote:
>>
>> The problem is, if you're trying to implement fine-grained parallelism, you
>> have to surround that code with some sort of scheduling mechanism, and that
>> overhead means you aren't going to get anywhere near 100% of the CPU at any
>> point.
> 
> The assumption here is that there are more tasks/threads/parallel things than 
> there are processors/cores. That's generally true now, but they way things are 
> going, it may not be true in the not-too-distant future (1-atom transistors 
> could make for a lot of cores). When you have a core per task you no longer need 
> scheduling.

You will still have synchronization issues to handle (real time, e.g. delay
until and to other tasks).

I assume that memory will not be shared because evidently that will be a
bottleneck for thousands of cores. Now if processors (not cores) are
synchronized by data exchange using some topology of connections, that will
be transputers reborn. In the heydays of transputers there existed switches
to connect transputer links dynamically, but that was slow and expensive.
Very likely the same problem and worse would emerge if we had architectures
of thousands of processors. The topology of the mesh will be rather fixed
with some few links left to reconnect dynamically.

Scheduling will remain but the objective will be to choose a processor with
physical connections satisfying most of synchronization constraints of the
task it will have to run. That might be a very different kind of parallel
programming than we know today.

But I agree with Randy that fine grained parallelism will never make it,
whatever architecture comes.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: basic question on Ada tasks and running on different cores

[Randy Brukardt]

"Jeffrey Carter" <spam.jrcarter.not@spam.not.acm.org> wrote in message 
news:jp1ivb$o6m$1@adenine.netfront.net...
> On 05/16/2012 05:11 PM, Randy Brukardt wrote:
>>
>> The problem is, if you're trying to implement fine-grained parallelism, 
>> you
>> have to surround that code with some sort of scheduling mechanism, and 
>> that
>> overhead means you aren't going to get anywhere near 100% of the CPU at 
>> any
>> point.
>
> The assumption here is that there are more tasks/threads/parallel things 
> than there are processors/cores. That's generally true now, but they way 
> things are going, it may not be true in the not-too-distant future (1-atom 
> transistors could make for a lot of cores). When you have a core per task 
> you no longer need scheduling.

Scheduling includes deciding *what* to run, as well as *when* to run it. If 
there are enough cores, *when* becomes trivial, but *what* still requires 
overhead (and data communication, as Dmitry noted). There are special cases 
were *what* is pre-determined (SIMD machines), but for general workloads, 
determining what to run will still add costs.

                                           Randy.