Re: Generators/coroutines in future Ada?

["Randy Brukardt" <randy@rrsoftware.com>]

"Shark8" <onewingedshark@gmail.com> wrote in message 
news:b6e36917-63f0-4c13-aeee-1a1a6f44d495@googlegroups.com...
On Tuesday, July 11, 2017 at 11:35:19 PM UTC-6, Randy Brukardt wrote:
>
>> The coming problem with "classic" programming languages like Ada is that
>> they don't map well to architectures with a lot of parallelism. The more
>> constructs that require sequential execution, the worse that Ada programs
>> will perform.
...
>Yes, I know that "automatic parallelization" of high-level language is 
>considered
> fairly hard right now, but less than 50 years ago an optimizing compiler 
> was
> considered hard... and we gained a LOT from allowing optimizing compilers
> + bootstrapping.

No, I think there is very little chance of it becoming possible. When I was 
a young buck at the University of Wisconsin, there was a lot of optimism and 
research into automatic parallelization of programs (particularly Fortran 
programs). I recall reading a number of papers in this area for classes.

Fast forward (gulp!) about 38 years, and most of the optimism is gone, and 
there has not been a lot of progress in the area of automatic 
parallelization. Certainly, compilers can recognize certain patterns and 
parallelize them (generally in cases where all of the "tasklets" run the 
same code), but general-purpose parallelization has defied solution (at 
least for conventional languages -- there might be more luck with a 
purpose-built language like Parasail -- but this is an Ada forum and such 
languages are irrelevant for Ada).

It should be clear that fine-grained parallelism (the kind that a compiler 
could conceivably do) is likely to be a failure, as scheduling takes a 
significant amount of overhead, on top of which a lot of conventional 
sequential code doesn't have much parallelism. (Remember that things like 
exceptions, assignment semantics, and the like add sequential requirements 
that probably aren't necessary to solve the actual problem.)

I would never want to say never, but I think it is far more likely that 
programming will cease to be a human activity at all (perhaps because 
everything is replaced by neural nets) than automatic parallelization of 
"classic" languages become possible.

>That's also why I'm also a bit dubious about the proposed PARALLEL / AND / 
>END
> blocks: it seems to me that the mistake here is to delve too far into the 
> minutia (the
> above "parallel assembly" idea) so as to make it difficult or impossible 
> to automatically
> optimize parallel code because of the more low-level view imposed by the 
> language...
> much like C's notion of arrays is fundamentally broken and undermines the 
> benefits
> of C++.

If you believe, like I do, that automatic parallelization is likely to be 
impossible in the reasonable term, the next best thing is to give the 
compiler hints. Moreover, nothing about "parallel" blocks and loops actually 
requires them to be run in parallel; it just suggests the possibility. It 
also includes checking that the "parallel" code is actually possible to 
execute in parallel, so you are actually giving an optimizer more help by 
ensuring that the code can really be run that way.

My goal for these constructs is that they are essentially semantically 
identical to the non-parallel versions, just with checking that parallel 
execution is possible. (I have no idea how close or far from that goal we 
will end up, as these constructs change a lot from meeting to meeting.)

> Now, I admit I could very well be wrong here, but there's a gut-feeling 
> that
> this is not the route we want to go down.

You're wrong here. ;-)

I think it is highly unlikely that the majority of Ada code could ever be 
parallelized automatically. Assignments (needed for results like 
reductions), and the handling of exceptions would prevent that. There is 
also the need (at least in the foreseeable future) to make your "chunks" of 
execution large enough so that scheduling overhead doesn't kill any 
parallelism gains. That's best handled with explicit code of some kind; 
Brad's libraries are a step in the right direction, but one would want 
checking for conflicts (one of the things that is extremely hard to do with 
existing Ada tasks) and more integration with the syntax.

                                           Randy.