Re: Real tasking problems with Ada.

[Robert Eachus <rieachus@comcast.net>]

On Tuesday, August 1, 2017 at 12:42:01 AM UTC-4, Randy Brukardt wrote:
 
> Use a discriminant of type CPU and (in Ada 2020) an 
> iterated_component_association. (This was suggested way back in Ada 9x, left 
> out in the infamous "scope reduction", and then forgotten about until 2013. 
> See AI12-0061-1 or the draft RM 
> http://www.ada-auth.org/standards/2xaarm/html/AA-4-3-3.html).

It is nice that this is in there, just in-time to be outmoded.  See my previous post, on most desktop and server processors assigning to alternate processor IDs is necessary.

> Why do you say this? Ada doesn't require the task that calls a protected 
> object to execute it (the execution can be handdled by the task that 
> services the barriers - I don't know if any implementation actually does 
> this, but the language rules are written to allow it).

Um, I say it because any other result is useless?  The use case is for the called task or protected object to be able to get the CPU number and do something with it.  The simple case right now would be a protected object whose only purpose is to make sure task assignments are compatible with the architecture.  Again with Zen, Epyx has multiprocessors with 64 cores, 128 threads that will show as CPUs.  There are favored pairings of threads so that they share caches, or have shorter paths to parts of L3.  Intel says they will support up to eight sockets, with 28 CPU cores and 56 threads per socket.  I don't believe a system with 448 CPU threads will be realistic, even 224 will probably stress scheduling.  (Note that many of these "big" systems use VMware to create lots of virtual machines with four or eight threads.)

> Say what? You're using Get_Id, so clearly you're using something there. 
> Get_Id (like the rest of dispatching domains is likely to be expensive, so 
> you don't want it dragged into all programs. (And CPU is effectively part of 
> all programs.)

Sigh! Get_Id as defined is heavy only because of the default initial value:

   function Get_CPU
      (T   : Ada.Task_Identification.Task_Id :=
                 Ada.Task_Identification.Current_Task)
           return CPU_Range;

but a parameterless Get_CPU could compile to a single load instruction.  I'm not asking for the current function to be eliminated, there are situations where it is needed.  But it doesn't need all that baggage for the normal use.

> 
> > 2) Allow a task to  its CPU assignment after it has started execution.  It
> > is no big deal if a task starts on a different CPU than the one it will 
> > spend
> > the rest of its life on.  At a minimum Set_CPU(Current_CPU) or just
> > Set_CPU should cause the task to be anchored to its current CPU core.
> >  Note that again you can do this with Dispatching_Domains.

Left out two words above.  Should read "Allow a task to statically set its CPU assignment...
> 
> So the capability already exists, but you don't like having to with an extra 
> package to use it? Have you lost your freaking mind? You want us to add 
> operations that ALREADY EXIST to another package, with all of the 
> compatibility problems that doing so would cause (especially for people that 
> had withed and used Dispatching_Domains)? When there are lots of problems 
> that can't be solved portably at all?

No, I don't want compilers putting in extra code when it is not necessary.  If a task has a (static) CPU assignment then again, Get_CPU is essentially free.  Is it cheaper than fishing some index out of main memory because it got flushed there? Probably.  Yes, I can make a function My_CPU which does this.  I'm just making too many of those workarounds right now. 
> 
> No, you're missing the point. Ada is about writing portable code. Nothing at 
> the level of "cache lines" is EVER going to be portable in any way. Either 
> one writes "safe" code and hopefully the compiler and runtime can take into 
> account the characteristics of the target. (Perhaps parallel loop constructs 
> will help with that.)
> 

> Or otherwise, one writes bleeding edge code that is not portable and not 
> safe. And you're on your own in such a case; no programming language could 
> possibly help you.

I am trying to write portable code.  Portable enough to run on all modern AMD64 and EM64T (Intel) CPUs.  A table of processor IDs with the associated values for the numbers I need in the program is not going to happen.  Most of the parameters I need can be found in Ada now.  Cache line size is one that is not there.
> 
> >A function Cache_Line_Size in System or System.Multiprocessors seems right.
> 
> No,  it doesn't. It assumes a particular memory organization, and one thing 
> that's pretty clear is that whatever memory organization is common now will 
> not be common in a bunch of years. Besides, so many systems have multiple 
> layers of caches, that a single result won't be enough. And there is no way 
> for a general implementation to find this out (neither CPUs nor kernels 
> describe such information).

Um. No.  Systems may have multiple levels of caches of different sizes and different numbers of "ways" per cache.  But the actual cache line size is almost locked in, and is the same for all caches in a system.  Most systems with DDR3 and DDR4 use 64 byte cache lines because it matches the memory burst length.  But other values are possible.  Right now HBM2 is pushing GPUs (not CPUs) to 256 byte cache lines.  Will we eventually have Ada compilers generating code for heterogeneous systems?  Possible.  What I am working on is building the blocks that can be used with DirectCompute, OpenCL 2.0, and perhaps other GPU software interfaces. 
> 
> >Is adding these features to Ada worth the effort?
> 
> No way. They're much too low level, and they actually aren't enough to allow 
> parallelization. You want a language which allows fine-grained parallelism 
> from the start (like Parasail); trying to retrofit that on Ada (which is 
> mainly sequential, only having coarse parallelism) just will make a mess. 
> You might get a few problems solved (those using actual arrays, as opposed 
> to containers or user-defined types -- which one hopes are far more common 
> in today's programs), but there is nothing general, nor anything that fits 
> into Ada's building block approach, at the level that you're discussing.
> 
For now we can agree to disagree.  The difference is the size of the arrays we have to deal with.  When arrays get to tens of millions of entries, and operations on them can take tens of billions operations, I don't think I am talking about fine grained parallelism.  The main characteristics of the operations I want to get working: matrix multiplication and inversion, linear programming, FFT, FLIR and FLRadar, Navier-Stokes, all have the form of a set of huge data arrays constant once loaded, and can be parallelized across large numbers of CPU cores, or GPUs.