Re: RFC: Prototype for a user threading library in Ada

[Hadrien Grasland <hadrien.grasland@gmail.com>]

Le jeudi 23 juin 2016 03:42:50 UTC+2, Randy Brukardt a écrit :
> "Hadrien Grasland" wrote :
> ...
> >I agree that implementation support for coroutines would be extremely 
> >valuable, if it
> >were available at the language level (as in Python, C#, Go...) or even in 
> >specific
> >implementations (as in Visual C++).
> 
> Coincidentally, we just spent quite substantial portion of the most recent 
> ARG meeting discussing this. (See AI12-0197-1; there are proposed 
> alternatives as well but those won't get posted for a few weeks - probably 
> along with the minutes.)

Count me pleasantly surprised!


> The problem with such proposals is that they are quite expensive to 
> implement, and they don't seem to buy that much above the existing Ada 
> tasking model. [Especially as the proposal explicitly does not support any 
> concurrency; one has to use POs/atomics in the normal way if concurrency is 
> needed.] (After all, if you really want coroutines in Ada, just use 
> Janus/Ada and regular tasks as it implements all tasks that way. :-)
> 
> The problem with the Janus/Ada implementation is the inability to use 
> threads to implement that; that's fixable but I'd need a customer to help 
> support the work. (I'd use a scheme internal to the task supervisor similar 
> to your "events" rather than trying to assign tasks to threads.)

I would be happy to beta-test that feature if you also integrated Ada 2012 support along the way! :)

That aside, let me explain what I think coroutines are good for. When people turn to threads, they usually look for some of the following things:

1. Exploiting the concurrent processing abilities of modern hardware (multicore, hyper-threading)
2. Providing the illusion of simultaneously running tasks to their users, in a fashion that extends beyond actual hardware concurrency.
3. Hide various kinds of latencies (IO, decision-making) by doing other processing in the meantime.
4. Handle IO-heavy workloads, the typical example being a web server going through millions of requests per second.

Unfortunately, no single threading implementation can be good at all of these. And outside the embedded world, the average modern OS is optimized to provide the best possible illusion of infinite multitasking through round-robin thread scheduling, and managing threads at the kernel level so that the kernel may quickly switch between them on clock interrupts instead of delegating that task to user processes.

Sadly, this setup is terrible for concurrent application performance, as can be easily tested by running a multithreaded computation with overcommitted CPU resources. If you allocate even just 2 times as many OS threads as you have hardware threads, you observe a huge performance drop. Why? Because instead of leaving computations alone, your OS keeps switching between threads during execution, each time doing a round trip through the kernel and trashing the CPU cache. There is no such thing as free concurrent lunch.

For IO-heavy applications, the situation is even worse: you will pay the aforementioned overhead not only at the scheduling rate of your round-robin algorithm (typically ~1 kHz), but every single time your application blocks for IO. This is why no web server application that allocates one OS thread per connection can scale to more than a couple thousand connections per second.

So if you don't desire the illusion of perfect multitasking, it is better to give up on the user convenience of round robin and use some batch-derived task scheduling algorithm instead. Which, because asking your customers to modify their OS kernel configuration is not usually acceptable, entails only allocating as many OS threads as there are hardware threads, and managing the remainder of your concurrency in user mode. Ergo, we need user threads, which are easiest to implement on top of language-level coroutine support.


> ...
> >However, I think that as it stands, we are just about as likely to see it 
> >happening in Ada as we are to get lambdas ...
> 
> We also talked about limited lambdas in Pisa: see AI12-0190-1. So you're 
> obviously right. ;-)

I stand once again pleasantly corrected, then :) Though I have to admit that in an Ada context, I miss first-class functions more than I miss lambdas: you can relatively easily replace a lambda with an expression function declared at the appropriate scope, but you need an awful lot of function-specific boilerplate in order to produce a standalone function object that can be easily transmitted to an outer scope after capturing some local state.


> The problem I have with the library approach (and the coroutines and the 
> like intended to support it) is that it does seem to solve any problems. I 
> understand why such approaches get used in languages that don't have a real 
> tasking model, but Ada hasn't had that problem since day 1. And the reasons 
> that writing tasking code in Ada is too hard aren't getting addressed by 
> these schemes (that is, race conditions, deadlocks [especially data 
> deadlocks], and the like).
>
> I'd prefer to concentrate on language features that make it as easy to write 
> (restricted and correct) parallel code as it is to write sequential code. I 
> don't see how libraries or coroutines or lambdas are getting us any closer 
> to that.
>
> I'd like to understand better the motivations for these features, so if you 
> (or anyone else) wants to try to explain them to me, feel free. (But keep in 
> mind that I tend to be hard to convince of anything these days, so don't 
> bother if you're going to give up easily. ;-)

See above. Being able to easily write highly concurrent code is of limited use if said code ends up running with terrible performance because modern OSs are not at all optimized for this kind of workload. We shouldn't need to worry about how our users' OS kernels are setup, and user threading and coroutines are a solution to this problem.

Not that I am against also providing abstractions that make concurrent code easier to write, mind you. I actually have plenty of ideas in that direction. It is just that I think this is something that can largely be done at the library level, without requiring too much help from the underlying programming language.