Re: For the AdaOS folks

["Warren W. Gay VE3WWG" <ve3wwg@NoSPAM.cogeco.ca>]

Nick Roberts wrote:
> "Warren W. Gay VE3WWG" <ve3wwg@NoSpam.cogeco.ca> wrote:
>>Its too bad that the L4 microkernel is not very secure yet. Apparently,
>>they are trying to fix that in L4ng, but it appears that they got so
>>focused on efficiency that they didn't get security. The efficiency of L4
>>otherwise makes it very attractive. 
> 
> The major problems with L4 that stopped me using it (or its interface) were
> its security problems, and the fact that it does not support process
> migration. 

By "process migration", are you referring to "thread-migration"?

In another related post, I offerred the following link for Mach:
   http://www.usenix.org/publications/library/proceedings/sf94/full_papers/ford.pdf

> Of course, that support could be added on top, in an extra layer;
> but obviously I felt there's no point in having that extra layer, I might
> just as well design a microkernel that supports it directly. There are a
> number of other minor problems with L4, too.

One of the remarks made in the above cited paper is that (paraphrasing)
QNX implements this by default, because of it doesn't support a queue (control
must immediately pass from sender to receiver). I know L4 works the same way,
but I also know that L4 talks of a Local-RPC. It would seem that Local-RPC
supports what you're looking for in L4.

> There are other problems. I want to write the microkernel (mostly) in Ada,

I support the concept ;-)

> so I would have to rewrite L4 anyway. It might have still been attractive to
> use the L4 interface, 

It is attractive to me because of the fact that they've ported it to
so many platforms. It is also very actively developed. But it has other
problems, obviously.

> but since there is only a binary interface and a C
> interface (no published Ada interface), this doesn't turn out to be such an
> advantage. Similar comments apply to other kernels, including Mach.

I looked at their API and it doesn't look too bad actually. They map
a number of things into MR (Message Registers), which would be real
easy to do in Ada, using the for Object'Address use ...;  A tiny bit
of _Asm would fix the rest. So I don't see that as much of a problem.
Define a package (or few) and the rest is a slam dunk.

>>I also don't like the way that L4 ties communications to threads. The Mach
>>approach is much easier to work with, with its single receiver and
>>multiple send and send-once rights.
> 
> This issue is related to the problem of process migration. Mach supports it
> by not tying communications to threads. But this creates all sorts of
> communications routing complexity inside the kernel. 

I am not entirely convinced of this. I am currently working with rtmk
(a stripped down Mach clone of sorts), and from the kernel side it
is not that difficult (I've had to mess with some of this ;-)

What it _does_ of course require is the need for formatted
messages. By this I mean that you cannot just send a stream of
bytes from one task to another, and include a Port somewhere in
the middle. The kernel must know when a port is being copied/moved,
so that it can play its kernel magic before the receiving task
gets the message (and thus inherit the port right). So from this
point of view, I would agree with you. But otherwise, I don't
see it as complex at all.

The thing is, depending upon design of course, you could easily
use thousands of send-rights, with little or no overhead. If you
must associate the message endpoint to a thread, barring lazy
allocation techniques, you impose all of the overheads that a
thread must impose (stacks, state etc.)  I just feel that the
port paradigm is more flexible for the OS designer (it costs
less). It is also conceptually cleaner to say you have two
end-points (ports) of a message queue, than saying you have
two-threads when threads aren't part of the abstraction.

> I think Bachar does
> much better: communications are thread-addressed, but a mechanism permits
> proxy threads (which can be in separate processes) to be transparently
> interposed when non-local communication is required (and removed when it is
> not). This way, you get simplicity and efficiency, but process migration is
> still properly supported.

But if you look at the Mach paper above, you can do the very
same thing with ports. Even without thread-migration, you can
interpose proxies. With thread-migration enabled RPC, you
achieve both and provide the OS designer a nicer interface.

For example, I might want a "notify port". Let's say I look up
the name server for the console service, and its not yet
registered (at boot time).  Rather than do a spin loop, the
client software obtains a notify port from the name server
first (it gets the receive port, while the ns keeps the send-once
right). Then the client tries to lookup the console, and if
its there, fine (it then destroys the notify port). If not,
then it waits for a message to be received from the ns on
the notify port, when something gets registered (something
has changed at the ns). Repeat if necessary.

If you tie that to threads, then things get messier. Ports
are easy to allocate (kernel just does some pointer arithmetic,
and increment reference counts), and easy to put into a
linked list for notifications.  A new thread isn't really
called for in this case, only the abstraction of a notify
port.

> Note that there are some other subtleties: all
> Bachar resources can be addressed (identified) in a way that can be exactly
> reproduced in the target machine when a process is migrated; 

Actually, Mach does this as well. However, they will tell you however,
that you must beware:  When writing a pager shared among multiple hosts
for example, you must be aware that the VM page size may vary according
to the type of equipment it is run on. For Intel 4K, but another platform
(I forget which), will use 8K.

Obviously, it will be difficult to migrate an Intel thread onto a PPC
platform ;-)

> most
> microkernels do not seem to support this functionality. Bachar provides
> functionality that allows any process to be frozen, dissected, reconstructed
> (on the target machine), and then reanimated. These functions can be done by
> a higher layer, but it makes more sense for them to be supported directly by
> the kernel.

I was planning to do this for my OS outside of the mk. The primary reason
for this is perhaps two-fold:

  1. I want to choose my own form of networking for the purpose
  2. It may require some other OS "coordination"

So I still favour a minimalist microkernel, but allowing for those nicer
abstractions above it.

>>   1. Clustered systems with all the same endianness and page size
>>   2. "Networked Clusters" of #1 groupings
>>
>>In this way, it would be efficiently possible to cluster similar
>>equipment, and yet still provide "one-ness" in the different computer
>>cases, with the necessary overhead and complexity. This would also permit
>>a tiered development approach.
> 
> Interesting idea. But I am assuming a network made up entirely of
> workstations sufficiently similar in architecture that a process can be
> migrated from any one to any other. In practice, to start with, that means a
> set of PCs.

Yes. You obviously have to track the capabilities in mixed Intel
environments for example, so you might have some host affinities
for i686 vs i386 etc. among the nodes. But yes, the general idea
is to allow the pager to page-fault a process over to another
cluster host, assuming that some sort of automatic load-balancing
can work out who it should page fault to.

To make this work, as I envision it, requires that all participating
nodes in a cluster see the file system the same way. IOW, there
can only be one root file system, which obviously is only local
to one node. To all other nodes, root will be access NFS-like (ie.
over the network cable). However, this is not to say that other
nodes will not have their own local disk, but they will of course
be logically mounted on top of root, or another node's filesystem(s).
This consistent file view is necessary to make tasks mobile to
any node in the cluster.

> Obviously we will support heterogeneous networking, but I think we might as
> well use IP to do this, and all the existing (reliable) technologies built
> on top of it. I'd like to support both IPv4 and IPv6.

Anything on top of AdaOS or whatever-OS, can do as it pleases. My
intitial concern is what the mk supports, and how can I get to
the longer range goal of supporting clusters.

Everyone collects old Intel boxes these days. Wouldn't it be
nice to be able to plug them all into a common hub, and boot
a common image and say "go forth and cluster!"  The goal is
obviously do this with minimal system administration with
no PhD required.

> A widely-distributed (heterogeneous architecture) filesystem and database
> system would be a very nice idea, however. I'm hoping that once we get a
> core AdaOS working, people will develop all sorts of nice ideas for it.

As soon as you go out of your own protected network, a whole new
can of worms come to roost. I'll let someone else do the
network security research side of things ;-)

>>I wonder if networking really belongs in the microkernel. This is
>>necessary for example if like Mach, you say that ports can speak to other
>>Mach machines on the network/cluster.
> 
> I'm sorry, I didn't mean to imply that Bachar will do any networking. It
> won't. It really will be a microkernel (maybe not as tiny as L4).
> 
> Networking will be implemented in device drivers and layers just above the
> kernel, especially in the ORB engine (called Avarus).

Yes, I agree.

>>I always get scoffed at for suggesting this, and I have no love for M$,
>>but the one idea they have had, which is useful, is the registry. I think
>>there are better ways of doing this however, but the general idea is good.
>>You can see that GNOME and Mozilla both use it, so it obviously satisfies
>>a need. I think any OS today that is going to support point and click
>>administration, is going to need some sort of a registry, even if it is
>>just layering it on a RieserFS.
> 
> Exactly. The main problem with the Registry is that it is not a proper
> database system.

Well, define "proper" and define what the "database system" solves that
the present system doesn't?

SQL databases are designed to deal with large volumes of similar data,
in SELECTs, joins etc. However, when you look at what goes into the
registry, a lot of it is custom, hierarchically structured information.

So I don't dispute there may be a better paradigm for this information,
but I haven't seen it yet.

Happy New Year!
-- 
Warren W. Gay VE3WWG
http://home.cogeco.ca/~ve3wwg