For the AdaOS folks

For the AdaOS folks

[Wes Groleau]

or anyone else with similar ambitions.

Read "Kill the operating System"
page 32 of September 2003 Technology Review

Not a prescription, but something to think about.

-- 
Wes Groleau

Always listen to experts.  They'll tell you
what can't be done and why.  Then do it.
                     -- Robert A. Heinlein

Re: For the AdaOS folks

[Florian Weimer]

* Wes Groleau:

> or anyone else with similar ambitions.
>
> Read "Kill the operating System"
> page 32 of September 2003 Technology Review
>
> Not a prescription, but something to think about.

Apparently, that's it:

<http://www.simson.net/clips/2003/2003.TR.09.KillTheOperatingSystem.pdf>

Re: For the AdaOS folks

[Georg Bauhaus]

Florian Weimer <fw@deneb.enyo.de> wrote:
: * Wes Groleau:
: 
:> or anyone else with similar ambitions.
:>
:> Read "Kill the operating System"
:> page 32 of September 2003 Technology Review
:>
:> Not a prescription, but something to think about.

If you would like to also try it, install Plan 9 :-)
The system does what the author describes in column 2 (among other
things.)


: Apparently, that's it:
: 
: <http://www.simson.net/clips/2003/2003.TR.09.KillTheOperatingSystem.pdf>

-- 
---
Microsoft Windows--a fresh perspective on information hiding

Re: For the AdaOS folks

[Florian Weimer]

* Georg Bauhaus:

> Florian Weimer <fw@deneb.enyo.de> wrote:
> : * Wes Groleau:
> : 
> :> or anyone else with similar ambitions.
> :>
> :> Read "Kill the operating System"
> :> page 32 of September 2003 Technology Review
> :>
> :> Not a prescription, but something to think about.
>
> If you would like to also try it, install Plan 9 :-)
> The system does what the author describes in column 2 (among other
> things.)

UNIX itself comes pretty close.  It's just not as graphical as
Hollywood likes it, and far less modal.  If he used a less
professional drawing program instead of Adobe Illustrator, the
embedding he describes already worked in 1995 (maybe even a few years
earlier if you were ready to use truly experimental software).

Of course, Garfinkel knows this, and I believe this article is just a
troll.

Re: For the AdaOS folks

[Wes Groleau]

Florian Weimer wrote:
> * Wes Groleau:
>>Read "Kill the operating System"
>>page 32 of September 2003 Technology Review
>>
>>Not a prescription, but something to think about.
> 
> Apparently, that's it:
> <http://www.simson.net/clips/2003/2003.TR.09.KillTheOperatingSystem.pdf>

Yes, same article.  Thanks.


-- 
Wes Groleau

    After the christening of his baby brother in church, Jason sobbed
    all the way home in the back seat of the car.  His father asked him
    three times what was wrong.  Finally, the boy replied, "That preacher
    said he wanted us brought up in a Christian home, and I wanted to
    stay with you guys."

Re: For the AdaOS folks

[Adrien Plisson]

Wes Groleau wrote:
> or anyone else with similar ambitions.
> 
> Read "Kill the operating System"
> page 32 of September 2003 Technology Review
> 
> Not a prescription, but something to think about.
> 

This is a nice article.

he could also go a LOT further: most people are running PCs, a computer 
architecture dating from an old age of computing. did anyone thought of 
a newer architecture ?

the interresting thing is : we can all think about it, but we also ARE 
ABLE TO DO IT. the concept of applications working together has been 
successfully treated (see Oberon <http://www.oberon.ethz.ch/>), the 
concept of another way of organizing files has been implemented (see 
TheBrain <http://www.thebrain.com/>), and many other new ideas already 
exist that this article does not cover...

so where is the problem ? why do we still use those weird old computing 
standards when we could have much much more ?

the answer is in the text: "Computing’s standard model owes its success 
to the economics of the computer industry". people are investing in 
development. they don't want to lose their investment, so they target 
what is widely used: game vendors target the windows market, cobol 
compiler vendors target the mainframe market... so we are in an infinite 
loop: game vendors target windows because game players owns most windows 
systems, but game players buys windows systems because it is easier to 
find a good game running on windows. and what about macintosh owners ? 
they CAN'T play recent games (quake II was released on macintosh only 
when quake III came out on windows). we are stuck with this model until 
someone is kind enough to take some risks.

so do investors never take any risk ?

yes they do. sometimes something really new, something revolutionnary 
comes out. it exists for some time, then disappear (see BeOS). those 
thing so not disappear because their ideas are bad, no. they disappear 
because people don't want to change: they don't want to risk investing 
in something new. they wait and see if it really is worth the 
investment. so investors that invested in the develoment of the new 
ideas don't get any return, the idea fails on the market, the investors 
abandon the idea, the idea disappear.

and then people say: "i was right not investing in this new idea: it 
disappeared !". what they don't understand is that it disappeared 
BECAUSE they did not invest in it.

so how to change this ? how to get new ideas accepted by all ? this is 
the real question to ask...

-- 
rien

Re: For the AdaOS folks

[Georg Bauhaus]

Adrien Plisson <aplisson-news@stochastique.net> wrote:
 
: development. they don't want to lose their investment, so they target 
: what is widely used:

I don't think there is a technical OS-related problem.
Windows has a lot under the hood. You can have a document centered
approach. You can forget about applications if you wish and instead
combine frames on one and the same virtual sheet of paper, where each
frame's content is managed by different modules. I.e. you choose data,
not programs (OO vs. application).
You can have "smart" drag and drop.

But all this isn't usually in the *showcase* when people are told
how they can use Windows. (One neighbor/collegue showing another, etc.)
So it is hardly known that there is OLE, COM, COM+, DCOM, ActiveX, I
forget the most recent name of the same OO concept. Have you ever seen
anyone recently moving a file with a mouse from one folder to another,
and not using File->Save As?
 I think it is just a lack "social visibility" of features that make
the "modern" features less used.

Example: You want to load a document in a text editor.
Typical solution: File->Open etc.
Possible solution: drag the file icon from the folder onto the editor window
Windows merges: The File->Open dialog *is* a folder view, with a slightly
  different look. Clever, eh?
(Programmer's IDE solution: the file in question is loaded *automatically*
  when you instruct the computer to show you the declaraion of some
  identifier.)

Unfortunately, the traditional appearance of file open folders etc.
doesn't invite people to quickly abandon the use of a specialised file
centered dialog window. They could because all the functionality *and*
*more* is already provided by the OS shell.

Secondary effect: programmers who do no know the OO side of folders and
icons (etc.)  start copying the traditional ways (File->Open). But they
do not copy the possibilities hidden in the object model of folders,
frames, etc.

For example, the File->* dialog in stand-alone Gimp 1.x was terribly
outdated technically. Why? Now Gimp is frequently associated with the
GNOME, and GNOME stands for GNU Network Object Model Environment, but
where are the system objects in Gimp's file manipulation dialogs? Do
they look like system folders, the same as hopefully in all other GNOME
programs? And then, why do we need File->* dialogs at all when there
is a system component for file manipulation?

The GNOME pages (Human Interface Guidelines) still speak of associations
between files and applications. Pity because there is so much more under
the hood than files and applications. The components view deserves
more attention I think. Maybe Mono will help to achieve this.

-- Georg

Re: Microkernels & Ada (Was for the AdaOS folks)

[Warren W. Gay VE3WWG]

Adrien Plisson wrote:
> Wes Groleau wrote:
> 
>> or anyone else with similar ambitions.
>>
>> Read "Kill the operating System"
>> page 32 of September 2003 Technology Review
>>
>> Not a prescription, but something to think about.
> 
> This is a nice article.

I agree with an earlier post that there
is a certain "troll factor" to this article.
Maybe that is required from time to time to get people
motivated enough to do something about it, or, for lack of
action, to at least discuss it.

> he could also go a LOT further: most people are running PCs, a computer 
> architecture dating from an old age of computing. did anyone thought of 
> a newer architecture ?

I think people are working on new architectures, but perhaps
not as actively as 10-20 years ago. I am finally relieved that
some hardware support is going into CPU instruction sets to
alleviate the buffer overrun problem. However, more needs to
be done.

I've recently developed an interest in microkernel
design and have been reading much of the
monolithic argument against microkernels etc. Much of
the argument is of course in support of Linux's design,
it seems, to justify their design approach.

The last article I read where Linus Torvalds critisized
microkernels was on the general grounds (paraphrased) that
"they promised to be more portable, but cannot be..". He
makes a point there, and I do see references to that in
the orignal literature. But portability was never the entire
point of microkernels.

Others criticize microkernels on the performance issue. This
point is harder to ignore, but I personally believe that this
also will become an insignificant issue with time. Each year
we see more powerfull CPU chips, and in a short period of
time, it looks like we may soon have dual, quad and
octo-core CPU chips. The efficiency factor becomes less
important with these types of advances in hardware.

The other side of the efficiency argument IMHO is this: if
people gripe about the inefficiency of Mach, and then run
Java desktops, then what are they griping about? Java is
inefficient too, yet some people find a reason to run
interpreted code, whether it be Java, Perl, TCL, Bash,
Python or BASIC. So the point I want to make is that if
microkernels lead to safer and better operating systems,
then why do we shy away from it on efficiency grounds?

Here is my last point on the efficiency of microkernel
design: Because the O/S is running in a bunch of user-mode
servers, all passing messages etc., in nice modular fashion,
it obviously does suffer from a number of costly VM TLB
flushes, to carry out a user service (each user context
switch). What I'd like to suggest, is that it is time
for hardware people to look at this problem and improve
upon the TLB flush problem. For example, cannot a CPU
carry more than one TLB set of resources, so that a
total flush is not always required? If increasing
amounts of cache is held on chip, why not set aside
more resources for say 8 TLB register sets to help
with microkernel user-mode context switches?

You can bet on the fact that if everyone used a microkernel
designed O/S today, Intel, AMD, IBM and others would be
working hard to fix some of that "overhead" problem. The
rest of the overhead may not be worth going after, but the
big problems would get attention.

I have recently started experimenting with Ada user
mode modules on top of a microkernel (GNAT). I would
like to see more projects like this in the future (I
plan to share the project and encourage participation).
So far, I have proved to myself that it can be done
with GNAT and with open sourced tools.

Perhaps someday, someone else will provide the microkernel
itself in Ada, in Mach or L4 microkernel like terms (or
something better).

> the interresting thing is : we can all think about it, but we also ARE 
> ABLE TO DO IT. 

My time is limited, but what I feel I _can_ do, is get an
O/S project started, using an _existing_ microkernel.
I suggested as much this last summer. After a month or
so, the idea bother me so much, that I had to try
something.

I firmly believe that others could do the same thing. The
point is that more _research_ needs to be done on _SAFE_
and _SECURE_ operating systems. We have enough fully
featured (M$) and fast (*NIX) systems, but I'd like to
see a secure and rootless O/S for starters.

Anyway, I'd like to see more O/S research that employed
Ada, and to repeat myself, one way to do this is to
start with a microkernel, and use Ada on top of it.

> so where is the problem ? why do we still use those weird old computing 
> standards when we could have much much more ?

Two problems, IMHO:

   1. The user wants one interface, and programs need another (programs
      don't like fuzzy file name references etc., yet for users this
      can be very helpful).

   2. People are often stuck on what they know (look at all of
      the C/C++-like languages that get invented every year).

While POSIX is good for UNIX and portability, when starting a
new O/S, I think you need to say "who cares about POSIX?" Work
from the requirements back to the implementation. POSIX embodies
a lot of good ideas, and I don't suggest that we want to be
different without good reasons for it, but I think some fresh
thinking would be welcome.

> the answer is in the text: "Computing’s standard model owes its success 
> to the economics of the computer industry". people are investing in 
> development. they don't want to lose their investment, so they target 
> what is widely used: 

This wasn't true when UNIX was being initially developed. It
need not be the case for a new design. The design just needs
to solve a problem. For example, a secure O/S is needed for
firewalls (hence my interest in a rootless O/S).

It doesn't need to take the world by storm either. It took
years before suddenly everyone realized they needed a *NIX
server (thanks to the Internet & www). I am sure that other
opportunities are in the wing.

> so do investors never take any risk ?

Take your savings and invest it, and then you'll know what
they are thinking. Everyone has a limited tolerance for
risk, especially when it is your investment dollar. But
where there is risk, there is opportunity also, if for
no other reason than nobody wants to go there.

> yes they do. sometimes something really new, something revolutionnary 
> comes out. it exists for some time, then disappear (see BeOS). those 
> thing so not disappear because their ideas are bad, no. they disappear 
> because people don't want to change: they don't want to risk investing 
> in something new. they wait and see if it really is worth the 
> investment. so investors that invested in the develoment of the new 
> ideas don't get any return, the idea fails on the market, the investors 
> abandon the idea, the idea disappear.

With the desktop, you may be right. But that is not the only
frontier for an O/S. There are millions of appliances
running operating systems, many of them Linux now.

With a microkernel approach, it _is_ possible to run
multiple environments at the same time (Linux, *BSD, & maybe
even M$ win32). Look for new ways to allow the user to migrate.
Try things, many things.

> so how to change this ? how to get new ideas accepted by all ? this is 
> the real question to ask...

Well, the last thing I'll say is that if nobody takes the risk
to produce a new O/S, then no one will risk using it ;-)
--
Warren W. Gay VE3WWG
http://home.cogeco.ca/~ve3wwg

Re: Microkernels & Ada (Was for the AdaOS folks)

[Adrien Plisson]

Warren W. Gay VE3WWG wrote:
> I agree with an earlier post that there
> is a certain "troll factor" to this article.

indeed, the proof is we are talking about it !

> Maybe that is required from time to time to get people
> motivated enough to do something about it, or, for lack of
> action, to at least discuss it.

right ! and as we are discussing it, are we doing soemthing about it ?
on your side, i know you do. unfortunately, on my side i only have 
ideas. they surely need a lot more thinking to get a project started.

> I've recently developed an interest in microkernel
> design and have been reading much of the
> monolithic argument against microkernels etc. Much of
> the argument is of course in support of Linux's design,
> it seems, to justify their design approach.

well, it would have felt strange if it didn't... but this is holy war 
afterall.

 > [...]
> Others criticize microkernels on the performance issue. This
> point is harder to ignore, but I personally believe that this
> also will become an insignificant issue with time. Each year
> we see more powerfull CPU chips, and in a short period of
> time, it looks like we may soon have dual, quad and
> octo-core CPU chips. The efficiency factor becomes less
> important with these types of advances in hardware.

that is part of the problem. we count to much on CPU power. when 
computers were slow, we had powerful and efficient software. now 
computers have 512 times more memory and are more than 300 times faster. 
what i see is: software are getting bigger and bigger with no really 
added features, and they perform almost as slowly as 20 years ago, if 
not slower. in those dark ages, developpers were taking great care not 
to cripple software with useless features and wrote efficient software.
nowadays, developpers don't think a bit when designing software and rely 
on CPUs power growth to take care it.

> [...]
> You can bet on the fact that if everyone used a microkernel
> designed O/S today, Intel, AMD, IBM and others would be
> working hard to fix some of that "overhead" problem. The
> rest of the overhead may not be worth going after, but the
> big problems would get attention.

and what about them designing something new too ? Intel once had very 
good ideas with the i432. why don't they try again ?

> [...]
> My time is limited, but what I feel I _can_ do, is get an
> O/S project started, using an _existing_ microkernel.
> I suggested as much this last summer. After a month or
> so, the idea bother me so much, that I had to try
> something.

i'm thinking hard on it too. but i don't know if i will be able to 
really try it.

> I firmly believe that others could do the same thing. The
> point is that more _research_ needs to be done on _SAFE_
> and _SECURE_ operating systems. We have enough fully
> featured (M$) and fast (*NIX) systems, but I'd like to
> see a secure and rootless O/S for starters.

i disagree here ! i am facing people everyday that don't know how to use 
their computer. they are just lost in the middle of all the features 
available to them. this kind of people need something simple and 
efficient. an OS which provides them with the tools they need to do what 
they WANT to do, and nothing more. in the vein of the macintosh some 
years ago...

(and if Man was not that violent we wouldn't need any security feature)

 > [...]
> While POSIX is good for UNIX and portability, when starting a
> new O/S, I think you need to say "who cares about POSIX?" Work
> from the requirements back to the implementation. POSIX embodies
> a lot of good ideas, and I don't suggest that we want to be
> different without good reasons for it, but I think some fresh
> thinking would be welcome.

i do often think of something NEW from scratch. if i could, i would even 
reinvent networks and protocols... but i'm a bit of an extremist on this 
point.

> This wasn't true when UNIX was being initially developed. It
> need not be the case for a new design. The design just needs
> to solve a problem. For example, a secure O/S is needed for
> firewalls (hence my interest in a rootless O/S).

you are right ! we should stop developping full-featured monsters we 
can't manage, let's keep things simple.

> It doesn't need to take the world by storm either. It took
> years before suddenly everyone realized they needed a *NIX
> server (thanks to the Internet & www). I am sure that other
> opportunities are in the wing.

i don't see any... as you said earlier: "People are often stuck on what 
they know". so will they really create something new as internet and the 
www was ? but i hope one day we will see something changing. or maybe we 
will create our own oppotunity !

> [...]
> Well, the last thing I'll say is that if nobody takes the risk
> to produce a new O/S, then no one will risk using it ;-)

this makes a great quote !

-- 
rien

    "if nobody takes the risk to produce a new O/S,
     then no one will risk using it"
                                      Warren W. Gay

Re: Microkernels & Ada (Was for the AdaOS folks)

[Warren W. Gay VE3WWG]

Adrien Plisson wrote:
> Warren W. Gay VE3WWG wrote:
> 
>> Others criticize microkernels on the performance issue. This
>> point is harder to ignore, but I personally believe that this
>> also will become an insignificant issue with time. Each year
>> we see more powerfull CPU chips, and in a short period of
>> time, it looks like we may soon have dual, quad and
>> octo-core CPU chips. The efficiency factor becomes less
>> important with these types of advances in hardware.
> 
> that is part of the problem. we count to much on CPU power. when 
> computers were slow, we had powerful and efficient software. now 
> computers have 512 times more memory and are more than 300 times faster. 
> what i see is: software are getting bigger and bigger with no really 
> added features, and they perform almost as slowly as 20 years ago, if 
> not slower. in those dark ages, developpers were taking great care not 
> to cripple software with useless features and wrote efficient software.
> nowadays, developpers don't think a bit when designing software and rely 
> on CPUs power growth to take care it.

I don't disagree completely on this, because there certainly is
an amount of useless featuritis in products like M$ Office etc.
This is mostly a matter of getting people to pay again for
software they already have (marketing).

But some of the bloat is justifiable. A developer's time is
costly, and because CPU time is cheaper now, it doesn't make
sense to have developers counting bytes unless it multiplies
into sizable quantities (I'm talking of the general purpose
(GP) programming areas - not embedded programming). Another
example is that I run my programs with all of the Ada checks
and assertions on, unless efficiency is more important. I
can afford to do that because CPUs are fast enough that I
don't have to care.

>> You can bet on the fact that if everyone used a microkernel
>> designed O/S today, Intel, AMD, IBM and others would be
>> working hard to fix some of that "overhead" problem. The
>> rest of the overhead may not be worth going after, but the
>> big problems would get attention.
> 
> and what about them designing something new too ? Intel once had very 
> good ideas with the i432. why don't they try again ?

If everyone started using Ada based O/Ss, perhaps the idea
would get ressurrected at Intel. The trouble with radical
hardware designs, is they must pay off in a shorter period
of time, and of course need a "market". I can't see that
happening until its O/S starts to "demand" better/matching
hardware.

>> I firmly believe that others could do the same thing. The
>> point is that more _research_ needs to be done on _SAFE_
>> and _SECURE_ operating systems. We have enough fully
>> featured (M$) and fast (*NIX) systems, but I'd like to
>> see a secure and rootless O/S for starters.
> 
> i disagree here ! i am facing people everyday that don't know how to use 
> their computer. they are just lost in the middle of all the features 
> available to them. this kind of people need something simple and 
> efficient. an OS which provides them with the tools they need to do what 
> they WANT to do, and nothing more. in the vein of the macintosh some 
> years ago...

I won't disagree with this point. Obviously, there is a
wide range of users, when it comes to capability. Given
the needs of new/simple users, perhaps there is an
opportunity to do something here.

Your mention of the old "MacIntosh" is a good point. My late
father-in-law left DOS kicking and screaming, and eventually
got the hang of Windows. He found DOS much easier.

The problem IMHO, is rooted in the range of features (choices),
that a user must be aware of. It is too much for a new user.
Its like the old joke about Model-T Fords where you had the
choice of colour as "black, black or black". Too many choices
(options), bewilder new users. Or here's another bad example:
McAfee antivirus cannot be setup ahead of time, so the
poor grandmother using the PC is later prompted about what
should be allowed on TCP/IP ports. How is she to know how
to answer those prompts?

The old MacIntosh only allowed the user to do one thing at
a time. You didn't have to teach the Mac user about minimising,
multi-processing, and how to get those minimised windows
back etc.

So yes, I would agree that "user interface" designs can be
improved. Once such improvement might be a "novice mode", to
simplify what the user must know. Do single-tasking for
novice users perhaps.

> (and if Man was not that violent we wouldn't need any security feature)

True, and if man didn't steal, you wouldn't need car keys
either. But we must accept the given situation and provide
solutions for them.

>> This wasn't true when UNIX was being initially developed. It
>> need not be the case for a new design. The design just needs
>> to solve a problem. For example, a secure O/S is needed for
>> firewalls (hence my interest in a rootless O/S).
> 
> you are right ! we should stop developping full-featured monsters we 
> can't manage, let's keep things simple.

A large part of it is about "control". Going back to DOS,
when you only had maybe 50 files, it was easy to know and
administer that environment inside and out. When I do a
virus scan on my family computer and I see 127,000 files.
How can I know about spyware files, except that I run a
spyware utility. How do I know the utility got them all?
Take away IE cache files, and it is still over 37,000 files
to manage.

But # of files is not the whole issue. I think we have to
accept that there will be increasing amounts of "software"
as we demand more functionality. But how do we do this and
make it manageable? Perhaps DLL files (and share libaries)
should be put into archives (ar). Win32 could start
applying real permissions on a better organized file
systems, as another example. These are nearly trivial
things to do technically, yet they are left undone.

M$ needs to stop with this "gee whiz, look at what we
can do" and be more sensible. People told them that
running Active-X controls in IE was bad news a
long time ago, yet they refuse to address that issue.
This is clearly a "trade safety for features" issue
(one could argue that marketing wins over safety).

>> It doesn't need to take the world by storm either. It took
>> years before suddenly everyone realized they needed a *NIX
>> server (thanks to the Internet & www). I am sure that other
>> opportunities are in the wing.
> 
> i don't see any... as you said earlier: "People are often stuck on what 
> they know". so will they really create something new as internet and the 
> www was ? but i hope one day we will see something changing. or maybe we 
> will create our own oppotunity !

Linus created his own opportunity, so why not. Thomas Edison
had many failures before he got a successful light bulb. By
trying ideas in new operating systems, you will probably find
out many things that don't work well. But along the way, you
might hit on some winning new ideas.
--
Warren W. Gay VE3WWG
http://home.cogeco.ca/~ve3wwg

Re: For the AdaOS folks

[Nick Roberts]

Wes Groleau <groleau+news@freeshell.org> wrote:

> or anyone else with similar ambitions.
> 
> Read "Kill the operating System" page 32 of September 2003 Technology
> Review
> 
> Not a prescription, but something to think about.

It's interesting that Garfinkel's comments correspond to many ideas that I
have had -- some in a vaguer form than others -- during my musings on OS
design.

One of the biggest challenges is to design an OS that works in a new and
better way, and yet is still capable of leveraging the power of an existing
software base.

One of the reasons, perhaps the biggest reason, why I decided on using a
microkernel was for security. I read the 'orange book' (the TCSEC) and
others which described how the world of computer science (especially within
governemntal and military sectors in the USA and the UK) foresaw the
evolution of computer technology. Remember that this is early 1990s, before
the domination of Microsoft. The overwhelming consensus was on a microkernel
based design, because this allowed the 'trusted computing base' (TCB) -- the
part of the overall system's software that had to be trusted not to subvert
security -- to be minimised.

The consensus was also on a 'fully distributed' system -- a network of
computers (always termed 'workstations' regardless of whether they had
screens, keyboards, etc.) that operated exactly like a single big computer,
from the point of view of normal users -- and so AdaOS is a
microkernel-based, fully distributed design.

However, I intend to design the microkernel so that it compromises on
minimality (unlike some other microkernel designs) to achieve reasonable
efficiency (whilst removing much of unnecessary detritus that bogged down
experimental microkernels), and I intend to ensure that administrative
division of the network is fully supported, as well as network partioning
robustness (if some workstations go down, the rest still work).

For a very long time, IBM mainframes have supported (very efficient)
programmatic access to a system database, as an alternative to file-based
data storage. I would like provide both forms of storage in AdaOS, in
addition to 'persistent objects' in some form.

At the moment, the place where we would like people to discuss AdaOS issues
(or anything at all related to OS design) is:

   http://adaos.multiply.com/

You have to register with Multiply, but that is a very simple process (and
you seem to be protected from junk mail etc.).

-- 
Nick Roberts

Re: For the AdaOS folks

[Warren W. Gay VE3WWG]

Nick Roberts wrote:
> Wes Groleau <groleau+news@freeshell.org> wrote:
>>or anyone else with similar ambitions.
>>
>>Read "Kill the operating System" page 32 of September 2003 Technology
>>Review
>>
>>Not a prescription, but something to think about.
...
> One of the reasons, perhaps the biggest reason, why I decided on using a
> microkernel was for security. I read the 'orange book' (the TCSEC) and
> others which described how the world of computer science (especially within
> governemntal and military sectors in the USA and the UK) foresaw the
> evolution of computer technology. Remember that this is early 1990s, before
> the domination of Microsoft. The overwhelming consensus was on a microkernel
> based design, because this allowed the 'trusted computing base' (TCB) -- the
> part of the overall system's software that had to be trusted not to subvert
> security -- to be minimised.

Its too bad that the L4 microkernel is not very secure yet. Apparently,
they are trying to fix that in L4ng, but it appears that they got so
focused on efficiency that they didn't get security. The efficiency of
L4 otherwise makes it very attractive. The following link describes
some of the things that they are looking at in L4ng :

   http://i30www.ira.uka.de/teaching/coursedocuments/105/l4ng-apr28.pdf

It still seems unnecessarily more complex than the Mach port approach.
And of course, and Ada microkernel would be better by definition ;-)

I also don't like the way that L4 ties communications to threads. The
Mach approach is much easier to work with, with its single receiver
and multiple send and send-once rights.

> The consensus was also on a 'fully distributed' system -- a network of
> computers (always termed 'workstations' regardless of whether they had
> screens, keyboards, etc.) that operated exactly like a single big computer,
> from the point of view of normal users -- and so AdaOS is a
> microkernel-based, fully distributed design.

I've been mulling this over myself. Unfortunately, as soon as you say
that all computers must work together, you introduce all sorts of
problems and overheads. Endianness and page size are two issues,
between two computers. I have been wondering if this shouldn't be
tiered:

   1. Clustered systems with all the same endianness and page size
   2. "Networked Clusters" of #1 groupings

In this way, it would be efficiently possible to cluster similar
equipment, and yet still provide "one-ness" in the different
computer cases, with the necessary overhead and complexity. This
would also permit a tiered development approach.

> However, I intend to design the microkernel so that it compromises on
> minimality (unlike some other microkernel designs)

Yes. Mach is too fat. They shouldn't have included device drivers
for example.

> to achieve reasonable
> efficiency (whilst removing much of unnecessary detritus that bogged down
> experimental microkernels), and I intend to ensure that administrative
> division of the network is fully supported, as well as network partioning
> robustness (if some workstations go down, the rest still work).

I wonder if networking really belongs in the microkernel. This is
necessary for example if like Mach, you say that ports can speak
to other Mach machines on the network/cluster.

However, you might want to take the tact that networking is done
outside of the microkernel. This way, in situations where you don't
want networking (embedded systems), it need not be there. Leave the
networking up to the user of the microkernel. This would be my
preference anyway. It also leaves it up to the OS designer what
type of networking will be supported.

Reading below, perhaps you are speaking collectively for AdaOS, for
microkernel and OS.

> For a very long time, IBM mainframes have supported (very efficient)
> programmatic access to a system database, as an alternative to file-based
> data storage. I would like provide both forms of storage in AdaOS, in
> addition to 'persistent objects' in some form.

I always get scoffed at for suggesting this, and I have no love for M$,
but the one idea they have had, which is useful, is the registry. I
think there are better ways of doing this however, but the general idea
is good. You can see that GNOME and Mozilla both use it, so it obviously
satisfies a need. I think any OS today that is going to support point
and click administration, is going to need some sort of a registry,
even if it is just layering it on a RieserFS.

--
Warren W. Gay VE3WWG
http://home.cogeco.ca/~ve3wwg

Re: For the AdaOS folks

[Dmitry A. Kazakov]

On Thu, 30 Dec 2004 08:58:23 -0500, Warren W. Gay VE3WWG wrote:

> Nick Roberts wrote:

>> However, I intend to design the microkernel so that it compromises on
>> minimality (unlike some other microkernel designs)
> 
> Yes. Mach is too fat. They shouldn't have included device drivers
> for example.

Implementations of or the generic interface for? I think the latter would
be a mistake. DOS->Windows tried this path. MS consistently refused to
specify device classes other than hard drive and keyboard.

>> to achieve reasonable
>> efficiency (whilst removing much of unnecessary detritus that bogged down
>> experimental microkernels), and I intend to ensure that administrative
>> division of the network is fully supported, as well as network partioning
>> robustness (if some workstations go down, the rest still work).
> 
> I wonder if networking really belongs in the microkernel. This is
> necessary for example if like Mach, you say that ports can speak
> to other Mach machines on the network/cluster.

That is an interesting question. I think that the kernel should be able to
tunnel its requests through higher order protocols.

> However, you might want to take the tact that networking is done
> outside of the microkernel. This way, in situations where you don't
> want networking (embedded systems), it need not be there.

There are lot of embedded systems which need networking, starting from
boards interconnected through dual ported RAM, to various sensors and
actors devices attached via Ethernet.

> Leave the
> networking up to the user of the microkernel. This would be my
> preference anyway. It also leaves it up to the OS designer what
> type of networking will be supported.
> 
> Reading below, perhaps you are speaking collectively for AdaOS, for
> microkernel and OS.
> 
>> For a very long time, IBM mainframes have supported (very efficient)
>> programmatic access to a system database, as an alternative to file-based
>> data storage. I would like provide both forms of storage in AdaOS, in
>> addition to 'persistent objects' in some form.
> 
> I always get scoffed at for suggesting this, and I have no love for M$,
> but the one idea they have had, which is useful, is the registry.

It might look attractive and is definitely better than Unix chaos, but the
concept of registry is not good. Try to reformat an Windows box and restore
all settings of say MS VC back.

Registry makes program's settings a property of the local machine or user.
The problem is that tree view. It should be at least relational. But better
would to have different views.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: For the AdaOS folks

[Warren W. Gay VE3WWG]

Dmitry A. Kazakov wrote:
> On Thu, 30 Dec 2004 08:58:23 -0500, Warren W. Gay VE3WWG wrote:
>>Nick Roberts wrote:
>>>However, I intend to design the microkernel so that it compromises on
>>>minimality (unlike some other microkernel designs)
>>
>>Yes. Mach is too fat. They shouldn't have included device drivers
>>for example.
> 
> Implementations of or the generic interface for? I think the latter would
> be a mistake. DOS->Windows tried this path. MS consistently refused to
> specify device classes other than hard drive and keyboard.

I'm not sure exactly what you're supporting here (clearly not M$), but
IMHO, the device driver(s) can be written on top of the microkernel,
and need to be inside it. This also helps to keep the mk small.

>>>to achieve reasonable
>>>efficiency (whilst removing much of unnecessary detritus that bogged down
>>>experimental microkernels), and I intend to ensure that administrative
>>>division of the network is fully supported, as well as network partioning
>>>robustness (if some workstations go down, the rest still work).
>>
>>I wonder if networking really belongs in the microkernel. This is
>>necessary for example if like Mach, you say that ports can speak
>>to other Mach machines on the network/cluster.
> 
> That is an interesting question. I think that the kernel should be able to
> tunnel its requests through higher order protocols.

Yes, they can be provided on top of the microkernel, as you seem to
be suggesting.

>>However, you might want to take the tact that networking is done
>>outside of the microkernel. This way, in situations where you don't
>>want networking (embedded systems), it need not be there.
> 
> There are lot of embedded systems which need networking, starting from
> boards interconnected through dual ported RAM, to various sensors and
> actors devices attached via Ethernet.

I am not disputing that. But that is different than saying every
embedded device needs it. So if you want to save some memory, then
the first thing that can be dumped (if the option exists) is
networking if you don't require it. If it is included in a
given mk, it should be optional.

However, IMHO, the microkernel should be small as possible, and
leave networking up to the modules outside of it. After all, some may
perfer an IPv6 only version of networking for example (in light
of China's recent network announcment). In the end, I don't want
to be "stuck" with what a brand of microkernel gives me. Just
give me the basics and let me decide what is included.

>>I always get scoffed at for suggesting this, and I have no love for M$,
>>but the one idea they have had, which is useful, is the registry.
> 
> It might look attractive and is definitely better than Unix chaos, 

So we agree that it is an improvement over little text files that
you must parse/edit etc..

> but the
> concept of registry is not good. Try to reformat an Windows box and restore
> all settings of say MS VC back.

You base your registry criticisms upon M$ _implementation_ of
a registry. But consider this:

If each registry item were a file (on a RieserFS so there is little
or no waste), then you can backup, secure and restore settings
just like any other file. You can do so with all of those familiar
tools, including tar and cpio.

> Registry makes program's settings a property of the local machine or user.
> The problem is that tree view. It should be at least relational. But better
> would to have different views.

I don't believe it need be treated any differently than a file
system. To support a registry on the RieserFS for example, one
merely need to add a nice programming API to make it easier for
programs to query and set values. You can still use links/symlinks
(if you must) etc.
--
Warren W. Gay VE3WWG
http://home.cogeco.ca/~ve3wwg

OT: Mach Ports (For the AdaOS folks)

[Warren W. Gay VE3WWG]

Dennis Lee Bieber wrote:
> On Thu, 30 Dec 2004 11:30:35 -0500, "Warren W. Gay VE3WWG"
> <ve3wwg@NoSpam.cogeco.ca> declaimed the following in comp.lang.ada:
>>However, IMHO, the microkernel should be small as possible, and
>>leave networking up to the modules outside of it. After all, some may
>>perfer an IPv6 only version of networking for example (in light
>>of China's recent network announcment). In the end, I don't want
>>to be "stuck" with what a brand of microkernel gives me. Just
>>give me the basics and let me decide what is included.

> 	Pardon my slipping in, but...

Discussion is always welcome here.

> 	The more I read in this thread, the more it sounds like my old
> Amiga.

I believe however, there is at least one major difference ;-)

> 	The Amiga EXEC handled only a very small list of functions:
> memory management (the weak point for the Amiga -- flat, shared, memory
> space; no process protection),

This _is_ the major difference. Mach, L4 and other microkernels
provide memory management for you, such that the mk itself,
which is kept very small, is the only piece that runs in "real
mode".

All other modules run in user-mode, preventing security
breaches and corruption from programming errors. Part of the
idea is to keep "real-mode code" to a minimum.

 > task scheduling (including
> semaphore/signal stuff) and process creation, shared library loading
> (note: not the I/O to load the library into memory, but rather the
> linking of a library into system lists), and IPC (message ports:
> essentially linked-lists on which a task may insert a "message" -- a
> structure with a pointer to the task specific data structure; this is
> why the flat memory model was pretty much needed).
> 
> 	All I/O, including keyboard/display, relied upon
> libraries/device drivers which had to be loaded (yes, it sounds
> chicken&egg here -- the Amiga ROMs did contain more than just EXEC, to
> include the core file system handlers, device drivers, etc. but they
> were logically independent units and could be replaced at run-time by
> libraries with identical interfaces).
> 
> 	Everything was done via message ports... An application disk
> read request became a message to the file handler, which then generated
> a message for the disk device driver, etc...

I find that the Mach message interface to be especially
convenient. I think that this one Mach concept is
perhaps the most brilliant part of that work.

Now, one might wonder how a Mach message queue (essentially that
is what it is within the kernel) differs from a UNIX (SysV IPC)
message queue? The differences include the following:

   SysV IPC Message Queue               Mach
   ===============================      =================================
   - can have multiple receivers        - only 1 receiving port per queue
                                          (hence only 1 receiver)
   - can have multiple senders          - same
   - does not have a send-once          - supports a send-once "right"
   - access is limited only by          - you must be "given" a right
     IPC permissions                      (or must have access to the
                                           "task" port that you are going
                                           to acquire a "right")
   - Can configure how much a           - In Mach, I think you can
     queue can hold                        configure between 1-12
                                           "message backlog"
   - Not supported                      - Can force one message (only)
                                          if a queue is full
   - Not supported                      - Dead port notification (when
                                          there are no receivers)
   - The IPC "queue" is removed         - Only the "receiver" port is
                                          destroyed to destroy the queue
   - Messages can be prioritized        - No priorities : FIFO

What makes Mach's port mechanism secure is that you cannot send to
a port, unless you are _given_ a "send-right" (as send-only port to
the message queue). There is one exception WRT the ownership of a
task port (see below).

In essence, there can only be _one_ "receive" right - ie. only
one task can own this receive-right (aka port) (a "task" in
Mach, is the address space, with or without threads). You cannot
duplicate a receive right, but you can "move" it to another task,
by "sending" it to another task, on another message queue.

A send-right OTOH, can be duplicated as often as you like (it only
increments the send reference count on the queue). A send-once
right works the same way, except that you can only use that port
to obviously send a message once (all other duplicated send-once
rights become dead ports, after the message is sent on any of them,
IIRC).

So how does task B get a send right to a port in task A?  It must
have a send-right delivered to task B, by a message. But there is
a bit of a chicken and egg problem here. To solve that,
typically, one module serves as a "name server".

When task B is created by the parent task, a send-right is
"inserted into task B". This allows it to query the
name server; to acquire a send right to A (provided that A has
registered a send right in the name server under some agreed
upon key). The name server duplicates the registered send-right
and "moves" it by message to the caller (client).

The "exception" that I refered to above is this: The parent
"task" receives a send-right of the child task it creates. As
the parent task, it is free to insert ports or to remove ports,
as long as it holds a send-right to that child task's task
port (by which other operating system calls are possible).
Once the parent task closes the child-task's task port
(send-right), it then cannot meddle any further (it cannot
insert/remove any further ports etc.).

So the bootstrap process might make the first "task" (module)
the name server. All other bootstrapped modules would have
a send-right inserted into it, which allows them to contact
the name server. The name server acts as a runtime registry
to accept registrations of services and allows clients to
query and obtain send-rights to these services.

What's neat about this is that it is secure. Knowing a
port number in task A as 87, is of no use to task B. The
port numbers are local to the task, so there is no
global ID that can be used to access the port (service).
A send-right in A as 87, might be 63 in task B, and 45
in task C. The port numbers are only for local task use.
This is one of the problems that the L4 mk has, since its
port IDs are global and can be guessed/determined. Knowing
the ID in L4, gains you access enough to send messages.

I should explain the send-once right: sometimes you
want some follow-up notification, but you only expect
one message. A send-once right guarantees this. This
helps the receiver as well as other possible sources of
the send-once (they all become dead-ports, once someone
uses it to send a message). Thus, a send-once right
guarantees that only one message is queued, and at the
same time, notifies all other cloned send-once rights that
they are now dead (and thus are no longer needed).

Mach 3.x dispensed with a complicated "ownership-right",
which existed in addition to the receiver port in
earlier releases.

I assume that the Mach designers eventually came to the
conclusion that it was a concept that was redundant. Certainly
with its removal, the port concepts become simpler (now when
a receive right is destroyed, all send-rights become
dead ports, notifying all clients of a service, that the
service is now "offline").

Anyway, if you want a deeper understanding of all this good
stuff, there is a book you can get used that does an excellent
job at describing Mach concepts (ports, tasks, threads and
memory objects):

Programming Under Mach:
http://www.amazon.com/exec/obidos/tg/detail/-/0201527391/103-1295179-1094257?v=glance

Unfortunately, it does centre on Mach 2.5, but mentions
Mach 3.x changes (it will cover the important 3.x features).

You will also have to overlook a certain amount of "UNIX
doesn't have threads" discussion, since it was published
in 1992. Otherwise, this is a recommended read for anyone
who is contemplating messing with microkernels, particularly
of the Mach variety.
--
Warren W. Gay VE3WWG
http://home.cogeco.ca/~ve3wwg

Re: For the AdaOS folks

[Dmitry A. Kazakov]

On Thu, 30 Dec 2004 11:30:35 -0500, Warren W. Gay VE3WWG wrote:

> Dmitry A. Kazakov wrote:
>> On Thu, 30 Dec 2004 08:58:23 -0500, Warren W. Gay VE3WWG wrote:
>>>Nick Roberts wrote:
>>>>However, I intend to design the microkernel so that it compromises on
>>>>minimality (unlike some other microkernel designs)
>>>
>>>Yes. Mach is too fat. They shouldn't have included device drivers
>>>for example.
>> 
>> Implementations of or the generic interface for? I think the latter would
>> be a mistake. DOS->Windows tried this path. MS consistently refused to
>> specify device classes other than hard drive and keyboard.
> 
> I'm not sure exactly what you're supporting here (clearly not M$), but
> IMHO, the device driver(s) can be written on top of the microkernel,
> and need to be inside it. This also helps to keep the mk small.

So it is about implementations then? But that is not what makes the kernel
fat. The problem is that OS should have generic interfaces for all kinds of
devices. They should be in. You can try to solve it by declaring
"everything is a file": UNIX (tm), or by doing nothing Windows (tm), but in
both cases the result is pretty same. I think only a native OO design
*exposed* in the kernel interface can help.

>>>>to achieve reasonable
>>>>efficiency (whilst removing much of unnecessary detritus that bogged down
>>>>experimental microkernels), and I intend to ensure that administrative
>>>>division of the network is fully supported, as well as network partioning
>>>>robustness (if some workstations go down, the rest still work).
>>>
>>>I wonder if networking really belongs in the microkernel. This is
>>>necessary for example if like Mach, you say that ports can speak
>>>to other Mach machines on the network/cluster.
>> 
>> That is an interesting question. I think that the kernel should be able to
>> tunnel its requests through higher order protocols.
> 
> Yes, they can be provided on top of the microkernel, as you seem to
> be suggesting.

Yes, but networking should be known to the kernel as something it can work
with. So you cannot completely throw it away.

>>>However, you might want to take the tact that networking is done
>>>outside of the microkernel. This way, in situations where you don't
>>>want networking (embedded systems), it need not be there.
>> 
>> There are lot of embedded systems which need networking, starting from
>> boards interconnected through dual ported RAM, to various sensors and
>> actors devices attached via Ethernet.
> 
> I am not disputing that. But that is different than saying every
> embedded device needs it. So if you want to save some memory, then
> the first thing that can be dumped (if the option exists) is
> networking if you don't require it. If it is included in a
> given mk, it should be optional.

I'm not sure if that could be possible. I have designed not an OS, but a
large middleware project. I tried to do the thing you describe, i.e. to
isolate all networking and remove it from the system core. It turned to be
impossible. In the final version it appears as a "normal driver", but it
still has some secret wires leading to the core. So I failed. BTW, there
were other things which seemed to be no less coupled. For example, I
managed to isolate the system data logger. So in principle, I agree with
you, but...

> However, IMHO, the microkernel should be small as possible, and
> leave networking up to the modules outside of it. After all, some may
> perfer an IPv6 only version of networking for example (in light
> of China's recent network announcment). In the end, I don't want
> to be "stuck" with what a brand of microkernel gives me. Just
> give me the basics and let me decide what is included.

Agree. Protocols must be decoupled from the system, though that is easier
to do, than to remove networking as a whole. Though it is related to an
interesting OO problem. When protocol implementation is buried somewhere in
an inheritance path, then it becomes difficult to switch from one protocol
to another. It looks like abstraction inversion.

>>>I always get scoffed at for suggesting this, and I have no love for M$,
>>>but the one idea they have had, which is useful, is the registry.
>> 
>> It might look attractive and is definitely better than Unix chaos, 
> 
> So we agree that it is an improvement over little text files that
> you must parse/edit etc..

Sure

>> but the
>> concept of registry is not good. Try to reformat an Windows box and restore
>> all settings of say MS VC back.
> 
> You base your registry criticisms upon M$ _implementation_ of
> a registry. But consider this:
> 
> If each registry item were a file (on a RieserFS so there is little
> or no waste), then you can backup, secure and restore settings
> just like any other file. You can do so with all of those familiar
> tools, including tar and cpio.

The problem is not an ability to backup, in UNIX one can backup /etc, /var
... The problem is complexity. There are dependencies between repository
objects, which cannot be mapped to a simple tree structure, whether that be
chaotic files or registry. Consider a simple diamond diagram. Application A
uses B and C, they in turn use D. This cannot be represented by a tree. So
you need a folder to keep them all. Let's name it "/etc". Welcome to back
to chaos.

>> Registry makes program's settings a property of the local machine or user.
>> The problem is that tree view. It should be at least relational. But better
>> would to have different views.
> 
> I don't believe it need be treated any differently than a file
> system. To support a registry on the RieserFS for example, one
> merely need to add a nice programming API to make it easier for
> programs to query and set values. You can still use links/symlinks
> (if you must) etc.

In my view there should be no file systems at all. OS should be fully
memory-mapped with persistent objects. A FS would be just a "low-level"
format then. (Kind of data base discussion again! (:-))

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: For the AdaOS folks

[Warren W. Gay VE3WWG]

Dmitry A. Kazakov wrote:
> On Thu, 30 Dec 2004 11:30:35 -0500, Warren W. Gay VE3WWG wrote:
>>Dmitry A. Kazakov wrote:
>>
>>>On Thu, 30 Dec 2004 08:58:23 -0500, Warren W. Gay VE3WWG wrote:
>>>
>>>>Nick Roberts wrote:
>>>>
>>>>>However, I intend to design the microkernel so that it compromises on
>>>>>minimality (unlike some other microkernel designs)
>>>>
>>>>Yes. Mach is too fat. They shouldn't have included device drivers
>>>>for example.
>>>
>>>Implementations of or the generic interface for? I think the latter would
>>>be a mistake. DOS->Windows tried this path. MS consistently refused to
>>>specify device classes other than hard drive and keyboard.
>>
>>I'm not sure exactly what you're supporting here (clearly not M$), but
>>IMHO, the device driver(s) can be written on top of the microkernel,
>>and need to be inside it. This also helps to keep the mk small.
> 
> So it is about implementations then? But that is not what makes the kernel
> fat. 

To support drivers in the mk, there was additional API added. This
fattens the core somewhat, perhaps not to the extent that other
things did.

The problem is that OS should have generic interfaces for all kinds of
> devices. They should be in. 

The problem then is that the drivers add to the code in "real-mode",
which is one thing that the mk tries to avoid (for additional safety
and modularity).

>>>That is an interesting question. I think that the kernel should be able to
>>>tunnel its requests through higher order protocols.
>>
>>Yes, they can be provided on top of the microkernel, as you seem to
>>be suggesting.
> 
> Yes, but networking should be known to the kernel as something it can work
> with. So you cannot completely throw it away.

The kernel only needs to be able to provide "hooks". It is the
user land OS modules that neeed the networking, not the mk. This
discussion is basically a disagreement on "core functionality"
for the mk. I just don't see networking as being "core", IMO.

>>>>However, you might want to take the tact that networking is done
>>>>outside of the microkernel. This way, in situations where you don't
>>>>want networking (embedded systems), it need not be there.
>>>
>>>There are lot of embedded systems which need networking, starting from
>>>boards interconnected through dual ported RAM, to various sensors and
>>>actors devices attached via Ethernet.
>>
>>I am not disputing that. But that is different than saying every
>>embedded device needs it. So if you want to save some memory, then
>>the first thing that can be dumped (if the option exists) is
>>networking if you don't require it. If it is included in a
>>given mk, it should be optional.
> 
> I'm not sure if that could be possible. I have designed not an OS, but a
> large middleware project. I tried to do the thing you describe, i.e. to
> isolate all networking and remove it from the system core. It turned to be
> impossible. In the final version it appears as a "normal driver", but it
> still has some secret wires leading to the core. So I failed. BTW, there
> were other things which seemed to be no less coupled. For example, I
> managed to isolate the system data logger. So in principle, I agree with
> you, but...

I view "impossible" as a strong word. It may have been true
with certain kinds of constraints imposed upon you that
it was impossible, but I have trouble generally accepting
"immpossible", if enough time and effort is permitted.

>>You base your registry criticisms upon M$ _implementation_ of
>>a registry. But consider this:
>>
>>If each registry item were a file (on a RieserFS so there is little
>>or no waste), then you can backup, secure and restore settings
>>just like any other file. You can do so with all of those familiar
>>tools, including tar and cpio.
> 
> The problem is not an ability to backup, in UNIX one can backup /etc, /var
> ... The problem is complexity. There are dependencies between repository
> objects, which cannot be mapped to a simple tree structure, whether that be
> chaotic files or registry. 

This is a different problem (complexity). No amount of registry/
database/repository design is going to change that for you. You
have complexity because of the applications -- not because of the
registry. At best, you might organize away some of the complexity,
but storage of registry values and the complexity of the same
are two different problem domains.

My point is simply that programs (including installers), need a
programmatic way of testing what is installed etc. on a system.
This is impractical with little text files, all in different
formats. The registry approach is a very practical solution
to this problem.

 > Consider a simple diamond diagram. Application A
> uses B and C, they in turn use D. This cannot be represented by a tree. So
> you need a folder to keep them all. Let's name it "/etc". Welcome to back
> to chaos.

I don't see any unsolvable problem here. Links and symlinks can
create any kind of network you need.

>>I don't believe it need be treated any differently than a file
>>system. To support a registry on the RieserFS for example, one
>>merely need to add a nice programming API to make it easier for
>>programs to query and set values. You can still use links/symlinks
>>(if you must) etc.
> 
> In my view there should be no file systems at all. OS should be fully
> memory-mapped with persistent objects. A FS would be just a "low-level"
> format then. (Kind of data base discussion again! (:-))

All it becomes is just different terminology, but you'll still have
similar levels of abstraction (short-term memory vs long-term etc.
for example). In the meantime, file systems provide a very useful
solution as a hierarchical database ;-)
--
Warren W. Gay VE3WWG
http://home.cogeco.ca/~ve3wwg

Re: For the AdaOS folks

[Dmitry A. Kazakov]

On Fri, 31 Dec 2004 06:30:51 -0500, Warren W. Gay VE3WWG wrote:

> The problem then is that the drivers add to the code in "real-mode",
> which is one thing that the mk tries to avoid (for additional safety
> and modularity).

I am still not sure what you mean here. If you mean modular kernel design
and more fine-grained system of modes than all-or-nothing, then yes, sure.
There is no reason, why a driver should not be treated as an application. 

>>>>That is an interesting question. I think that the kernel should be able to
>>>>tunnel its requests through higher order protocols.
>>>
>>>Yes, they can be provided on top of the microkernel, as you seem to
>>>be suggesting.
>> 
>> Yes, but networking should be known to the kernel as something it can work
>> with. So you cannot completely throw it away.
> 
> The kernel only needs to be able to provide "hooks". It is the
> user land OS modules that neeed the networking, not the mk. This
> discussion is basically a disagreement on "core functionality"
> for the mk. I just don't see networking as being "core", IMO.

But how could it become distributed? That "hook" should be there. An
implementation could be a module, no problem, but all system objects,
synchronization mechanisms, queues etc, should be designed in a way that
would allow distributed behavior (no matter would it be local symmetric, in
a cluster, random networking). This is what I would define as "true"
networking. From this perspective neither Windows nor Unix are networking.

> I view "impossible" as a strong word. It may have been true
> with certain kinds of constraints imposed upon you that
> it was impossible, but I have trouble generally accepting
> "immpossible", if enough time and effort is permitted.

Of course.
 
>> The problem is not an ability to backup, in UNIX one can backup /etc, /var
>> ... The problem is complexity. There are dependencies between repository
>> objects, which cannot be mapped to a simple tree structure, whether that be
>> chaotic files or registry. 
> 
> This is a different problem (complexity). No amount of registry/
> database/repository design is going to change that for you. You
> have complexity because of the applications -- not because of the
> registry. At best, you might organize away some of the complexity,
> but storage of registry values and the complexity of the same
> are two different problem domains.

Right, but if registry does not help much you to organize, and it does not,
then it means that registry provides too low abstraction level. Technically
it is the same level of abstraction as raw files. So you cannot get here
much more support than from a file hierarchy. It only adds a bit type
safety (one can query the type of a key). This means that registry is
unsuitable to serve as an interface for any complex object persistence, as
carrier, yes, as an interface no.

> My point is simply that programs (including installers), need a
> programmatic way of testing what is installed etc. on a system.

Yes, but I would formulate it in more general way:

1. object persistence (must be)
2. object containers. An application can be viewed as a container for its
"mortal" instances and "immortal" persistent data. 

> This is impractical with little text files, all in different
> formats. The registry approach is a very practical solution
> to this problem.

The very first tiny step, in my view.

>> Consider a simple diamond diagram. Application A
>> uses B and C, they in turn use D. This cannot be represented by a tree. So
>> you need a folder to keep them all. Let's name it "/etc". Welcome to back
>> to chaos.
> 
> I don't see any unsolvable problem here. Links and symlinks can
> create any kind of network you need.

Compare it with a network of pointers in C (memory~files, pointers~links).
Not an unsolvable problem, right, but...

>>>I don't believe it need be treated any differently than a file
>>>system. To support a registry on the RieserFS for example, one
>>>merely need to add a nice programming API to make it easier for
>>>programs to query and set values. You can still use links/symlinks
>>>(if you must) etc.
>> 
>> In my view there should be no file systems at all. OS should be fully
>> memory-mapped with persistent objects. A FS would be just a "low-level"
>> format then. (Kind of data base discussion again! (:-))
> 
> All it becomes is just different terminology, but you'll still have
> similar levels of abstraction (short-term memory vs long-term etc.
> for example).

Yes, but I do not want to expose it in API. It is comparable with evolution
of programming languages: from assemble with clear distinction between
registers and memory, modern languages which hide it. Caching should be
OS's business, down with files and I/O!

> In the meantime, file systems provide a very useful
> solution as a hierarchical database ;-)

(:-))

Happy New Year!

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: For the AdaOS folks

[Warren W. Gay VE3WWG]

Dmitry A. Kazakov wrote:
> On Fri, 31 Dec 2004 06:30:51 -0500, Warren W. Gay VE3WWG wrote:
>>The problem then is that the drivers add to the code in "real-mode",
>>which is one thing that the mk tries to avoid (for additional safety
>>and modularity).
> 
> I am still not sure what you mean here. If you mean modular kernel design
> and more fine-grained system of modes than all-or-nothing, then yes, sure.
> There is no reason, why a driver should not be treated as an application. 

That's right. Just remember that the purpose of the mk is to keep the
"priviledged-mode" code to a bare minimum (hence the "micro").

>>The kernel only needs to be able to provide "hooks". It is the
>>user land OS modules that neeed the networking, not the mk. This
>>discussion is basically a disagreement on "core functionality"
>>for the mk. I just don't see networking as being "core", IMO.
> 
> But how could it become distributed? That "hook" should be there. 

The hook can forward message requests to a networking "module", which
is run outside of the mk. As far as the mk is concerned, it need not
worry about more than tasks, threads, ports and perhaps mk "memory
objects" (for paging).

Don't forget that you can play amazing "port tricks" using Mach
styled message ports. To get a flavour of this, read about the Hurd
"Translator Mechanism" concept as described at:

   http://www.cs.pdx.edu/~trent/gnu/hurd/hurd-paper.html

As long as you can insert a "networking module" between the
two endpoints, the mk itself may never need to know about TCP/IP
(or whatever you choose) to network between the two. All it needs
to care about are message ports (internally message queues).

Obviously applications need to worry about networking details,
but the mk itself may never need to. The inserted "networking module"
can take care of that (acting as a proxy).

> An
> implementation could be a module, no problem, but all system objects,
> synchronization mechanisms, queues etc, should be designed in a way that
> would allow distributed behavior (no matter would it be local symmetric, in
> a cluster, random networking). This is what I would define as "true"
> networking. From this perspective neither Windows nor Unix are networking.

There is an "application view" and a "microkernel view". I think you
are mixing the two.

All of the infrastructure that you talk about is built outside of
the mk. The mk's purpose is to provide the basic needs for an OS, but
_not_ be the OS itself. In fact, using a mk approach, application and
OS modules conceptually run beside each other, perhaps only differing
in the privileges they have (ports that they have rights to).

>>This is a different problem (complexity). No amount of registry/
>>database/repository design is going to change that for you. You
>>have complexity because of the applications -- not because of the
>>registry. At best, you might organize away some of the complexity,
>>but storage of registry values and the complexity of the same
>>are two different problem domains.
> 
> Right, but if registry does not help much you to organize, and it does not,
> then it means that registry provides too low abstraction level. Technically
> it is the same level of abstraction as raw files. So you cannot get here
> much more support than from a file hierarchy. It only adds a bit type
> safety (one can query the type of a key). This means that registry is
> unsuitable to serve as an interface for any complex object persistence, as
> carrier, yes, as an interface no.

Well, there are two general approaches to API development, that I
can think of:

  1. Develop a general API to leave flexibility in the hands of the designer
  2. Develop a strict API to "straight-jacket" the users into a certain
     set of "policies".

I tend to shy away from #2, if there is no good reason for it. But in
either case, the user can choose to use it wisely or abuse it. It becomes
more difficult to abuse in #2, but abuse is usually always possible.

The point I make is that software cannot organize things for you, unless
some person has worked it out for you ahead of time. That person can't
do that for my software, and I doubt he/she can for yours either.

To date, the registry idea has gone further than the /etc/file approach,
and I have yet to see a better practical solution proposed. Of course I
would be glad to hear a better approach if one exists.  ;-)

>>My point is simply that programs (including installers), need a
>>programmatic way of testing what is installed etc. on a system.
> 
> Yes, but I would formulate it in more general way:
> 
> 1. object persistence (must be)
> 2. object containers. An application can be viewed as a container for its
> "mortal" instances and "immortal" persistent data. 

What does "object persistence" have more than a "set of values"? How
is an "object container" any different than a directory/subdirectories
containing "values"?

>>>Consider a simple diamond diagram. Application A
>>>uses B and C, they in turn use D. This cannot be represented by a tree. So
>>>you need a folder to keep them all. Let's name it "/etc". Welcome to back
>>>to chaos.
>>
>>I don't see any unsolvable problem here. Links and symlinks can
>>create any kind of network you need.
> 
> Compare it with a network of pointers in C (memory~files, pointers~links).
> Not an unsolvable problem, right, but...

But you have not proposed any solution.  You keep saying that
"I don't want to be responsible", but there is nobody/nothing
left to fill that void.

Not everything manifests itself as a diamond (obviously). If a
hierarchy or "node network" doesn't satisfy your needs, then
I'd like to hear just what other structure can.  You can't
just close your eyes and say "just make it happen". We don't
enjoy that luxury! 8->

>>All it becomes is just different terminology, but you'll still have
>>similar levels of abstraction (short-term memory vs long-term etc.
>>for example).
> 
> Yes, but I do not want to expose it in API. It is comparable with evolution
> of programming languages: from assemble with clear distinction between
> registers and memory, modern languages which hide it. Caching should be
> OS's business, down with files and I/O!

So you don't want an API, and yet you want it to happen somehow. Somehow,
some piece of software/hardware/system is supposed to know what pieces
of information are configurable elements, know how to organize it,
know how to identify it and share it in a safe way with other processes
and users on the same system/network?

Well, its ok to consider these kinds of questions, but if you really expect
that to happen any time soon, I think you're more of a dreamer
than I!

> Happy New Year!

Happy New Year to yourself, and users of Ada everywhere.
-- 
Warren W. Gay VE3WWG
http://home.cogeco.ca/~ve3wwg

Re: For the AdaOS folks

[Marven Lee]

Warren W. Gay VE3WWG wrote:
> Dmitry A. Kazakov wrote:
>> Warren W. Gay VE3WWG wrote:
>>> The problem then is that the drivers add to the code in "real-mode",
>>> which is one thing that the mk tries to avoid (for additional safety
>>> and modularity).
>>
>> I am still not sure what you mean here. If you mean modular kernel
>> design and more fine-grained system of modes than all-or-nothing,
>> then yes, sure. There is no reason, why a driver should not be
>> treated as an application.
>
> That's right. Just remember that the purpose of the mk is to keep the
> "priviledged-mode" code to a bare minimum (hence the "micro").

Some microkernels and single address space operating systems
implement cross-domain call mechanisms that enables a thread
to cross from one protection domain into another in a similar
way as a segmented system performs far calls or interrupts
and traps can transfer from user-mode to kernel-mode.

In Mach it's called "migrating threads".
The Spring Nucleus (and Solaris) has cross-domain calls through "Doors".
In the Pebble Operating System it's called "Portal Traversal"
Mungi has "Protection Domain eXtensions".
I think the Grasshopper kernel has something similar.
Maybe LPC in Windows is a form of cross-domain call, I'm not sure.

You can end up with a microkernel that only handles cross-domain
calls.  Everything else, including what you normally think
a microkernel should at least include, such as memory management,
scheduling and IPC can be implemented outside of the microkernel.



Marv 

Re: For the AdaOS folks

[Warren W. Gay VE3WWG]

Marven Lee wrote:
> Warren W. Gay VE3WWG wrote:
>>Dmitry A. Kazakov wrote:
>>>Warren W. Gay VE3WWG wrote:
>>>
>>>>The problem then is that the drivers add to the code in "real-mode",
>>>>which is one thing that the mk tries to avoid (for additional safety
>>>>and modularity).
>>>
>>>I am still not sure what you mean here. If you mean modular kernel
>>>design and more fine-grained system of modes than all-or-nothing,
>>>then yes, sure. There is no reason, why a driver should not be
>>>treated as an application.
>>
>>That's right. Just remember that the purpose of the mk is to keep the
>>"priviledged-mode" code to a bare minimum (hence the "micro").
> 
> Some microkernels and single address space operating systems
> implement cross-domain call mechanisms that enables a thread
> to cross from one protection domain into another in a similar
> way as a segmented system performs far calls or interrupts
> and traps can transfer from user-mode to kernel-mode.
> 
> In Mach it's called "migrating threads".

Yes, I've seen references to this in the Mach
literature.

> The Spring Nucleus (and Solaris) has cross-domain calls through "Doors".
> In the Pebble Operating System it's called "Portal Traversal"
> Mungi has "Protection Domain eXtensions".
> I think the Grasshopper kernel has something similar.
> Maybe LPC in Windows is a form of cross-domain call, I'm not sure.
> 
> You can end up with a microkernel that only handles cross-domain
> calls.  Everything else, including what you normally think
> a microkernel should at least include,

I'm glad you said "normally" here. ;-)

> such as memory management,
> scheduling and IPC can be implemented outside of the microkernel.
> 
> Marv 

Exokernel?

   http://www.pdos.lcs.mit.edu/exo.html

I've never been too keen on the single address space systems. But
I did like the way the ring mechanism worked in PrimOS (not a mk),
which was probably borrowed from Multics. I liked the way the
links were "snapped" and the way that you could call into different
levels of protection.

Cross domain threads is an interesting idea though. I'll check it
out.
-- 
Warren W. Gay VE3WWG
http://home.cogeco.ca/~ve3wwg

Re: For the AdaOS folks

[Warren W. Gay VE3WWG]

Warren W. Gay VE3WWG wrote:
> Marven Lee wrote:
> 
>> Warren W. Gay VE3WWG wrote:
>>
>>> Dmitry A. Kazakov wrote:
>>>
>>>> Warren W. Gay VE3WWG wrote:
>>>>
>>>>> The problem then is that the drivers add to the code in "real-mode",
>>>>> which is one thing that the mk tries to avoid (for additional safety
>>>>> and modularity).
>>>>
>>>>
>>>> I am still not sure what you mean here. If you mean modular kernel
>>>> design and more fine-grained system of modes than all-or-nothing,
>>>> then yes, sure. There is no reason, why a driver should not be
>>>> treated as an application.
>>>
>>>
>>> That's right. Just remember that the purpose of the mk is to keep the
>>> "priviledged-mode" code to a bare minimum (hence the "micro").
>>
>>
>> Some microkernels and single address space operating systems
>> implement cross-domain call mechanisms that enables a thread
>> to cross from one protection domain into another in a similar
>> way as a segmented system performs far calls or interrupts
>> and traps can transfer from user-mode to kernel-mode.
>>
>> In Mach it's called "migrating threads".
> 
> 
> Yes, I've seen references to this in the Mach
> literature.
> 
>> The Spring Nucleus (and Solaris) has cross-domain calls through "Doors".
>> In the Pebble Operating System it's called "Portal Traversal"
>> Mungi has "Protection Domain eXtensions".
>> I think the Grasshopper kernel has something similar.
>> Maybe LPC in Windows is a form of cross-domain call, I'm not sure.
>>
>> You can end up with a microkernel that only handles cross-domain
>> calls.  Everything else, including what you normally think
>> a microkernel should at least include,
> 
> 
> I'm glad you said "normally" here. ;-)
> 
>> such as memory management,
>> scheduling and IPC can be implemented outside of the microkernel.
>>
>> Marv 
> 
> 
> Exokernel?
> 
>   http://www.pdos.lcs.mit.edu/exo.html
> 
> I've never been too keen on the single address space systems. But
> I did like the way the ring mechanism worked in PrimOS (not a mk),
> which was probably borrowed from Multics. I liked the way the
> links were "snapped" and the way that you could call into different
> levels of protection.
> 
> Cross domain threads is an interesting idea though. I'll check it
> out.

For those interested, a Mach paper on migrating threads is
available at:

http://www.usenix.org/publications/library/proceedings/sf94/full_papers/ford.pdf

In the Mach case, it is only a modification of the thread
model (semantics are changed somewhat). A major benefit was
the speedup it offers RPC calls. However the overall microkernel
structure does not change (you still have priv-mode code in mk,
and user-mode code is outside).

-- 
Warren W. Gay VE3WWG
http://home.cogeco.ca/~ve3wwg

Re: For the AdaOS folks

[Marven Lee]

Warren W. Gay VE3WWG wrote:
> Marven Lee wrote:
>> In Mach it's called "migrating threads".
>
> Yes, I've seen references to this in the Mach
> literature.

Yes, and from your other post you've already found the
paper on migrating threads in Mach.

http://www.brynosaurus.com/pub.html

It is interesting to note that he wrote some libraries and
apps on AmigaOS.  I think he works for Microsoft now.



>> You can end up with a microkernel that only handles cross-domain
>> calls.  Everything else, including what you normally think
>> a microkernel should at least include,
>
> I'm glad you said "normally" here. ;-)
>
>> such as memory management,
>> scheduling and IPC can be implemented outside of the microkernel.
>
> Exokernel?

No, I'd still call it a microkernel or perhaps a trampoline.

It would be a bit like Ingo Molnar's 4GB kernel / 4GB user
patch for Linux.

http://lkml.org/lkml/2003/7/8/246

This moved the kernel from the top 1GB of every process
into a 4GB address space of its own.  a 16MB trampoline
(microkernel) exists at the top of every address space
to transfer syscalls and interrupts into the address space
that the Linux kernel resides in.

In some ways this is similar to a single server Linux personality
running on a microkernel like L4Linux. I think the difference
between the trampoline and L4Linux can be summed up as
L4 uses an active object model and the trampoline/migrating
threads uses a passive object model.

I'm not sure if or why the 4G4G in Linux needs 16MB, it
can be done in less than a page of code and a few bytes of data.
The trampoline should take up something in the region of 8KB
of the address space.

You could replace Linux with Mach as the personality.  Yes,
it sounds a bit wierd having Mach run on top of a microkernel
or trampoline.   Although Mach would be running as a single
server personality this doesn't stop multiple servers existing, they
just have to make cross-domain calls into the Mach personality
to make use of Mach's IPC.  It is single server only from the
point of view of cross-domain calls through the trampoline
microkernel.

So you can think of this trampoline/microkernel as a fancy
syscall redirection mechanism that moves a kernel from
kernel-mode in all address spaces to its own address space.

The trampoline copies small messages in registers, if
you want to copy more then you have to change functions
in the personality such as copy_to_user() and copy_from_user()
to map pages from the application process into the personality
and copy the data from there.



By using a more sophisticated trampoline that keeps track
of what protection domains a thread has permission to
migrate to you can have a multiserver cross-domain call
mechanism.  The trampoline would require more storage
space than the simple single server approach.

I've dug up an old thread from alt.os.development where
KVP describes a single server trampoline and I follow
up with a rough idea of how to implement a
trampoline-microkernel that supports multiple servers.

http://tinyurl.com/6gb5b

That was an old posting of mine and I'd probably change the
"activation" structure around a bit.  I'd more likely use
array indices to reference other activations instead of pointers.
I'd also change some of the needed system calls as well.



Onto AmigaOS.  Let me make this clear from the outset
to any Amigans reading that what I'm about to write below isn't
about  how to add memory protection to AmigaOS, it's only
about how the structure of AmigaOS relates to microkernels
that make use of cross-domain calls / migrating threads.

I always think of migrating threads in terms of AmigaOS.
AmigaOS didn't have any protection and divided everything
into shared libraries.  But the libraries weren't like those
in other operating systems, they were more like the subsystems
of a kernel.  Ideally they should have been placed in their
own address spaces/protection domains and a cross-domain
call should have been used to access them.

The core library of AmigaOS, exec.library would become
a server, just like Mach in the above single server example.
All other libraries, including devices which are more or less
libraries with their own thread would also become servers.

It would be as if the a trampoline microkernel had replaced
the library jump tables it used.  In an earlier post in this
thread Dennis Lee Bieber said everything was done with
message ports, from my point of view I'd say everthing
was done with library calls.  After all the message passing
was done by library calls to exec.library.

In effect what they should be is, "Protected Shared Libraries."  *

I had wondered if anyone in the Amiga community had come up
with similar ideas and found this in a 1997 newsgroup posting
entitled  "p-OS & Multiprocessing"...


Michael van Elst wrote:
> Jeroen T. Vermeulen wrote:
>> One major problem is that the Amiga's shared libraries act as
>> OOP-style objects, matching responsibilities to context much more
>> elegantly than other OSs do (where process ID is everything).  A
>> process conceptually switches to a different protection domain when
>> it calls a library function.
>
> Correct. That's why a protected AmigaOS must support protection
> domains not only for processes but also for libraries. That's a
> generalization of the UNIX model. There you have protection domains
> for each process plus one protection domain for the kernel which is
> mainly a huge library. In AmigaOS every library would need its own
> protection domain and context switches must be as lightweight as
> possible.


I also like the way each thread in AmigaOS has access to exec.library.
Each thread could call the exec.library functions OpenLibrary() and
CloseLibrary() to load a library into memory and obtain a handle to
access it.  A similar way could be used for loading modules in a
system that uses cross-domain calls/migrating threads except that
it would need to create a stack in a server for a thread to use when
it has called into the server.

These "protected shared libraries" (PSLs from now on) needn't be
global to the entire system.  You could have global PSLs for
the memory manager, the filesystem, networking and GUI.  Per-
user PSLs for things like a clipboard and desktop.  Finally you
could have per application private PSLs so as to decompose
an application into a number of PSLs.



> I've never been too keen on the single address space systems.

I like the way you can pass areas of memory between address
spaces without having to search for a suitable place to stick
it in the destination address space, they can use the same
region of the address space in each address space.  This might
also be useful when copying data through a series of address
spaces.  I don't think having pointers meaning the same thing
in different address spaces is a benefit though, especially
in messages where a pointer could point outside a message
without any error checking.




Now for some suggested reading...

"High Performance Cross Address Space Communication"
and "Lightweight Remote Procedure Call" by B.Bershad.
One of the older papers that other papers cite.

"Evolving Mach 3.0 to a Migrating Thread Model" and
"Microkernels Should Support Passive Objects" by
B.Ford & J.Lepreau.  The second one puts forward
some reasons why microkernels should use migrating
threads.

Sun's "Spring nucleus, A Microkernel for Objects"
by G. Hamilton.  They took the idea of Doors from
Spring and later added them to Solaris.


The "Pebble Operating System"
http://www.bell-labs.com/project/pebble/

has a few papers, "The Pebble Component-Based Operating
System" is the main one.

"The Mungi Single-Address-Space Operating System"
by G.Heiser.  Although I think it runs on top of L4
it has a "protected procedure call" mechanism called PDX.

There is also the Grasshopper kernel but I'm not sure
what the main research papers on it are called.

* "Protected Shared Libraries" by A.Banerju and D.L. Cohn.

Then there are kernel-less systems that have some
similarities to cross-domain calls.

"Anonymous RPC" by C.Yarvin et al.  Imagine the Amiga's
shared libraries randomly scattered around a 64-bit address
space.  Imagine also that the Amiga library jump tables
are execute-only and not readable, and that it was running
on a RISC chip with instruction-alignment constraints.
Then you've got a idea of what "Anonymous RPC" and
probablistic memory protection is about.

"Implementing Operating Systems without Kernels" and
"Efficient Cross-domain Mechanisms for Building
Kernel-less Operating Systems" by D.Probert and J.Bruno.
I read it some time ago, I think it is more or less about
the trampoline/microkernel being implemented in hardware.




Marv

Re: For the AdaOS folks

[Luke A. Guest]

On Sun, 02 Jan 2005 15:09:36 +0000, Marven Lee wrote:

> Onto AmigaOS.  Let me make this clear from the outset
> to any Amigans reading that what I'm about to write below isn't
> about  how to add memory protection to AmigaOS, it's only
> about how the structure of AmigaOS relates to microkernels
> that make use of cross-domain calls / migrating threads.
> 
> I always think of migrating threads in terms of AmigaOS.
> AmigaOS didn't have any protection and divided everything
> into shared libraries.  But the libraries weren't like those
> in other operating systems, they were more like the subsystems
> of a kernel.  Ideally they should have been placed in their
> own address spaces/protection domains and a cross-domain
> call should have been used to access them.

Wrong. They would be equivalent of a server/service in a microkernel that
supports memory protection. Each library would be mapped into the
application's address space when it is opened.

> The core library of AmigaOS, exec.library would become
> a server, just like Mach in the above single server example.
> All other libraries, including devices which are more or less
> libraries with their own thread would also become servers.

Wrong. This is the only library that wouldn't be mapped into the
application's address space when opened, because it is always open.
Remember, exec.library is always available, you never have to open it, it
resides at address 0x00000004.

The exec.library is the "microkernel" of the AmigaOS, thus all of the
functions inside exec.library would become syscalls and it would already
be in memory, in kernel space, mapped into every running application's
address space, implicitly.

> It would be as if the a trampoline microkernel had replaced the library
> jump tables it used.  In an earlier post in this thread Dennis Lee
> Bieber said everything was done with message ports, from my point of
> view I'd say everthing was done with library calls.  After all the
> message passing was done by library calls to exec.library.

Everything was done with message ports, any app/lib that needed to
implement any kind of messaging, could extend the exec's message type and
set up a port to send it to, see intuition.library for an example.

> In effect what they should be is, "Protected Shared Libraries."  *
> 
> I had wondered if anyone in the Amiga community had come up with similar
> ideas and found this in a 1997 newsgroup posting entitled  "p-OS &
> Multiprocessing"...
> 
> 
> Michael van Elst wrote:
>> Jeroen T. Vermeulen wrote:
>>> One major problem is that the Amiga's shared libraries act as
>>> OOP-style objects, matching responsibilities to context much more
>>> elegantly than other OSs do (where process ID is everything).  A
>>> process conceptually switches to a different protection domain when it
>>> calls a library function.
>>
>> Correct. That's why a protected AmigaOS must support protection domains
>> not only for processes but also for libraries. That's a generalization
>> of the UNIX model. There you have protection domains for each process
>> plus one protection domain for the kernel which is mainly a huge
>> library. In AmigaOS every library would need its own protection domain
>> and context switches must be as lightweight as possible.

I certainly wouldn't do it like that. On hardware that has an MMU (most
these days), that would result in a very slow system due to the amount of
context switches, this is why the libraries need to be mapped into the
address space of the running app, i.e. shared between applications.

> I also like the way each thread in AmigaOS has access to exec.library.
> Each thread could call the exec.library functions OpenLibrary() and
> CloseLibrary() to load a library into memory and obtain a handle to
> access it.  A similar way could be used for loading modules in a system
> that uses cross-domain calls/migrating threads except that it would need
> to create a stack in a server for a thread to use when it has called
> into the server.
> 
> These "protected shared libraries" (PSLs from now on) needn't be global
> to the entire system.  You could have global PSLs for the memory
> manager, the filesystem, networking and GUI.  Per- user PSLs for things
> like a clipboard and desktop.  Finally you could have per application
> private PSLs so as to decompose an application into a number of PSLs.

Which is essentially the way the AmigaOS worked.
 
>> I've never been too keen on the single address space systems.
> 
> I like the way you can pass areas of memory between address spaces
> without having to search for a suitable place to stick it in the
> destination address space, they can use the same region of the address
> space in each address space.  This might also be useful when copying
> data through a series of address spaces.  I don't think having pointers
> meaning the same thing in different address spaces is a benefit though,
> especially in messages where a pointer could point outside a message
> without any error checking.

Single address space OSes are only useful if you have loads of memory and
you're not using that many apps at any one time. So you need to be more
careful with the sizes of apps. You will run out of memory, especially
considering the size of apps these days.

Luke.

Re: For the AdaOS folks

[Warren W. Gay VE3WWG]

Luke A. Guest wrote:
> On Sun, 02 Jan 2005 15:09:36 +0000, Marven Lee wrote:
>>Michael van Elst wrote:
>>>Jeroen T. Vermeulen wrote:
>>>>One major problem is that the Amiga's shared libraries act as
>>>>OOP-style objects, matching responsibilities to context much more
>>>>elegantly than other OSs do (where process ID is everything).  A
>>>>process conceptually switches to a different protection domain when it
>>>>calls a library function.
>>>
>>>Correct. That's why a protected AmigaOS must support protection domains
>>>not only for processes but also for libraries. That's a generalization
>>>of the UNIX model. There you have protection domains for each process
>>>plus one protection domain for the kernel which is mainly a huge
>>>library. In AmigaOS every library would need its own protection domain
>>>and context switches must be as lightweight as possible.
> 
> I certainly wouldn't do it like that. On hardware that has an MMU (most
> these days), that would result in a very slow system due to the amount of
> context switches, this is why the libraries need to be mapped into the
> address space of the running app, i.e. shared between applications.

There is no argument about the overhead in this case. However, I
maintain that if enough people start using operating systems
that are implemented this way, they'll finally enhance the
hardware to correct this problem. Until then, people may as well
continue to say "we can't do it that way". I believe the problem
is fixable in hardware.

>>>I've never been too keen on the single address space systems.
>>
>>I like the way you can pass areas of memory between address spaces
>>without having to search for a suitable place to stick it in the
>>destination address space, they can use the same region of the address
>>space in each address space.  This might also be useful when copying
>>data through a series of address spaces.  I don't think having pointers
>>meaning the same thing in different address spaces is a benefit though,
>>especially in messages where a pointer could point outside a message
>>without any error checking.
> 
> Single address space OSes are only useful if you have loads of memory and
> you're not using that many apps at any one time. So you need to be more
> careful with the sizes of apps. You will run out of memory, especially
> considering the size of apps these days.
> 
> Luke.

Even when you have address space to "burn", you can quickly use it
up if you decide to map files into memory. Security is another
potential issue (you would have to map out pages that you don't
want others to see etc.)
--
Warren W. Gay VE3WWG
http://home.cogeco.ca/~ve3wwg

Re: For the AdaOS folks

[Luke A. Guest]

On Sun, 02 Jan 2005 22:13:47 -0500, Warren W. Gay VE3WWG wrote:

>>>>Correct. That's why a protected AmigaOS must support protection domains
>>>>not only for processes but also for libraries. That's a generalization
>>>>of the UNIX model. There you have protection domains for each process
>>>>plus one protection domain for the kernel which is mainly a huge
>>>>library. In AmigaOS every library would need its own protection domain
>>>>and context switches must be as lightweight as possible.
>> 
>> I certainly wouldn't do it like that. On hardware that has an MMU (most
>> these days), that would result in a very slow system due to the amount of
>> context switches, this is why the libraries need to be mapped into the
>> address space of the running app, i.e. shared between applications.
> 
> There is no argument about the overhead in this case. However, I
> maintain that if enough people start using operating systems
> that are implemented this way, they'll finally enhance the
> hardware to correct this problem. Until then, people may as well
> continue to say "we can't do it that way". I believe the problem
> is fixable in hardware.

Yeah but, what you were saying was that you wanted OSes to use protection
domains for libraries *as well*, this is the wrong way to go. The
libraries are (and should be) mapped into an applications address space,
libraries *should never* have their own address spaces. It is that idea
which is wrong and would be slow, not the *proper* way to do it.

>> Single address space OSes are only useful if you have loads of memory and
>> you're not using that many apps at any one time. So you need to be more
>> careful with the sizes of apps. You will run out of memory, especially
>> considering the size of apps these days.
>> 
>> Luke.
> 
> Even when you have address space to "burn", you can quickly use it
> up if you decide to map files into memory. Security is another
> potential issue (you would have to map out pages that you don't
> want others to see etc.)

Well, if you have a single address space OS, you're unlikely to have
virtual memory, that's what these OSes are used for. You can still provide
memory protection on the pages that belong to each application without any
problems.

Luke.

Re: For the AdaOS folks

[Marven Lee]

Luke A. Guest wrote...
> Yeah but, what you were saying was that you wanted OSes to use
> protection domains for libraries *as well*, this is the wrong way to
> go. The libraries are (and should be) mapped into an applications
> address space, libraries *should never* have their own address
> spaces. It is that idea which is wrong and would be slow, not the
> *proper* way to do it.

I'm not on about "normal" libraries becoming "protected shared
libraries".

"Protected shared libraries" are the same as servers in other microkernel
systems.  I could have called them servers, however the term
"protected shared libraries" seems to emphasise a passive object model
where a thread makes a cross-domain call into a server as opposed
to having a seperate thread in the client and a seperate pool of threads
in the server which would be an active object model.

You get to keep your normal libraries.

Just like a server, protected shared libraries keep data shared between
multiple processes private and you'd use them wherever you use servers.

Migrating threads / cross-domain calls / LRPC / thread tunnelling whatever
you want to call it allows for a more streamlined IPC between processes.
There is no need for a rendezvous between threads, there is no need to
block threads and the kernel path becomes simpler.

Re: For the AdaOS folks

[Warren W. Gay VE3WWG]

Luke A. Guest wrote:
> On Sun, 02 Jan 2005 22:13:47 -0500, Warren W. Gay VE3WWG wrote:
>>>>>Correct. That's why a protected AmigaOS must support protection domains
>>>>>not only for processes but also for libraries. That's a generalization
>>>>>of the UNIX model. There you have protection domains for each process
>>>>>plus one protection domain for the kernel which is mainly a huge
>>>>>library. In AmigaOS every library would need its own protection domain
>>>>>and context switches must be as lightweight as possible.
>>>
>>>I certainly wouldn't do it like that. On hardware that has an MMU (most
>>>these days), that would result in a very slow system due to the amount of
>>>context switches, this is why the libraries need to be mapped into the
>>>address space of the running app, i.e. shared between applications.
>>
>>There is no argument about the overhead in this case. However, I
>>maintain that if enough people start using operating systems
>>that are implemented this way, they'll finally enhance the
>>hardware to correct this problem. Until then, people may as well
>>continue to say "we can't do it that way". I believe the problem
>>is fixable in hardware.
> 
> Yeah but, what you were saying was that you wanted OSes to use protection
> domains for libraries *as well*, this is the wrong way to go. The
> libraries are (and should be) mapped into an applications address space,
> libraries *should never* have their own address spaces. It is that idea
> which is wrong and would be slow, not the *proper* way to do it.

Oh, no - I don't know where you got that from (perhaps you've mixed me
up with a different poster). Shared libraries have an important
role to play.
--
Warren W. Gay VE3WWG
http://home.cogeco.ca/~ve3wwg

Re: For the AdaOS folks

[Ad Buijsen]

Warren W. Gay VE3WWG wrote:
> Luke A. Guest wrote:

>> I certainly wouldn't do it like that. On hardware that has an MMU (most
>> these days), that would result in a very slow system due to the amount of
>> context switches, this is why the libraries need to be mapped into the
>> address space of the running app, i.e. shared between applications.
> 
> 
> There is no argument about the overhead in this case. However, I
> maintain that if enough people start using operating systems
> that are implemented this way, they'll finally enhance the
> hardware to correct this problem. Until then, people may as well
> continue to say "we can't do it that way". I believe the problem
> is fixable in hardware.
> 

It is, by tagging the TLB lines with an address space identifier. This 
is supported by MIPS CPUs with a R4000-style MMU (ASId), Alpha 21164 and 
21264 (ASN; the 21264 also has a tagged instruction cache), 
PowerPC-BookE (TID) and probably some mainframe CPUs.
Tagged TLBs can be simulated on IA32 and 'classic' PPC by segmentation 
tricks; this was done for L4.  I suspect that the Athlon64 has hidden 
tags to implement the flush filters.

The next problem would be the cost (in cycles) of kernel entry...

Ad Buijsen

Re: For the AdaOS folks

[Warren W. Gay VE3WWG]

Ad Buijsen wrote:
> Warren W. Gay VE3WWG wrote:
>> Luke A. Guest wrote:
>>> I certainly wouldn't do it like that. On hardware that has an MMU (most
>>> these days), that would result in a very slow system due to the 
>>> amount of
>>> context switches, this is why the libraries need to be mapped into the
>>> address space of the running app, i.e. shared between applications.
>>
>> There is no argument about the overhead in this case. However, I
>> maintain that if enough people start using operating systems
>> that are implemented this way, they'll finally enhance the
>> hardware to correct this problem. Until then, people may as well
>> continue to say "we can't do it that way". I believe the problem
>> is fixable in hardware.
> 
> It is, by tagging the TLB lines with an address space identifier. This 
> is supported by MIPS CPUs with a R4000-style MMU (ASId), Alpha 21164 and 
> 21264 (ASN; the 21264 also has a tagged instruction cache), 
> PowerPC-BookE (TID) and probably some mainframe CPUs.
> Tagged TLBs can be simulated on IA32 and 'classic' PPC by segmentation 
> tricks; this was done for L4.  I suspect that the Athlon64 has hidden 
> tags to implement the flush filters.

Glad to hear it. Perhaps AMD will push Intel to do something ;-)
--
Warren W. Gay VE3WWG
http://home.cogeco.ca/~ve3wwg

Re: For the AdaOS folks

[Marven Lee]

Luke A. Guest <laguest@n_o_p_o_r_k_a_n_d_h_a_m.abyss2.demon.co.uk>
> The exec.library is the "microkernel" of the AmigaOS, thus all of the
> functions inside exec.library would become syscalls and it would
> already be in memory, in kernel space, mapped into every running
> application's address space, implicitly.

Or you could move it out of the kernel part of the address space
and  into a server/protected shared library.

Just like the single server Linux and Mach examples I gave.

The only thing my microkernel does is replace your syscall into your Exec
microkernel with a cross-domain call from one address space to another.

The microkernel becomes a glorified interrupt/syscall handler transferring
a thread across address spaces, much like an interrupt/kernel trap
transfers a thread from user-mode to kernel-mode.

Then you only need several system calls in the microkernel.
No need for threads, processes, message ports, memory management,
page tables, syncronisation primitives, scheduler or anything else
in the microkernel.  They are all implemented in one of the
servers/protected shared libraries.


Marv

Re: For the AdaOS folks

[Nick Roberts]

"Luke A. Guest" <laguest@n_o_p_o_r_k_a_n_d_h_a_m.abyss2.demon.co.uk> wrote:

> > > Correct. That's why a protected AmigaOS must support protection
> > > domains not only for processes but also for libraries. That's a
> > > generalization of the UNIX model. There you have protection domains
> > > for each process plus one protection domain for the kernel which is
> > > mainly a huge library. In AmigaOS every library would need its own
> > > protection domain and context switches must be as lightweight as
> > > possible.
> 
> I certainly wouldn't do it like that. On hardware that has an MMU (most
> these days), that would result in a very slow system due to the amount of
> context switches, this is why the libraries need to be mapped into the
> address space of the running app, i.e. shared between applications.

It is necessary to be able to take both approaches.

If you trust the code in a library never to deliberately (and to be unlikely
to accidentally) subvert the security of the data your own program could
have access to, you can map it into your address space, so that your calls
upon that code can be efficient (they do not have to cross any protection
boundary).

Otherwise, you must place some or all of the less-than-completely trusted
code in a separate protection space, so that it is prevented from accessing
data other than the data you explicitly permit to access. One popular way to
do this is to make the less-than-completely trusted code into a program that
executes as a 'server', and you, the 'client', make calls to it which do
transition a protection boundary.

Sometimes you need the efficiency (and you trust the code), sometimes you
need the protection (because you don't trust the code). So, both mechanisms
must be available.

> Single address space OSes are only useful if you have loads of memory and
> you're not using that many apps at any one time. So you need to be more
> careful with the sizes of apps. You will run out of memory, especially
> considering the size of apps these days.

Exactly why I've chosen a design, for the IA-32, that gives each process
(almost) the full 4 GiB.

-- 
Nick Roberts

Re: For the AdaOS folks

[Warren W. Gay VE3WWG]

Marven Lee wrote:
> Warren W. Gay VE3WWG wrote:
>>Marven Lee wrote:
>>>You can end up with a microkernel that only handles cross-domain
>>>calls.  Everything else, including what you normally think
>>>a microkernel should at least include,
>>
>>I'm glad you said "normally" here. ;-)

>>>such as memory management,
>>>scheduling and IPC can be implemented outside of the microkernel.
>>
>>Exokernel?
> 
> No, I'd still call it a microkernel or perhaps a trampoline.

Ok, I know what you mean now. But I _want_ the mk to take
care of VM and port management however, since I don't want
to be bothered with those. Having the VM done in the mk
makes it a little easier to port MyOS to another hardware
platform (mind you, you still have to port your other
hardware support, but it is one less worry).

But your point that a mk could be smaller is taken.

> I've dug up an old thread from alt.os.development where
> KVP describes a single server trampoline and I follow
> up with a rough idea of how to implement a
> trampoline-microkernel that supports multiple servers.
> 
> http://tinyurl.com/6gb5b
> 
> That was an old posting of mine and I'd probably change the
> "activation" structure around a bit.  I'd more likely use
> array indices to reference other activations instead of pointers.
> I'd also change some of the needed system calls as well.

It sounds like an interesting idea. Perhaps you'd like to
work on rtmk and extend it some?

I simply don't have the time to do the mk part myself. For
now, I am more interested in the Ada components on top of
it. But it sure would be nice to have an Ada microkernel
underneath it someday ;-)

> Now for some suggested reading...
> 
...
> "Anonymous RPC" by C.Yarvin et al.  Imagine the Amiga's
> shared libraries randomly scattered around a 64-bit address
> space.  Imagine also that the Amiga library jump tables
> are execute-only and not readable, and that it was running
> on a RISC chip with instruction-alignment constraints.
> Then you've got a idea of what "Anonymous RPC" and
> probablistic memory protection is about.

I don't know much about the Amiga myself. But
There are 2 strikes against this IMHO:

   1. Single address space (my own bias)
   2. Not secure.

For # 2 they state "it is _unlikely_ that a process will
be able to locate any other segment... impossible to
corrupt a segment without knowing its location."

The word "unlikely" is not good enough for me here.

My first priority is security (efficiency and everything
else is secondary). So any approach like this one, where
security is just "good enough", is not appropriate to my
own goals.

> "Implementing Operating Systems without Kernels" and
> "Efficient Cross-domain Mechanisms for Building
> Kernel-less Operating Systems" by D.Probert and J.Bruno.
> I read it some time ago, I think it is more or less about
> the trampoline/microkernel being implemented in hardware.

Again, I am not really interested in doing all of the details
of an O/S myself. That is why I rather like the layered
microckernel concept. It is also a nice way to subdivide the
problem for those adventuresome people that _do_ have the
time to fiddle with the lower level things ;-)
--
Warren W. Gay VE3WWG
http://home.cogeco.ca/~ve3wwg

Re: For the AdaOS folks

[Nick Roberts]

"Marven Lee" <mehs60@dsl.pipex.com> wrote:

> > > You can end up with a microkernel that only handles cross-domain
> > > calls.  Everything else, including what you normally think a
> > > microkernel should at least include,
> >
> > I'm glad you said "normally" here. ;-)
> >
> > > such as memory management, scheduling and IPC can be implemented
> > > outside of the microkernel.
> >
> > Exokernel?
> 
> No, I'd still call it a microkernel or perhaps a trampoline.

Bachar has cross-domain calls (I think ;-) although I've called them 'IPC
service calls', and describe them in terms of the client-server model.

A process that provides a service (the server process) obtains a kernel
resource called an 'IPC service', and publishes it somehow (how is a long
story). A process that wants to use the service (the client process) obtains
a kernel resource called a 'service key', whose target service is the
required IPC service. The client then calls the service key to obtain the
service. The server receives calls to its IPC service, performs the service,
and returns. The (calling thread of the) client remains blocked until this
is complete.

> I like the way you can pass areas of memory between address spaces without
> having to search for a suitable place to stick it in the destination
> address space, they can use the same region of the address space in each
> address space.  This might also be useful when copying data through a
> series of address spaces.  I don't think having pointers meaning the same
> thing in different address spaces is a benefit though, especially in
> messages where a pointer could point outside a message without any error
> checking.

AdaOS provides multiple memory 'regions'. A process can map multiple regions
into its address space at one time. Several processes can share a memory
region by obtaining kernel resources called 'region keys', and using those
keys to map the region. Again, a mechanism of publishing provides the means
for processes to co-ordinate this.

The idea is that one or more shared regions can be used to pass parameters
between processes in accompaniment to IPC service calls.

Because the regions for doing this are genuinely shared memory areas -- if
the the client and server both reside on the same workstation -- there is no
copying involved (although there may be marshalling on both sides, where the
format of data is generally changed).

However, the exact same IPC calls can be made trans-network, completely
transparently to both client and server, by the insertion of 'agent' (proxy)
processes: the client calls a client agent; the client agent communicates
the call across the network to the server agent; the server agent calls the
server; when the server finishes, the server agent communicates the reply to
the client agent; the client agent returns the reply to the client.
Distributed memory regions are used, so that pages of data get automatically
pulled around the network as they are used by client and server.

An asynchronous form of call is also supported (the client thread is not
blocked, and there is no reply).

This scheme is completely secure: no process can ever gain access to data it
is not permitted to access (it is prevented from getting a region key), and
no client can call a service it is not permitted to (it is prevented from
getting a service key).

-- 
Nick Roberts

Re: For the AdaOS folks

[Warren W. Gay VE3WWG]

Nick Roberts wrote:
> "Marven Lee" <mehs60@dsl.pipex.com> wrote:
>>>>You can end up with a microkernel that only handles cross-domain
>>>>calls.  Everything else, including what you normally think a
>>>>microkernel should at least include,
>>>
>>>I'm glad you said "normally" here. ;-)
>>>
>>>>such as memory management, scheduling and IPC can be implemented
>>>>outside of the microkernel.
>>>
>>>Exokernel?
>>
>>No, I'd still call it a microkernel or perhaps a trampoline.
> 
> Bachar has cross-domain calls (I think ;-) although I've called them 'IPC
> service calls', and describe them in terms of the client-server model.
> 
> A process that provides a service (the server process) obtains a kernel
> resource called an 'IPC service', and publishes it somehow (how is a long
> story). A process that wants to use the service (the client process) obtains
> a kernel resource called a 'service key', whose target service is the
> required IPC service. The client then calls the service key to obtain the
> service. The server receives calls to its IPC service, performs the service,
> and returns. The (calling thread of the) client remains blocked until this
> is complete.

How is this "IPC service key" different than a Mach port? What you
describe sounds just like Mach RPC (with or without migrating-threads).

>>I like the way you can pass areas of memory between address spaces without
>>having to search for a suitable place to stick it in the destination
>>address space, they can use the same region of the address space in each
>>address space.  This might also be useful when copying data through a
>>series of address spaces.  I don't think having pointers meaning the same
>>thing in different address spaces is a benefit though, especially in
>>messages where a pointer could point outside a message without any error
>>checking.
> 
> AdaOS provides multiple memory 'regions'. A process can map multiple regions
> into its address space at one time. Several processes can share a memory
> region by obtaining kernel resources called 'region keys', and using those
> keys to map the region. Again, a mechanism of publishing provides the means
> for processes to co-ordinate this.

How does this differ from Mach using its ports to map in regions
of memory?

> The idea is that one or more shared regions can be used to pass parameters
> between processes in accompaniment to IPC service calls.

Mach has been doing this with its "out-of-line" data as an optional
RPC feature (the rtmk microkernel that I am using does this for
example - it was inspired by Mach).

> Because the regions for doing this are genuinely shared memory areas -- if
> the the client and server both reside on the same workstation -- there is no
> copying involved (although there may be marshalling on both sides, where the
> format of data is generally changed).
> 
> However, the exact same IPC calls can be made trans-network, completely
> transparently to both client and server, by the insertion of 'agent' (proxy)
> processes: the client calls a client agent; the client agent communicates
> the call across the network to the server agent; the server agent calls the
> server; when the server finishes, the server agent communicates the reply to
> the client agent; the client agent returns the reply to the client.
> Distributed memory regions are used, so that pages of data get automatically
> pulled around the network as they are used by client and server.

In some Mach literature this is referred to as External Memory
Management (EMM). The most easiest version of this is the
1-writer + n-readers case. > 1 writers leads to a number of
problems and significant overhead, depending upon the sequence
of events. You'll find a 1992 example of the 1-writer EMM
in the book "Programming under Mach".

> An asynchronous form of call is also supported (the client thread is not
> blocked, and there is no reply).

Mach of course, supports this out of the box.

> This scheme is completely secure: no process can ever gain access to data it
> is not permitted to access (it is prevented from getting a region key), and
> no client can call a service it is not permitted to (it is prevented from
> getting a service key).

It sounds pretty much deja-Mach to me, which is ok. What I have
trouble understanding is the need for new names.
--
Warren W. Gay VE3WWG
http://home.cogeco.ca/~ve3wwg

Re: For the AdaOS folks

[Nick Roberts]

"Warren W. Gay VE3WWG" <ve3wwg@NoSpam.cogeco.ca> wrote:

> ...
> How is this "IPC service key" different than a Mach port? What you
> describe sounds just like Mach RPC (with or without migrating-threads).

I think there are quite a lot of differences in the details. Certainly
thereare similarities.

> ...
> How does this differ from Mach using its ports to map in regions of
> memory?

AdaOS regions are more flexible, in that they can exist outside any process
address space, they can be bigger than 4 MiB, and they can be partially
mapped into address spaces. Different regions can have a different memory
managers.

> It sounds pretty much deja-Mach to me, which is ok. What I have trouble
> understanding is the need for new names.

The basic design for Bachar predates anything (I know of) published about
Mach. I have stuck to the terminology I have used for all the years I have
been evolving the design of Bachar. This design is significantly different
to Mach.

-- 
Nick Roberts

Re: For the AdaOS folks

[Dmitry A. Kazakov]

On Fri, 31 Dec 2004 11:24:40 -0500, Warren W. Gay VE3WWG wrote:

> Dmitry A. Kazakov wrote:
>> On Fri, 31 Dec 2004 06:30:51 -0500, Warren W. Gay VE3WWG wrote:

>>>The kernel only needs to be able to provide "hooks". It is the
>>>user land OS modules that neeed the networking, not the mk. This
>>>discussion is basically a disagreement on "core functionality"
>>>for the mk. I just don't see networking as being "core", IMO.
>> 
>> But how could it become distributed? That "hook" should be there. 
> 
> The hook can forward message requests to a networking "module", which
> is run outside of the mk.

Yes, that is what I meant.

>> An
>> implementation could be a module, no problem, but all system objects,
>> synchronization mechanisms, queues etc, should be designed in a way that
>> would allow distributed behavior (no matter would it be local symmetric, in
>> a cluster, random networking). This is what I would define as "true"
>> networking. From this perspective neither Windows nor Unix are networking.
> 
> There is an "application view" and a "microkernel view". I think you
> are mixing the two.
> 
> All of the infrastructure that you talk about is built outside of
> the mk. The mk's purpose is to provide the basic needs for an OS, but
> _not_ be the OS itself. In fact, using a mk approach, application and
> OS modules conceptually run beside each other, perhaps only differing
> in the privileges they have (ports that they have rights to).

I agree in principle, but disagree in details, in what are these basic
needs. I think that a kind of "protected object" should be provided already
by the kernel as a building block.

>>>This is a different problem (complexity). No amount of registry/
>>>database/repository design is going to change that for you. You
>>>have complexity because of the applications -- not because of the
>>>registry. At best, you might organize away some of the complexity,
>>>but storage of registry values and the complexity of the same
>>>are two different problem domains.
>> 
>> Right, but if registry does not help much you to organize, and it does not,
>> then it means that registry provides too low abstraction level. Technically
>> it is the same level of abstraction as raw files. So you cannot get here
>> much more support than from a file hierarchy. It only adds a bit type
>> safety (one can query the type of a key). This means that registry is
>> unsuitable to serve as an interface for any complex object persistence, as
>> carrier, yes, as an interface no.
> 
> Well, there are two general approaches to API development, that I
> can think of:
> 
>   1. Develop a general API to leave flexibility in the hands of the designer
>   2. Develop a strict API to "straight-jacket" the users into a certain
>      set of "policies".
> 
> I tend to shy away from #2, if there is no good reason for it. But in
> either case, the user can choose to use it wisely or abuse it. It becomes
> more difficult to abuse in #2, but abuse is usually always possible.
> 
> The point I make is that software cannot organize things for you, unless
> some person has worked it out for you ahead of time. That person can't
> do that for my software, and I doubt he/she can for yours either.

[ In general neither 1 nor 2 is right. When we know little and just start
to explore a new field of computing, we always start with 1. Then we gather
knowledge, get frustrated with "flexibility" of making errors and switch 2.
That happens when we already know how-to. But having reached the stage 2,
we find it possible to do things we could not approach before and so open a
new field. Then again first 1, etc. ]

The point is, it seems that with installation/properties/profile issue we
have reached the stage when we could start to think about 2.

> To date, the registry idea has gone further than the /etc/file approach,
> and I have yet to see a better practical solution proposed. Of course I
> would be glad to hear a better approach if one exists.  ;-)
> 
>>>My point is simply that programs (including installers), need a
>>>programmatic way of testing what is installed etc. on a system.
>> 
>> Yes, but I would formulate it in more general way:
>> 
>> 1. object persistence (must be)
>> 2. object containers. An application can be viewed as a container for its
>> "mortal" instances and "immortal" persistent data. 
> 
> What does "object persistence" have more than a "set of values"? How
> is an "object container" any different than a directory/subdirectories
> containing "values"?

The difference is that the above is untyped or weakly typed. An object is
not just a value but also the methods to apply. This firstly would prevent
abuses, and secondly, we could define somewhere an abstract interface to be
implemented and so enforce proper behavior.

>>>>Consider a simple diamond diagram. Application A
>>>>uses B and C, they in turn use D. This cannot be represented by a tree. So
>>>>you need a folder to keep them all. Let's name it "/etc". Welcome to back
>>>>to chaos.
>>>
>>>I don't see any unsolvable problem here. Links and symlinks can
>>>create any kind of network you need.
>> 
>> Compare it with a network of pointers in C (memory~files, pointers~links).
>> Not an unsolvable problem, right, but...
> 
> But you have not proposed any solution.  You keep saying that
> "I don't want to be responsible", but there is nobody/nothing
> left to fill that void.

In OO the object is responsible to provide an implementation. A minimal
kernel should give an ability to define some system object. A pair levels
above there should be defined an abstract interface of application
persistent data. Each application that possesses such data have to
implement that interface etc.

> Not everything manifests itself as a diamond (obviously). If a
> hierarchy or "node network" doesn't satisfy your needs, then
> I'd like to hear just what other structure can.  You can't
> just close your eyes and say "just make it happen". We don't
> enjoy that luxury! 8->

Node network, relational data base, tree-like structure with links, all
that are just implementations of an abstract interface of what a repository
should provide. We will go no further if we will try to put the cart before
the horse. First we must define the interface, its implementation is a
minor problem.

>>>All it becomes is just different terminology, but you'll still have
>>>similar levels of abstraction (short-term memory vs long-term etc.
>>>for example).
>> 
>> Yes, but I do not want to expose it in API. It is comparable with evolution
>> of programming languages: from assemble with clear distinction between
>> registers and memory, modern languages which hide it. Caching should be
>> OS's business, down with files and I/O!
> 
> So you don't want an API, and yet you want it to happen somehow.

No, I do not want to see Read_Block and Write_Block in API. Writing an Ada
program, I do not see LDA, STR, MOV. They are still there, somewhere
beneath in the Underwolrd, but I don't care. 

> Somehow,
> some piece of software/hardware/system is supposed to know what pieces
> of information are configurable elements, know how to organize it,
> know how to identify it and share it in a safe way with other processes
> and users on the same system/network?

The answer is ADT. As long these pieces are just bytes, there will be
nothing much better than Windows/UNIX. It is time to make a technological
leap.

> Well, its ok to consider these kinds of questions, but if you really expect
> that to happen any time soon, I think you're more of a dreamer
> than I!

Yes I am.

I also fully agree with Nick Roberts when he says that developing an OS is
tightly related with the language. It is clear to me that Ada should be
used for new OS developing. Purely theoretically it could be C, but
practically the result will be null. Then even Ada is still unsuitable for
this, as long as there is no great type unification, which would allow to
develop user-defined protected objects, tasks, access types etc. This
should become the basic gear to build OS types. I also cannot imagine any
decent OO design of OS without MI, MD and interface inheritance from
concrete objects. Alas there is no visible efforts in these directions.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: For the AdaOS folks

[Warren W. Gay VE3WWG]

Dmitry A. Kazakov wrote:
> On Fri, 31 Dec 2004 11:24:40 -0500, Warren W. Gay VE3WWG wrote:
>>Dmitry A. Kazakov wrote:
>>>On Fri, 31 Dec 2004 06:30:51 -0500, Warren W. Gay VE3WWG wrote:
>>All of the infrastructure that you talk about is built outside of
>>the mk. The mk's purpose is to provide the basic needs for an OS, but
>>_not_ be the OS itself. In fact, using a mk approach, application and
>>OS modules conceptually run beside each other, perhaps only differing
>>in the privileges they have (ports that they have rights to).
> 
> I agree in principle, but disagree in details, in what are these basic
> needs. I think that a kind of "protected object" should be provided already
> by the kernel as a building block.

Ok, but if your Ada RTL provides it, do you really care that
much whether it is in or out of the OS?

> When we know little and just start
> to explore a new field of computing, we always start with 1. Then we gather
> knowledge, get frustrated with "flexibility" of making errors and switch 2.

This _assumes_ flexibility = errors. I don't believe this to
be generally true; but I'll agree that it _can_ be.

> That happens when we already know how-to. But having reached the stage 2,
> we find it possible to do things we could not approach before and so open a
> new field. Then again first 1, etc. ]

Perhaps, but it is not always good or necessary to straight-jacket
APIs. It should only be applied to curtail abuse, without limiting
the programming capability.

> The point is, it seems that with installation/properties/profile issue we
> have reached the stage when we could start to think about 2.

Sure, we can think about it, but IMO I don't think anyone has
gone beyond the registry in proposed solutions yet.  You say
"OO will do it better", but haven't supported the "why" it will.
You avoid that by not talking about "how". If I avoid answering
the "how", I could propose a lot of "better" ways.

>>What does "object persistence" have more than a "set of values"? How
>>is an "object container" any different than a directory/subdirectories
>>containing "values"?
> 
> The difference is that the above is untyped or weakly typed. An object is
> not just a value but also the methods to apply. This firstly would prevent
> abuses, and secondly, we could define somewhere an abstract interface to be
> implemented and so enforce proper behavior.

But a persistent object is nothing more than a blob of data
recorded in a file (or something acting as a container). The
fact that it might have methods makes no real difference. Who provides
the methods?  If you have to provide them, then you make matters
worse (we're back to everybody writing their own way to save
and retrieve data in their own way). If you don't, then this
cannot possibly anticipate your needs, unless you use something
like Ada stream I/O. But you don't want to commit to "how" this
is done, so it remains an unsubstantiated claim, and we cannot
progress in that discussion ;-)

>>But you have not proposed any solution.  You keep saying that
>>"I don't want to be responsible", but there is nobody/nothing
>>left to fill that void.
> 
> In OO the object is responsible to provide an implementation. 

But who writes the object? The kernel writer can't. How is he
going to anticipate all of your configuration needs for your
application and mine?

You already have Ada.Streams, but so what. I don't see that as
an improvement for this purpose.

> A minimal
> kernel should give an ability to define some system object. A pair levels
> above there should be defined an abstract interface of application
> persistent data. Each application that possesses such data have to
> implement that interface etc.

This still doesn't say "how" it is better, or "how" it will
be done. Without saying "how", you could promise the moon.

>>Not everything manifests itself as a diamond (obviously). If a
>>hierarchy or "node network" doesn't satisfy your needs, then
>>I'd like to hear just what other structure can.  You can't
>>just close your eyes and say "just make it happen". We don't
>>enjoy that luxury! 8->
> 
> Node network, relational data base, tree-like structure with links, all
> that are just implementations of an abstract interface of what a repository
> should provide. 

So what? Eventually everyone has to "choose" something.

 > We will go no further if we will try to put the cart before
> the horse. First we must define the interface, its implementation is a
> minor problem.

As part of any abstract discussion, you do have to eventually
consider the practical aspect of "how" it will be done. Otherwise
people can make all kinds of promises that can't be kept.

>>So you don't want an API, and yet you want it to happen somehow.
> 
> No, I do not want to see Read_Block and Write_Block in API. 

Sure, but who was proposing that?

> Writing an Ada
> program, I do not see LDA, STR, MOV. They are still there, somewhere
> beneath in the Underwolrd, but I don't care. 

Of course. But when you want to know where GNAT is installed, it
is very easy in Windows to lookup a registry key and get the
pathname for it. If I want to know where APQ should be installed
as an Ada binding, I can do another registry query and get the
top level GNAT directory for bindings.

Is that so complicated?  Does it really need to be more
complicated than that?

>>Somehow,
>>some piece of software/hardware/system is supposed to know what pieces
>>of information are configurable elements, know how to organize it,
>>know how to identify it and share it in a safe way with other processes
>>and users on the same system/network?
> 
> The answer is ADT. As long these pieces are just bytes, there will be
> nothing much better than Windows/UNIX. It is time to make a technological
> leap.

You keep saying "just bytes". Even the windows registry is better than
that, and you know that. Call it "weak typing" if you like, but you
can't say it is a bad idea because it _can_ be worse. Discuss the idea
on the merits of what it _can_ be.

How strongly typed do you make your pathnames in your programs? Do you
really define a different string type for different pathnames and
file names? Most people would find a String or Unbounded_String
acceptable.

...
> Purely theoretically it could be C, but
> practically the result will be null. Then even Ada is still unsuitable for
> this, as long as there is no great type unification, which would allow to
> develop user-defined protected objects, tasks, access types etc.
 > This should become the basic gear to build OS types.

Yes of course - ie. it must support the Ada RTL, at least.

 > I also cannot imagine any
> decent OO design of OS without MI, 

But the O/S has nothing to do with MI, unless it wants to sport
an OO interface (which IMHO is unnecessary, at least for many
APIs).

> MD and interface inheritance from
> concrete objects. 

If you force everything into an object metaphore, you make
it difficult for other languages. Frankly, for the services
that you get from an O/S, I don't see much point in putting
an OO face on it.

If you look around, you'll see a number of people are getting
less fanatical about objects, and in some cases are talking
"services" instead. And that is largely what the operating system
provides. But if you say you must have OO interfaces, then
the Ada RTL is there to let you model any way you see it.
You could even write packages for this, or if you don't like
the one you have write alternatives.

> Alas there is no visible efforts in these directions.

Practical ideas tend to get implemented (and thus become
quite visible). Impractical onces either remain dreams or
wait for the right opportunity. Either or both could apply,
depending upon your expectations ;-)
--
Warren W. Gay VE3WWG
http://home.cogeco.ca/~ve3wwg

Re: For the AdaOS folks

[Dmitry A. Kazakov]

On Sat, 01 Jan 2005 19:31:56 -0500, Warren W. Gay VE3WWG wrote:

> Dmitry A. Kazakov wrote:
>> On Fri, 31 Dec 2004 11:24:40 -0500, Warren W. Gay VE3WWG wrote:
>>>Dmitry A. Kazakov wrote:
>>>>On Fri, 31 Dec 2004 06:30:51 -0500, Warren W. Gay VE3WWG wrote:
>>>All of the infrastructure that you talk about is built outside of
>>>the mk. The mk's purpose is to provide the basic needs for an OS, but
>>>_not_ be the OS itself. In fact, using a mk approach, application and
>>>OS modules conceptually run beside each other, perhaps only differing
>>>in the privileges they have (ports that they have rights to).
>> 
>> I agree in principle, but disagree in details, in what are these basic
>> needs. I think that a kind of "protected object" should be provided already
>> by the kernel as a building block.
> 
> Ok, but if your Ada RTL provides it, do you really care that
> much whether it is in or out of the OS?

Yes, because Ada's protected objects are local and cooperative. It'd like
to see system services exposed as protected objects, with entry queues and
data fully memory protected. Same with tasks. I like OO interface to OS, no
compromises.

>>>What does "object persistence" have more than a "set of values"? How
>>>is an "object container" any different than a directory/subdirectories
>>>containing "values"?
>> 
>> The difference is that the above is untyped or weakly typed. An object is
>> not just a value but also the methods to apply. This firstly would prevent
>> abuses, and secondly, we could define somewhere an abstract interface to be
>> implemented and so enforce proper behavior.
> 
> But a persistent object is nothing more than a blob of data
> recorded in a file (or something acting as a container). The
> fact that it might have methods makes no real difference. Who provides
> the methods?

The type of the object is known. So the operations defined on it as well.

> If you have to provide them, then you make matters
> worse (we're back to everybody writing their own way to save
> and retrieve data in their own way).

No, it is fundamentally different. First because there is no "save and
retrieve". The object is persistent. There is only create and delete. The
latter is automatic because of garbage collection through reference
counting. Any other operations apart from construction, copy-construction
and destruction are of no interest for the first level of abstraction. On
the second level we could define some relationships between application,
host and user objects.

>>>But you have not proposed any solution.  You keep saying that
>>>"I don't want to be responsible", but there is nobody/nothing
>>>left to fill that void.
>> 
>> In OO the object is responsible to provide an implementation. 
> 
> But who writes the object? The kernel writer can't. How is he
> going to anticipate all of your configuration needs for your
> application and mine?

Where is any problem? If the kernel provides persistence and your
application needs, say an integer parameter. Derive from the base
Application_Data_Type add a field, here you are.

> You already have Ada.Streams, but so what. I don't see that as
> an improvement for this purpose.

Yes, Ada.Streams illustrates the idea. But problems with it are:

1. Not portable
2. No constructors/destructors
3. Oriented on sequential I/O
4. Double dispatch is hard-wired
5. Cannot deal with object dependencies (not inheritance, but references)

Basically, Ada's ADT should be extended in a way, which would allow user
types acting in a way similar to Ada.Streams.

>>>So you don't want an API, and yet you want it to happen somehow.
>> 
>> No, I do not want to see Read_Block and Write_Block in API. 
> 
> Sure, but who was proposing that?

Is Read_String and Write_String any better?

>> Writing an Ada
>> program, I do not see LDA, STR, MOV. They are still there, somewhere
>> beneath in the Underwolrd, but I don't care. 
> 
> Of course. But when you want to know where GNAT is installed, it
> is very easy in Windows to lookup a registry key and get the
> pathname for it. If I want to know where APQ should be installed
> as an Ada binding, I can do another registry query and get the
> top level GNAT directory for bindings.
> 
> Is that so complicated?

Yes it is. You have to know the names you are looking for! The first step
is of course to define some naming convention. This is a bad way, because
it is not enforced. See what happens with /etc and HKEY_LOCAL_MACHINE. So
the second step is to provide API to query that names from an application.
But that is still not good. A better way, as I said, is OO. You ask an
application: give me your settings and the result is an object of those.
The last step is to ask yourself what for? There is GNAT, an object why
cannot I talk to it directly?

> Does it really need to be more
> complicated than that?

No it need to be much more simpler!

> How strongly typed do you make your pathnames in your programs? Do you
> really define a different string type for different pathnames and
> file names?

Why there should be any pathnames? A name is used only to get an object
from a text map, provided that there should be one. Once you have the
object forget about its name. It has a type and so the methods to call.

>> I also cannot imagine any
>> decent OO design of OS without MI, 
> 
> But the O/S has nothing to do with MI, unless it wants to sport
> an OO interface (which IMHO is unnecessary, at least for many
> APIs).

Yes, this is the real source of our disagreement. As long as OS API are
non-OO you will fall back to bits and bytes or String and Unbounded_String,
if you want.
 
>> MD and interface inheritance from
>> concrete objects. 
> 
> If you force everything into an object metaphore, you make
> it difficult for other languages.

Did UNIX care much about Ada? We'll make them see! (:-)) OK, Visual Basic
users will have a virtual machine, it is a conventional ways to do such
things.

> Frankly, for the services
> that you get from an O/S, I don't see much point in putting
> an OO face on it.
> 
> If you look around, you'll see a number of people are getting
> less fanatical about objects, and in some cases are talking
> "services" instead.

Call it service, that's no matter. I use the word OO only because it is a
known buzz word. Actually I meant ADT. The service should be a typed
object. Its entry points should have typed parameters. Type checks have to
be enforced both by the language and by the kernel. Further most of the
calls to the entry points have to be dispatching. Dispatching tables should
be memory-protected.

> And that is largely what the operating system
> provides. But if you say you must have OO interfaces, then
> the Ada RTL is there to let you model any way you see it.
> You could even write packages for this, or if you don't like
> the one you have write alternatives.

No, I cannot. The problem is that tasks and protected objects are not
tagged. As you probably saw, I do not want to build a fat sandwich
OO-interface->procedural API->OO-kernel. It clumsy, ugly, buggy and
inefficient. I want to directly talk to OS. For this its API have to
provide, say, protected objects. They need to be implemented on very
different basis, than the buil-in ones. If some of them have to be memory
protected, they might require context switches, if some of them are remote,
then queuing would require networking, data marshaling etc

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: For the AdaOS folks

[Warren W. Gay VE3WWG]

Dmitry A. Kazakov wrote:
> On Sat, 01 Jan 2005 19:31:56 -0500, Warren W. Gay VE3WWG wrote:
>>Dmitry A. Kazakov wrote:
>>>On Fri, 31 Dec 2004 11:24:40 -0500, Warren W. Gay VE3WWG wrote:
>>>>Dmitry A. Kazakov wrote:
>>>>>On Fri, 31 Dec 2004 06:30:51 -0500, Warren W. Gay VE3WWG wrote:

> Yes, because Ada's protected objects are local and cooperative. It'd like
> to see system services exposed as protected objects, with entry queues and
> data fully memory protected. Same with tasks. I like OO interface to OS, no
> compromises.

If you use Mach ports, you already get all of that (just
think of the port as the handle to the object).

> Where is any problem? If the kernel provides persistence and your
> application needs, say an integer parameter. Derive from the base
> Application_Data_Type add a field, here you are.

The problem is where is the improvement over the registry
API that says create/delete an integer? Put your own OO
framework around it if that's what you want.

>>Of course. But when you want to know where GNAT is installed, it
>>is very easy in Windows to lookup a registry key and get the
>>pathname for it. If I want to know where APQ should be installed
>>as an Ada binding, I can do another registry query and get the
>>top level GNAT directory for bindings.
>>
>>Is that so complicated?
> 
> Yes it is. You have to know the names you are looking for! The first step
> is of course to define some naming convention. This is a bad way, because
> it is not enforced. See what happens with /etc and HKEY_LOCAL_MACHINE. So
> the second step is to provide API to query that names from an application.
> But that is still not good. A better way, as I said, is OO. You ask an
> application: give me your settings and the result is an object of those.
> The last step is to ask yourself what for? There is GNAT, an object why
> cannot I talk to it directly?

And how do your identify your GNAT object? At the end of the day,
you still have to grapple with "names" and "identities". Only
the details change.

> Why there should be any pathnames? A name is used only to get an object
> from a text map, provided that there should be one. 

So just how do you expect to identify what piece of software
you want to know about, without using a name (or identity if
you prefer)? OO doesn't do away with this requirement.

 > Once you have the
 > object forget about its name. It has a type and so the methods to call.

Sure, like an open file you use its handle or file descriptor. Under
Mach you use a port. So where's the problem? But when you open a
file system object, you still have to specify which object to
open by name. And a pathname is actually just a fancy composite
name, because you are naming several objects all at once (from
root directory down to the actual object).

> Yes, this is the real source of our disagreement. As long as OS API are
> non-OO you will fall back to bits and bytes or String and Unbounded_String,
> if you want.

Every strongly typed Ada entity is based upon bits, bytes, words and
arrays of such as base types. Ie. every type has a base type. OS RTL
libraries can strengthen the typing of the Ada interface. For a
C interface, not much really changes.

The O/S itself can present things in basic terms (registers and software
interrupts for example), but the OS RTL can present it in any form
you want. A binding. It is just that for an Ada based O/S, you
want the RTL to support it in a first class citizen manner.

> Did UNIX care much about Ada? We'll make them see! (:-)) 

The only problem with that is that you can make things hard for
yourself. For example, I have no grand plans that I or anyone
else I know, is going to convert the X-Window system to MyOS
anytime soon. The best compromise is to allow an optional POSIX
module on top of the mk, working with MyOS, to make it easy to
port the X-Window system, if that's what I want. GNAT/GCC is
another motive in that direction.

> Call it service, that's no matter. I use the word OO only because it is a
> known buzz word. Actually I meant ADT. The service should be a typed
> object. Its entry points should have typed parameters. Type checks have to
> be enforced both by the language and by the kernel. Further most of the
> calls to the entry points have to be dispatching. Dispatching tables should
> be memory-protected.

Well, maybe I lack vision on this point, but I don't see it. The
RTL can provide anything you lack at the O/S interface.

>>And that is largely what the operating system
>>provides. But if you say you must have OO interfaces, then
>>the Ada RTL is there to let you model any way you see it.
>>You could even write packages for this, or if you don't like
>>the one you have write alternatives.
> 
> No, I cannot. The problem is that tasks and protected objects are not
> tagged. 

So? Have the RTL put a tagged face on it. Call it a binding.
Essentially even traditional UNIX uses a thin libc binding
for O/S services for C programs.

> As you probably saw, I do not want to build a fat sandwich
> OO-interface->procedural API->OO-kernel. It clumsy, ugly, buggy and
> inefficient. I want to directly talk to OS. 

A couple of things. First of all, most microkernels that I am
aware of (or at least the ones I plan to use), all use a
procedural API (albeit with handles, if you want to call
them that).

Using a Mach type of approach, you write a bunch of executables
that behave as "servers". Application programs become "clients"
of these. This leaves the interface between the client user
mode program and the server(s), as a RPC API interface set.

Now MyOS (I'll just use that for example), would then of
course sport a nice native Ada interface for application
programs. However, underneath this "binding" (which is what
it really is), it will be calling an RPC message API that is
supported by the procedural API microkernel.

User-mode-client-OO->Ada Binding->RPC->Server(s)->microkernel

Can native objects be exported from operating systems? A
quick survey produces this list:

   - integer file descriptor (POSIX)
   - handles (Windows)
   - ports (Mach)
   - identities (L4)
   - ipc ID (UNIX SysV) for message queues, shared mem etc.

These are all fancy "handles" of one sort or another. You won't
get much more than that from any O/S. Why? How do you share an
object in protected-mode with a user-mode process? You can't.
Even if you could, there would be parts of objects that the
O/S wouldn't want you to mess with, or perhaps even look at.
Maybe you can't because the object is just a Mach port.

The best API answer seems to be to just provide a handle that
can be used to refer to the object (like an Ada access type),
or in the case of a Mach port, be the object metaphore. Then
you call varied API set (effectively methods) on that object,
using the given handle(s).

Then if you want it to look different than that, you provide
a binding. The general idea however, is that an Ada based
O/S will have native bindings. The packages won't be
adaptations of C header files etc.
--
Warren W. Gay VE3WWG
http://home.cogeco.ca/~ve3wwg

Re: For the AdaOS folks

[Dmitry A. Kazakov]

On Sun, 02 Jan 2005 17:04:29 -0500, Warren W. Gay VE3WWG wrote:

> Dmitry A. Kazakov wrote:
>> On Sat, 01 Jan 2005 19:31:56 -0500, Warren W. Gay VE3WWG wrote:

>> Where is any problem? If the kernel provides persistence and your
>> application needs, say an integer parameter. Derive from the base
>> Application_Data_Type add a field, here you are.
> 
> The problem is where is the improvement over the registry
> API that says create/delete an integer?

What if my application needs My_Fancy_Type, soundtrack of James Bond and
another application? 

> Put your own OO framework around it if that's what you want.

It is Turing complete, I know! (:-))

>> Yes it is. You have to know the names you are looking for! The first step
>> is of course to define some naming convention. This is a bad way, because
>> it is not enforced. See what happens with /etc and HKEY_LOCAL_MACHINE. So
>> the second step is to provide API to query that names from an application.
>> But that is still not good. A better way, as I said, is OO. You ask an
>> application: give me your settings and the result is an object of those.
>> The last step is to ask yourself what for? There is GNAT, an object why
>> cannot I talk to it directly?
> 
> And how do your identify your GNAT object? At the end of the day,
> you still have to grapple with "names" and "identities". Only
> the details change.

The "detail" is that you identify not some GNAT installation file #1534,
but GNAT. GNAT itself is easy to find, its components are not. They may
vary from version to version. It is GNAT's responsibility to know them, not
of GNAT user.

>> Why there should be any pathnames? A name is used only to get an object
>> from a text map, provided that there should be one. 
> 
> So just how do you expect to identify what piece of software
> you want to know about, without using a name (or identity if
> you prefer)? OO doesn't do away with this requirement.

ADT allows to organize it in a proper way. We are using types to forget
about bytes. In OSes we are still using "bytes" = files.

>> Did UNIX care much about Ada? We'll make them see! (:-)) 
> 
> The only problem with that is that you can make things hard for
> yourself. For example, I have no grand plans that I or anyone
> else I know, is going to convert the X-Window system to MyOS
> anytime soon. The best compromise is to allow an optional POSIX
> module on top of the mk, working with MyOS, to make it easy to
> port the X-Window system, if that's what I want. GNAT/GCC is
> another motive in that direction.

X11 and GCC are just applications. GNAT needs text files. Let you have an
Ada_Source object derived from what I know. Somewhere you will also have
Abstract_Text_Stream. If Ada_Source is not already a descendant of
Abstract_Text_Stream then derive a new type from both. End of story.

Ada run-time is another issue. I suppose you'd better have OS-native
tasking than tunnel it through POSIX. After all, it was you who started APQ
instead of using ODBC, did you? (:-))

>> As you probably saw, I do not want to build a fat sandwich
>> OO-interface->procedural API->OO-kernel. It clumsy, ugly, buggy and
>> inefficient. I want to directly talk to OS. 
> 
> A couple of things. First of all, most microkernels that I am
> aware of (or at least the ones I plan to use), all use a
> procedural API (albeit with handles, if you want to call
> them that).
> 
> Using a Mach type of approach, you write a bunch of executables
> that behave as "servers". Application programs become "clients"
> of these. This leaves the interface between the client user
> mode program and the server(s), as a RPC API interface set.
> 
> Now MyOS (I'll just use that for example), would then of
> course sport a nice native Ada interface for application
> programs. However, underneath this "binding" (which is what
> it really is), it will be calling an RPC message API that is
> supported by the procedural API microkernel.
> 
> User-mode-client-OO->Ada Binding->RPC->Server(s)->microkernel
> 
> Can native objects be exported from operating systems? A
> quick survey produces this list:
> 
>    - integer file descriptor (POSIX)
>    - handles (Windows)
>    - ports (Mach)
>    - identities (L4)
>    - ipc ID (UNIX SysV) for message queues, shared mem etc.
> 
> These are all fancy "handles" of one sort or another. You won't
> get much more than that from any O/S. Why? How do you share an
> object in protected-mode with a user-mode process? You can't.
> Even if you could, there would be parts of objects that the
> O/S wouldn't want you to mess with, or perhaps even look at.
> Maybe you can't because the object is just a Mach port.
> 
> The best API answer seems to be to just provide a handle that
> can be used to refer to the object (like an Ada access type),
> or in the case of a Mach port, be the object metaphore. Then
> you call varied API set (effectively methods) on that object,
> using the given handle(s).

It is not like Ada access type, because handle is not a pointer. It is a
key in some associative array. To "dereference" it you have to switch to
supervisor mode. Then you have to check it, because somebody could mess
with its value etc. It is very inefficient. You have to do all this even if
you call a function of a protected object, which could be safely performed
in user-mode. BTW, the very notion of user-mode is also of old procedural
fashion. A protected object is passive. You should be able to access it in
any "mode". But when you want to queue to its entry or to call its
procedure, only then you have to switch the context to make the object data
available to write. It is not kernel business anymore.

Your RPC services is another model. They are monitors. These can be exposed
in API as tasks to have rendezvous with. Again OO/ADT could bring great
advantages here. Imagine extensible tasks. Presently, if you have some
sequence of actions to be performed on one service, you have to care about
locking, serializing, then you implement it as separate inefficient RPCs to
the server. Service extensions would be sort of stored procedures in DB
terms.

> Then if you want it to look different than that, you provide
> a binding. The general idea however, is that an Ada based
> O/S will have native bindings. The packages won't be
> adaptations of C header files etc.

OK, but that is more like C programming in Ada to me. This new OS will not
become a breakthrough. It will be a stable decent OS, but who needs that?
See what happened with VMS? The only way to kill the Windows/UNIX monster
to offer something really revolutionary.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: For the AdaOS folks

[Warren W. Gay VE3WWG]

Dmitry A. Kazakov wrote:

> On Sun, 02 Jan 2005 17:04:29 -0500, Warren W. Gay VE3WWG wrote:
>>Dmitry A. Kazakov wrote:
>>>On Sat, 01 Jan 2005 19:31:56 -0500, Warren W. Gay VE3WWG wrote:
>>And how do your identify your GNAT object? At the end of the day,
>>you still have to grapple with "names" and "identities". Only
>>the details change.
> 
> The "detail" is that you identify not some GNAT installation file #1534,
> but GNAT. GNAT itself is easy to find, 

Is it? What if you 2 or 3 versions of it?  There is an interesting
situation now with windows as well, since you can have a cygwin
version in addition to the usual gnat-3.15p type of release.

> its components are not. They may
> vary from version to version. It is GNAT's responsibility to know them, not
> of GNAT user.

If I want to install a binding, I need to know where GNAT has installed
its bindings (like win32). The registry makes this a trivial thing to
do (I know, because I have used it).

>>So just how do you expect to identify what piece of software
>>you want to know about, without using a name (or identity if
>>you prefer)? OO doesn't do away with this requirement.
> 
> ADT allows to organize it in a proper way. We are using types to forget
> about bytes. In OSes we are still using "bytes" = files.

You haven't answered how you are going to identify your
object. This doesn't explain it away.

>>Can native objects be exported from operating systems? A
>>quick survey produces this list:
>>
>>   - integer file descriptor (POSIX)
>>   - handles (Windows)
>>   - ports (Mach)
>>   - identities (L4)
>>   - ipc ID (UNIX SysV) for message queues, shared mem etc.
>>
>>These are all fancy "handles" of one sort or another. You won't
>>get much more than that from any O/S. Why? How do you share an
>>object in protected-mode with a user-mode process? You can't.
>>Even if you could, there would be parts of objects that the
>>O/S wouldn't want you to mess with, or perhaps even look at.
>>Maybe you can't because the object is just a Mach port.
>>
>>The best API answer seems to be to just provide a handle that
>>can be used to refer to the object (like an Ada access type),
>>or in the case of a Mach port, be the object metaphore. Then
>>you call varied API set (effectively methods) on that object,
>>using the given handle(s).
> 
> It is not like Ada access type, because handle is not a pointer. 

I am not suggesting they are the same, but merely that they are
used for the same purpose.

> It is a
> key in some associative array. To "dereference" it you have to switch to
> supervisor mode. Then you have to check it, because somebody could mess
> with its value etc. It is very inefficient. You have to do all this even if
> you call a function of a protected object, which could be safely performed
> in user-mode. BTW, the very notion of user-mode is also of old procedural
> fashion. A protected object is passive. You should be able to access it in
> any "mode". But when you want to queue to its entry or to call its
> procedure, only then you have to switch the context to make the object data
> available to write. It is not kernel business anymore.

But if you were to replace the file descriptor with an object that
implements the equivalent of a v-node, you wouldn't be able to do
this in "user mode".

> Your RPC services is another model. They are monitors. These can be exposed
> in API as tasks to have rendezvous with. 

Or they could be simple procedure/function calls in Ada. It is merely
a point of presentation.

> Again OO/ADT could bring great
> advantages here. Imagine extensible tasks. Presently, if you have some
> sequence of actions to be performed on one service, you have to care about
> locking, serializing, then you implement it as separate inefficient RPCs to
> the server. Service extensions would be sort of stored procedures in DB
> terms.

Again, you can package it any way you like already. Just wrap it
in the trimmings you want.

>>Then if you want it to look different than that, you provide
>>a binding. The general idea however, is that an Ada based
>>O/S will have native bindings. The packages won't be
>>adaptations of C header files etc.
> 
> OK, but that is more like C programming in Ada to me. 

Why?

> This new OS will not
> become a breakthrough. It will be a stable decent OS, but who needs that?

Firewalls for one.

> See what happened with VMS? The only way to kill the Windows/UNIX monster
> to offer something really revolutionary.

I am not on any quest for world domination, though I don't
discourage others from trying to do so ;-)
--
Warren W. Gay VE3WWG
http://home.cogeco.ca/~ve3wwg

Re: For the AdaOS folks

[Dmitry A. Kazakov]

On Mon, 03 Jan 2005 11:36:34 -0500, Warren W. Gay VE3WWG wrote:

> Dmitry A. Kazakov wrote:
> 
>> On Sun, 02 Jan 2005 17:04:29 -0500, Warren W. Gay VE3WWG wrote:
>>>Dmitry A. Kazakov wrote:
>>>>On Sat, 01 Jan 2005 19:31:56 -0500, Warren W. Gay VE3WWG wrote:
>>>And how do your identify your GNAT object? At the end of the day,
>>>you still have to grapple with "names" and "identities". Only
>>>the details change.
>> 
>> The "detail" is that you identify not some GNAT installation file #1534,
>> but GNAT. GNAT itself is easy to find, 
> 
> Is it? What if you 2 or 3 versions of it?  There is an interesting
> situation now with windows as well, since you can have a cygwin
> version in addition to the usual gnat-3.15p type of release.

OK, these will be different objects. An application should probably be a
factory object of its versions. So you will ask GNAT factory: give me an
instance of GNAT with the parameters so and so.

>> its components are not. They may
>> vary from version to version. It is GNAT's responsibility to know them, not
>> of GNAT user.
> 
> If I want to install a binding, I need to know where GNAT has installed
> its bindings (like win32). The registry makes this a trivial thing to
> do (I know, because I have used it).

Nowhere. It is installed and available.

>>>So just how do you expect to identify what piece of software
>>>you want to know about, without using a name (or identity if
>>>you prefer)? OO doesn't do away with this requirement.
>> 
>> ADT allows to organize it in a proper way. We are using types to forget
>> about bytes. In OSes we are still using "bytes" = files.
> 
> You haven't answered how you are going to identify your
> object. This doesn't explain it away.

The same way you identify objects in your program:

declare
   GNAT_Installation : Application_Type'Class :=
      Get (Local_Machine, "GNAT");
begin
   ...

>> It is a
>> key in some associative array. To "dereference" it you have to switch to
>> supervisor mode. Then you have to check it, because somebody could mess
>> with its value etc. It is very inefficient. You have to do all this even if
>> you call a function of a protected object, which could be safely performed
>> in user-mode. BTW, the very notion of user-mode is also of old procedural
>> fashion. A protected object is passive. You should be able to access it in
>> any "mode". But when you want to queue to its entry or to call its
>> procedure, only then you have to switch the context to make the object data
>> available to write. It is not kernel business anymore.
> 
> But if you were to replace the file descriptor with an object that
> implements the equivalent of a v-node, you wouldn't be able to do
> this in "user mode".

Your application may have read-only mapping to the public parts of the
descriptor.

>> Your RPC services is another model. They are monitors. These can be exposed
>> in API as tasks to have rendezvous with. 
> 
> Or they could be simple procedure/function calls in Ada. It is merely
> a point of presentation.

You cannot make a conditional or timed call to a procedure. You cannot
requeue to a procedure. Why not to use the advantages Ada gives?

>> Again OO/ADT could bring great
>> advantages here. Imagine extensible tasks. Presently, if you have some
>> sequence of actions to be performed on one service, you have to care about
>> locking, serializing, then you implement it as separate inefficient RPCs to
>> the server. Service extensions would be sort of stored procedures in DB
>> terms.
> 
> Again, you can package it any way you like already. Just wrap it
> in the trimmings you want.

Turing completeness...

>> This new OS will not
>> become a breakthrough. It will be a stable decent OS, but who needs that?
> 
> Firewalls for one.

Huh, firewalls only exist because of that crippled, buggy, unsafe OSes!
 
>> See what happened with VMS? The only way to kill the Windows/UNIX monster
>> to offer something really revolutionary.
> 
> I am not on any quest for world domination, though I don't
> discourage others from trying to do so ;-)

Seriously, I do think that Windows/UNIX impact on history of humankind is
yet to estimate. But it is another story...

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: For the AdaOS folks

[Warren W. Gay VE3WWG]

Dmitry A. Kazakov wrote:

> On Mon, 03 Jan 2005 11:36:34 -0500, Warren W. Gay VE3WWG wrote:
>>Dmitry A. Kazakov wrote:
>>>On Sun, 02 Jan 2005 17:04:29 -0500, Warren W. Gay VE3WWG wrote:
>>>>Dmitry A. Kazakov wrote:
>>>>>On Sat, 01 Jan 2005 19:31:56 -0500, Warren W. Gay VE3WWG wrote:
>>>>
>>>>And how do your identify your GNAT object? At the end of the day,
>>>>you still have to grapple with "names" and "identities". Only
>>>>the details change.
>>>
>>>The "detail" is that you identify not some GNAT installation file #1534,
>>>but GNAT. GNAT itself is easy to find, 
>>
>>Is it? What if you 2 or 3 versions of it?  There is an interesting
>>situation now with windows as well, since you can have a cygwin
>>version in addition to the usual gnat-3.15p type of release.
> 
> OK, these will be different objects. An application should probably be a
> factory object of its versions. So you will ask GNAT factory: give me an
> instance of GNAT with the parameters so and so.

Aha! The name/identification is couched in the terms
"parameters so and so".  You cannot dispense with names.

>>>its components are not. They may
>>>vary from version to version. It is GNAT's responsibility to know them, not
>>>of GNAT user.
>>
>>If I want to install a binding, I need to know where GNAT has installed
>>its bindings (like win32). The registry makes this a trivial thing to
>>do (I know, because I have used it).
> 
> Nowhere. It is installed and available.

Pardon me? The GNAT win32 binding is installed in a very
specific location. I place APQ in the same parent directory.

>>>>So just how do you expect to identify what piece of software
>>>>you want to know about, without using a name (or identity if
>>>>you prefer)? OO doesn't do away with this requirement.
>>>
>>>ADT allows to organize it in a proper way. We are using types to forget
>>>about bytes. In OSes we are still using "bytes" = files.
>>
>>You haven't answered how you are going to identify your
>>object. This doesn't explain it away.
> 
> The same way you identify objects in your program:
> 
> declare
>    GNAT_Installation : Application_Type'Class :=
>       Get (Local_Machine, "GNAT");
> begin
>    ...

Which version of two does this get?

>>But if you were to replace the file descriptor with an object that
>>implements the equivalent of a v-node, you wouldn't be able to do
>>this in "user mode".
> 
> Your application may have read-only mapping to the public parts of the
> descriptor.

But then it won't function!  For example, how can you
use a kernel mode pointer within the v-node in user mode?

>>>This new OS will not
>>>become a breakthrough. It will be a stable decent OS, but who needs that?
>>
>>Firewalls for one.
> 
> Huh, firewalls only exist because of that crippled, buggy, unsafe OSes!

Ah, not necessarily. To enforce policy, it makes sense to throttle
all of a company's access to the world-wild-net through one "gate",
rather than try to administer it on the bases of every host, pc,
and laptop within an organization.

Even at home, there is much more safety in doing things this way.
--
Warren W. Gay VE3WWG
http://home.cogeco.ca/~ve3wwg

Re: For the AdaOS folks

[Dmitry A. Kazakov]

On Mon, 03 Jan 2005 14:01:42 -0500, Warren W. Gay VE3WWG wrote:

> Dmitry A. Kazakov wrote:
> 
>> On Mon, 03 Jan 2005 11:36:34 -0500, Warren W. Gay VE3WWG wrote:
>>>Dmitry A. Kazakov wrote:
>>>>On Sun, 02 Jan 2005 17:04:29 -0500, Warren W. Gay VE3WWG wrote:
>>>>>Dmitry A. Kazakov wrote:
>>>>>>On Sat, 01 Jan 2005 19:31:56 -0500, Warren W. Gay VE3WWG wrote:
>>>>>
>>>>>And how do your identify your GNAT object? At the end of the day,
>>>>>you still have to grapple with "names" and "identities". Only
>>>>>the details change.
>>>>
>>>>The "detail" is that you identify not some GNAT installation file #1534,
>>>>but GNAT. GNAT itself is easy to find, 
>>>
>>>Is it? What if you 2 or 3 versions of it?  There is an interesting
>>>situation now with windows as well, since you can have a cygwin
>>>version in addition to the usual gnat-3.15p type of release.
>> 
>> OK, these will be different objects. An application should probably be a
>> factory object of its versions. So you will ask GNAT factory: give me an
>> instance of GNAT with the parameters so and so.
> 
> Aha! The name/identification is couched in the terms
> "parameters so and so".  You cannot dispense with names.

The parameters needed to identify your version. BTW I am not sure that it
is the best possible way to deal with versions. It is yet another story. In
any case it would be easier to handle versions using objects instead of
names and paths.

>>>>its components are not. They may
>>>>vary from version to version. It is GNAT's responsibility to know them, not
>>>>of GNAT user.
>>>
>>>If I want to install a binding, I need to know where GNAT has installed
>>>its bindings (like win32). The registry makes this a trivial thing to
>>>do (I know, because I have used it).
>> 
>> Nowhere. It is installed and available.
> 
> Pardon me? The GNAT win32 binding is installed in a very
> specific location. I place APQ in the same parent directory.

Forget about locations. GNAT object will have a hook (or a container) to
attach plug-ins. Your APQ is just a new object to attach there.

>>>>>So just how do you expect to identify what piece of software
>>>>>you want to know about, without using a name (or identity if
>>>>>you prefer)? OO doesn't do away with this requirement.
>>>>
>>>>ADT allows to organize it in a proper way. We are using types to forget
>>>>about bytes. In OSes we are still using "bytes" = files.
>>>
>>>You haven't answered how you are going to identify your
>>>object. This doesn't explain it away.
>> 
>> The same way you identify objects in your program:
>> 
>> declare
>>    GNAT_Installation : Application_Type'Class :=
>>       Get (Local_Machine, "GNAT");
>> begin
>>    ...
> 
> Which version of two does this get?

All. You continue then:

Get_Conform_To (GNAT_Installation, Required_Version);

>>>But if you were to replace the file descriptor with an object that
>>>implements the equivalent of a v-node, you wouldn't be able to do
>>>this in "user mode".
>> 
>> Your application may have read-only mapping to the public parts of the
>> descriptor.
> 
> But then it won't function!  For example, how can you
> use a kernel mode pointer within the v-node in user mode?

Your application will be mapped there. To obtain an object is like loading
a library. The pointer you will use to read public data will be a user-mode
one.

>>>>This new OS will not
>>>>become a breakthrough. It will be a stable decent OS, but who needs that?
>>>
>>>Firewalls for one.
>> 
>> Huh, firewalls only exist because of that crippled, buggy, unsafe OSes!
> 
> Ah, not necessarily. To enforce policy, it makes sense to throttle
> all of a company's access to the world-wild-net through one "gate",
> rather than try to administer it on the bases of every host, pc,
> and laptop within an organization.

But in our hypothetical OS each possible way of access will be represented
by some safe system object. These objects, when properly designed will
provide necessary administrative services. Do you have one "gate" for hard
drive I/O? Do you need a firewall to tunnel open/close/read/write to floppy
drives? It would be nonsense. The problem is that network protocols do not
have safety of a file system. The foundations of file systems were
established long before the disaster named MS.

> Even at home, there is much more safety in doing things this way.

It an imaginary safety.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: For the AdaOS folks

[Warren W. Gay VE3WWG]

Dmitry A. Kazakov wrote:
> On Mon, 03 Jan 2005 14:01:42 -0500, Warren W. Gay VE3WWG wrote:
>>Dmitry A. Kazakov wrote:
>>>On Mon, 03 Jan 2005 11:36:34 -0500, Warren W. Gay VE3WWG wrote:
>>>>Dmitry A. Kazakov wrote:
>>>>>On Sun, 02 Jan 2005 17:04:29 -0500, Warren W. Gay VE3WWG wrote:
>>>>>>Dmitry A. Kazakov wrote:
>>>>>>>On Sat, 01 Jan 2005 19:31:56 -0500, Warren W. Gay VE3WWG wrote:
>>>Huh, firewalls only exist because of that crippled, buggy, unsafe OSes!
>>
>>Ah, not necessarily. To enforce policy, it makes sense to throttle
>>all of a company's access to the world-wild-net through one "gate",
>>rather than try to administer it on the bases of every host, pc,
>>and laptop within an organization.
> 
> But in our hypothetical OS each possible way of access will be represented
> by some safe system object. These objects, when properly designed will
> provide necessary administrative services. 

If you are a night watchman for a Mall, which situation makes it
easier to sleep at night when you've locked up and gone home?

   1. A mall with one or two doors on the outside to be
      locked and checked.
   2. A mall with thousands of doors on the outside to be
      locked and checked.

The answer is obvious. Sure, it is ok for other doors to exist
inside the mall (for each store), which can be locked, but it
only makes sense to choke the security at a minimal number
of points.

> Do you have one "gate" for hard
> drive I/O? 

Yes, actually. The kernel controls the issuing of the IDE
commands, so that no process can permanently destroy the
IDE drive (which can be done, if certain commands are issued).
Not to mention that partition scope(s) must be enforced.

File systems mitigate access to the thousands of objects
that exist within the file system. In a hierarchical system
of directories, you have upper levels of choke points (in
parent directories), as well as the ability to control
access on the object itself.

> Do you need a firewall to tunnel open/close/read/write to floppy
> drives? It would be nonsense. 

Maybe its not your floppy. Maybe it belongs to
another user (perhaps a student/coworker/spouse).

> The problem is that network protocols do not
> have safety of a file system. 

A file system is confined. A network is exposed by
definition. That is the element that makes network
security so difficult. It has very little to do
with which came first.

>>Even at home, there is much more safety in doing things this way.
> 
> It an imaginary safety.

Not at all. While it is not the entire answer to network
security, you court disaster without one. You will not find
one network security expert to suggest what you are promoting.
--
Warren W. Gay VE3WWG
http://home.cogeco.ca/~ve3wwg

Re: For the AdaOS folks

[Randy Brukardt]

"Warren W. Gay VE3WWG" <ve3wwg@NoSpam.cogeco.ca> wrote in message
news:z7iCd.15318$Y_4.1372013@read2.cgocable.net...
> > But in our hypothetical OS each possible way of access will be
represented
> > by some safe system object. These objects, when properly designed will
> > provide necessary administrative services.
>
> If you are a night watchman for a Mall, which situation makes it
> easier to sleep at night when you've locked up and gone home?
>
>    1. A mall with one or two doors on the outside to be
>       locked and checked.
>    2. A mall with thousands of doors on the outside to be
>       locked and checked.
>
> The answer is obvious. Sure, it is ok for other doors to exist
> inside the mall (for each store), which can be locked, but it
> only makes sense to choke the security at a minimal number
> of points.

There are other requirements than security! I think the fire dept. would be
pretty unhappy with (1) - at least if you ever let anybody in. Same holds
for OSes; you're going to need multiple ways to get things done; you can try
to minimize the number of "doors", but its unlikely that you can get down to
just one.

BTW, I think Dmitry's grand vision is on the right track. The tough part (as
always) is mapping that to something that still is practical. Why shouldn't
an Ada OS take advantage of Ada's features to provide persistence,
synchronization, and the like? Sure, you'll have to go beyond that, too, but
why not start with Ada's fine features in this area?

                      Randy.

Re: For the AdaOS folks

[Warren W. Gay VE3WWG]

Randy Brukardt wrote:
> "Warren W. Gay VE3WWG" <ve3wwg@NoSpam.cogeco.ca> wrote in message
>>>But in our hypothetical OS each possible way of access will be
>>>represented
>>>by some safe system object. These objects, when properly designed will
>>>provide necessary administrative services.
>>
>>If you are a night watchman for a Mall, which situation makes it
>>easier to sleep at night when you've locked up and gone home?
>>
>>   1. A mall with one or two doors on the outside to be
>>      locked and checked.
>>   2. A mall with thousands of doors on the outside to be
>>      locked and checked.
>>
>>The answer is obvious. Sure, it is ok for other doors to exist
>>inside the mall (for each store), which can be locked, but it
>>only makes sense to choke the security at a minimal number
>>of points.
> 
> There are other requirements than security! 

Of course. Did I ever claim that this was the
only requirement?

> I think the fire dept. would be
> pretty unhappy with (1) - at least if you ever let anybody in. Same holds
> for OSes; you're going to need multiple ways to get things done; you can try
> to minimize the number of "doors", but its unlikely that you can get down to
> just one.

Yes, practical issues prevail, but to use a different example, I doubt that
banks would like the idea of buying vaults with more than one door.

I was illustrating a "general principle", and I stand by it as a principle.
Will there be exceptions to it? Probably and it depends on the specific
problem you are solving. Exceptions are generally understood to exist
when discussing general principles.

> BTW, I think Dmitry's grand vision is on the right track. The tough part (as
> always) is mapping that to something that still is practical. 

Exactly. We can all talk about grand visions until the cows come home,
but that doesn't get us anywhere. I don't have a problem with "vision",
but it needs to be discussed in the context of "how are we going
to accomplish it?"  Otherwise, it is a vision that will never be.

> Why shouldn't
> an Ada OS take advantage of Ada's features to provide persistence,
> synchronization, and the like? 

I'm not against these basic needs, and certainly not against Ada,
or Ada methods (this is the whole reason I embarked on this project).
But Ada BTW, is where some of the work belongs (for example Ada.Streams).
In the mean time, the project will have to at least consider alternatives
if portability becomes one of the goals (I am not sure that it must be).

> Sure, you'll have to go beyond that, too, but
> why not start with Ada's fine features in this area?

Randy, this is precisely the issue (going beyond that). How?  No
"how" is being offered in the present discussion. I am pretty
open minded about things that make sense to do, provided that
they _can_ be done (and provided that all of the extra work doesn't
fall to me.)

I can talk about a lot of pie-in-the-sky stuff too. But if we don't
ever take it beyond that, it will just become another empty project
on Source Forge. There are enough of those already.

In fact, the "registry" issue always seems to bring out strong
feelings. So I say, go off and SF a project to fix it the way
_you_ want it to work. It need not require a new O/S to prototype
So if someone wants to prove me wrong with a prototype - I
welcome the event!

As a side note, my little project will eventually be put up on
SF, so that others can meddle with or otherwise contribute to it,
if they want. But my suspicion is that a lot of hand waving will
disappear when it comes to actually implementing something. ;-)
-- 
Warren W. Gay VE3WWG
http://home.cogeco.ca/~ve3wwg

Re: For the AdaOS folks

[Nick Roberts]

Sorry for the brief reply here. I'm going to try to write a few 'technology
papers' on AdaOS as quickly as I can, which will then be posted on the AdaOS
web site.

-- 
Nick Roberts

Re: For the AdaOS folks

[Dmitry A. Kazakov]

On Mon, 03 Jan 2005 15:44:17 -0500, Warren W. Gay VE3WWG wrote:

> Dmitry A. Kazakov wrote:
>> 
>> But in our hypothetical OS each possible way of access will be represented
>> by some safe system object. These objects, when properly designed will
>> provide necessary administrative services. 
> 
> If you are a night watchman for a Mall, which situation makes it
> easier to sleep at night when you've locked up and gone home?
> 
>    1. A mall with one or two doors on the outside to be
>       locked and checked.
>    2. A mall with thousands of doors on the outside to be
>       locked and checked.
> 
> The answer is obvious. Sure, it is ok for other doors to exist
> inside the mall (for each store), which can be locked, but it
> only makes sense to choke the security at a minimal number
> of points.

But you can approach the problem in other ways. You could change people to
make impossible for somebody to steal. You could make objects unusable when
stolen etc.

>> Do you have one "gate" for hard drive I/O? 
> 
> Yes, actually. The kernel controls the issuing of the IDE
> commands, so that no process can permanently destroy the
> IDE drive (which can be done, if certain commands are issued).
> Not to mention that partition scope(s) must be enforced.

It is no different from handling TCP/IP sockets. So the problem lies
elsewhere above. Anybody may try to open a file.

> File systems mitigate access to the thousands of objects
> that exist within the file system. In a hierarchical system
> of directories, you have upper levels of choke points (in
> parent directories), as well as the ability to control
> access on the object itself.

Yes, that is the point. Files are primitive, but objects. It is much easier
to enforce security in a hierarchical system than in a flat sea of
unstructured data.

>> Do you need a firewall to tunnel open/close/read/write to floppy
>> drives? It would be nonsense. 
> 
> Maybe its not your floppy. Maybe it belongs to
> another user (perhaps a student/coworker/spouse).

But how a tunnel might help with that? It does not know who is the owner.

>> The problem is that network protocols do not
>> have safety of a file system. 
> 
> A file system is confined.

Come on, there were multi-user OSes before Windows. Even UNIX pretended to
be one.

> A network is exposed by
> definition. That is the element that makes network
> security so difficult. It has very little to do
> with which came first.
> 
>>>Even at home, there is much more safety in doing things this way.
>> 
>> It an imaginary safety.
> 
> Not at all. While it is not the entire answer to network
> security, you court disaster without one. You will not find
> one network security expert to suggest what you are promoting.

Sure, why should they kill a hen carrying the gold eggs? (:-)) Did you ever
hear from any company selling anti-virus software that the only problem
with viruses is OS?

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: For the AdaOS folks

[Warren W. Gay VE3WWG]

Dmitry A. Kazakov wrote:

> On Mon, 03 Jan 2005 15:44:17 -0500, Warren W. Gay VE3WWG wrote:
>>Dmitry A. Kazakov wrote:
>>
>>>But in our hypothetical OS each possible way of access will be represented
>>>by some safe system object. These objects, when properly designed will
>>>provide necessary administrative services. 
>>
>>If you are a night watchman for a Mall, which situation makes it
>>easier to sleep at night when you've locked up and gone home?
>>
>>   1. A mall with one or two doors on the outside to be
>>      locked and checked.
>>   2. A mall with thousands of doors on the outside to be
>>      locked and checked.
>>
>>The answer is obvious. Sure, it is ok for other doors to exist
>>inside the mall (for each store), which can be locked, but it
>>only makes sense to choke the security at a minimal number
>>of points.
> 
> But you can approach the problem in other ways. You could change people to
> make impossible for somebody to steal. You could make objects unusable when
> stolen etc.

How much chance do you think that this has of working with PCs,
laptops, servers etc. that might run an new O/S?  You're not
a practical man.

>>>Do you have one "gate" for hard drive I/O? 
>>
>>Yes, actually. The kernel controls the issuing of the IDE
>>commands, so that no process can permanently destroy the
>>IDE drive (which can be done, if certain commands are issued).
>>Not to mention that partition scope(s) must be enforced.
> 
> It is no different from handling TCP/IP sockets. So the problem lies
> elsewhere above. Anybody may try to open a file.

I'm just going to bite my lip on this one.

>>File systems mitigate access to the thousands of objects
>>that exist within the file system. In a hierarchical system
>>of directories, you have upper levels of choke points (in
>>parent directories), as well as the ability to control
>>access on the object itself.
> 
> Yes, that is the point. Files are primitive, but objects. It is much easier
> to enforce security in a hierarchical system than in a flat sea of
> unstructured data.

But a firewall prevents you from accessing any of my files at home ;-)
and my files at work.

Sure, there is also an account+password, more networking, and
more controls behind it. But the one I really count on Dmitry, is
that firewall.

>>>Do you need a firewall to tunnel open/close/read/write to floppy
>>>drives? It would be nonsense. 
>>
>>Maybe its not your floppy. Maybe it belongs to
>>another user (perhaps a student/coworker/spouse).
> 
> But how a tunnel might help with that? It does not know who is the owner.

Not a problem. I can determine who accesses the floppy
when it is mounted (look up the mount command).

>>>The problem is that network protocols do not
>>>have safety of a file system. 
>>
>>A file system is confined.
> 
> Come on, there were multi-user OSes before Windows. Even UNIX pretended to
> be one.

So? Who gets an account? (approved folk).

Who is on the internet? (everyone, including hackers, nobody excluded)

There is a difference, and there are other differences also.

>>Not at all. While it is not the entire answer to network
>>security, you court disaster without one. You will not find
>>one network security expert to suggest what you are promoting.
> 
> Sure, why should they kill a hen carrying the gold eggs? (:-)) 

It sounds like the golden egg is on your system(s) - especially
if you don't believe in firewalls ;-)

> Did you ever
> hear from any company selling anti-virus software that the only problem
> with viruses is OS?

I'm not going to bite. I'll just bite my lip instead ;-)
-- 
Warren W. Gay VE3WWG
http://home.cogeco.ca/~ve3wwg

Re: For the AdaOS folks

[Dmitry A. Kazakov]

On Tue, 04 Jan 2005 13:00:04 -0500, Warren W. Gay VE3WWG wrote:

> Dmitry A. Kazakov wrote:
> 
>You're not a practical man.

Nor you are. We both stick to Ada! (:-))

>>>File systems mitigate access to the thousands of objects
>>>that exist within the file system. In a hierarchical system
>>>of directories, you have upper levels of choke points (in
>>>parent directories), as well as the ability to control
>>>access on the object itself.
>> 
>> Yes, that is the point. Files are primitive, but objects. It is much easier
>> to enforce security in a hierarchical system than in a flat sea of
>> unstructured data.
> 
> But a firewall prevents you from accessing any of my files at home ;-)
> and my files at work.
> 
> Sure, there is also an account+password, more networking, and
> more controls behind it. But the one I really count on Dmitry, is
> that firewall.

But the only need in firewall is the policy of trusting behind it. Any
program may read your address book. Why your address book allows that? The
problem of the firewall approach is that the firewall has to know all
possible ways of misusing all possible system resources. Everything in me
cries that this is a wrong design, per definition wrong.

>>>>Do you need a firewall to tunnel open/close/read/write to floppy
>>>>drives? It would be nonsense. 
>>>
>>>Maybe its not your floppy. Maybe it belongs to
>>>another user (perhaps a student/coworker/spouse).
>> 
>> But how a tunnel might help with that? It does not know who is the owner.
> 
> Not a problem. I can determine who accesses the floppy
> when it is mounted (look up the mount command).

Yes, but once mounted it is accessible for all. Actually it is the file
system with its access rights to the files, that makes access safe, not
only the mount command.

>>>>The problem is that network protocols do not
>>>>have safety of a file system. 
>>>
>>>A file system is confined.
>> 
>> Come on, there were multi-user OSes before Windows. Even UNIX pretended to
>> be one.
> 
> So? Who gets an account? (approved folk).
>
> Who is on the internet? (everyone, including hackers, nobody excluded)

Stop, the definition of a true multi-user system is that ideally you should
be unable to observe any effects of actions of other people (if you do not
want to, of course.) If a hacker cannot influence your work, do you care
whether he has an account or not? The real difference is that in the
internet everybody is "root".

>>>Not at all. While it is not the entire answer to network
>>>security, you court disaster without one. You will not find
>>>one network security expert to suggest what you are promoting.
>> 
>> Sure, why should they kill a hen carrying the gold eggs? (:-)) 
> 
> It sounds like the golden egg is on your system(s) - especially
> if you don't believe in firewalls ;-)

One my colleague adamantly refused to replace Windows NT 4.0 with XP on his
box. He argued that though MS does not plan any new service packs for NT,
neither do viruses developers! (:-))

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: For the AdaOS folks

[Warren W. Gay VE3WWG]

Dmitry A. Kazakov wrote:
> On Tue, 04 Jan 2005 13:00:04 -0500, Warren W. Gay VE3WWG wrote:
>>Dmitry A. Kazakov wrote:
>>
>>You're not a practical man.
> 
> Nor you are. We both stick to Ada! (:-))

OK.

> But the only need in firewall is the policy of trusting behind it.

That is all I need to keep you from messing with my files ;-)

> Any
> program may read your address book. Why your address book allows that? The
> problem of the firewall approach is that the firewall has to know all
> possible ways of misusing all possible system resources. Everything in me
> cries that this is a wrong design, per definition wrong.

The firewall is one cog in the security plan. It is like the root
directory, that is quite capable of preventing people from
gaining access to subdirectories and files. It is like the
first "wall" that you hit (hence the name).

>>Not a problem. I can determine who accesses the floppy
>>when it is mounted (look up the mount command).
> 
> Yes, but once mounted it is accessible for all. Actually it is the file
> system with its access rights to the files, that makes access safe, not
> only the mount command.

You didn't do your homework on this one:

Mount options for fat

   uid=value and gid=value
     Set the owner and group of all files. (Default: the uid
     and gid of the current process.)

>>>>>The problem is that network protocols do not
>>>>>have safety of a file system. 
>>>>
>>>>A file system is confined.
>>>
>>>Come on, there were multi-user OSes before Windows. Even UNIX pretended to
>>>be one.
>>
>>So? Who gets an account? (approved folk).
>>
>>Who is on the internet? (everyone, including hackers, nobody excluded)
> 
> 
> Stop, the definition of a true multi-user system is that ideally you should
> be unable to observe any effects of actions of other people (if you do not
> want to, of course.) If a hacker cannot influence your work, do you care
> whether he has an account or not? 

I forget how we got here, but I do agree that a secure O/S should
permit "hostile user accounts". This is one my goals actually.

But even if I had such a secure system, I would not dispense with
the firewall. If you disagree, then fine - we'll leave at that.

> The real difference is that in the
> internet everybody is "root".

I think I understand the point you are making, but to be fair,
even this is not quite equivalent. Having root means having
access to the account. On the net, you are hoping to acquire
access (usually to root, directly or indirectly), by
observation.

> One my colleague adamantly refused to replace Windows NT 4.0 with XP on his
> box. He argued that though MS does not plan any new service packs for NT,
> neither do viruses developers! (:-))

You are lucky if you can install Win98, and
get the service packs/updates before it gets riddled with
viruses. Without a firewall, you might be good for 10 minutes,
if you're lucky. Picture a Clint Eastwood dialog box saying
"Do you feel lucky punk!?" ;-)
-- 
Warren W. Gay VE3WWG
http://home.cogeco.ca/~ve3wwg

Re: For the AdaOS folks

[Nick Roberts]

"Warren W. Gay VE3WWG" <ve3wwg@NoSPAM.cogeco.ca> wrote:

> Dmitry A. Kazakov wrote:
> > But the only need in firewall is the policy of trusting behind it.
> 
> That is all I need to keep you from messing with my files ;-)

I think I side with Dmitry on this one.

When reading a variety of authoritative documents, papers, and books on the
subject of computer security, one of the basic principles they all espouse
is that of 'minimum necessary privilege'. In other words, access is denied
by default, of every object (file, database table, etc.) to every subject
(person, program). Access is granted between an object and a subject only
when these is a specific need.

Okay, I think this principle needs to be taken as a guideline, rather than a
strict rule. It's not likely to be practical on a very fine-grained, highly
dynamic level. Nevertheless, I intend to make the security mechanisms
capable of supporting this principle, to a reasonable degree, and to make
the default security policies implement it.

In practice, that means that, for example, when a user creates a new file
(and saves it), the new file is, by default, inaccessible to (and invisible
to) all other unprivileged users.

By the same token, I intend to make it easy to deliberately share things.
For example, a user can create a kind of directory or folder called a 'file
group', share the group with others users (by simple drag-and-drop), and
then make a file part of the group (also by simple drag-and-drop). The other
users can then see and access the file. The group can be made a 'read-only'
type or a 'full access' type.

When somebody uses an internet service in AdaOS, they do so with a certain
'role' of a certain user. This restricts their privileges (to that role of
that user). If that role is not permitted to access a file, the user of the
internet service is not, either. Of course, typically, things will be
arranged to permit minimum necessary access by internet services. For
example, a web server will be permitted to access the files (and other data)
which make up a web site, but nothing else.

The necessity for a separate firewall seems to be obviated by this
arrangement. The whole system is acting as a big firewall in itself. In
particular, AdaOS will not have any holes or back doors in its security. The
security mechanisms will be hermetically sealed. (This may be somewhat in
contrast to other operating systems.)

-- 
Nick Roberts

Re: For the AdaOS folks

[Warren W. Gay VE3WWG]

Nick Roberts wrote:

>"Warren W. Gay VE3WWG" <ve3wwg@NoSPAM.cogeco.ca> wrote:
>>Dmitry A. Kazakov wrote:
>>>But the only need in firewall is the policy of trusting behind it.
>>
>>That is all I need to keep you from messing with my files ;-)
> 
> I think I side with Dmitry on this one.

Thankfully, my firewall protects me from you as well ;-)

> When reading a variety of authoritative documents, papers, and books on the
> subject of computer security, one of the basic principles they all espouse
> is that of 'minimum necessary privilege'. In other words, access is denied
> by default, of every object (file, database table, etc.) to every subject
> (person, program). Access is granted between an object and a subject only
> when these is a specific need.

Ok, but how does that eliminate the concept of a firewall? It does
precisely this (deny all access) by default, allowing the minimum
necessary permission. Under perfect circumstances, I think you are
saying that a firewall is redundant. But in practice, it'll never
be redundant.

> Okay, I think this principle needs to be taken as a guideline, rather than a
> strict rule. It's not likely to be practical on a very fine-grained, highly
> dynamic level. Nevertheless, I intend to make the security mechanisms
> capable of supporting this principle, to a reasonable degree, and to make
> the default security policies implement it.
> 
> In practice, that means that, for example, when a user creates a new file
> (and saves it), the new file is, by default, inaccessible to (and invisible
> to) all other unprivileged users.

I am not disagreeing with this - and never have.  But are you going
to trust 100s/1000s of CPUs to all be properly locked down to the
outside world?

> When somebody uses an internet service in AdaOS, they do so with a certain
> 'role' of a certain user. This restricts their privileges (to that role of
> that user). If that role is not permitted to access a file, the user of the
> internet service is not, either. Of course, typically, things will be
> arranged to permit minimum necessary access by internet services. For
> example, a web server will be permitted to access the files (and other data)
> which make up a web site, but nothing else.

These are merely different grades of access controls. And as such
I am not against them (and never have been). It could be the best
security ever invented, but if I have to administer 1000s of these,
I will not trust them all to be entirely correct. Worse, other
people may administer some of them - firewall helps to enforce
the company position on access policy!

> The necessity for a separate firewall seems to be obviated by this
> arrangement. The whole system is acting as a big firewall in itself. In
> particular, AdaOS will not have any holes or back doors in its security. The
> security mechanisms will be hermetically sealed. (This may be somewhat in
> contrast to other operating systems.)

Its not quite as simple as that. For example, if you were to
support the ftp service, it doesn't matter how secure
the AdaOS is. The first time someone uses ftp to login
to a server, that account is potentially compromised!
Userid and password information is sniffed by every
machine that has a LAN card listening to the same wire.
The OS itself is _not_ the complete answer to security
(this is where firewalls help).

Even though ssh2 might provide reasonable security today,
any hardened "sealed" AdaOS may still be vulnerable to
developed ssh2 weaknesses in the future.

If you have only 1 windows machine, or 1 Mac or Linux (or
whatever with ftp or other weak clients), then you are
wide open for attack.

So yes, in a pie-in-the-sky world, where all machines use
only the safest of protocols, and are perfectly secure,
you might stand a chance of that working without an outer
firewall.

Would I trust my enterprise to the net this way? Would the
US military trust their secrets without a firewall on their
network? No way. They'll run a firewall anyway, just so that
people can sleep at night. I'll continue to do the same,
thank-you very much.

Because for all I know, this may be one elabourite phishing
scam, trying to get me to drop my firewall ;-)
--
Warren W. Gay VE3WWG
http://home.cogeco.ca/~ve3wwg

Re: For the AdaOS folks

[Nick Roberts]

"Warren W. Gay VE3WWG" <ve3wwg@NoSpam.cogeco.ca> wrote:

> Ok, but how does that eliminate the concept of a firewall? It does
> precisely this (deny all access) by default, allowing the minimum
> necessary permission. Under perfect circumstances, I think you are saying
> that a firewall is redundant. But in practice, it'll never be redundant.

No, in practice it really will be redundant.

> > In practice, that means that, for example, when a user creates a new
> > file (and saves it), the new file is, by default, inaccessible to (and
> > invisible to) all other unprivileged users.
> 
> I am not disagreeing with this - and never have.  But are you going to
> trust 100s/1000s of CPUs to all be properly locked down to the outside
> world?

Yes.

> These are merely different grades of access controls. And as such I am not
> against them (and never have been). It could be the best security ever
> invented, but if I have to administer 1000s of these, I will not trust
> them all to be entirely correct. Worse, other people may administer some
> of them - firewall helps to enforce the company position on access policy!

No, the firewall is worse. The finer grades of access control provide better
and more comprehensive security than a firewall can. Tools can provide the
necessary administrative control, as well as mandatory security controls.

> > The necessity for a separate firewall seems to be obviated by this
> > arrangement. The whole system is acting as a big firewall in itself. In
> > particular, AdaOS will not have any holes or back doors in its security.
> > The security mechanisms will be hermetically sealed. (This may be
> > somewhat in contrast to other operating systems.)
> 
> Its not quite as simple as that.

Yes it is, actually.

> For example, if you were to support the ftp service ...

Obviously we will /not/ support the FTP service, except for anonymous login.
For password-protected file transfer, we will support only SFTP (or perhaps
something that supersedes SFTP).

> The OS itself is _not_ the complete answer to security (this is where
> firewalls help).

I think you are basing that judgement on poor existing operating systems,
and are perhaps therefore unable to comprehend that an OS can really be
watertight.

> Even though ssh2 might provide reasonable security today, any hardened
> "sealed" AdaOS may still be vulnerable to developed ssh2 weaknesses in the
> future.

But I am sure that a firewall would provide no greater protection from such
weaknesses than the OS.

> If you have only 1 windows machine, or 1 Mac or Linux (or whatever with
> ftp or other weak clients), then you are wide open for attack.

Not true. By definition, an AdaOS network will comprise either machines that
are running AdaOS or machines which can communicate with AdaOS only through
the secure IP boundary. If an AdaOS machine is compromised, it could leave
the whole AdaOS network compromised, yes. If one of the other machines is
compromised, this will have no effect within the AdaOS network.

> So yes, in a pie-in-the-sky world, where all machines use only the safest
> of protocols, and are perfectly secure, you might stand a chance of that
> working without an outer firewall.

Warren, with respect, you sound like a horseman who, upon seeing a motor car
for the first time in his life, simply cannot understand that there isn't
anywhere for the saddle to go.

AdaOS /will/ be watertight, and that /will/ obviate the need for a firewall.

-- 
Nick Roberts

Re: For the AdaOS folks

[Warren W. Gay VE3WWG]

Nick Roberts wrote:

> "Warren W. Gay VE3WWG" <ve3wwg@NoSpam.cogeco.ca> wrote:
...
> Warren, with respect, you sound like a horseman who, upon seeing a motor car
> for the first time in his life, simply cannot understand that there isn't
> anywhere for the saddle to go.

This doesn't hold water. You are professing a belief that there will
be a perfect O/S someday - I am simply saying that this is impractical.

> AdaOS /will/ be watertight, and that /will/ obviate the need for a firewall.

Cue SuperTramp singing "Dreamer... silly little dreamer!"

We obviously differ on this point ;-)
-- 
Warren W. Gay VE3WWG
http://home.cogeco.ca/~ve3wwg

Re: For the AdaOS folks

[Nick Roberts]

"Warren W. Gay VE3WWG" <ve3wwg@NoSPAM.cogeco.ca> wrote:

> We obviously differ on this point ;-)

All right, but I hope to be able to prove my point by actual demonstration
one day. I suppose that will (or won't) be the final proof.

-- 
Nick Roberts

Re: For the AdaOS folks

[Wes Groleau]

Nick Roberts wrote:
> AdaOS /will/ be watertight, and that /will/ obviate the need for a firewall.

I have utmost confidence that AdaOS people are actually doing
appropriate "engineering."

Remember all the claims of Java security?
And yet it's history shows that the only security
Java has is against threats the developers thought
of along the way.

AdaOS will be the same.  The difference will be
(I hope) that the Java people obviously didn't
think hard enough about that.

-- 
Wes Groleau

    In any formula, constants (especially those obtained
    from handbooks) are to be treated as variables.

Re: For the AdaOS folks

[Dmitry A. Kazakov]

On Wed, 05 Jan 2005 20:29:31 -0500, Wes Groleau wrote:

> Nick Roberts wrote:
>> AdaOS /will/ be watertight, and that /will/ obviate the need for a firewall.
> 
> I have utmost confidence that AdaOS people are actually doing
> appropriate "engineering."
> 
> Remember all the claims of Java security?
> And yet it's history shows that the only security
> Java has is against threats the developers thought
> of along the way.

That is because Java didn't use the technology it should. One cannot have
security within old paradigm of firewalls. Placing a guard before the
brothel doesn't it make a church!

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: For the AdaOS folks

[Dmitry A. Kazakov]

On Tue, 04 Jan 2005 14:57:39 -0500, Warren W. Gay VE3WWG wrote:

> Dmitry A. Kazakov wrote:

>>>Not a problem. I can determine who accesses the floppy
>>>when it is mounted (look up the mount command).
>> 
>> Yes, but once mounted it is accessible for all. Actually it is the file
>> system with its access rights to the files, that makes access safe, not
>> only the mount command.
> 
> You didn't do your homework on this one:
> 
> Mount options for fat
> 
>    uid=value and gid=value
>      Set the owner and group of all files. (Default: the uid
>      and gid of the current process.)

Huh, these options are provided for a FAT FS. The reason is clear MS-DOS
never was a proper multi-user OS. Use a normal FS! (I think under Linux we
could define FS "normal" if that has no uid/gid options in mount. (:-))

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: For the AdaOS folks

[Warren W. Gay VE3WWG]

Dmitry A. Kazakov wrote:
> On Tue, 04 Jan 2005 14:57:39 -0500, Warren W. Gay VE3WWG wrote:
>>Dmitry A. Kazakov wrote:
> 
>>>>Not a problem. I can determine who accesses the floppy
>>>>when it is mounted (look up the mount command).
>>>
>>>Yes, but once mounted it is accessible for all. Actually it is the file
>>>system with its access rights to the files, that makes access safe, not
>>>only the mount command.
>>
>>You didn't do your homework on this one:
>>
>>Mount options for fat
>>
>>   uid=value and gid=value
>>     Set the owner and group of all files. (Default: the uid
>>     and gid of the current process.)
> 
> Huh, these options are provided for a FAT FS. The reason is clear MS-DOS
> never was a proper multi-user OS. Use a normal FS! (I think under Linux we
> could define FS "normal" if that has no uid/gid options in mount. (:-))

What did you expect on a floppy? ReiserFS? 8-/
--
Warren W. Gay VE3WWG
http://home.cogeco.ca/~ve3wwg

Re: For the AdaOS folks

[Dmitry A. Kazakov]

On Wed, 05 Jan 2005 06:20:44 -0500, Warren W. Gay VE3WWG wrote:

> Dmitry A. Kazakov wrote:
>> On Tue, 04 Jan 2005 14:57:39 -0500, Warren W. Gay VE3WWG wrote:
>>>Dmitry A. Kazakov wrote:
>> 
>>>>>Not a problem. I can determine who accesses the floppy
>>>>>when it is mounted (look up the mount command).
>>>>
>>>>Yes, but once mounted it is accessible for all. Actually it is the file
>>>>system with its access rights to the files, that makes access safe, not
>>>>only the mount command.
>>>
>>>You didn't do your homework on this one:
>>>
>>>Mount options for fat
>>>
>>>   uid=value and gid=value
>>>     Set the owner and group of all files. (Default: the uid
>>>     and gid of the current process.)
>> 
>> Huh, these options are provided for a FAT FS. The reason is clear MS-DOS
>> never was a proper multi-user OS. Use a normal FS! (I think under Linux we
>> could define FS "normal" if that has no uid/gid options in mount. (:-))
> 
> What did you expect on a floppy? ReiserFS? 8-/

mke2fs /dev/fd0

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: For the AdaOS folks

[Warren W. Gay VE3WWG]

Dmitry A. Kazakov wrote:
> On Wed, 05 Jan 2005 06:20:44 -0500, Warren W. Gay VE3WWG wrote:
> 
> 
>>Dmitry A. Kazakov wrote:
>>
>>>On Tue, 04 Jan 2005 14:57:39 -0500, Warren W. Gay VE3WWG wrote:
>>>
>>>>Dmitry A. Kazakov wrote:
>>>
>>>>>>Not a problem. I can determine who accesses the floppy
>>>>>>when it is mounted (look up the mount command).
>>>>>
>>>>>Yes, but once mounted it is accessible for all. Actually it is the file
>>>>>system with its access rights to the files, that makes access safe, not
>>>>>only the mount command.
>>>>
>>>>You didn't do your homework on this one:
>>>>
>>>>Mount options for fat
>>>>
>>>>  uid=value and gid=value
>>>>    Set the owner and group of all files. (Default: the uid
>>>>    and gid of the current process.)
>>>
>>>Huh, these options are provided for a FAT FS. The reason is clear MS-DOS
>>>never was a proper multi-user OS. Use a normal FS! (I think under Linux we
>>>could define FS "normal" if that has no uid/gid options in mount. (:-))
>>
>>What did you expect on a floppy? ReiserFS? 8-/
> 
> mke2fs /dev/fd0

So what? I showed you how it could be done. If you don't
like it, that's your problem 8-P
-- 
Warren W. Gay VE3WWG
http://home.cogeco.ca/~ve3wwg

Re: For the AdaOS folks

[zest_fien]

i keep seeing reviews and raves about this
http://www.naturalisproducts.com and
http://www.organiconline.com.sg . many people are discussing in beauty
forums and magazines have positive reviews on this . but this thing
ain't new, its
been around for many years! anyone tried can feedback to me on exactly
how good it is?

----------------------------------------
can anyone help me please, am looking for the local distributor or any
shop selling the naturalis range of skin and body care products, from
this company http://www.naturalisproducts.com . looking for this
urgently. for those who have not come across it, its some foodbased
anti-aging products. i googled for this and received result
showing its available at http://www.organiconline.com.sg. i need this
urgently but shipping from singapore will take some time, if anyone is
distributing this please contact me at g...@raterenterprise.com
urgently. i have a group of us looking to buy this. thanks!

Re: For the AdaOS folks

[Larry Kilgallen]

In article <1104945397.086080.304650@c13g2000cwb.googlegroups.com>, zest_fien@yahoo.com writes:
> i keep seeing reviews and raves about this
> http://www.naturalisproducts.com and
> http://www.organiconline.com.sg . many people are discussing in beauty
> forums and magazines have positive reviews on this . but this thing

For those of you who did not read the headers of that post,
they said:

> X-Complaints-To: groups-abuse@google.com

Now that the European mail gateway has been cleaned up,
this seems to be the major source of newsgroup spam to
comp.os.vms.

Send a copy of each off topic post to the X-Complaints-To:
in the headers of that post.

Re: For the AdaOS folks

[Nick Roberts]

"Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> wrote:

> I think only a native OO design *exposed* in the kernel interface can
> help.

As I currently have it, AdaOS is very OO from the level just above the
microkernel -- device drivers -- upwards, making full use of Annex E and
remote access types.

> > Yes, they can be provided on top of the microkernel, as you seem to be
> > suggesting.
> 
> Yes, but networking should be known to the kernel as something it can work
> with. So you cannot completely throw it away.

Do you really mean the (micro)kernel, Dmitry, or the TCB (Trusted Computing
Base)?  In Linux and BSD, for example, the kernel /is/ the TCB, but this is
not the case with AdaOS.

> Agree. Protocols must be decoupled from the system, though that is easier
> to do, than to remove networking as a whole. Though it is related to an
> interesting OO problem. When protocol implementation is buried somewhere
> in an inheritance path, then it becomes difficult to switch from one
> protocol to another. It looks like abstraction inversion.

This is why I think the introduction of interfaces in Ada 2005 will be
important. Not only will this help support multiple inheritance, but it will
also help decouple interface hierarchies from implementational hierarchies.

> > You base your registry criticisms upon M$ _implementation_ of a
> > registry. But consider this:
> > 
> > If each registry item were a file (on a RieserFS so there is little or
> > no waste), then you can backup, secure and restore settings just like
> > any other file. You can do so with all of those familiar tools,
> > including tar and cpio.
> 
> The problem is not an ability to backup, in UNIX one can backup /etc, /var
> ... The problem is complexity. There are dependencies between repository
> objects, which cannot be mapped to a simple tree structure, whether that
> be chaotic files or registry. Consider a simple diamond diagram.
> Application A uses B and C, they in turn use D. This cannot be represented
> by a tree. So you need a folder to keep them all. Let's name it "/etc".
> Welcome to back to chaos.

Dmitry seems to be echoing what I was saying in another post in this thread.

> > I don't believe it need be treated any differently than a file system.
> > To support a registry on the RieserFS for example, one merely need to
> > add a nice programming API to make it easier for programs to query and
> > set values. You can still use links/symlinks (if you must) etc.
> 
> In my view there should be no file systems at all. OS should be fully
> memory-mapped with persistent objects. A FS would be just a "low-level"
> format then. (Kind of data base discussion again! (:-))

As I currently have it, that is how AdaOS is designed, in essence. A program
classed as a 'local storage manager' (LSM) provides a way to cause a memory
region to be stored on the hard disk, each such 'store' identified by a
number. At a low level (device drivers) a simple namesystem maps names to
stores, providing an elementary file system.

A programs classed as a 'distributed storage managers' (DSM) will provide
'distributed stores'. A distributed store is identified by a number that is
the same across the whole network, and the pages of a store are
automatically moved (or copied, for R/O pages) to workstations upon demand.
Distributed programs use DSMs (only) to provide their memory regions and
stores.

At a higher level (including application programs), files are organised as
objects (records) in database tables; the 'data' of a file will be stored in
a field of type 'binary large object' (BLOB); each large BLOB will be kept
in a separate distributed store (small ones will be agglomerated, together
with other 'small' database data). However, nothing will prevent stores
being accessed by other means, if necessary. 

-- 
Nick Roberts

Re: For the AdaOS folks

[Dmitry A. Kazakov]

On Tue, 4 Jan 2005 20:09:00 +0000, Nick Roberts wrote:

> "Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> wrote:
>
>>> Yes, they can be provided on top of the microkernel, as you seem to be
>>> suggesting.
>> 
>> Yes, but networking should be known to the kernel as something it can work
>> with. So you cannot completely throw it away.
> 
> Do you really mean the (micro)kernel, Dmitry, or the TCB (Trusted Computing
> Base)?  In Linux and BSD, for example, the kernel /is/ the TCB, but this is
> not the case with AdaOS.

Kernel. Warren and I agreed on hooks for networking. But it seems that we
meant different consequences. Especially because of security issues. I
suppose he tend to move them far away to the upper abstraction layers.
Firewall is a relatively high level. I wished to see them already in the
kernel. In this case "hooks" might become very fat.

>> Agree. Protocols must be decoupled from the system, though that is easier
>> to do, than to remove networking as a whole. Though it is related to an
>> interesting OO problem. When protocol implementation is buried somewhere
>> in an inheritance path, then it becomes difficult to switch from one
>> protocol to another. It looks like abstraction inversion.
> 
> This is why I think the introduction of interfaces in Ada 2005 will be
> important. Not only will this help support multiple inheritance, but it will
> also help decouple interface hierarchies from implementational hierarchies.

Yes, I think it is a very important pattern, not only for an OS, but also
for any complex layered system.

>>> I don't believe it need be treated any differently than a file system.
>>> To support a registry on the RieserFS for example, one merely need to
>>> add a nice programming API to make it easier for programs to query and
>>> set values. You can still use links/symlinks (if you must) etc.
>> 
>> In my view there should be no file systems at all. OS should be fully
>> memory-mapped with persistent objects. A FS would be just a "low-level"
>> format then. (Kind of data base discussion again! (:-))
> 
> As I currently have it, that is how AdaOS is designed, in essence. A program
> classed as a 'local storage manager' (LSM) provides a way to cause a memory
> region to be stored on the hard disk, each such 'store' identified by a
> number. At a low level (device drivers) a simple namesystem maps names to
> stores, providing an elementary file system.
> 
> A programs classed as a 'distributed storage managers' (DSM) will provide
> 'distributed stores'. A distributed store is identified by a number that is
> the same across the whole network, and the pages of a store are
> automatically moved (or copied, for R/O pages) to workstations upon demand.
> Distributed programs use DSMs (only) to provide their memory regions and
> stores.
> 
> At a higher level (including application programs), files are organised as
> objects (records) in database tables; the 'data' of a file will be stored in
> a field of type 'binary large object' (BLOB); each large BLOB will be kept
> in a separate distributed store (small ones will be agglomerated, together
> with other 'small' database data). However, nothing will prevent stores
> being accessed by other means, if necessary.

That sounds really great.

The other means would be by mapping everything to L/DSM. The problem is how
to achieve different views on the same storage. As I mentioned in
discussion with Warren, I would like to have something like Ada protected
object with protection enforced by memory mapping.

Then there is of course the issue of reset/restart/upgrade. System objects
have to be organized in a way which would allow restart failed system
parts. One would never be able to restart the whole [distributed] system
once it runs.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: For the AdaOS folks

[Nick Roberts]

"Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> wrote:

> The other means would be by mapping everything to L/DSM. The problem is
> how to achieve different views on the same storage.

I think one would build layers on top of the database to achieve this.

For example, you might have a table for text files with the fields "Name",
"Encoding", and "Data", of types Char Varying, Enumeration, and BLOB,
respectively. The Encoding field would say what the charcater encoding of
the data was, so that the binary data in the Data field characters can be
interpreted as characters. One might then have a class (of system objects)
"Text_File" which has a method to read a sequence of characters; this method
would invisibly use the Encoding field to select the correct algorithm to
interpret the Data field.

> As I mentioned in discussion with Warren, I would like to have something
> like Ada protected object with protection enforced by memory mapping.

What I envisage is a locking (and transaction) protocol based on the model
described by the SQL-92 standard. This would achieve the same essential
objectives as an Ada protected object, but with the added functionality of
transaction rollback, and more implementational flexibility. I'm not wedded
to the use of the SQL-92 standard, however.

One possibility is a translator that would generate Ada code from a
specification written in a language designed to conveniently describe how
database tables (or views) are to accessed in a program. The generator would
generate both an Ada package specification and its corresponding body. The
spec could contain an interface (to a database table) that would be similar
to the interface of a protected object. The body would contain the nasty
messing about necessary to access the database.

> Then there is of course the issue of reset/restart/upgrade. System objects
> have to be organized in a way which would allow restart failed system
> parts. One would never be able to restart the whole [distributed] system
> once it runs.

This is why a comprehensively transactional system would be so important. 

The system would be designed so that transactions were guaranteed to be
effectively atomic, even for changes spread across the network; in
particular, rolling back would be guaranteed atomic as well as committal.

This way, any failure anywhere in the network would cause the affected
transactions to be rolled back -- and this would be guaranteed to succeed in
getting the system to a stable (consistent) state -- and then the remainder
of the network could proceed correctly from there, with software components
restarted as necessary.

-- 
Nick Roberts

Re: For the AdaOS folks

[Dmitry A. Kazakov]

On Wed, 5 Jan 2005 18:33:53 +0000, Nick Roberts wrote:

> "Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> wrote:
> 
>> As I mentioned in discussion with Warren, I would like to have something
>> like Ada protected object with protection enforced by memory mapping.
> 
> What I envisage is a locking (and transaction) protocol based on the model
> described by the SQL-92 standard. This would achieve the same essential
> objectives as an Ada protected object, but with the added functionality of
> transaction rollback, and more implementational flexibility. I'm not wedded
> to the use of the SQL-92 standard, however.

OK. One note though. In fact Ada's entry points have some equivalent of
rollbacks. When an entry call is aborted it is basically same as rolling
back the transaction. Especially if internally the call is routed through
many queues. Of course all internal "requeue"s should be "with abort" and
the implementation should take care about internal state of the objects
involved.

> One possibility is a translator that would generate Ada code from a
> specification written in a language designed to conveniently describe how
> database tables (or views) are to accessed in a program. The generator would
> generate both an Ada package specification and its corresponding body. The
> spec could contain an interface (to a database table) that would be similar
> to the interface of a protected object. The body would contain the nasty
> messing about necessary to access the database.

Much work in the linker/loader field, also.

>> Then there is of course the issue of reset/restart/upgrade. System objects
>> have to be organized in a way which would allow restart failed system
>> parts. One would never be able to restart the whole [distributed] system
>> once it runs.
> 
> This is why a comprehensively transactional system would be so important. 
> 
> The system would be designed so that transactions were guaranteed to be
> effectively atomic, even for changes spread across the network; in
> particular, rolling back would be guaranteed atomic as well as committal.
> 
> This way, any failure anywhere in the network would cause the affected
> transactions to be rolled back -- and this would be guaranteed to succeed in
> getting the system to a stable (consistent) state -- and then the remainder
> of the network could proceed correctly from there, with software components
> restarted as necessary.

Agree.

Also there must be a deeper hierarchy of nested objects and scopes where
system exceptions could propagate. Presently it is basically the system and
applications. When the system cannot handle an exception propagated out of
some application it crashes.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: For the AdaOS folks

[Nick Roberts]

"Warren W. Gay VE3WWG" <ve3wwg@NoSpam.cogeco.ca> wrote:

> Its too bad that the L4 microkernel is not very secure yet. Apparently,
> they are trying to fix that in L4ng, but it appears that they got so
> focused on efficiency that they didn't get security. The efficiency of L4
> otherwise makes it very attractive. 

The major problems with L4 that stopped me using it (or its interface) were
its security problems, and the fact that it does not support process
migration. Of course, that support could be added on top, in an extra layer;
but obviously I felt there's no point in having that extra layer, I might
just as well design a microkernel that supports it directly. There are a
number of other minor problems with L4, too.

There are other problems. I want to write the microkernel (mostly) in Ada,
so I would have to rewrite L4 anyway. It might have still been attractive to
use the L4 interface, but since there is only a binary interface and a C
interface (no published Ada interface), this doesn't turn out to be such an
advantage. Similar comments apply to other kernels, including Mach.

> I also don't like the way that L4 ties communications to threads. The Mach
> approach is much easier to work with, with its single receiver and
> multiple send and send-once rights.

This issue is related to the problem of process migration. Mach supports it
by not tying communications to threads. But this creates all sorts of
communications routing complexity inside the kernel. I think Bachar does
much better: communications are thread-addressed, but a mechanism permits
proxy threads (which can be in separate processes) to be transparently
interposed when non-local communication is required (and removed when it is
not). This way, you get simplicity and efficiency, but process migration is
still properly supported. Note that there are some other subtleties: all
Bachar resources can be addressed (identified) in a way that can be exactly
reproduced in the target machine when a process is migrated; most
microkernels do not seem to support this functionality. Bachar provides
functionality that allows any process to be frozen, dissected, reconstructed
(on the target machine), and then reanimated. These functions can be done by
a higher layer, but it makes more sense for them to be supported directly by
the kernel.

> I've been mulling this over myself. Unfortunately, as soon as you say that
> all computers must work together, you introduce all sorts of problems and
> overheads. Endianness and page size are two issues, between two computers.
> I have been wondering if this shouldn't be tiered:
> 
>    1. Clustered systems with all the same endianness and page size
>    2. "Networked Clusters" of #1 groupings
> 
> In this way, it would be efficiently possible to cluster similar
> equipment, and yet still provide "one-ness" in the different computer
> cases, with the necessary overhead and complexity. This would also permit
> a tiered development approach.

Interesting idea. But I am assuming a network made up entirely of
workstations sufficiently similar in architecture that a process can be
migrated from any one to any other. In practice, to start with, that means a
set of PCs.

Obviously we will support heterogeneous networking, but I think we might as
well use IP to do this, and all the existing (reliable) technologies built
on top of it. I'd like to support both IPv4 and IPv6.

A widely-distributed (heterogeneous architecture) filesystem and database
system would be a very nice idea, however. I'm hoping that once we get a
core AdaOS working, people will develop all sorts of nice ideas for it.

> I wonder if networking really belongs in the microkernel. This is
> necessary for example if like Mach, you say that ports can speak to other
> Mach machines on the network/cluster.

I'm sorry, I didn't mean to imply that Bachar will do any networking. It
won't. It really will be a microkernel (maybe not as tiny as L4).

Networking will be implemented in device drivers and layers just above the
kernel, especially in the ORB engine (called Avarus).

> I always get scoffed at for suggesting this, and I have no love for M$,
> but the one idea they have had, which is useful, is the registry. I think
> there are better ways of doing this however, but the general idea is good.
> You can see that GNOME and Mozilla both use it, so it obviously satisfies
> a need. I think any OS today that is going to support point and click
> administration, is going to need some sort of a registry, even if it is
> just layering it on a RieserFS.

Exactly. The main problem with the Registry is that it is not a proper
database system.

-- 
Nick Roberts

Re: For the AdaOS folks

[Warren W. Gay VE3WWG]

Nick Roberts wrote:
> "Warren W. Gay VE3WWG" <ve3wwg@NoSpam.cogeco.ca> wrote:
>>Its too bad that the L4 microkernel is not very secure yet. Apparently,
>>they are trying to fix that in L4ng, but it appears that they got so
>>focused on efficiency that they didn't get security. The efficiency of L4
>>otherwise makes it very attractive. 
> 
> The major problems with L4 that stopped me using it (or its interface) were
> its security problems, and the fact that it does not support process
> migration. 

By "process migration", are you referring to "thread-migration"?

In another related post, I offerred the following link for Mach:
   http://www.usenix.org/publications/library/proceedings/sf94/full_papers/ford.pdf

> Of course, that support could be added on top, in an extra layer;
> but obviously I felt there's no point in having that extra layer, I might
> just as well design a microkernel that supports it directly. There are a
> number of other minor problems with L4, too.

One of the remarks made in the above cited paper is that (paraphrasing)
QNX implements this by default, because of it doesn't support a queue (control
must immediately pass from sender to receiver). I know L4 works the same way,
but I also know that L4 talks of a Local-RPC. It would seem that Local-RPC
supports what you're looking for in L4.

> There are other problems. I want to write the microkernel (mostly) in Ada,

I support the concept ;-)

> so I would have to rewrite L4 anyway. It might have still been attractive to
> use the L4 interface, 

It is attractive to me because of the fact that they've ported it to
so many platforms. It is also very actively developed. But it has other
problems, obviously.

> but since there is only a binary interface and a C
> interface (no published Ada interface), this doesn't turn out to be such an
> advantage. Similar comments apply to other kernels, including Mach.

I looked at their API and it doesn't look too bad actually. They map
a number of things into MR (Message Registers), which would be real
easy to do in Ada, using the for Object'Address use ...;  A tiny bit
of _Asm would fix the rest. So I don't see that as much of a problem.
Define a package (or few) and the rest is a slam dunk.

>>I also don't like the way that L4 ties communications to threads. The Mach
>>approach is much easier to work with, with its single receiver and
>>multiple send and send-once rights.
> 
> This issue is related to the problem of process migration. Mach supports it
> by not tying communications to threads. But this creates all sorts of
> communications routing complexity inside the kernel. 

I am not entirely convinced of this. I am currently working with rtmk
(a stripped down Mach clone of sorts), and from the kernel side it
is not that difficult (I've had to mess with some of this ;-)

What it _does_ of course require is the need for formatted
messages. By this I mean that you cannot just send a stream of
bytes from one task to another, and include a Port somewhere in
the middle. The kernel must know when a port is being copied/moved,
so that it can play its kernel magic before the receiving task
gets the message (and thus inherit the port right). So from this
point of view, I would agree with you. But otherwise, I don't
see it as complex at all.

The thing is, depending upon design of course, you could easily
use thousands of send-rights, with little or no overhead. If you
must associate the message endpoint to a thread, barring lazy
allocation techniques, you impose all of the overheads that a
thread must impose (stacks, state etc.)  I just feel that the
port paradigm is more flexible for the OS designer (it costs
less). It is also conceptually cleaner to say you have two
end-points (ports) of a message queue, than saying you have
two-threads when threads aren't part of the abstraction.

> I think Bachar does
> much better: communications are thread-addressed, but a mechanism permits
> proxy threads (which can be in separate processes) to be transparently
> interposed when non-local communication is required (and removed when it is
> not). This way, you get simplicity and efficiency, but process migration is
> still properly supported.

But if you look at the Mach paper above, you can do the very
same thing with ports. Even without thread-migration, you can
interpose proxies. With thread-migration enabled RPC, you
achieve both and provide the OS designer a nicer interface.

For example, I might want a "notify port". Let's say I look up
the name server for the console service, and its not yet
registered (at boot time).  Rather than do a spin loop, the
client software obtains a notify port from the name server
first (it gets the receive port, while the ns keeps the send-once
right). Then the client tries to lookup the console, and if
its there, fine (it then destroys the notify port). If not,
then it waits for a message to be received from the ns on
the notify port, when something gets registered (something
has changed at the ns). Repeat if necessary.

If you tie that to threads, then things get messier. Ports
are easy to allocate (kernel just does some pointer arithmetic,
and increment reference counts), and easy to put into a
linked list for notifications.  A new thread isn't really
called for in this case, only the abstraction of a notify
port.

> Note that there are some other subtleties: all
> Bachar resources can be addressed (identified) in a way that can be exactly
> reproduced in the target machine when a process is migrated; 

Actually, Mach does this as well. However, they will tell you however,
that you must beware:  When writing a pager shared among multiple hosts
for example, you must be aware that the VM page size may vary according
to the type of equipment it is run on. For Intel 4K, but another platform
(I forget which), will use 8K.

Obviously, it will be difficult to migrate an Intel thread onto a PPC
platform ;-)

> most
> microkernels do not seem to support this functionality. Bachar provides
> functionality that allows any process to be frozen, dissected, reconstructed
> (on the target machine), and then reanimated. These functions can be done by
> a higher layer, but it makes more sense for them to be supported directly by
> the kernel.

I was planning to do this for my OS outside of the mk. The primary reason
for this is perhaps two-fold:

  1. I want to choose my own form of networking for the purpose
  2. It may require some other OS "coordination"

So I still favour a minimalist microkernel, but allowing for those nicer
abstractions above it.

>>   1. Clustered systems with all the same endianness and page size
>>   2. "Networked Clusters" of #1 groupings
>>
>>In this way, it would be efficiently possible to cluster similar
>>equipment, and yet still provide "one-ness" in the different computer
>>cases, with the necessary overhead and complexity. This would also permit
>>a tiered development approach.
> 
> Interesting idea. But I am assuming a network made up entirely of
> workstations sufficiently similar in architecture that a process can be
> migrated from any one to any other. In practice, to start with, that means a
> set of PCs.

Yes. You obviously have to track the capabilities in mixed Intel
environments for example, so you might have some host affinities
for i686 vs i386 etc. among the nodes. But yes, the general idea
is to allow the pager to page-fault a process over to another
cluster host, assuming that some sort of automatic load-balancing
can work out who it should page fault to.

To make this work, as I envision it, requires that all participating
nodes in a cluster see the file system the same way. IOW, there
can only be one root file system, which obviously is only local
to one node. To all other nodes, root will be access NFS-like (ie.
over the network cable). However, this is not to say that other
nodes will not have their own local disk, but they will of course
be logically mounted on top of root, or another node's filesystem(s).
This consistent file view is necessary to make tasks mobile to
any node in the cluster.

> Obviously we will support heterogeneous networking, but I think we might as
> well use IP to do this, and all the existing (reliable) technologies built
> on top of it. I'd like to support both IPv4 and IPv6.

Anything on top of AdaOS or whatever-OS, can do as it pleases. My
intitial concern is what the mk supports, and how can I get to
the longer range goal of supporting clusters.

Everyone collects old Intel boxes these days. Wouldn't it be
nice to be able to plug them all into a common hub, and boot
a common image and say "go forth and cluster!"  The goal is
obviously do this with minimal system administration with
no PhD required.

> A widely-distributed (heterogeneous architecture) filesystem and database
> system would be a very nice idea, however. I'm hoping that once we get a
> core AdaOS working, people will develop all sorts of nice ideas for it.

As soon as you go out of your own protected network, a whole new
can of worms come to roost. I'll let someone else do the
network security research side of things ;-)

>>I wonder if networking really belongs in the microkernel. This is
>>necessary for example if like Mach, you say that ports can speak to other
>>Mach machines on the network/cluster.
> 
> I'm sorry, I didn't mean to imply that Bachar will do any networking. It
> won't. It really will be a microkernel (maybe not as tiny as L4).
> 
> Networking will be implemented in device drivers and layers just above the
> kernel, especially in the ORB engine (called Avarus).

Yes, I agree.

>>I always get scoffed at for suggesting this, and I have no love for M$,
>>but the one idea they have had, which is useful, is the registry. I think
>>there are better ways of doing this however, but the general idea is good.
>>You can see that GNOME and Mozilla both use it, so it obviously satisfies
>>a need. I think any OS today that is going to support point and click
>>administration, is going to need some sort of a registry, even if it is
>>just layering it on a RieserFS.
> 
> Exactly. The main problem with the Registry is that it is not a proper
> database system.

Well, define "proper" and define what the "database system" solves that
the present system doesn't?

SQL databases are designed to deal with large volumes of similar data,
in SELECTs, joins etc. However, when you look at what goes into the
registry, a lot of it is custom, hierarchically structured information.

So I don't dispute there may be a better paradigm for this information,
but I haven't seen it yet.

Happy New Year!
-- 
Warren W. Gay VE3WWG
http://home.cogeco.ca/~ve3wwg

Re: For the AdaOS folks

[Nick Roberts]

"Warren W. Gay VE3WWG" <ve3wwg@NoSPAM.cogeco.ca> wrote:

> By "process migration", are you referring to "thread-migration"?

No, I'm talking about moving an entire process (its memory, threads, and all
other resources held) from one workstation to another (within the network)
at /any/ arbitrary point during its execution (the execution of its
threads).

I'm assuming that the threads within a process will be closely-coupled --
they will share the same memory address space -- so as to support the
typical monoprocessor and SMP configurations of contemporary PCs.

As such, Mach's concept of thread migration seems inappropriate. Mach was
designed to accomodate experimentation with NUMA, but NUMA machines have not
come into the mainstream marketplace yet, so this is one way in which the
experimental nature of Mach's design is less than ideal, to me.

> > Of course, that support could be added on top, in an extra layer; but
> > obviously I felt there's no point in having that extra layer, I might
> > just as well design a microkernel that supports it directly. There are a
> > number of other minor problems with L4, too.
> 
> One of the remarks made in the above cited paper is that (paraphrasing)
> QNX implements this by default, because of it doesn't support a queue
> (control must immediately pass from sender to receiver). I know L4 works
> the same way, but I also know that L4 talks of a Local-RPC. It would seem
> that Local-RPC supports what you're looking for in L4.

Since priorities must be enforced by the AdaOS kernel at all times (to
prevent a thread subverting a more privileged time-critical thread), there
must be a change of priority associated with /any/ transfer of control to
another thread. This implies queueing of some kind. But I have studied L4
carefully, and I am sure that would need to put in a layer above it for all
processes that needed to be distributed (migratable).

> > but since there is only a binary interface and a C interface (no
> > published Ada interface), this doesn't turn out to be such an advantage.
> > Similar comments apply to other kernels, including Mach.
> 
> I looked at their API and it doesn't look too bad actually. They map a
> number of things into MR (Message Registers), which would be real easy to
> do in Ada, using the for Object'Address use ...;  A tiny bit of _Asm would
> fix the rest. So I don't see that as much of a problem. Define a package
> (or few) and the rest is a slam dunk.

They take the attitude that the C interface must remain the same, regardless
of the machine. I think that's impractical. I've designed the Bachar
interface so that it will use real machine registers, rather than pretend
registers. It means the interface changes from machine to machine, but it
also means that the interface is as efficient as possible for each machine.
Note that the differences in Bachar's interface, from machine to machine,
will vary only in certain details (such as call convention and parameter
passing).

> > This issue is related to the problem of process migration. Mach supports
> > it by not tying communications to threads. But this creates all sorts of
> > communications routing complexity inside the kernel.
> 
> I am not entirely convinced of this. I am currently working with rtmk (a
> stripped down Mach clone of sorts), and from the kernel side it is not
> that difficult (I've had to mess with some of this ;-)
>
> What it _does_ of course require is the need for formatted messages. By
> this I mean that you cannot just send a stream of bytes from one task to
> another, and include a Port somewhere in the middle. The kernel must know
> when a port is being copied/moved, so that it can play its kernel magic
> before the receiving task gets the message (and thus inherit the port
> right). So from this point of view, I would agree with you. But otherwise,
> I don't see it as complex at all.

Assume we are considering one distributed process making an RPC to another
one. Every such call must be dynamically routed, since the target process
could be on any workstation in the network. Not only must the router store a
table of the location (which workstation) of every process, but there must
also be a protocol that allows a router on another workstation to forward
the call (because the process has just moved) and notify the originating
router (so it updates its locations table). It is also necessary to deal
with network partitions (a partial breakdown of the network), and it is
necessary in AdaOS for such RPCs to be made in the context of transactions,
so that if the call unrecoverably fails, the transaction can be aborted. I
don't think all that network routing complexity should be inside a
microkernel.

> The thing is, depending upon design of course, you could easily use
> thousands of send-rights, with little or no overhead. If you must
> associate the message endpoint to a thread, barring lazy allocation
> techniques, you impose all of the overheads that a thread must impose
> (stacks, state etc.)  I just feel that the port paradigm is more flexible
> for the OS designer (it costs less). It is also conceptually cleaner to
> say you have two end-points (ports) of a message queue, than saying you
> have two-threads when threads aren't part of the abstraction.

I have taken the approach of local IPC being supported by the microkernel
(and lower levels of TCB software), and network IPC being supported by a
higher layer of TCB software (Avarus).

> > I think Bachar does much better: communications are thread-addressed,
> > but a mechanism permits proxy threads (which can be in separate
> > processes) to be transparently interposed when non-local communication
> > is required (and removed when it is not). This way, you get simplicity
> > and efficiency, but process migration is still properly supported.
> 
> But if you look at the Mach paper above, you can do the very same thing
> with ports. Even without thread-migration, you can interpose proxies. With
> thread-migration enabled RPC, you achieve both and provide the OS designer
> a nicer interface.

But if we did that, what would be the advantage?

> > Note that there are some other subtleties: all Bachar resources can be
> > addressed (identified) in a way that can be exactly reproduced in the
> > target machine when a process is migrated;
> 
> Actually, Mach does this as well. However, they will tell you however,
> that you must beware:  When writing a pager shared among multiple hosts
> for example, you must be aware that the VM page size may vary according to
> the type of equipment it is run on. For Intel 4K, but another platform (I
> forget which), will use 8K.
> 
> Obviously, it will be difficult to migrate an Intel thread onto a PPC
> platform ;-)

I've made what I think is the reasonable assumption that the page size and
processor architecture remain the same throughout the network (or the
administrative division of the network within which process migration is to
occur).

I know that Mach supports migration (it was designed to), but not in the
form I want it, as I've described above.

> > most microkernels do not seem to support this functionality. Bachar
> > provides functionality that allows any process to be frozen, dissected,
> > reconstructed (on the target machine), and then reanimated. These
> > functions can be done by a higher layer, but it makes more sense for
> > them to be supported directly by the kernel.
> 
> I was planning to do this for my OS outside of the mk. The primary reason
> for this is perhaps two-fold:
> 
>   1. I want to choose my own form of networking for the purpose
>   2. It may require some other OS "coordination"
> 
> So I still favour a minimalist microkernel, but allowing for those nicer
> abstractions above it.

I don't see the point in making the microkernel /that/ minimalist, to be
honest. Whatever the upper layer did, it would have to be dependent upon
intimate details of the kernel. So why not just lump them together?

> >>   1. Clustered systems with all the same endianness and page size
> >>   2. "Networked Clusters" of #1 groupings
> >>
> > > In this way, it would be efficiently possible to cluster similar
> > > equipment, and yet still provide "one-ness" in the different computer
> > > cases, with the necessary overhead and complexity. This would also
> > > permit a tiered development approach.
> > 
> > Interesting idea. But I am assuming a network made up entirely of
> > workstations sufficiently similar in architecture that a process can be
> > migrated from any one to any other. In practice, to start with, that
> > means a set of PCs.
> 
> Yes. You obviously have to track the capabilities in mixed Intel
> environments for example, so you might have some host affinities for i686
> vs i386 etc. among the nodes. But yes, the general idea is to allow the
> pager to page-fault a process over to another cluster host, assuming that
> some sort of automatic load-balancing can work out who it should page
> fault to.

Yes. I intend to take the lowest common denominator approach for the IA-32.
My idea is to have an auxiliary program that Avarus uses to give it
recommendations for the migration of processes.

> To make this work, as I envision it, requires that all participating nodes
> in a cluster see the file system the same way. IOW, there can only be one
> root file system, which obviously is only local to one node. To all other
> nodes, root will be access NFS-like (ie. over the network cable).

This is a fundamental precept of fully distributed networking.

> However, this is not to say that other nodes will not have their own local
> disk, but they will of course be logically mounted on top of root, or
> another node's filesystem(s).

AdaOS will support local processes, that are never migrated. These will be
processes which are closely related to one particular workstation (for
example, a process closely connected with a peripheral connected to the
workstation). Local processes will additionally have access to a local
filesystem (local to the workstation).

> This consistent file view is necessary to make tasks mobile to any node in
> the cluster.

Not only must the filesystem view by fully distributed (the same from every
workstation in the network), but also the temporary memory view must be.
Since AdaOS allows a process to access multiple memory spaces (called
'regions'), distributed processes must have a distributed way of accessing
all of its regions. This is provided a program calssed as a 'distributed
storage manager' (DSM); there will be a DSM program called Cortex, which
automatically shuffles pages around the network on demand.

> Everyone collects old Intel boxes these days. Wouldn't it be nice to be
> able to plug them all into a common hub, and boot a common image and say
> "go forth and cluster!"  The goal is obviously do this with minimal system
> administration with no PhD required.

Exactly what I hope to achieve.

> > Exactly. The main problem with the Registry is that it is not a proper
> > database system.
> 
> Well, define "proper" and define what the "database system" solves that
> the present system doesn't?

The Windows Registry doesn't provide indexing, joining, field selection, or
filtering (record selection). These would all be useful capabilities. It
also doesn't provide the kind of administrative functionality (e.g. backup)
that a full database system usually does.

> SQL databases are designed to deal with large volumes of similar data, in
> SELECTs, joins etc. However, when you look at what goes into the registry,
> a lot of it is custom, hierarchically structured information.

No, the kind of data that goes into the Registry would be naturally
organised in many complex ways. For example, many different applications
store their own font information, but a font disinstaller may well wish to
be able to 'cut across' this font data, to make appropriate adjustments (for
all the applications). As another example, some applications may wish to
store data for several different users, but an adminstrative program may
wish to change all the data (across many applications) for a particular
user.

> So I don't dispute there may be a better paradigm for this information,
> but I haven't seen it yet.

Well, I'm not a big fan of SQL or typical relational databases. But I'm
intending to build a database engine for AdaOS, called Carrot, which
provides the essential facilties: multiple fields, and field selection; a
good coverage of data types (including BLOB), and some basic operations on
them; indexing (on most of the different types); joining; filtering. I might
base it upon the XML Database model.

> Happy New Year!

Thank you, and you too.

-- 
Nick Roberts

Re: For the AdaOS folks

[Warren W. Gay VE3WWG]

Nick Roberts wrote:

> "Warren W. Gay VE3WWG" <ve3wwg@NoSPAM.cogeco.ca> wrote:
>>By "process migration", are you referring to "thread-migration"?
> 
> No, I'm talking about moving an entire process (its memory, threads, and all
> other resources held) from one workstation to another (within the network)
> at /any/ arbitrary point during its execution (the execution of its
> threads).

Ok, it wasn't clear from the context.

> I'm assuming that the threads within a process will be closely-coupled --
> they will share the same memory address space -- so as to support the
> typical monoprocessor and SMP configurations of contemporary PCs.
> 
> As such, Mach's concept of thread migration seems inappropriate. Mach was
> designed to accomodate experimentation with NUMA, but NUMA machines have not
> come into the mainstream marketplace yet, so this is one way in which the
> experimental nature of Mach's design is less than ideal, to me.

Mach's migrating threads do in fact cross into other
"tasks". Instead of a caller thread stopping when a message is
queued, and the receiving thread starting (from a blocked state),
a migrating thread continues from the caller, works
through the server code, and then returns back to the caller (in
the caller's "task"). This is a simplification of what happens,
because there are "activations" and stack games that must happen
to make this work - but it is a migrating thread.

>>>but since there is only a binary interface and a C interface (no
>>>published Ada interface), this doesn't turn out to be such an advantage.
>>>Similar comments apply to other kernels, including Mach.
>>
>>I looked at their API and it doesn't look too bad actually. They map a
>>number of things into MR (Message Registers), which would be real easy to
>>do in Ada, using the for Object'Address use ...;  A tiny bit of _Asm would
>>fix the rest. So I don't see that as much of a problem. Define a package
>>(or few) and the rest is a slam dunk.
> 
> They take the attitude that the C interface must remain the same, regardless
> of the machine. I think that's impractical. 

In what way?

> I've designed the Bachar
> interface so that it will use real machine registers, rather than pretend
> registers. 

But don't forget the MRs are immediately loaded into registers
after all of your "deposits" are made. I don't think you
can level valid efficiency arguments against L4. AFAICS, it
is the best performing microkernel available.

> It means the interface changes from machine to machine, but it
> also means that the interface is as efficient as possible for each machine.
> Note that the differences in Bachar's interface, from machine to machine,
> will vary only in certain details (such as call convention and parameter
> passing).

I think L4 has done a nice job of saving the programmer from the
hassle of potential processor differences. They've apparently
done a good job of it, because it performs quite well in the
benchmark tests compared to the others.

>>What it _does_ of course require is the need for formatted messages. By
>>this I mean that you cannot just send a stream of bytes from one task to
>>another, and include a Port somewhere in the middle. The kernel must know
>>when a port is being copied/moved, so that it can play its kernel magic
>>before the receiving task gets the message (and thus inherit the port
>>right). So from this point of view, I would agree with you. But otherwise,
>>I don't see it as complex at all.
> 
> Assume we are considering one distributed process making an RPC to another
> one. Every such call must be dynamically routed, since the target process
> could be on any workstation in the network. Not only must the router store a
> table of the location (which workstation) of every process, but there must
> also be a protocol that allows a router on another workstation to forward
> the call (because the process has just moved) and notify the originating
> router (so it updates its locations table). It is also necessary to deal
> with network partitions (a partial breakdown of the network), and it is
> necessary in AdaOS for such RPCs to be made in the context of transactions,
> so that if the call unrecoverably fails, the transaction can be aborted. I
> don't think all that network routing complexity should be inside a
> microkernel.

Agreed. What you talk about is at least one abstraction layer above
the one I was speaking of.

>>The thing is, depending upon design of course, you could easily use
>>thousands of send-rights, with little or no overhead. If you must
>>associate the message endpoint to a thread, barring lazy allocation
>>techniques, you impose all of the overheads that a thread must impose
>>(stacks, state etc.)  I just feel that the port paradigm is more flexible
>>for the OS designer (it costs less). It is also conceptually cleaner to
>>say you have two end-points (ports) of a message queue, than saying you
>>have two-threads when threads aren't part of the abstraction.
> 
> I have taken the approach of local IPC being supported by the microkernel
> (and lower levels of TCB software), and network IPC being supported by a
> higher layer of TCB software (Avarus).

Yes, I agree with that layered approach. I was just stating that the
Mach message port interface is extremely convenient at the microkernel
level, for building your higher level layers.

>>But if you look at the Mach paper above, you can do the very same thing
>>with ports. Even without thread-migration, you can interpose proxies. With
>>thread-migration enabled RPC, you achieve both and provide the OS designer
>>a nicer interface.
> 
> But if we did that, what would be the advantage?

Its cheaper. To have 1000 send rights, costs near nothing. 1000 send
rights in one task costs nothing (just increments ref count). 1000 in
a 1000 different tasks, costs one pointer in each task. Nothing further.

1000 threads OTOH, costs you state information for each of those
1000 threads (same task or different). The amount of state
information needed will vary with platform.

>>>most microkernels do not seem to support this functionality. Bachar
>>>provides functionality that allows any process to be frozen, dissected,
>>>reconstructed (on the target machine), and then reanimated. These
>>>functions can be done by a higher layer, but it makes more sense for
>>>them to be supported directly by the kernel.
>>
>>I was planning to do this for my OS outside of the mk. The primary reason
>>for this is perhaps two-fold:
>>
>>  1. I want to choose my own form of networking for the purpose
>>  2. It may require some other OS "coordination"
>>
>>So I still favour a minimalist microkernel, but allowing for those nicer
>>abstractions above it.
> 
> I don't see the point in making the microkernel /that/ minimalist, to be
> honest. Whatever the upper layer did, it would have to be dependent upon
> intimate details of the kernel. So why not just lump them together?

But in your answer:

 > I have taken the approach of local IPC being supported by the microkernel
 > (and lower levels of TCB software), and network IPC being supported by a
 > higher layer of TCB software (Avarus).

If I understand correctly, you are saying that your network "layer"
is already built on top of lower levels (like microkernel).

>>To make this work, as I envision it, requires that all participating nodes
>>in a cluster see the file system the same way. IOW, there can only be one
>>root file system, which obviously is only local to one node. To all other
>>nodes, root will be access NFS-like (ie. over the network cable).
> 
> 
> This is a fundamental precept of fully distributed networking.

It would seem so, but I think there is room for creative
thinking here. Perhaps a "context sensitive view" is
also possible.

>>This consistent file view is necessary to make tasks mobile to any node in
>>the cluster.
> 
> Not only must the filesystem view by fully distributed (the same from every
> workstation in the network), but also the temporary memory view must be.

Good point.

> Since AdaOS allows a process to access multiple memory spaces (called
> 'regions'), distributed processes must have a distributed way of accessing
> all of its regions. This is provided a program calssed as a 'distributed
> storage manager' (DSM); there will be a DSM program called Cortex, which
> automatically shuffles pages around the network on demand.

This is nothing new of course. An example program was given the
Programming under Mach in 1992, and I am sure others did this
well before that book came out.

>>>Exactly. The main problem with the Registry is that it is not a proper
>>>database system.
>>
>>Well, define "proper" and define what the "database system" solves that
>>the present system doesn't?
> 
> The Windows Registry doesn't provide indexing, joining, field selection, or

   - Indexes are a performance issue (not an API issue, per se)
   - Joining (hard pressed to think of any registry entry I wanted
     to do a join for - I can't think of any)
   - Field selection (already part of the registry API)

> filtering (record selection).

   - I've never needed this. My programs usually just ask questions
     like "where is object such-n-such".

> These would all be useful capabilities. It
> also doesn't provide the kind of administrative functionality (e.g. backup)
> that a full database system usually does.

File systems are easily backed up, and so are individual files. This
isn't a difficult problem to solve, no matter what the form of the
registry.

>>SQL databases are designed to deal with large volumes of similar data, in
>>SELECTs, joins etc. However, when you look at what goes into the registry,
>>a lot of it is custom, hierarchically structured information.
> 
> No, the kind of data that goes into the Registry would be naturally
> organised in many complex ways. For example, many different applications
> store their own font information, but a font disinstaller may well wish to
> be able to 'cut across' this font data, to make appropriate adjustments (for
> all the applications). As another example, some applications may wish to
> store data for several different users, but an adminstrative program may
> wish to change all the data (across many applications) for a particular
> user.

The same argument can be made for changing file systems into databases.
So far, the filesystems approach seems to be the clear winner for
that kind of data.
--
Warren W. Gay VE3WWG
http://home.cogeco.ca/~ve3wwg

Re: For the AdaOS folks

[Nick Roberts]

"Warren W. Gay VE3WWG" <ve3wwg@NoSpam.cogeco.ca> wrote:

> Mach's migrating threads do in fact cross into other "tasks". Instead of a
> caller thread stopping when a message is queued, and the receiving thread
> starting (from a blocked state), a migrating thread continues from the
> caller, works through the server code, and then returns back to the caller
> (in the caller's "task"). This is a simplification of what happens,
> because there are "activations" and stack games that must happen to make
> this work - but it is a migrating thread.

I'm not quite sure what you're getting at, Warren. I don't think it's very
surprising that Mach has capabilities that are similar to Bachar, since they
are both aimed at doing roughly the same thing. But there are significant
differences in the details. I did begin designing (what has become) Bachar a
long time ago (pre-1990s).

> > They take the attitude that the C interface must remain the same,
> > regardless of the machine. I think that's impractical.
> 
> In what way?

There are sufficient differences between some architectures (e.g. a CISC
such as the IA-32 and a RISC such as the PowerPC) to warrant a difference in
the interface between a microkernel intended for one and a microkernel
intended for the other, even if the software on top is intended to be the
same. Some of these differences cannot be hidden by C (more of them can be
hidden by Ada, in fact).

> > I've designed the Bachar interface so that it will use real machine
> > registers, rather than pretend registers.
> 
> But don't forget the MRs are immediately loaded into registers after all
> of your "deposits" are made. I don't think you can level valid efficiency
> arguments against L4. AFAICS, it is the best performing microkernel
> available.

One of the advantages of using real registers that an Ada compiler's ability
to optimise register usage can be brought into play when calling kernel
services. Admittedly, this might not be important in practice, but
theoretically it is an advantage. The L4 people don't seem to have
considered the issues concerning the use of a language other than C.

As for performance, I suppose ultimately the proof of the pudding will be in
the eating. I suspect that Bachar will do quite well against L4. It won't be
as fast, but it does more anyway.

> I think L4 has done a nice job of saving the programmer from the hassle of
> potential processor differences. They've apparently done a good job of it,
> because it performs quite well in the benchmark tests compared to the
> others.

Yes, but AdaOS will do a much better job of saving the programmer from the
hassle of potential processor differences, but the simple token of allowing
the programmer to program in Ada!  Most such differences will vanish from
the programmer's horizon, because they are taken care of by the Ada
compiler, in a way that a C compiler cannot achieve.

> > > But if you look at the Mach paper above, you can do the very same
> > > thing with ports. Even without thread-migration, you can interpose
> > > proxies. With thread-migration enabled RPC, you achieve both and
> > > provide the OS designer a nicer interface.
> > 
> > But if we did that, what would be the advantage?
> 
> Its cheaper. To have 1000 send rights, costs near nothing. 1000 send
> rights in one task costs nothing (just increments ref count). 1000 in a
> 1000 different tasks, costs one pointer in each task. Nothing further.
> 
> 1000 threads OTOH, costs you state information for each of those 1000
> threads (same task or different). The amount of state information needed
> will vary with platform.

I have to guess somewhat, Warren, at the point you are making (maybe it's my
fault). Are you suggesting that if I have 1000 client processes (or
threads?) performing trans-network IPC, it will be necessary to have 1000
(or 2000?) proxy processes (or threads)? On AdaOS, in practice, there will
only be one proxy process (the program Avarus, running once on each
workstation).

> > I don't see the point in making the microkernel /that/ minimalist, to be
> > honest. Whatever the upper layer did, it would have to be dependent upon
> > intimate details of the kernel. So why not just lump them together?
> 
> But in your answer:
> 
> > I have taken the approach of local IPC being supported by the
> > microkernel (and lower levels of TCB software), and network IPC being
> > supported by a higher layer of TCB software (Avarus).
> 
> If I understand correctly, you are saying that your network "layer" is
> already built on top of lower levels (like microkernel).

Yes, that's correct.

> > > To make this work, as I envision it, requires that all participating
> > > nodes in a cluster see the file system the same way. IOW, there can
> > > only be one root file system, which obviously is only local to one
> > > node. To all other nodes, root will be access NFS-like (ie. over the
> > > network cable).
> > 
> > This is a fundamental precept of fully distributed networking.
> 
> It would seem so, but I think there is room for creative thinking here.
> Perhaps a "context sensitive view" is also possible.

Indeed, for legacy software it may be necessary to do that sort of thing. I
think I will attempt to organise the canonical file hierarchy according to
the FHS. However, for a program written to be run on a Windows computer, for
example, it may be necessary to translate '\' into '/', as well as drive
letters into the appropriate mounted drive names, and so on.

> > > SQL databases are designed to deal with large volumes of similar data,
> > > in SELECTs, joins etc. However, when you look at what goes into the
> > > registry, a lot of it is custom, hierarchically structured
> > > information.
> > 
> > No, the kind of data that goes into the Registry would be naturally
> > organised in many complex ways. For example, many different applications
> > store their own font information, but a font disinstaller may well wish
> > to be able to 'cut across' this font data, to make appropriate
> > adjustments (for all the applications). As another example, some
> > applications may wish to store data for several different users, but an
> > adminstrative program may wish to change all the data (across many
> > applications) for a particular user.
> 
> The same argument can be made for changing file systems into databases. So
> far, the filesystems approach seems to be the clear winner for that kind
> of data.

On the contrary, the database approach has been used by IBM (on their
mainframes and minis) for decades, and appears to be a clear winner over
using files. Microsoft have had plans for quite a long time now to introduce
a database-based OS as a replacement to Windows.

-- 
Nick Roberts

Re: For the AdaOS folks

[Warren W. Gay VE3WWG]

Nick Roberts wrote:
> "Warren W. Gay VE3WWG" <ve3wwg@NoSpam.cogeco.ca> wrote:
>>Mach's migrating threads do in fact cross into other "tasks". Instead of a
>>caller thread stopping when a message is queued, and the receiving thread
>>starting (from a blocked state), a migrating thread continues from the
>>caller, works through the server code, and then returns back to the caller
>>(in the caller's "task"). This is a simplification of what happens,
>>because there are "activations" and stack games that must happen to make
>>this work - but it is a migrating thread.
> 
> I'm not quite sure what you're getting at, Warren. I don't think it's very
> surprising that Mach has capabilities that are similar to Bachar, since they
> are both aimed at doing roughly the same thing. But there are significant
> differences in the details. I did begin designing (what has become) Bachar a
> long time ago (pre-1990s).

Mach's roots go back to 1975, which is quite some time ago ;-)

>>>>But if you look at the Mach paper above, you can do the very same
>>>>thing with ports. Even without thread-migration, you can interpose
>>>>proxies. With thread-migration enabled RPC, you achieve both and
>>>>provide the OS designer a nicer interface.
>>>
>>>But if we did that, what would be the advantage?
>>
>>Its cheaper. To have 1000 send rights, costs near nothing. 1000 send
>>rights in one task costs nothing (just increments ref count). 1000 in a
>>1000 different tasks, costs one pointer in each task. Nothing further.
>>
>>1000 threads OTOH, costs you state information for each of those 1000
>>threads (same task or different). The amount of state information needed
>>will vary with platform.
> 
> I have to guess somewhat, Warren, at the point you are making (maybe it's my
> fault). Are you suggesting that if I have 1000 client processes (or
> threads?) performing trans-network IPC, it will be necessary to have 1000
> (or 2000?) proxy processes (or threads)? On AdaOS, in practice, there will
> only be one proxy process (the program Avarus, running once on each
> workstation).

However, the "name service" for example, will often have many clients
(only one receive port for the name service, but at least one send-right
for every client "task" on the system). This is one example where
the numbers of send-rights can be large.
-- 
Warren W. Gay VE3WWG
http://home.cogeco.ca/~ve3wwg

Re: For the AdaOS folks

[Nick Roberts]

"Warren W. Gay VE3WWG" <ve3wwg@NoSPAM.cogeco.ca> wrote:

> Mach's roots go back to 1975, which is quite some time ago ;-)

Blimey, I didn't know that. I started designing the microkernel that has
become Bachar in the mid 1980s. Certainly I had never heard of Mach at that
time.

> However, the "name service" for example, will often have many clients
> (only one receive port for the name service, but at least one send-right
> for every client "task" on the system). This is one example where the
> numbers of send-rights can be large.

My current plan is that every process (except one, the Top Process) will
have recourse to a 'parent' (actually an 'inner parent') process, from
which, among other things, it can request a resolution of a named
environment variable. Normally, name servers would be obtained, queried, and
then discarded, to resolve each step of a path (starting at an environment
variable).

However, this means that every process will have to be issued with at least
one service key -- the one to call its parent -- for the lifetime of the
process. Avarus is the process that will have the most direct (inner) child
processes, typically in the ball park of 50 to 100 per workstation, so
that's 100 service keys for a start.

All of those service keys /could/ be serviced by just one thread (and one
IPC service). However, it might be better to have each serviced by its own
separate thread (operating at an appropriate priority). So, that might
require 100 threads within Avarus just for that purpose alone; there would
surely be many more threads for other purposes.

Looking at my Windows XP Task Manager just now, I see 47 processes and 446
threads. Interesting. I'm not sure if it's a problem. Currently, I
anticipate capacity for 500 processes and 1500 threads (per workstation),
and for 1000 IPC services and 3000 service keys. Each (normal) thread will
eat up about 300 bytes of RAM (for the TSS and ring 0, 1, and 2 stacks). The
RAM used up for an IPC service or service key is negligible.

How would Mach do it better?

-- 
Nick Roberts

Re: For the AdaOS folks

[Warren W. Gay VE3WWG]

Nick Roberts wrote:
> "Warren W. Gay VE3WWG" <ve3wwg@NoSPAM.cogeco.ca> wrote:
>>Mach's roots go back to 1975, which is quite some time ago ;-)
> 
> Blimey, I didn't know that. I started designing the microkernel that has
> become Bachar in the mid 1980s. Certainly I had never heard of Mach at that
> time.

There is a little Mach history blurb available at:

  http://hurd.gnufans.org/bin/view/Mach/MachHistory

Presumably if the project started in 1975, the ideas must have been
floating around for a time prior to that.

>>However, the "name service" for example, will often have many clients
>>(only one receive port for the name service, but at least one send-right
>>for every client "task" on the system). This is one example where the
>>numbers of send-rights can be large.
...
> Looking at my Windows XP Task Manager just now, I see 47 processes and 446
> threads. Interesting. I'm not sure if it's a problem. Currently, I
> anticipate capacity for 500 processes and 1500 threads (per workstation),
> and for 1000 IPC services and 3000 service keys. Each (normal) thread will
> eat up about 300 bytes of RAM (for the TSS and ring 0, 1, and 2 stacks). The
> RAM used up for an IPC service or service key is negligible.
> 
> How would Mach do it better?

The only suggestion I was making is that if you separate the port from
the thread (the "Mach way" instead of the L4 way where port=thread), then
you don't have the "thread memory overhead" for each "port".  A port
becomes much like a UNIX file descriptor, where a dup(2)-ed file
descriptor just shares an existing file entry (virtually no overhead
- just a pointer and reference count). A port just becomes a nearly
cost-free handle.

As soon as you say every communication end-point implies a thread, then
I as a designer, have to start counting the cost.
--
Warren W. Gay VE3WWG
http://home.cogeco.ca/~ve3wwg