* Ada OS Kernel features @ 2001-09-05 5:58 Brian Catlin 2001-09-05 10:15 ` Jacob Sparre Andersen ` (3 more replies) 0 siblings, 4 replies; 41+ messages in thread From: Brian Catlin @ 2001-09-05 5:58 UTC (permalink / raw) What sort of features would be desirable/required in the kernel? Processor architectures: What sort of processor architectures should be supported? 32-bit obviously, but what about 64-bit, or 16-bit? Little-endian only, or big-endian too? (personally, I view big-endian as a crime against nature, and would vote against it - besides there are not many of them out there compared to little-endian processors) Deciding on these issues early is very important, as it will greatly influence the rest of the design and implementation of the kernel. All the candidate processors need to be examined, and the features that are present in all (and all subsequent architectures that will be supported) need to be identified. Processors that do not implement certain required features must be able to emulate these features in software, or the processor cannot be supported. Scheduler: I think that the kernel should support three types of tasks: isochronous, real-time, and general purpose. An isochronous task is a periodic time-driven task with performance requirements for bounded latency and jitter, and guaranteed throughput. A real-time task has performance requirements for low latency and high throughput. Both isochronous and real-time tasks have bounded execution times, so that the system can bound latency and jitter for other such tasks. In contrast, a general purpose task is characterized by lengthy or unbounded execution time, and may not have strict requirements for latency or throughput; however, the system does guarantee some progress for such tasks. Tasks should be grouped into processes. Symmetric multi-processing: This is the same capability general-purpose operating systems such as NT, VMS, etc. have. Each processor is of the same type, and has the same view of memory. It then becomes irrelevant which processor is running a particular task. Synchronization primitives: How will tasks synchronize with each other? Synchronization of tasks within the same process can use existing Ada methods; but what about tasks in different processes, or on different processors? What about application written in languages other than Ada? Memory management: A demand-paged memory manager would seem to be required. However, I would not like to see paging made a requirement - as long as there is enough physical memory, there should be no need. This would allow the operating system to be tailored to run on a very small diskless system. It is the memory manager that really differentiates one process from another, by providing a unique address space that is unreachable by any other process. The memory manager should support sharing pages between processes to facilitate fast inter-process communications. Mapping a file on the disk directly into a range of addresses in a process would also be a big benefit: all disk reads and writes would be implemented as reads and writes to a process' address space. File system cache: What sort of file system cache would be appropriate? A logical block cache uses much less memory than a virtual block cache, but the virtual block cache has the ability to read ahead. A logical block cache will cache the blocks on a disk (or partition) _after_ an application has already read them in. A logical block cache knows nothing about files. A virtual block cache will cache blocks in a file, which allows read ahead (because the next blocks in a file will always be known). I/O subsystem: What form are drivers to take? A packet-driven system would seems to provide the best performance. In a packet-driven I/O system, a context block (generally referred to as an IRP (I/O Request Packet)) containing all the information to process an I/O request is passed around between drivers. What form should a driver take? Implementing a driver as one or more tasks in the kernel seems to be the most natural. The layering of device drivers (as implemented) in NT would also seem to be a very valuable feature. -Brian ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Ada OS Kernel features 2001-09-05 5:58 Ada OS Kernel features Brian Catlin @ 2001-09-05 10:15 ` Jacob Sparre Andersen 2001-09-05 11:16 ` Larry Kilgallen ` (2 subsequent siblings) 3 siblings, 0 replies; 41+ messages in thread From: Jacob Sparre Andersen @ 2001-09-05 10:15 UTC (permalink / raw) Brian: > Processor architectures: > What sort of processor architectures should be supported? IA32, HP-Alpha (or whatever it is called today) and m68k (I think that's all I have in my office at the moment ;-). Sparc would be cool because of the open Sparc clone from ESA. > Synchronization primitives: > How will tasks synchronize with each other? Synchronization of tasks within > the same process can use existing Ada methods; but what about tasks in different > processes, or on different processors? Entries and protected types should, if it is possible, be supported on the OS level (but I don't know how). > What about application written in > languages other than Ada? Think about that later. Jacob -- http://jacob.sparre.dk/Ada/ ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Ada OS Kernel features 2001-09-05 5:58 Ada OS Kernel features Brian Catlin 2001-09-05 10:15 ` Jacob Sparre Andersen @ 2001-09-05 11:16 ` Larry Kilgallen 2001-09-05 17:06 ` Brian Catlin ` (2 more replies) 2001-09-05 14:04 ` Ted Dennison 2001-09-05 16:02 ` Alfred Hilscher 3 siblings, 3 replies; 41+ messages in thread From: Larry Kilgallen @ 2001-09-05 11:16 UTC (permalink / raw) In article <9n4euv$t9m$1@slb6.atl.mindspring.net>, "Brian Catlin" <briancatlin@mindspring.com> writes: > What sort of features would be desirable/required in the kernel? > Symmetric multi-processing: > This is the same capability general-purpose operating systems such as NT, VMS, > etc. have. Each processor is of the same type, and has the same view of memory. > It then becomes irrelevant which processor is running a particular task. You leave out a lot of big systems if you don't support NUMA. ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Ada OS Kernel features 2001-09-05 11:16 ` Larry Kilgallen @ 2001-09-05 17:06 ` Brian Catlin 2001-09-06 14:35 ` Marin David Condic 2001-09-05 17:55 ` David Starner 2001-09-12 6:47 ` Mats Karlssohn 2 siblings, 1 reply; 41+ messages in thread From: Brian Catlin @ 2001-09-05 17:06 UTC (permalink / raw) "Larry Kilgallen" <Kilgallen@SpamCop.net> wrote in message news:OCbelAvsFWui@eisner.encompasserve.org... > In article <9n4euv$t9m$1@slb6.atl.mindspring.net>, "Brian Catlin" <briancatlin@mindspring.com> writes: > > What sort of features would be desirable/required in the kernel? > > > Symmetric multi-processing: > > This is the same capability general-purpose operating systems such as NT, VMS, > > etc. have. Each processor is of the same type, and has the same view of memory. > > It then becomes irrelevant which processor is running a particular task. > > You leave out a lot of big systems if you don't support NUMA. True, but how many people that would be participating in the implementation of the Ada OS have access to such machines? Supporting these machines initially would require quite a bit of effort, and might be biting off more than we can chew. Perhaps adding support for them later in the project would be better (yes, I know that it will require lots of rework, but if the project is too big initially, it will never get started) -Brian ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Ada OS Kernel features 2001-09-05 17:06 ` Brian Catlin @ 2001-09-06 14:35 ` Marin David Condic 2001-09-07 9:31 ` Tarjei T. Jensen 0 siblings, 1 reply; 41+ messages in thread From: Marin David Condic @ 2001-09-06 14:35 UTC (permalink / raw) Help me pound the drum some more: KEEP THE GOALS TO SOMETHING ACHIEVABLE! :-) If it is possible to leave the design open-ended enough so that a later revision can support more general requirements, then O.K. This ought to be reasonably simple since we're all experts in the notion of abstraction & information hiding. If there is a memory manager package that hides the details of what kind of memory is there and where does it reside and all that, it should be easy to replace the body with something that supports whatever you like. But the first cut ought to go towards a simple, common architecture. Think PC as the first target. Think "embeddable" since if all you have is a basic core (no file system, no elaborate GUI stuff, etc.) you still have something that can be *used* in real world apps. If there were some core piece of an OS that could boot on a PC from a floppy and maybe do some simple command-line communication through a serial port then you'd have something that was embeddable & runable on an old PC where a hobbyist could have something to work with. That gets the "Quixote OS" a *LOT* farther than all the wonderful, theoretical OS's floating around in the minds of many engineers. MDC -- Marin David Condic Senior Software Engineer Pace Micro Technology Americas www.pacemicro.com Enabling the digital revolution e-Mail: marin.condic@pacemicro.com Web: http://www.mcondic.com/ "Brian Catlin" <briancatlin@mindspring.com> wrote in message news:9n5m26$tb6$1@slb6.atl.mindspring.net... > > True, but how many people that would be participating in the implementation of > the Ada OS have access to such machines? Supporting these machines initially > would require quite a bit of effort, and might be biting off more than we can > chew. Perhaps adding support for them later in the project would be better > (yes, I know that it will require lots of rework, but if the project is too big > initially, it will never get started) > ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Ada OS Kernel features 2001-09-06 14:35 ` Marin David Condic @ 2001-09-07 9:31 ` Tarjei T. Jensen 0 siblings, 0 replies; 41+ messages in thread From: Tarjei T. Jensen @ 2001-09-07 9:31 UTC (permalink / raw) Marin David Condic wrote in message <9n81k3$oqt$1@nh.pace.co.uk>... >Help me pound the drum some more: KEEP THE GOALS TO SOMETHING ACHIEVABLE! >:-) Right on commander! ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Ada OS Kernel features 2001-09-05 11:16 ` Larry Kilgallen 2001-09-05 17:06 ` Brian Catlin @ 2001-09-05 17:55 ` David Starner 2001-09-05 18:42 ` Darren New 2001-09-12 6:47 ` Mats Karlssohn 2 siblings, 1 reply; 41+ messages in thread From: David Starner @ 2001-09-05 17:55 UTC (permalink / raw) On 5 Sep 2001 06:16:07 -0500, Larry Kilgallen <Kilgallen@SpamCop.net> wrote: > In article <9n4euv$t9m$1@slb6.atl.mindspring.net>, "Brian Catlin" <briancatlin@mindspring.com> writes: >> What sort of features would be desirable/required in the kernel? > >> Symmetric multi-processing: >> This is the same capability general-purpose operating systems such as NT, VMS, >> etc. have. Each processor is of the same type, and has the same view of memory. >> It then becomes irrelevant which processor is running a particular task. > > You leave out a lot of big systems if you don't support NUMA. And? Starting out trying to write an OS for everything from an Atari to a multimillion dollar mainframe or supercomputer just seems like a bad idea. I rather drop stuff off the top (where equipment and developers are scarce) rather than the bottom (where equipment and developers are plentful.) It can always be added in latter. (Cf. Linux, an OS written specifically for Linus Torvald's 386, now running on almost anything with 32+ bits.) -- David Starner - dstarner98@aasaa.ofe.org Pointless website: http://dvdeug.dhis.org "I don't care if Bill personally has my name and reads my email and laughs at me. In fact, I'd be rather honored." - Joseph_Greg ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Ada OS Kernel features 2001-09-05 17:55 ` David Starner @ 2001-09-05 18:42 ` Darren New 0 siblings, 0 replies; 41+ messages in thread From: Darren New @ 2001-09-05 18:42 UTC (permalink / raw) > And? Starting out trying to write an OS for everything from an Atari to > a multimillion dollar mainframe or supercomputer just seems like a bad > idea. I rather drop stuff off the top (where equipment and developers are > scarce) rather than the bottom (where equipment and developers are plentful.) > It can always be added in latter. (Cf. Linux, an OS written specifically for > Linus Torvald's 386, now running on almost anything with 32+ bits.) The trick is to make sure the API works for all that, then make it for a subset of useful machines. I.e., if you're doing persistant addresses, don't use 32 (or 64) bits. If you're doing files, make sure you define the semantics for networked files. If you're doing tasking, make sure you can specify what you need to to make NUMA work. -- Darren New San Diego, CA, USA (PST). Cryptokeys on demand. ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Ada OS Kernel features 2001-09-05 11:16 ` Larry Kilgallen 2001-09-05 17:06 ` Brian Catlin 2001-09-05 17:55 ` David Starner @ 2001-09-12 6:47 ` Mats Karlssohn 2 siblings, 0 replies; 41+ messages in thread From: Mats Karlssohn @ 2001-09-12 6:47 UTC (permalink / raw) Larry Kilgallen wrote: > > In article <9n4euv$t9m$1@slb6.atl.mindspring.net>, "Brian Catlin" <briancatlin@mindspring.com> writes: > > What sort of features would be desirable/required in the kernel? > > > Symmetric multi-processing: > > This is the same capability general-purpose operating systems such as NT, VMS, > > etc. have. Each processor is of the same type, and has the same view of memory. > > It then becomes irrelevant which processor is running a particular task. > > You leave out a lot of big systems if you don't support NUMA. Yes, have you seen (or maybe even published :-) any good paper(s) on what kinds of OS support is needed for NUMA machines ? I must have looked in the wrong direction... -- Mats Karlssohn, developer mailto:mats@mida.se Mida Systemutveckling AB http://www.mida.se Box 64, S-732 22 ARBOGA, SWEDEN Phone: +46-(0)589-89808 Fax: +46-(0)589-89809 ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Ada OS Kernel features 2001-09-05 5:58 Ada OS Kernel features Brian Catlin 2001-09-05 10:15 ` Jacob Sparre Andersen 2001-09-05 11:16 ` Larry Kilgallen @ 2001-09-05 14:04 ` Ted Dennison 2001-09-05 17:23 ` Brian Catlin 2001-09-05 16:02 ` Alfred Hilscher 3 siblings, 1 reply; 41+ messages in thread From: Ted Dennison @ 2001-09-05 14:04 UTC (permalink / raw) In article <9n4euv$t9m$1@slb6.atl.mindspring.net>, Brian Catlin says... >Scheduler: > I think that the kernel should support three types of tasks: isochronous, >real-time, and general purpose. An isochronous task is a periodic time-driven >task with performance requirements for bounded latency and jitter, and >guaranteed throughput. A real-time task has performance requirements for low >latency and high throughput. Both isochronous and real-time tasks have bounded >execution times, so that the system can bound latency and jitter for other such >tasks. In contrast, a general purpose task is characterized by lengthy or Interesting. I've never seen them broken out that way. Is that a typical way of thinking about real-time tasks these days? In fact, I have trouble finding good references for dealing with RT tasks that have jitter requirements. As near as I can tell, any such task has to be scheduled in such a way that it is the first task to get run after its designated clock tick. Obviously this means you can't create a whole lot of these... >be the most natural. The layering of device drivers (as implemented) in NT >would also seem to be a very valuable feature. I've started to look at NT drivers recently. I do think thier layered approach is a cool idea, but I have one major problem with it. There doesn't appear to be any good reason to require any but the lowest-level driver to run in kernel mode (ring 3, supervisor, whatever you call it...) Such programs are much tougher to debug, and bugs in them are much more catastrophic. It appears that they are just having them run in kernel mode because that's what device drivers are supposed to do. --- T.E.D. homepage - http://www.telepath.com/dennison/Ted/TED.html home email - mailto:dennison@telepath.com ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Ada OS Kernel features 2001-09-05 14:04 ` Ted Dennison @ 2001-09-05 17:23 ` Brian Catlin 2001-09-05 20:17 ` Ted Dennison 0 siblings, 1 reply; 41+ messages in thread From: Brian Catlin @ 2001-09-05 17:23 UTC (permalink / raw) "Ted Dennison" <dennison@telepath.com> wrote in message news:Khql7.4732$4z.14760@www.newsranger.com... > In article <9n4euv$t9m$1@slb6.atl.mindspring.net>, Brian Catlin says... > >Scheduler: > > I think that the kernel should support three types of tasks: isochronous, > >real-time, and general purpose. An isochronous task is a periodic time-driven > >task with performance requirements for bounded latency and jitter, and > >guaranteed throughput. A real-time task has performance requirements for low > >latency and high throughput. Both isochronous and real-time tasks have bounded > >execution times, so that the system can bound latency and jitter for other such > >tasks. In contrast, a general purpose task is characterized by lengthy or > > Interesting. I've never seen them broken out that way. Is that a typical way of > thinking about real-time tasks these days? In fact, I have trouble finding good > references for dealing with RT tasks that have jitter requirements. As near as I > can tell, any such task has to be scheduled in such a way that it is the first > task to get run after its designated clock tick. Obviously this means you can't > create a whole lot of these... An isochronously scheduled task means that the task will get a fixed amount of CPU time over time (think of it as getting a fixed percentage of the CPU over time). Supporting isochronous tasks is a requirement if you want to be able to handle streaming multimedia, such as audio or video. A simple way to implement this, would have the scheduler use a weighted round robin scheme for scheduling real-time and general purpose tasks. Isochronous tasks would be driven off timer interrupts. The kernel would assign a scheduling flag and a weight to each real-time task. To schedule realtime tasks, the would scheduler poll a set of flags, each of which indicates a pending task. For each flag that is set, the scheduler would invoke the corresponding real-time task, with the assigned weight as a parameter. A scheduling flag may be set by an interrupt service routine as well as any realtime or general purpose task in the system. As a further refinement, the kernel could introduce the notion of preemption windows. Isochronous and real-time tasks can only be preempted during their preemption windows. An isochronous task could be preempted by higher priority isochronous tasks; real-time tasks could be preempted by isochronous tasks. Preemption windows would be placed at the completion of a unit of work, where little state needs to be saved in context-switching to the new task. A real-time task would be expected to process at most the number of work units equal to the weight passed to it as a parameter. Upon completion, the task saves its state, if necessary, and voluntarily returns to the scheduler. After having completed one round of polling for real-time tasks, the scheduler would switch to the list of general purpose tasks. This task runs until it blocks or the quantum timer expires. If the timer expires, the scheduler saves the tasks's context and goes back to servicing real-time tasks. > >be the most natural. The layering of device drivers (as implemented) in NT > >would also seem to be a very valuable feature. > > I've started to look at NT drivers recently. I do think thier layered approach > is a cool idea, but I have one major problem with it. There doesn't appear to be > any good reason to require any but the lowest-level driver to run in kernel mode > (ring 3, supervisor, whatever you call it...) Such programs are much tougher to > debug, and bugs in them are much more catastrophic. It appears that they are > just having them run in kernel mode because that's what device drivers are > supposed to do. Yes, it makes them more difficult to debug, but placing the drivers in the kernel has several benefits: 1. Processing an I/O request requires access to privileged system routines and data structures. Allowing access to these resources from user-mode is a *BAD* idea (this is one of the major reasons why Win98 was so unstable; applications could step all over the kernel) 2. Putting drivers in user-mode means increasing the number of transitions between user-mode and kernel-mode, which is *VERY* time-consuming 3. A driver will need access to the requesting process' address space, but if the "driver" is running as part of another process, the requesting process' address space is inaccessible. -Brian ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Ada OS Kernel features 2001-09-05 17:23 ` Brian Catlin @ 2001-09-05 20:17 ` Ted Dennison 2001-09-05 21:10 ` Brian Catlin 0 siblings, 1 reply; 41+ messages in thread From: Ted Dennison @ 2001-09-05 20:17 UTC (permalink / raw) In article <9n5n33$ke5$1@nntp9.atl.mindspring.net>, Brian Catlin says... > >An isochronously scheduled task means that the task will get a fixed amount of >CPU time over time (think of it as getting a fixed percentage of the CPU over >time). Supporting isochronous tasks is a requirement if you want to be able to >handle streaming multimedia, such as audio or video. A simple way to implement Ahhh. That explains it. The multimedia folks do seem to like that term. They used it in pretty much the same sense in the FireWire spec (reserving a % of the bus bandwidth). However, I don't see anything in there that really speaks about jitter. Perhaps jitter reduction (making sure the task starts regularly at the exact same time every cycle) is just an implementation detail they don't like to talk about explicitly? The only easy way I've found to prevent jitter is to schedule a task to be the first one that runs after the clock tick. Clearly the amount of tasks one can use this solution on is limited by the product of your clock period and the task frequency (eg: 240Hz clock w/ 60Hz threads gives you at most 4 possible jitter-free tasks). However, I heartily agree that the scheduling algorithm would need to be different for RT and non-RT tasks. >Yes, it makes them more difficult to debug, but placing the drivers in the >kernel has several benefits: >1. Processing an I/O request requires access to privileged system routines and .. >2. Putting drivers in user-mode means increasing the number of transitions .. >3. A driver will need access to the requesting process' address space, but if .. Interesting. But I'm pretty sure that Mach puts *all* device drivers into user space, as well as large parts of the kernel. --- T.E.D. homepage - http://www.telepath.com/dennison/Ted/TED.html home email - mailto:dennison@telepath.com ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Ada OS Kernel features 2001-09-05 20:17 ` Ted Dennison @ 2001-09-05 21:10 ` Brian Catlin 2001-09-06 6:45 ` Tarjei T. Jensen 0 siblings, 1 reply; 41+ messages in thread From: Brian Catlin @ 2001-09-05 21:10 UTC (permalink / raw) "Ted Dennison" <dennison@telepath.com> wrote in message news:VLvl7.5404$4z.21201@www.newsranger.com... > In article <9n5n33$ke5$1@nntp9.atl.mindspring.net>, Brian Catlin says... > > > >An isochronously scheduled task means that the task will get a fixed amount of > >CPU time over time (think of it as getting a fixed percentage of the CPU over > >time). Supporting isochronous tasks is a requirement if you want to be able to > >handle streaming multimedia, such as audio or video. A simple way to implement > > Ahhh. That explains it. The multimedia folks do seem to like that term. They > used it in pretty much the same sense in the FireWire spec (reserving a % of the > bus bandwidth). > > However, I don't see anything in there that really speaks about jitter. Perhaps > jitter reduction (making sure the task starts regularly at the exact same time > every cycle) is just an implementation detail they don't like to talk about > explicitly? > > The only easy way I've found to prevent jitter is to schedule a task to be the > first one that runs after the clock tick. Clearly the amount of tasks one can > use this solution on is limited by the product of your clock period and the task > frequency (eg: 240Hz clock w/ 60Hz threads gives you at most 4 possible > jitter-free tasks). This is the point at which Rate Monotonic Analysis (RMA) enters the picture. The behavior of real time (and isochronous) tasks need to be well understood in order to schedule them effectively. Usually, RMA is the provence of real-time systems, but to effectively support a _few_ isochronous and real-time tasks in a general purpose operating system, we should build some tools (performance counters and such) for measuring the behavior of these tasks to help the author better optimize the code (and overall system performance). > However, I heartily agree that the scheduling algorithm would need to be > different for RT and non-RT tasks. > > >Yes, it makes them more difficult to debug, but placing the drivers in the > >kernel has several benefits: > >1. Processing an I/O request requires access to privileged system routines and > .. > >2. Putting drivers in user-mode means increasing the number of transitions > .. > >3. A driver will need access to the requesting process' address space, but if > .. > > Interesting. But I'm pretty sure that Mach puts *all* device drivers into user > space, as well as large parts of the kernel. Well, I wouldn't hold Mach up as the end-all or be-all of operating system kernels; its performance is nothing to brag about. It certainly has many interesting concepts, but its overhead is too high for my taste (I haven't looked at it in several years, so it may have been cleaned up; I'll have to go check it out again). As I mentioned in a previous post, by implementing drivers as essentially applications in their own processes, the system will incur a huge amount of overhead invalidating the translation lookaside buffer and the accompanying multiple mode transitions for process context switches. -Brian ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Ada OS Kernel features 2001-09-05 21:10 ` Brian Catlin @ 2001-09-06 6:45 ` Tarjei T. Jensen 2001-09-06 6:56 ` Brian Catlin 0 siblings, 1 reply; 41+ messages in thread From: Tarjei T. Jensen @ 2001-09-06 6:45 UTC (permalink / raw) Brian Catlin wrote >Well, I wouldn't hold Mach up as the end-all or be-all of operating system >kernels; its performance is nothing to brag about. It certainly has many >interesting concepts, but its overhead is too high for my taste (I haven't >looked at it in several years, so it may have been cleaned up; I'll have to go >check it out again). As I recall Mach is supposed to represent a snapshot of microkernel technology at a relative early stage. I believe that at least IBM has advanced the state of the art considerably since mach was released. Greetings, ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Ada OS Kernel features 2001-09-06 6:45 ` Tarjei T. Jensen @ 2001-09-06 6:56 ` Brian Catlin 2001-09-06 14:05 ` Ted Dennison 0 siblings, 1 reply; 41+ messages in thread From: Brian Catlin @ 2001-09-06 6:56 UTC (permalink / raw) "Tarjei T. Jensen" <tarjei.jensen@kvaerner.com> wrote in message news:9n762l$7ah4@news.kvaerner.com... > > Brian Catlin wrote > >Well, I wouldn't hold Mach up as the end-all or be-all of operating system > >kernels; its performance is nothing to brag about. It certainly has many > >interesting concepts, but its overhead is too high for my taste (I haven't > >looked at it in several years, so it may have been cleaned up; I'll have to > go > >check it out again). > > As I recall Mach is supposed to represent a snapshot of microkernel > technology at a relative early stage. I believe that at least IBM has > advanced the state of the art considerably since mach was released. Yes, I just looked at RT-Mach (Real-Time Mach), and it is quite impressive - as far as microkernels go. The big problem with microkernels is that you have to use RPC to communicate between services, which is much more expensive than a simple CALL in a monolithic kernel. -Brian ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Ada OS Kernel features 2001-09-06 6:56 ` Brian Catlin @ 2001-09-06 14:05 ` Ted Dennison 0 siblings, 0 replies; 41+ messages in thread From: Ted Dennison @ 2001-09-06 14:05 UTC (permalink / raw) In article <9n76r2$nkl$1@slb4.atl.mindspring.net>, Brian Catlin says... > >Yes, I just looked at RT-Mach (Real-Time Mach), and it is quite impressive - as >far as microkernels go. The big problem with microkernels is that you have to >use RPC to communicate between services, which is much more expensive than a >simple CALL in a monolithic kernel. From what I read, Mach supposedly tried to solve that by recognising when one can be transformed into the other. But that recognition code takes CPU cycles too... --- T.E.D. homepage - http://www.telepath.com/dennison/Ted/TED.html home email - mailto:dennison@telepath.com ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Ada OS Kernel features 2001-09-05 5:58 Ada OS Kernel features Brian Catlin ` (2 preceding siblings ...) 2001-09-05 14:04 ` Ted Dennison @ 2001-09-05 16:02 ` Alfred Hilscher 2001-09-05 16:19 ` Jacob Sparre Andersen 2001-09-05 17:44 ` Brian Catlin 3 siblings, 2 replies; 41+ messages in thread From: Alfred Hilscher @ 2001-09-05 16:02 UTC (permalink / raw) Brian Catlin wrote: > > What sort of features would be desirable/required in the kernel? > > Processor architectures: > What sort of processor architectures should be supported? 32-bit obviously, > but what about 64-bit, or 16-bit? Little-endian only, or big-endian too? 32-bit open for future 64-bit. Both, _big_ and _little_ endian. > (personally, I view big-endian as a crime against nature, and would vote against Why ? > processors) Deciding on these issues early is very important, as it will > greatly influence the rest of the design and implementation of the kernel. All In which manner? Where do you think that there will be any influence? > Memory management: > A demand-paged memory manager would seem to be required. However, I would not > like to see paging made a requirement - as long as there is enough physical > memory, there should be no need. Thats what I liked on OS/2, one was able to switch it off. > process an I/O request is passed around between drivers. What form should a > driver take? Implementing a driver as one or more tasks in the kernel seems to > be the most natural. The layering of device drivers (as implemented) in NT > would also seem to be a very valuable feature. You should be able to load/unload a driver dynamically (I hate rebooting because of driver change). You should be able to "overload" a driver. What I mean ? Lets assume you have a simple grafic driver on bootup, then you load a "better" (more complex, higher resolution, 3D excelerator ...) one. If this one crashes, then it should simply be unloaded and the system should continue work with the (simple) default driver - instead of showing a "blue screen" ;-) ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Ada OS Kernel features 2001-09-05 16:02 ` Alfred Hilscher @ 2001-09-05 16:19 ` Jacob Sparre Andersen 2001-09-05 17:58 ` Brian Catlin 2001-09-05 17:44 ` Brian Catlin 1 sibling, 1 reply; 41+ messages in thread From: Jacob Sparre Andersen @ 2001-09-05 16:19 UTC (permalink / raw) Alfred: > Brian Catlin wrote: > > > > What sort of features would be desirable/required in the kernel? [...] > You should be able to load/unload a driver dynamically (I hate rebooting > because of driver change). And it would be nice if the kernel was prepared for kernel updates on-the-fly (load new kernel image, copy process information to the new kernel image, hand over operations to the new kernel). For most uses it is of course just a gimmick, but it is likely to win some of the "uptime freaks" from the Linux camp. ;-) Jacob -- <URL: sm�snak://arbejde/gangm�de/...> ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Ada OS Kernel features 2001-09-05 16:19 ` Jacob Sparre Andersen @ 2001-09-05 17:58 ` Brian Catlin 0 siblings, 0 replies; 41+ messages in thread From: Brian Catlin @ 2001-09-05 17:58 UTC (permalink / raw) "Jacob Sparre Andersen" <sparre@nbi.dk> wrote in message news:3B965086.B9D8E007@nbi.dk... > Alfred: > > > Brian Catlin wrote: > > > > > > What sort of features would be desirable/required in the kernel? > [...] > > You should be able to load/unload a driver dynamically (I hate rebooting > > because of driver change). > > And it would be nice if the kernel was prepared for kernel > updates on-the-fly (load new kernel image, copy process > information to the new kernel image, hand over operations to > the new kernel). A hugely difficult problem. Think about any running program. Your want to replace it, while it is running, with a new and updated version of itself. The new version has different data structures. How do you stop the running program (which a user might be using), and start up the new one transparently to the user, with the same state as the old program? It is possible to do, however, the notion of a checkpoint needs to be added to the system and applications. When a checkpoint happens, all internal state needs to be saved somewhere, then the new program/system has to be loaded, and then the old state information needs to be interpreted to put the new program/system in the same state. This will not be an instantaneous process. I've been thinking about how to implement this for the last few years, and I've been looking for the opportunity to implement it. One of the ramifications of implementing this type of checkpointing, is that you now have the ability to move running processes (applications) from one system to another (of the same architecture) at run-time. This opens the door for true "compute servers". You could start an application on your system, and then decide to move it off to another machine if its taking too long, or you needed to reboot your machine. Obviously, there are many constraints to such a thing, but the 24x7 crowd would love it. -Brian ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Ada OS Kernel features 2001-09-05 16:02 ` Alfred Hilscher 2001-09-05 16:19 ` Jacob Sparre Andersen @ 2001-09-05 17:44 ` Brian Catlin 2001-09-05 17:58 ` Darren New ` (4 more replies) 1 sibling, 5 replies; 41+ messages in thread From: Brian Catlin @ 2001-09-05 17:44 UTC (permalink / raw) "Alfred Hilscher" <Alfred.Hilscher@icn.siemens.de> wrote in message news:3B964C7A.BC04374E@icn.siemens.de... > > > Brian Catlin wrote: > > > > What sort of features would be desirable/required in the kernel? > > > > Processor architectures: > > What sort of processor architectures should be supported? 32-bit obviously, > > but what about 64-bit, or 16-bit? Little-endian only, or big-endian too? > > 32-bit open for future 64-bit. > > Both, _big_ and _little_ endian. > > > (personally, I view big-endian as a crime against nature, and would vote against > > Why ? I don't want to start a flame war, or get too far off topic, but big-endian just doesn't make sense (at least compared to little-endian). In little-endian, you count your bits and bytes from right to left, all the way to infinity. In big-endian, what purpose does swapping the bytes around have? Also, communications between a big-endian and little-endian system is a pain in the neck. Intel's x86 may be a lousy architecture, but in my opinion it has done at least one good thing: it has virtually eliminated big-endian machines. The only big companies making big-endian systems are IBM, Motorola, and Sun. While it is true that they could be called "heavy weights", their market share as a percentage of the x86 is miniscule. > > processors) Deciding on these issues early is very important, as it will > > greatly influence the rest of the design and implementation of the kernel. All > > In which manner? Where do you think that there will be any influence? This is one of the lessons learned from NT, which at one time supported 4 very different processor architectures. Unfortunately, they chose the MIPS as one of their targets, and the repercussions were felt all over the operating system, as the MIPS was a lot less capable that the other architectures (x86, PPC, Alpha). One of the most notable problems was that the early MIPS processors didn't support 4 processor modes (as I remember, it only supported two; user mode and supervisor mode). > > Memory management: > > A demand-paged memory manager would seem to be required. However, I would not > > like to see paging made a requirement - as long as there is enough physical > > memory, there should be no need. > > Thats what I liked on OS/2, one was able to switch it off. > > > process an I/O request is passed around between drivers. What form should a > > driver take? Implementing a driver as one or more tasks in the kernel seems to > > be the most natural. The layering of device drivers (as implemented) in NT > > would also seem to be a very valuable feature. > > You should be able to load/unload a driver dynamically (I hate rebooting > because of driver change). Agreed. This is not simple to implement, but it is well worth the trouble > You should be able to "overload" a driver. What I mean ? Lets assume > you have a simple grafic driver on bootup, then you load a "better" > (more complex, higher resolution, 3D excelerator ...) one. If this one > crashes, then it should simply be unloaded and the system should > continue work with the (simple) default driver - instead of showing a > "blue screen" ;-) My first reaction to this was "Not Possible". However, that isn't entirely true; it is just *VERY VERY* difficult. A driver runs in kernel mode, and has access to system data structures. If a driver corrupts a system data structure, how do you detect this, repair it, and continue? In such instances, it is much better to bugcheck (blue screen) the system than try to continue. Consider, if the system is slightly corrupted and continues to operate, there is the very real possibility that your data will be corrupted without your knowledge. This was Win98's philosophy, and it was a disaster. VMS and NT (and others) stop the system dead in its tracks to prevent hidden corruption. -Brian ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Ada OS Kernel features 2001-09-05 17:44 ` Brian Catlin @ 2001-09-05 17:58 ` Darren New 2001-09-05 18:51 ` Larry Kilgallen ` (3 subsequent siblings) 4 siblings, 0 replies; 41+ messages in thread From: Darren New @ 2001-09-05 17:58 UTC (permalink / raw) Brian Catlin wrote: > I don't want to start a flame war, or get too far off topic, but big-endian just > doesn't make sense (at least compared to little-endian). In little-endian, you > count your bits and bytes from right to left, all the way to infinity. In > big-endian, what purpose does swapping the bytes around have? There's no "left" and "right" in bits. That's an artifact of how you write things. If your normal english text was written right to left (like arabic) the whole discussion would never have started. :-) It's also an artifact of writing stuff from top to bottom. Do you number a hex dump with the lower addresses lower on the page? If you do, you discover that "endian" issues are completely different, all of a sudden. -- Darren New San Diego, CA, USA (PST). Cryptokeys on demand. ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Ada OS Kernel features 2001-09-05 17:44 ` Brian Catlin 2001-09-05 17:58 ` Darren New @ 2001-09-05 18:51 ` Larry Kilgallen 2001-09-05 19:25 ` chris.danx ` (2 subsequent siblings) 4 siblings, 0 replies; 41+ messages in thread From: Larry Kilgallen @ 2001-09-05 18:51 UTC (permalink / raw) In article <9n5o9n$37a$1@slb7.atl.mindspring.net>, "Brian Catlin" <briancatlin@mindspring.com> writes: > I don't want to start a flame war, or get too far off topic, but big-endian just > doesn't make sense (at least compared to little-endian). In little-endian, you > count your bits and bytes from right to left, all the way to infinity. An Ada Operating System should work with existing hardware. That includes both types of hardware. ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Ada OS Kernel features 2001-09-05 17:44 ` Brian Catlin 2001-09-05 17:58 ` Darren New 2001-09-05 18:51 ` Larry Kilgallen @ 2001-09-05 19:25 ` chris.danx 2001-09-05 20:07 ` Darren New ` (2 more replies) 2001-09-06 13:42 ` Alfred Hilscher 2001-09-08 3:55 ` Kenneth Almquist 4 siblings, 3 replies; 41+ messages in thread From: chris.danx @ 2001-09-05 19:25 UTC (permalink / raw) > > You should be able to load/unload a driver dynamically (I hate rebooting > > because of driver change). > > Agreed. This is not simple to implement, but it is well worth the trouble > > > You should be able to "overload" a driver. What I mean ? Lets assume > > you have a simple grafic driver on bootup, then you load a "better" > > (more complex, higher resolution, 3D excelerator ...) one. If this one > > crashes, then it should simply be unloaded and the system should > > continue work with the (simple) default driver - instead of showing a > > "blue screen" ;-) > > My first reaction to this was "Not Possible". However, that isn't entirely > true; it is just *VERY VERY* difficult. Only in the "drivers in supervisor mode" model. > A driver runs in kernel mode, Why? Why not just have it in user mode? It makes more sense to have them in user mode, at least to me. They can only corrupt themselves then, etc. > and has access to system data structures. Why should it? In your model a driver can screw a system up good and proper, but if you put the driver in user mode then the associated problems go away. New ones do crop up, but there's ways and means to deal with them. > If a driver corrupts a system data structure, how do you detect this, repair it, and continue? I really don't get why a driver must have access to system structures or atleast those in kernel space, can you explain this? ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Ada OS Kernel features 2001-09-05 19:25 ` chris.danx @ 2001-09-05 20:07 ` Darren New 2001-09-05 20:14 ` Larry Kilgallen 2001-09-05 20:18 ` Brian Catlin 2 siblings, 0 replies; 41+ messages in thread From: Darren New @ 2001-09-05 20:07 UTC (permalink / raw) > I really don't get why a driver must have access to system structures or > atleast those in kernel space, can you explain this? Generally because interrupt handling, munging of memory maps, and I/O instructions are all privledged operations on most machines. How do you let the user-space video driver change video modes without letting someone else do so? -- Darren New San Diego, CA, USA (PST). Cryptokeys on demand. ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Ada OS Kernel features 2001-09-05 19:25 ` chris.danx 2001-09-05 20:07 ` Darren New @ 2001-09-05 20:14 ` Larry Kilgallen 2001-09-06 13:45 ` Alfred Hilscher 2001-09-06 18:06 ` chris.danx 2001-09-05 20:18 ` Brian Catlin 2 siblings, 2 replies; 41+ messages in thread From: Larry Kilgallen @ 2001-09-05 20:14 UTC (permalink / raw) In article <3%ul7.3362$9z1.440040@news6-win.server.ntlworld.com>, "chris.danx" <chris.danx@ntlworld.com> writes: > >> > You should be able to load/unload a driver dynamically (I hate rebooting >> > because of driver change). >> >> Agreed. This is not simple to implement, but it is well worth the trouble >> >> > You should be able to "overload" a driver. What I mean ? Lets assume >> > you have a simple grafic driver on bootup, then you load a "better" >> > (more complex, higher resolution, 3D excelerator ...) one. If this one >> > crashes, then it should simply be unloaded and the system should >> > continue work with the (simple) default driver - instead of showing a >> > "blue screen" ;-) >> >> My first reaction to this was "Not Possible". However, that isn't > entirely >> true; it is just *VERY VERY* difficult. > > Only in the "drivers in supervisor mode" model. > >> A driver runs in kernel mode, > > Why? Why not just have it in user mode? It makes more sense to have them > in user mode, at least to me. They can only corrupt themselves then, etc. A driver is part of the TCB. It cannot be part of the TCB in user mode. (For anyone unfamiliar with the concept, the TCB is the Trusted Computing Base which implements the security policy of the system.) > I really don't get why a driver must have access to system structures or > atleast those in kernel space, can you explain this? A driver has access to raw hardware. Therefore it necessarily has access to the system -- it could rewrite the copy of the OS on disk if it chose. ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Ada OS Kernel features 2001-09-05 20:14 ` Larry Kilgallen @ 2001-09-06 13:45 ` Alfred Hilscher 2001-09-06 18:06 ` chris.danx 1 sibling, 0 replies; 41+ messages in thread From: Alfred Hilscher @ 2001-09-06 13:45 UTC (permalink / raw) Larry Kilgallen wrote: > > I really don't get why a driver must have access to system structures or > > atleast those in kernel space, can you explain this? > > A driver has access to raw hardware. Therefore it necessarily has > access to the system -- it could rewrite the copy of the OS on disk > if it chose. Yes, but only a hacker would do it. A serious driver developer does only what makes sense. No driver for NIC would offer the possibility to format the disk, or do you know one ? ;-) ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Ada OS Kernel features 2001-09-05 20:14 ` Larry Kilgallen 2001-09-06 13:45 ` Alfred Hilscher @ 2001-09-06 18:06 ` chris.danx 2001-09-06 19:41 ` Larry Kilgallen 2001-09-07 8:04 ` Dmitry Kazakov 1 sibling, 2 replies; 41+ messages in thread From: chris.danx @ 2001-09-06 18:06 UTC (permalink / raw) > > I really don't get why a driver must have access to system structures or > > atleast those in kernel space, can you explain this? > > A driver has access to raw hardware. Therefore it necessarily has > access to the system -- it could rewrite the copy of the OS on disk > if it chose. Perhaps, but that's a problem that could be solved, in part, by restricting what the drivers allowed to access. Video drivers have no business using the disk, so their access to disk access ports should be denied. Similar schemes should be used when appropriate. It should not be possible for a driver to access things willy nilly. Chris ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Ada OS Kernel features 2001-09-06 18:06 ` chris.danx @ 2001-09-06 19:41 ` Larry Kilgallen 2001-09-06 22:32 ` chris.danx 2001-09-07 8:04 ` Dmitry Kazakov 1 sibling, 1 reply; 41+ messages in thread From: Larry Kilgallen @ 2001-09-06 19:41 UTC (permalink / raw) In article <UWOl7.11352$592.628767@news2-win.server.ntlworld.com>, "chris.danx" <chris.danx@ntlworld.com> writes: > >> > I really don't get why a driver must have access to system structures or >> > atleast those in kernel space, can you explain this? >> >> A driver has access to raw hardware. Therefore it necessarily has >> access to the system -- it could rewrite the copy of the OS on disk >> if it chose. > > Perhaps, but that's a problem that could be solved, in part, by restricting > what the drivers allowed to access. Video drivers have no business using > the disk, so their access to disk access ports should be denied. Similar > schemes should be used when appropriate. It should not be possible for a > driver to access things willy nilly. My impression was that the goal was to use existing hardware that is widely available. ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Ada OS Kernel features 2001-09-06 19:41 ` Larry Kilgallen @ 2001-09-06 22:32 ` chris.danx 2001-09-07 11:04 ` Larry Kilgallen 0 siblings, 1 reply; 41+ messages in thread From: chris.danx @ 2001-09-06 22:32 UTC (permalink / raw) "Larry Kilgallen" <Kilgallen@SpamCop.net> wrote in message news:d8$8gnF$GFEk@eisner.encompasserve.org... > In article <UWOl7.11352$592.628767@news2-win.server.ntlworld.com>, "chris.danx" <chris.danx@ntlworld.com> writes: > > > >> > I really don't get why a driver must have access to system structures or > >> > atleast those in kernel space, can you explain this? > >> > >> A driver has access to raw hardware. Therefore it necessarily has > >> access to the system -- it could rewrite the copy of the OS on disk > >> if it chose. > > > > Perhaps, but that's a problem that could be solved, in part, by restricting > > what the drivers allowed to access. Video drivers have no business using > > the disk, so their access to disk access ports should be denied. Similar > > schemes should be used when appropriate. It should not be possible for a > > driver to access things willy nilly. > > My impression was that the goal was to use existing hardware that is > widely available. Yes. why? Do you think that won't work across all hardware? ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Ada OS Kernel features 2001-09-06 22:32 ` chris.danx @ 2001-09-07 11:04 ` Larry Kilgallen 0 siblings, 0 replies; 41+ messages in thread From: Larry Kilgallen @ 2001-09-07 11:04 UTC (permalink / raw) In article <pQSl7.12306$592.1131328@news2-win.server.ntlworld.com>, "chris.danx" <chris.danx@ntlworld.com> writes: > > "Larry Kilgallen" <Kilgallen@SpamCop.net> wrote in message > news:d8$8gnF$GFEk@eisner.encompasserve.org... >> In article <UWOl7.11352$592.628767@news2-win.server.ntlworld.com>, > "chris.danx" <chris.danx@ntlworld.com> writes: >> > >> >> > I really don't get why a driver must have access to system structures > or >> >> > atleast those in kernel space, can you explain this? >> >> >> >> A driver has access to raw hardware. Therefore it necessarily has >> >> access to the system -- it could rewrite the copy of the OS on disk >> >> if it chose. >> > >> > Perhaps, but that's a problem that could be solved, in part, by > restricting >> > what the drivers allowed to access. Video drivers have no business > using >> > the disk, so their access to disk access ports should be denied. > Similar >> > schemes should be used when appropriate. It should not be possible for > a >> > driver to access things willy nilly. >> >> My impression was that the goal was to use existing hardware that is >> widely available. > > Yes. why? Do you think that won't work across all hardware? Because access to IO pages must be protected by access mode, and hardware granularity does not allow protecting the disk device from the video driver. ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Ada OS Kernel features 2001-09-06 18:06 ` chris.danx 2001-09-06 19:41 ` Larry Kilgallen @ 2001-09-07 8:04 ` Dmitry Kazakov 2001-09-07 8:18 ` Mattias Svensson 1 sibling, 1 reply; 41+ messages in thread From: Dmitry Kazakov @ 2001-09-07 8:04 UTC (permalink / raw) On Thu, 6 Sep 2001 19:06:29 +0100, "chris.danx" <chris.danx@ntlworld.com> wrote: > >> > I really don't get why a driver must have access to system structures or >> > atleast those in kernel space, can you explain this? >> >> A driver has access to raw hardware. Therefore it necessarily has >> access to the system -- it could rewrite the copy of the OS on disk >> if it chose. > >Perhaps, but that's a problem that could be solved, in part, by restricting >what the drivers allowed to access. Video drivers have no business using >the disk, so their access to disk access ports should be denied. Similar >schemes should be used when appropriate. It should not be possible for a >driver to access things willy nilly. Any shared device should be not allowed to be directly used in the user mode. If a video driver operates in the user mode, then it is possible to have an application that reads and updates the parts of the screen of another application. Now consider that another application is a part of home-banking software. Wellcome in the MS-Windows world! Regards, Dmitry Kazakov ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Ada OS Kernel features 2001-09-07 8:04 ` Dmitry Kazakov @ 2001-09-07 8:18 ` Mattias Svensson 2001-09-07 12:40 ` Dmitry Kazakov 2001-09-12 7:12 ` Mats Karlssohn 0 siblings, 2 replies; 41+ messages in thread From: Mattias Svensson @ 2001-09-07 8:18 UTC (permalink / raw) In article <3b987e18.2182875@news.cis.dfn.de>, Dmitry Kazakov wrote: >On Thu, 6 Sep 2001 19:06:29 +0100, "chris.danx" ><chris.danx@ntlworld.com> wrote: >Any shared device should be not allowed to be directly used in the >user mode. If a video driver operates in the user mode, then it is >possible to have an application that reads and updates the parts of >the screen of another application. Now consider that another >application is a part of home-banking software. Wellcome in the >MS-Windows world! Why? If the video driver has exclusive access to the video memory and just presents an API to other programs by some menas of IPC. In what way is this different from having the video driver in the kernel? The hard part I think is to give access to the video memory from userspace. //Mattias -- He's fast, he's strong, he's TCP/IP and he knows your address -----------------------------------\ Warriors of the net /- email: d97svema@dtek.chalmers.se sms: d97svema+sms@dtek.chalmers.se ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Ada OS Kernel features 2001-09-07 8:18 ` Mattias Svensson @ 2001-09-07 12:40 ` Dmitry Kazakov 2001-09-12 7:12 ` Mats Karlssohn 1 sibling, 0 replies; 41+ messages in thread From: Dmitry Kazakov @ 2001-09-07 12:40 UTC (permalink / raw) On 7 Sep 2001 08:18:21 GMT, d97svema@dtek.chalmers.se (Mattias Svensson) wrote: >In article <3b987e18.2182875@news.cis.dfn.de>, Dmitry Kazakov wrote: >>On Thu, 6 Sep 2001 19:06:29 +0100, "chris.danx" >><chris.danx@ntlworld.com> wrote: >>Any shared device should be not allowed to be directly used in the >>user mode. If a video driver operates in the user mode, then it is >>possible to have an application that reads and updates the parts of >>the screen of another application. Now consider that another >>application is a part of home-banking software. Wellcome in the >>MS-Windows world! > >Why? If the video driver has exclusive access to the video memory and >just presents an API to other programs by some menas of IPC. If the driver runs on the context of the caller's process, then there is no way to share the device safely. >In what >way is this different from having the video driver in the kernel? Not in the kernel, but in a separate process which has other privilegies than the caller. There IMO should be no monolitic kernel, but a set of objects having no more access than they should. >The hard part I think is to give access to the video memory from >userspace. Regards, Dmitry Kazakov ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Ada OS Kernel features 2001-09-07 8:18 ` Mattias Svensson 2001-09-07 12:40 ` Dmitry Kazakov @ 2001-09-12 7:12 ` Mats Karlssohn 1 sibling, 0 replies; 41+ messages in thread From: Mats Karlssohn @ 2001-09-12 7:12 UTC (permalink / raw) Mattias Svensson wrote: %< > Why? If the video driver has exclusive access to the video memory and > just presents an API to other programs by some menas of IPC. In what > way is this different from having the video driver in the kernel? > The hard part I think is to give access to the video memory from > userspace. Not really, at least on IA-32. The I/O address space can be delegated to user processes on a per-address basis. Memory can be delegated on a page (fixed size) or segment (can be of variable size) basis. -- Mats Karlssohn, developer mailto:mats@mida.se Mida Systemutveckling AB http://www.mida.se Box 64, S-732 22 ARBOGA, SWEDEN Phone: +46-(0)589-89808 Fax: +46-(0)589-89809 ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Ada OS Kernel features 2001-09-05 19:25 ` chris.danx 2001-09-05 20:07 ` Darren New 2001-09-05 20:14 ` Larry Kilgallen @ 2001-09-05 20:18 ` Brian Catlin 2001-09-06 6:48 ` Ole-Hjalmar Kristensen 2 siblings, 1 reply; 41+ messages in thread From: Brian Catlin @ 2001-09-05 20:18 UTC (permalink / raw) "chris.danx" <chris.danx@ntlworld.com> wrote in message news:3%ul7.3362$9z1.440040@news6-win.server.ntlworld.com... > > > > You should be able to "overload" a driver. What I mean ? Lets assume > > > you have a simple grafic driver on bootup, then you load a "better" > > > (more complex, higher resolution, 3D excelerator ...) one. If this one > > > crashes, then it should simply be unloaded and the system should > > > continue work with the (simple) default driver - instead of showing a > > > "blue screen" ;-) > > > > My first reaction to this was "Not Possible". However, that isn't > > entirely true; it is just *VERY VERY* difficult. > > Only in the "drivers in supervisor mode" model. > > > A driver runs in kernel mode, > > Why? Why not just have it in user mode? It makes more sense to have them > in user mode, at least to me. They can only corrupt themselves then, etc. > > > and has access to system data structures. > > Why should it? In your model a driver can screw a system up good and > proper, but if you put the driver in user mode then the associated problems > go away. New ones do crop up, but there's ways and means to deal with them. This has been well studied and the reasons will show up in just about any search of the relevant literature (in case my explanation does not make sense, or are not complete enough for you). A driver typically runs in two contexts, the context of the requesting process, because it needs to access the user's buffers, and "system" context (strictly, arbitrary process context) where the driver does not need access to the requesting process' address space. If a driver is running in its own process, how can it gain efficient access to the requesting process' buffers? Also, drivers spend most of their time running at elevated IPL (interrupt priority level), which can only be done in kernel mode. It is possible to build a general purpose operating system as you suggest, but the performance would suck. You would lose a lot of time doing translation buffer invalidates and switching between modes. > > If a driver corrupts a system data structure, how do you detect this, > repair it, and continue? > > I really don't get why a driver must have access to system structures or > atleast those in kernel space, can you explain this? A driver needs to access privileged APIs and data structures in the normal course of its work; for example, mapping DMA transfers, sending I/O requests to other drivers, etc. These APIs and data structures are specifically put in kernel mode to prevent user's from accessing them. -Brian ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Ada OS Kernel features 2001-09-05 20:18 ` Brian Catlin @ 2001-09-06 6:48 ` Ole-Hjalmar Kristensen 2001-09-06 6:59 ` Brian Catlin 0 siblings, 1 reply; 41+ messages in thread From: Ole-Hjalmar Kristensen @ 2001-09-06 6:48 UTC (permalink / raw) "Brian Catlin" <briancatlin@mindspring.com> writes: > "chris.danx" <chris.danx@ntlworld.com> wrote in message > news:3%ul7.3362$9z1.440040@news6-win.server.ntlworld.com... > > > > > > You should be able to "overload" a driver. What I mean ? Lets assume > > > > you have a simple grafic driver on bootup, then you load a "better" > > > > (more complex, higher resolution, 3D excelerator ...) one. If this one > > > > crashes, then it should simply be unloaded and the system should > > > > continue work with the (simple) default driver - instead of showing a > > > > "blue screen" ;-) > > > > > > My first reaction to this was "Not Possible". However, that isn't > > > entirely true; it is just *VERY VERY* difficult. > > > > Only in the "drivers in supervisor mode" model. > > > > > A driver runs in kernel mode, > > > > Why? Why not just have it in user mode? It makes more sense to have them > > in user mode, at least to me. They can only corrupt themselves then, etc. > > > > > and has access to system data structures. > > > > Why should it? In your model a driver can screw a system up good and > > proper, but if you put the driver in user mode then the associated problems > > go away. New ones do crop up, but there's ways and means to deal with them. > > This has been well studied and the reasons will show up in just about any > search of the relevant literature (in case my explanation does not make sense, > or are not complete enough for you). A driver typically runs in two contexts, > the context of the requesting process, because it needs to access the user's > buffers, and "system" context (strictly, arbitrary process context) where the > driver does not need access to the requesting process' address space. If a > driver is running in its own process, how can it gain efficient access to the > requesting process' buffers? Also, drivers spend most of their time running at It is possible (and has been done) to associate privileges with the memory segment you are currently executing in, not which process you are running in. In this way it is possible to get both speed and safety, by having special libraries with sufficient privileges. Kind of like the monitor/protected object idea, which lets you execute code within it without process switching. <stuff deleted> -- Kabelsalat ist gesund. Ole-Hj. Kristensen ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Ada OS Kernel features 2001-09-06 6:48 ` Ole-Hjalmar Kristensen @ 2001-09-06 6:59 ` Brian Catlin 2001-09-10 7:32 ` Ole-Hjalmar Kristensen 0 siblings, 1 reply; 41+ messages in thread From: Brian Catlin @ 2001-09-06 6:59 UTC (permalink / raw) [...snip...snip...] > It is possible (and has been done) to associate privileges with the > memory segment you are currently executing in, not which process you > are running in. In this way it is possible to get both speed and > safety, by having special libraries with sufficient privileges. Kind > of like the monitor/protected object idea, which lets you execute > code within it without process switching. Interesting. Do you have any references to any implementations? This seems that it would work for a small system, such as an RTOS, but I suspect not for a large general-purpose operating system. -Brian ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Ada OS Kernel features 2001-09-06 6:59 ` Brian Catlin @ 2001-09-10 7:32 ` Ole-Hjalmar Kristensen 0 siblings, 0 replies; 41+ messages in thread From: Ole-Hjalmar Kristensen @ 2001-09-10 7:32 UTC (permalink / raw) "Brian Catlin" <briancatlin@mindspring.com> writes: > [...snip...snip...] > > > It is possible (and has been done) to associate privileges with the > > memory segment you are currently executing in, not which process you > > are running in. In this way it is possible to get both speed and > > safety, by having special libraries with sufficient privileges. Kind > > of like the monitor/protected object idea, which lets you execute > > code within it without process switching. > > Interesting. Do you have any references to any implementations? This seems > that it would work for a small system, such as an RTOS, but I suspect not for a > large general-purpose operating system. > > -Brian > > > Seems my posting did not make it, so I'll try again. "Transaction Processing: Concepts and Techniques" by Gray and Reuter has a brief discussion on pages 64-65. AS400 is one of the systems mentioned there, but the System 38 had the same mechanism. I believe the concept is discussed other places in the book too. You probably need some special hardware to do this efficiently, although the segmentation check hardware in the Intel x86 architecture could probably be pressed into service. A pity the 432 never took off. -- Kabelsalat ist gesund. Ole-Hj. Kristensen ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Ada OS Kernel features 2001-09-05 17:44 ` Brian Catlin ` (2 preceding siblings ...) 2001-09-05 19:25 ` chris.danx @ 2001-09-06 13:42 ` Alfred Hilscher 2001-09-07 8:13 ` Dmitry Kazakov 2001-09-08 3:55 ` Kenneth Almquist 4 siblings, 1 reply; 41+ messages in thread From: Alfred Hilscher @ 2001-09-06 13:42 UTC (permalink / raw) Brian Catlin wrote: > > > You should be able to "overload" a driver. What I mean ? Lets assume > > you have a simple grafic driver on bootup, then you load a "better" > > (more complex, higher resolution, 3D excelerator ...) one. If this one > > crashes, then it should simply be unloaded and the system should > > continue work with the (simple) default driver - instead of showing a > > "blue screen" ;-) > > My first reaction to this was "Not Possible". However, that isn't entirely > true; it is just *VERY VERY* difficult. A driver runs in kernel mode, and has > access to system data structures. If a driver corrupts a system data structure, > how do you detect this, repair it, and continue? In such instances, it is much > better to bugcheck (blue screen) the system than try to continue. Consider, if > the system is slightly corrupted and continues to operate, there is the very > real possibility that your data will be corrupted without your knowledge. This > was Win98's philosophy, and it was a disaster. VMS and NT (and others) stop the > system dead in its tracks to prevent hidden corruption. Ok, I agree that there may be drivers where it would be hard. But - lets assume a driver for graphic card (the ones I've got most problems with). After the driver crashes, you reinitialize the card and continue work (e.g. with low resolution). How to do the switch over ? As far as I know drivers are accessed via a dispatch table (OS/2, Win). So if an other driver is loaded, "stack" the previous dispatch table. When it crashes then reload the code of the previous driver and "unstack" the dispatch table. So maybe some information gets lost (e.g. the screen gets blank), but it should be possible to either repeat the last action (if the driver calls are "logged"), or to wait for "autorepair" (e.g. a repaint message to all windows when a driver cahnge occures). For a NIC reinitialize its interrupts and DMA channels and continue with the old driver. Maybe a few packets get lost, but that can always happen in a network, so it is no new situation. For an SCSI device driver reinitialize the hardware, repeat the last action or accept loss of data. For harddisc the filesystem should be able to repair it, for scanner, the user should repeat the scan (if the PC reboots he must do it, too), and so on. I think failure on one transaction is less worse then a complete system crash. I don't think that every driver _must_ have write access to system internal data. An graphic driver for example does not need to write in the process-table (please correct me if I'm wrong). If there are drivers that need to do so, they should not access these data structures directly, but via access procedures (which could do checks). And even if a driver corrupts system internal data, then hopefully they belong only to one process and not to the inner kernel. In this case I think it would be more acceptable to kill one process instead of killing the whole system. So if there are a few drivers where this can not be done, then do it for the rest. 50% failure tolerance is even better than 0%. ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Ada OS Kernel features 2001-09-06 13:42 ` Alfred Hilscher @ 2001-09-07 8:13 ` Dmitry Kazakov 0 siblings, 0 replies; 41+ messages in thread From: Dmitry Kazakov @ 2001-09-07 8:13 UTC (permalink / raw) On Thu, 06 Sep 2001 15:42:13 +0200, Alfred Hilscher <Alfred.Hilscher@icn.siemens.de> wrote: >I don't think that every driver _must_ have write access to system >internal data. An graphic driver for example does not need to write in >the process-table (please correct me if I'm wrong). If there are drivers >that need to do so, they should not access these data structures >directly, but via access procedures (which could do checks). And even if >a driver corrupts system internal data, then hopefully they belong only >to one process and not to the inner kernel. In this case I think it >would be more acceptable to kill one process instead of killing the >whole system. > >So if there are a few drivers where this can not be done, then do it for >the rest. 50% failure tolerance is even better than 0%. I believe it is a goal of new OO OS design to get rid of all-or-nothing strategy. A driver should have access to only the system objects it should. No more and no less. An unhandled exception in the driver should be caught an handled by the driver's "owner". If this handling means "to kill a process", fine. Regards, Dmitry Kazakov ^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: Ada OS Kernel features 2001-09-05 17:44 ` Brian Catlin ` (3 preceding siblings ...) 2001-09-06 13:42 ` Alfred Hilscher @ 2001-09-08 3:55 ` Kenneth Almquist 4 siblings, 0 replies; 41+ messages in thread From: Kenneth Almquist @ 2001-09-08 3:55 UTC (permalink / raw) > My first reaction to this was "Not Possible". However, that isn't entirely > true; it is just *VERY VERY* difficult. A driver runs in kernel mode, and > has access to system data structures. If a driver corrupts a system data > structure, how do you detect this, repair it, and continue? In such > instances, it is much better to bugcheck (blue screen) the system than > try to continue. Consider, if the system is slightly corrupted and > continues to operate, there is the very real possibility that your data > will be corrupted without your knowledge. This was Win98's philosophy, > and it was a disaster. VMS and NT (and others) stop the system dead in > its tracks to prevent hidden corruption. There are several related risks here. One is system data structures being overwritten. The Intel x86 architecture maps segment addresses to linear addresses and then uses the page table to map linear addresses to physical addresses, so it is possible to give device drivers their own address spaces without invalidating the page table cache every time a device driver runs. However, if the device drivers are written in Ada then there is little need for hardware memory protection. Another risk is resource leaks if a device driver allocates a resource (e.g. allocates memory) and then crashes. This can be dealt with by providing debugging wrappers for kernel routines which allocate resources, which keep track of which device driver holds the resource. Then, when a device driver crashes the resources held by that driver can be reclaimed. I assume that the open routine for a device driver will return a tagged object which is used to perform device operations. Tracking down all the references to these objects may not be practical. One approach is to write a wrapper around the device driver. When you call the open routine for the wrapper, it calls the driver's open routine, and then allocates a wrapper object which points to the object returned by the driver's open routine. If the driver crashes, the wrapper switches to the backup driver. This is done by iterating through all the wrapper objects, freeing the objects they point to, and making them point to objects obtained by calling the open routine for the backup driver. When I say the driver "crashes," that means that one task or interrupt handler executing driver code raised an unhandled exception. There could be other tasks executing driver code at the same time. As long as these tasks do not block, they can be allowed to continue, but if they block then it is necessary to throw an exception in the task. This requires an extention to the Ada run time. In GNAT, aborting a task throws a special exception that cannot be caught, so that the basic logic required to raise an exception in another task is there. The execption should be caught by the wrapper, which will then retry the operation using the backup version of the driver. These ideas add up to a bit of work, but they should allow a new version of a device driver to be tested on a running system with only a small risk of disrupting system activity if the new version doesn't work. Whether this is worth doing or not is an open question. I wrote the initial implementation of modules for Linux, and didn't do any of this stuff. (Traps in loaded modules cause the system to crash, just like traps from the core kernel code.) But when Ada code throws an exception, you can be reasonably confident that it hasn't corrupted data managed by some unrelated piece of code, so there is less risk in keeping the system running that there is when C code goes awry. I would say, though, that dynamic loading of code into a running kernel is the big win. If mistakes which are not caught by the Ada type system cause the system to crash, that is still a lot better than having to reboot every time you want to test a changed line of code. Kenneth Almquist ^ permalink raw reply [flat|nested] 41+ messages in thread
end of thread, other threads:[~2001-09-12 7:12 UTC | newest] Thread overview: 41+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2001-09-05 5:58 Ada OS Kernel features Brian Catlin 2001-09-05 10:15 ` Jacob Sparre Andersen 2001-09-05 11:16 ` Larry Kilgallen 2001-09-05 17:06 ` Brian Catlin 2001-09-06 14:35 ` Marin David Condic 2001-09-07 9:31 ` Tarjei T. Jensen 2001-09-05 17:55 ` David Starner 2001-09-05 18:42 ` Darren New 2001-09-12 6:47 ` Mats Karlssohn 2001-09-05 14:04 ` Ted Dennison 2001-09-05 17:23 ` Brian Catlin 2001-09-05 20:17 ` Ted Dennison 2001-09-05 21:10 ` Brian Catlin 2001-09-06 6:45 ` Tarjei T. Jensen 2001-09-06 6:56 ` Brian Catlin 2001-09-06 14:05 ` Ted Dennison 2001-09-05 16:02 ` Alfred Hilscher 2001-09-05 16:19 ` Jacob Sparre Andersen 2001-09-05 17:58 ` Brian Catlin 2001-09-05 17:44 ` Brian Catlin 2001-09-05 17:58 ` Darren New 2001-09-05 18:51 ` Larry Kilgallen 2001-09-05 19:25 ` chris.danx 2001-09-05 20:07 ` Darren New 2001-09-05 20:14 ` Larry Kilgallen 2001-09-06 13:45 ` Alfred Hilscher 2001-09-06 18:06 ` chris.danx 2001-09-06 19:41 ` Larry Kilgallen 2001-09-06 22:32 ` chris.danx 2001-09-07 11:04 ` Larry Kilgallen 2001-09-07 8:04 ` Dmitry Kazakov 2001-09-07 8:18 ` Mattias Svensson 2001-09-07 12:40 ` Dmitry Kazakov 2001-09-12 7:12 ` Mats Karlssohn 2001-09-05 20:18 ` Brian Catlin 2001-09-06 6:48 ` Ole-Hjalmar Kristensen 2001-09-06 6:59 ` Brian Catlin 2001-09-10 7:32 ` Ole-Hjalmar Kristensen 2001-09-06 13:42 ` Alfred Hilscher 2001-09-07 8:13 ` Dmitry Kazakov 2001-09-08 3:55 ` Kenneth Almquist
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox