Re: A Customer's Request For Open Source Software

["Warren W. Gay VE3WWG" <ve3wwg@cogeco.ca>]

Robert I. Eachus wrote:
> Larry Kilgallen wrote:
> 
>> In article <3F53B88E.7040405@attbi.com>, "Robert I. Eachus" 
>> <rieachus@attbi.com> writes:
>>> As I said, most RDBMSs already have the ability to bypass the OS and 
>>> access disk drives directly.
> 
>> They do ?  On what operating systems ?
> 
> I guess we have a different idea of what bypassing the OS means. I was 
> thinking of for example, Oracle on Solaris.  You can go through the file 
> system, bypass the file system and have Oracle manage a partition (raw 
> mode), or give Oracle full control of the DISK device or devices.  But 
> even then you are right, the DBMS uses the standard device drivers.  The 
> difference is that if Oracle controls the entire disk, it can take 
> control of the head positioning and other driver level functions.

INFORMIX and Sybase can work with the raw devices as well. This
was in fact, a problem in early Linux kernels, as they did not
support raw devices (only "block" devices where the disk blocks
are cached -- database engines prefer to do their own caching
techniques). Linus didn't understand the need for these raw
(unbuffered) devices in the beginning. I think Oracle was
probably the source of enlightening on that front. ;-)

So on the early Linux kernels, only file system based versions of
INFORMIX and Sybase databases were possible. Now I would expect
that has all changed.

You might wonder why buffered "block" devices are not good
enough for the purpose. I can't answer to the specifics, but
only that database engines are in a position to better manage
the cache based upon what they "know" needs to be done. Another
important reason to control caching details is that when a
transaction is committed, you need to guarantee that the
data is written to the disk media (or can be recovered if
the database is to be restarted at that point).

The database
must be recoverable at any given point anyhow, and this
usually requires fine grained control over physical writes
to the media. This aspect and performance means that the
engine must balance performance with reliability (persistance),
which are conflicting goals when using disk.

It is this last area where oodles of persistent fast memory
(instead of disk), can make a world of difference. In this
case persistence = memory performance, which of course is
where the win is. If disks became obsolete (one can hope),
then I could see that new database engine design (internals)
will become much different than what it is today.  Certainly,
many of the present compromises would be eliminated.

-- 
Warren W. Gay VE3WWG
http://home.cogeco.ca/~ve3wwg