Re: creating database

[Robert A Duff <bobduff@shell01.TheWorld.com>]

tmoran@acm.org writes:

> > > don't know how many records there'll be so it seems inefficient to
> > > declare an array of a particular size.
> >
> > Sensible thinking.
> > ...
> > Instead of allocating a fixed size array - which on unix systems isn't
> > quite as bad an idea as it sounds - you can allocate and reallocate
>   A fixed array allocation may be the both the most efficient and the
> simplest on a virtual memory system.  The parts you don't use don't
> occupy any physical space, just address space, and the OS's allocation
> of physical space may well be difficult to beat in efficiency.

But I believe that by default, on both Unix and Windows, the OS will
allocate backing store (i.e. space in the paging file).
That can be a problem for really huge arrays.

To avoid that on Unix, use mmap with the NORESERVE flag,
and map the file /dev/zero -- then you'll get allocate-on-write
pages of virtual address space.

On Windows, it's a bit more complicated: use VirtualAlloc to reserve
pages, and then when you actually want to use them, use VirtualAlloc
again to "commit" pages.

- Bob