Re: Memory Mapped Storage Pools.

Re: Memory Mapped Storage Pools.

[Dr. Adrian Wrigley]

On Wed, 26 Oct 2005 20:08:44 -0400, Freejack wrote:

> Ada Storage Pools, in Unix/Posix type environments, are typically
> handled via the omnipresent C malloc() routine. While this undoubtedly
> simplifies compiler/runtime development, it has left me desiring a
> mechanism with a bit more control.
> 
> So I've begun working on a library that allocates storage pools through
> the lower level mmap() suite of system utilities. I'm tentatively calling
> it Custom_Storage.Mapped_Pools
> 
> It's intended use is for very large allocations that are intended to be
> returned to the system upon finalization, rather than having the extra
> memory linger in the program's process "space".

I did this once, a few years ago.

The objective were:

1) Allow region-based allocation (zero space overhead!)
2) Allow conventional "malloc/free" style allocation within each pool
3) task compatible ("thread safe")
4) Able to reload pools from the mmapped files on subsequent runs
5) Pools able to expand as necessary

I used a completely free, open source allocator for the malloc/free
part of the code (in C), and did the high-level stuff from the Ada.

The mmap'ed blocks were mapped as the pool expanded, getting
larget maps each time.  The pool kept a fixed-size table
of all the regions allocated.

I don't think I have the code to hand, since it was not mine :(

The objectives above probably apply to your code too.  I think that
region-based allocation is an important and under-appreciated
approach to memory management.

I can't remember the exact details of the interface, but I don't
think you need bindings to mmap, since they are part of Florist
(I still use these in my current code).

Good luck!
-- 
Adrian
 

Re: Memory Mapped Storage Pools.

[Dan Baysinger]

I use mmap'ed storage for essentially the same reasons that you gave. 
However, I just derived from System.Storage_Pools.Root_Storage_Pool and 
  overrode all of the subprograms to allocate the requested objects 
using mmap and deallocate with unmap.

The Root_Storage _Pool type was extended with a private pointer to a 
linked list of allocations made for the client's storage pool.  I used 
an opaque type (incomplete declaration in the spec's private area and 
completion in the body) for the elements of the allocation linked list 
to discourage (not actually prevent) users from trying to manipulate the 
allocations directly.

You need to override finalization so that it walks the allocation list 
and unmaps all allocations for the user's storage pool.

Also, I mmapped to /dev/zero so that an actual file was not created.

This has worked quite well for my purposes, and its usage is completely 
compatible with the storage_pool specification for access types, the 
'new' allocator, and Ada.Unchecked_Deallocation.  From the user's 
perspective the behavior is identical to that of a storage pool created 
System.Storage_Pools.Root_Storage_Pool.

Dan Baysinger




Freejack wrote:

> Ada Storage Pools, in Unix/Posix type environments, are typically
> handled via the omnipresent C malloc() routine. While this undoubtedly
> simplifies compiler/runtime development, it has left me desiring a
> mechanism with a bit more control.
> 
> So I've begun working on a library that allocates storage pools through
> the lower level mmap() suite of system utilities. I'm tentatively calling
> it Custom_Storage.Mapped_Pools
> 
> It's intended use is for very large allocations that are intended to be
> returned to the system upon finalization, rather than having the extra
> memory linger in the program's process "space".
> 
> Such a package's interfaces need to be carefully designed so as to prevent
> it's client developers from going ape shit every time  they need a clear
> definition of exactly what the routine is going to do to the rest of their
> process space. Hence I would like some advice. How would you like to see
> such a package laid out? Should I make a set of thick or thin bindings to
> the mmap() facilities to go along with the Mapped_Pools package? What
> things should I keep in mind as to making the package portable to other
> environments with an mmap() facility?
> 
> Any tips would be appreciated.
> 
> Freejack