Re: Ada.Containers and Storage_Pools

[msimonides@power.com.pl]

Matthew Heaney wrote:
> msimonides@power.com.pl wrote:
> > I'm working on a caching DNS resolver for our project. The structures
> > used to hold data in cache make use of Ada.Containers packages (e.g.
> > each node stores a map of list resource records for each data type, so
> > there is a lot of individual containers).
>
> OK, it sounds like you have a map container whose elements are list
> containers.  Is that right?

Yes, that is the basic idea.
There is a tree representing DNS zones. Each zone has a map of names to
child zone accesses (to be replaced by Hashed_Sets in the future) and a
map of resource types to resource record accesses. Each resource record
has a vector of data elements and some additional information.

> > The problem I'm facing (and that is certain to show up in other parts
> > of our project) is that I need to limit the size of cache in memory. To
> > do this I would need to be able to have the containers allocate their
> > nodes from a user-specified storage pool with some limiting mechanism
> > (I already have such a pool and use it for objects that are allocated
> > by my code).
>
> Is the pool bounded or unbounded?  In other words, can you run out of
> nodes?  If so, what should happen?

This storage pool is nothing fancy. It has a maximal size supplied at
creation of the pool object and at each allocation increases its
internal counter of used up Storage_Units and raises an error when the
limit is reached. Memory allocation itself is performed with "new", so
it is really a wrapper on the default allocator.

It is also possible to query the amount of memory left in the pool, it
is used to perform cleaning of the cache once a certain threshold is
reached. Two such pool objects are used for both access to zone and
access to resource record objects, which makes it possible to see which
structures use more memory.

Having such a pool for use by containers' internal structures would
enable us to choose more appropriate container types in terms of memory
usage (eg. we had a list of data elements in resource records but now
use vectors as they have less memory overhead on each element and are
as good performance-wise when the number of nodes is small. But there
is no (easy) way of measuring it).

> > So I have a question: is there a possibility, either in GNAT 3.4.5 or
> > in Ada 2005, to specify a custom storage pool for use by containers? If
> > not, is there any other solution to limiting the size of memory used by
> > containers?
>
[...]
>
> One possibility is to keep a list around and use it as a kind of storage
> pool for the list objects that are map elements.  You can use the Splice
> operation for lists to move list nodes between list objects (that are
> map elements) and the list object that serves as the pool.

Yes, but this is only a partial solution. Adapting it to thousands of
small vectors seems infeasible.

Ideally I could just specify the maximal size for all cache structures
together and have the storage pool enforce it.

> > The only solution that I have found is using some other containers
> > library (I've seen that Booch components allow for specifying custom
> > storage pools), but I'd really prefer to stick with the standard ones.
>
> I'd like to know more about what you're doing, and about the nature of
> your storage pool.  If you don't mind drop me some email so we can
> discuss your issues in more detail.

I hope I have included most of the important information in this post.
I will gladly provide more details if necessary.

Thank you for your help.
-- 
Marcin Simonides