Memory profiling

Memory profiling

[Alex R. Mosteo]

Hello,

I'm wondering if there exists some utility -maybe gdb based- that would 
allow to run a program and at a given time interrupt it and dump the 
heap usage classified by the pointer types. For example:

Heap allocation for process <blah>:

    Type_X_Access: 24.5 MB
    Type_Y_Access: 225.3 KB
    ...

I bet this could be very useful for memory usage profiling and 
optimization. I've just seen Massif from the Valgrind toolsuite and I'm 
going to read more about it. If someone can confirm this is in the right 
path...

I suppose something similar can be achieved using distinct storage pools 
for each access type being tracked, but I find this more inconvenient. 
Ummm, maybe a type holding a list of storage pools created on demand...

So, any suggestions in this direction?

Re: Memory profiling

[Robert A Duff]

"Alex R. Mosteo" <devnull@mailinator.com> writes:

> I suppose something similar can be achieved using distinct storage pools
> for each access type being tracked, but I find this more
> inconvenient. Ummm, maybe a type holding a list of storage pools created
> on demand...

All those storage pools can share the same underlying memory pool.  That
is, each pool just keeps track of whatever debugging/statistics info you
want, and then calls some underlying pool, or just does a "new", or
calls malloc, or whatever.  So gathering fine-grained (per type)
information does not need to imply that you have to actually allocate
type-segregated data.

- Bob

Re: Memory profiling

[Alex R. Mosteo]

Robert A Duff wrote:
> "Alex R. Mosteo" <devnull@mailinator.com> writes:
> 
> 
>>I suppose something similar can be achieved using distinct storage pools
>>for each access type being tracked, but I find this more
>>inconvenient. Ummm, maybe a type holding a list of storage pools created
>>on demand...
> 
> 
> All those storage pools can share the same underlying memory pool.  That
> is, each pool just keeps track of whatever debugging/statistics info you
> want, and then calls some underlying pool, or just does a "new", or
> calls malloc, or whatever.  So gathering fine-grained (per type)
> information does not need to imply that you have to actually allocate
> type-segregated data.

That's the idea I had in mind, but you still have to provide a new 
storage pool for each tracked access type. Nonetheless, I see 
interesting potential in this idea.

In other news, I've just tested the Massif module of Valgrind and while 
it doesn't do exactly what I wanted, it is a very valuable tool. What it 
does is to keep track of all allocations, so it gives and idea of the 
overall story instead of a particular moment in time. It works 
out-of-the-box with gnat executables compiled with debug info.

And I just have recalled about gnatmem, silly me.

Re: Memory profiling

[Alex R. Mosteo]

Alex R. Mosteo wrote:

> And I just have recalled about gnatmem, silly me.

Another thing I've recalled is that gnatmem may use humongous amounts of 
disk space when creating the dump file, and humongous amounts of memory 
when doing the post-mortem analysis. Valgrind seems a bit spartan so it 
may be a valid alternative.