newbie: can't read fast enough... :-) memory leaks...

newbie: can't read fast enough... :-) memory leaks...

[gdotone]

i know i'll get there, currently in chapter 3, of Ada 95 PSPD, anyway, can Ada code have/produce memory leaks?  well, maybe not "can" it but, does Ada, natively prevent memory leaks?

like, C seems to leak all over the place, if not managed correctly by the programmer.

Re: newbie: can't read fast enough... :-) memory leaks...

[Pascal Obry]

Le mercredi 03 septembre 2014 à 02:38 -0700, gdotone@gmail.com a
écrit : 
> i know i'll get there, currently in chapter 3, of Ada 95 PSPD, anyway, can Ada code have/produce memory leaks?  well, maybe not "can" it but, does Ada, natively prevent memory leaks?
> 
> like, C seems to leak all over the place, if not managed correctly by the programmer.

This depends on the implementation. Ada make it possible to use a
garbage collector but as far as I know no implementation is using one.
So yes, every Ada compiler around will leak memory if not properly
freed.

To free an allocated object one need to instantiate
Ada.Unchecked_Deallocation.

Note that well designed object can deallocate themselves when possible
by using Ada.Finalization API. As an example, Unbounded_String are using
dynamic allocation but you need need to worry about that as a user.

-- 
  Pascal Obry /  Magny Les Hameaux (78)

  The best way to travel is by means of imagination

  http://v2p.fr.eu.org
  http://www.obry.net

  gpg --keyserver keys.gnupg.net --recv-key F949BD3B

Re: newbie: can't read fast enough... :-) memory leaks...

[G.B.]

On 03.09.14 12:17, Pascal Obry wrote:
> This depends on the implementation. Ada make it possible to use a
> garbage collector but as far as I know no implementation is using one.

The implementations targeting the JVM (JGNAT/AppleMagic) or .NET (A#)
will have automatic memory management, I think.

Re: newbie: can't read fast enough... :-) memory leaks...

[Pascal Obry]

Le mercredi 03 septembre 2014 à 12:48 +0200, G.B. a écrit : 
> On 03.09.14 12:17, Pascal Obry wrote:
> > This depends on the implementation. Ada make it possible to use a
> > garbage collector but as far as I know no implementation is using one.
> 
> The implementations targeting the JVM (JGNAT/AppleMagic) or .NET (A#)
> will have automatic memory management, I think.

Sure, provided by the VM.

-- 
  Pascal Obry /  Magny Les Hameaux (78)

  The best way to travel is by means of imagination

  http://v2p.fr.eu.org
  http://www.obry.net

  gpg --keyserver keys.gnupg.net --recv-key F949BD3B

Re: newbie: can't read fast enough... :-) memory leaks...

[Robert A Duff]

"G.B." <bauhaus@futureapps.invalid> writes:

> On 03.09.14 12:17, Pascal Obry wrote:
>> This depends on the implementation. Ada make it possible to use a
>> garbage collector but as far as I know no implementation is using one.
>
> The implementations targeting the JVM (JGNAT/AppleMagic) or .NET (A#)
> will have automatic memory management, I think.

I have successfully used the BDW conservative garbage collector with
Ada.

- Bob

Re: newbie: can't read fast enough... :-) memory leaks...

[Pascal Obry]

Le mercredi 03 septembre 2014 à 13:28 -0400, Robert A Duff a écrit :
> I have successfully used the BDW conservative garbage collector with
> Ada.

Is that with GNAT?

I've never looked at garbage collector. Is it possible with Ada to
create a pool using BDW garbage collector and make it the default pool?
Would that work as expected?

-- 
  Pascal Obry /  Magny Les Hameaux (78)

  The best way to travel is by means of imagination

  http://v2p.fr.eu.org
  http://www.obry.net

  gpg --keyserver keys.gnupg.net --recv-key F949BD3B

Re: newbie: can't read fast enough... :-) memory leaks...

[gdotone]

so,
question: in systems where ada is being used that are critical, airplanes, medical devices, etc., a garbage collector should not be used?

is this why a garbage collector is part of the language? well part of the standard? or because of syntax and semantics of the language leaks are rare? 

Re: newbie: can't read fast enough... :-) memory leaks...

[gdotone]

rewrite:

 so,
 
 question: in systems where ada is being used that are critical, airplanes, medical devices, etc., a garbage collector should not be used?
 is this why a garbage collector is not part of the language? well part of the standard? or because of syntax and semantics of the language leaks are rare?

Re: newbie: can't read fast enough... :-) memory leaks...

[Dmitry A. Kazakov]

On Wed, 03 Sep 2014 13:28:22 -0400, Robert A Duff wrote:

> I have successfully used the BDW conservative garbage collector with
> Ada.

Could you sketch out how you did the mark phase?

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: newbie: can't read fast enough... :-) memory leaks...

[Dmitry A. Kazakov]

On Wed, 3 Sep 2014 12:19:00 -0700 (PDT), gdotone@gmail.com wrote:

> is this why a garbage collector is not part of the language? well part of
> the standard? or because of syntax and semantics of the language leaks are
> rare?

I would say because the cure is worse than the disease. But this is my
opinion...

In particular, finalization under a collector is performed with no regard
to interdependencies of the collected objects. The property of being
"unreachable" is not enough to determine the correct order of finalization.

If some order is established using references, then the gates of hell
opens, hard references, weak references, cyclic references etc.

And should you have references already, a reference counting schema would
be as good as automated collection.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: newbie: can't read fast enough... :-) memory leaks...

[Robert A Duff]

Pascal Obry <pascal@obry.net> writes:

> Le mercredi 03 septembre 2014 à 13:28 -0400, Robert A Duff a écrit :
>> I have successfully used the BDW conservative garbage collector with
>> Ada.
>
> Is that with GNAT?

Yes, it was.  This was years ago, so I don't remember details.
I think it was on Linux, but it might have been windows.

> I've never looked at garbage collector. Is it possible with Ada to
> create a pool using BDW garbage collector and make it the default pool?

I didn't use storage pools.  I think I hacked the GNAT runtimes a little
bit.  E.g. change System.Memory to use GC_malloc, which is Boehm's
version of malloc.  And I had to change how tasks are created to
call Boehm's version of thread-create.

> Would that work as expected?

I don't think so, at least not in a simple way.  GC can't work within a
single pool -- it would need to trace pointers that point into the pool
from outside it (from other pools, from the stack, registers, ...).

- Bob

Re: newbie: can't read fast enough... :-) memory leaks...

[Robert A Duff]

"Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> writes:

> On Wed, 03 Sep 2014 13:28:22 -0400, Robert A Duff wrote:
>
>> I have successfully used the BDW conservative garbage collector with
>> Ada.
>
> Could you sketch out how you did the mark phase?

BDW = Boehm-Demers-Weiser.  I didn't write it, so I didn't "do the mark
phase".  There's lots of info available via "google boehm gc", if you
want to know how it works.  Boehm's documentation is very well written.

As I recall, it first finds all the thread stacks, which is an
OS-dependent operation.  Those are the "roots".  It looks at each word
in the stacks, using supposedly-conservative heuristics to guess whether
that word is a pointer.  If so, it traces that, and follows all the
pointers in the heap object, transitively.

It uses "black listing" to avoid too many wrong guesses.  For example,
if you have a string containing "Hello, w" that's not a pointer.  It
treats that as a 64-bit number (e.g. on x86-64), and checks whether that
memory location is currently allocated.  If so, it assumes it's a
pointer (which is a wrong guess).  But if not, it guesses correctly that
this data is not a pointer, and also puts that memory location on a
black list, and never allocates anything there -- that way, it will
not guess wrong in the future.

There are all sorts of other tricks to avoid guessing wrong.

- Bob

Re: newbie: can't read fast enough... :-) memory leaks...

[Brad Moore]

On 2014-09-03 4:17 AM, Pascal Obry wrote:
> Le mercredi 03 septembre 2014 à 02:38 -0700, gdotone@gmail.com a
> écrit :
>> i know i'll get there, currently in chapter 3, of Ada 95 PSPD, anyway, can Ada code have/produce memory leaks?  well, maybe not "can" it but, does Ada, natively prevent memory leaks?
>>
>> like, C seems to leak all over the place, if not managed correctly by the programmer.
>
> This depends on the implementation. Ada make it possible to use a
> garbage collector but as far as I know no implementation is using one.
> So yes, every Ada compiler around will leak memory if not properly
> freed.
>
> To free an allocated object one need to instantiate
> Ada.Unchecked_Deallocation.
>
> Note that well designed object can deallocate themselves when possible
> by using Ada.Finalization API. As an example, Unbounded_String are using
> dynamic allocation but you need need to worry about that as a user.
>

A couple of points on heap allocation that I think newcomers to Ada 
often miss, or fail to appreciate.

Compared to languages like C and C++, I would say Ada reduces the need 
for heap allocation.

For example, In C, for a function that can return a string of variable 
length, a common approach is to allocate memory for the result from the 
heap, and then return the address of the allocated result to the caller, 
since in C, functions essentially return numeric values.
A numeric value might be a char, int, float, or an address of some type.

eg.

char * get_string()
{
    const char *s1 = "The return result";
    char * ptr = malloc(strlen(S1) + 1);
    strcpy(ptr, s1);
    return ptr;
}

or alternatively, pass the address of the buffer as a parameter into the 
function as in...

void get_string(char *buf, int buflen)
{
    const char *s1 = "The return result";
    if (strlen(s1) + 1 > buflen) {
       FATAL_ERR("Buffer too small");
    }
    strcpy(buf, s1);
}

But this relies on the caller knowing the maximum length of the result 
before making the call.

In Ada, unconstrained types such as string types and arrays can be 
easily returned on the stack as a result without allocating from the heap.

eg.

function Get_String return String is
begin
   return "The return result";
end Get_String;

The second point is that another alternative to calling 
Unchecked_Deallocation is to let the finalization of an access type free 
all heap allocations for that type, using a nested scope.

eg.
      declare
         type String_Access is access String;
         A, B : String_Access;
      begin
         A := new String'("String 1");
         B := new String'("String 2");
         ... -- Operate on the data.
      end;
      -- A and B are automatically freed here because the String_Access
      -- is finalized since the scope of the declaration of the access
      -- type is being exited, and finalizing an access type also
      -- finalizes its collection (allocated objects designated by that
      -- type).

This is a nice way to free up a bunch of objects that are no longer 
needed, if it can be guaranteed that the objects are only needed in a 
nested scope. I like this better than garbage collection, because the
deallocation occurs more consistently, when and where it is needed.
I view this as being less error prone, if one is able to take advantage 
of this approach.

There are other techniques and libraries that are helpful in this area,
but I think these two points are a good starting point.

Brad

Re: newbie: can't read fast enough... :-) memory leaks...

[Jeffrey Carter]

On 09/03/2014 09:57 PM, Brad Moore wrote:
> 
>      declare
>         type String_Access is access String;
>         A, B : String_Access;
>      begin
>         A := new String'("String 1");
>         B := new String'("String 2");
>         ... -- Operate on the data.
>      end;
>      -- A and B are automatically freed here because the String_Access
>      -- is finalized since the scope of the declaration of the access
>      -- type is being exited, and finalizing an access type also
>      -- finalizes its collection (allocated objects designated by that
>      -- type).
> 
> This is a nice way to free up a bunch of objects that are no longer needed, if
> it can be guaranteed that the objects are only needed in a nested scope. I like
> this better than garbage collection, because the
> deallocation occurs more consistently, when and where it is needed.
> I view this as being less error prone, if one is able to take advantage of this
> approach.

Recent comments here in another thread indicate that this only works if
'Storage_Size is defined for the access type.

-- 
Jeff Carter
"Hello! Smelly English K...niggets."
Monty Python & the Holy Grail
08

Re: newbie: can't read fast enough... :-) memory leaks...

[Dmitry A. Kazakov]

On Wed, 03 Sep 2014 16:31:12 -0400, Robert A Duff wrote:

> As I recall, it first finds all the thread stacks, which is an
> OS-dependent operation.  Those are the "roots".  It looks at each word
> in the stacks, using supposedly-conservative heuristics to guess whether
> that word is a pointer.  If so, it traces that, and follows all the
> pointers in the heap object, transitively.

Thanks. That answered my question.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: newbie: can't read fast enough... :-) memory leaks...

[Dmitry A. Kazakov]

On Wed, 03 Sep 2014 22:32:51 -0700, Jeffrey Carter wrote:

> On 09/03/2014 09:57 PM, Brad Moore wrote:
>> 
>>      declare
>>         type String_Access is access String;
>>         A, B : String_Access;
>>      begin
>>         A := new String'("String 1");
>>         B := new String'("String 2");
>>         ... -- Operate on the data.
>>      end;
>>      -- A and B are automatically freed here because the String_Access
>>      -- is finalized since the scope of the declaration of the access
>>      -- type is being exited, and finalizing an access type also
>>      -- finalizes its collection (allocated objects designated by that
>>      -- type).
>> 
>> This is a nice way to free up a bunch of objects that are no longer needed, if
>> it can be guaranteed that the objects are only needed in a nested scope. I like
>> this better than garbage collection, because the
>> deallocation occurs more consistently, when and where it is needed.
>> I view this as being less error prone, if one is able to take advantage of this
>> approach.
> 
> Recent comments here in another thread indicate that this only works if
> 'Storage_Size is defined for the access type.

An arena pool would do the job. The Storage_Pool attribute would statically
ensure that the access type won't outlive the pool.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: newbie: can't read fast enough... :-) memory leaks...

[Brad Moore]

On 2014-09-03 11:32 PM, Jeffrey Carter wrote:
> On 09/03/2014 09:57 PM, Brad Moore wrote:
>>
>>       declare
>>          type String_Access is access String;
>>          A, B : String_Access;
>>       begin
>>          A := new String'("String 1");
>>          B := new String'("String 2");
>>          ... -- Operate on the data.
>>       end;
>>       -- A and B are automatically freed here because the String_Access
>>       -- is finalized since the scope of the declaration of the access
>>       -- type is being exited, and finalizing an access type also
>>       -- finalizes its collection (allocated objects designated by that
>>       -- type).
>>
>> This is a nice way to free up a bunch of objects that are no longer needed, if
>> it can be guaranteed that the objects are only needed in a nested scope. I like
>> this better than garbage collection, because the
>> deallocation occurs more consistently, when and where it is needed.
>> I view this as being less error prone, if one is able to take advantage of this
>> approach.
>
> Recent comments here in another thread indicate that this only works if
> 'Storage_Size is defined for the access type.
>


That doesn't appear to be correct.

In the latest GNAP GPL, the following program outputs;

Finalizing
Finalizing
Exited Block


with Ada.Text_IO; use Ada.Text_IO;
with Ada.Finalization; use Ada;
procedure Test_Deallocation is
begin

    declare
       type T is new Finalization.Limited_Controlled with null record;
       overriding procedure Finalize   (Object : in out T);
       overriding procedure Finalize (Object : in out T) is
       begin
          Put_Line ("Finalizing ");
       end Finalize;

       type T_Access is access T;
       T1, T2 : T_Access;
    begin
       T1 := new T;
       T2 := new T;
    end;

    Put_Line ("Exited Block");

end Test_Deallocation;

Re: newbie: can't read fast enough... :-) memory leaks...

[Brad Moore]

On 2014-09-04 8:23 AM, Brad Moore wrote:
> On 2014-09-03 11:32 PM, Jeffrey Carter wrote:
>> On 09/03/2014 09:57 PM, Brad Moore wrote:
>>>
>>>       declare
>>>          type String_Access is access String;
>>>          A, B : String_Access;
>>>       begin
>>>          A := new String'("String 1");
>>>          B := new String'("String 2");
>>>          ... -- Operate on the data.
>>>       end;
>>>       -- A and B are automatically freed here because the String_Access
>>>       -- is finalized since the scope of the declaration of the access
>>>       -- type is being exited, and finalizing an access type also
>>>       -- finalizes its collection (allocated objects designated by that
>>>       -- type).
>>>
>>> This is a nice way to free up a bunch of objects that are no longer
>>> needed, if
>>> it can be guaranteed that the objects are only needed in a nested
>>> scope. I like
>>> this better than garbage collection, because the
>>> deallocation occurs more consistently, when and where it is needed.
>>> I view this as being less error prone, if one is able to take
>>> advantage of this
>>> approach.
>>
>> Recent comments here in another thread indicate that this only works if
>> 'Storage_Size is defined for the access type.
>>
>
>
> That doesn't appear to be correct.

Looking deeper, yes, the objects of access type are finalized when the 
access type goes out of scope, but it is implementation defined whether 
the storage for the objects is reclaimed.

I wonder how many implementations are out there that do not have a 
default storage pool that reclaims the storage of a collection of an 
nested access type when the master associated with of the access type 
declaration is exited.

I'm pretty sure that GNAT does for Windows and Linux, but perhaps they 
have a minimalist runtime for certain targets where this is not needed 
or desired?

Re: newbie: can't read fast enough... :-) memory leaks...

[G.B.]

On 04.09.14 16:23, Brad Moore wrote:
> On 2014-09-03 11:32 PM, Jeffrey Carter wrote:
>> On 09/03/2014 09:57 PM, Brad Moore wrote:
>>>
>>>       declare
>>>          type String_Access is access String;
>>>          A, B : String_Access;
>>>       begin
>>>          A := new String'("String 1");
>>>          B := new String'("String 2");
>>>          ... -- Operate on the data.
>>>       end;
>>>       -- A and B are automatically freed here because the String_Access
>>>       -- is finalized since the scope of the declaration of the access
>>>       -- type is being exited, and finalizing an access type also
>>>       -- finalizes its collection (allocated objects designated by that
>>>       -- type).
>>>
>>> This is a nice way to free up a bunch of objects that are no longer
>>> needed, if
>>> it can be guaranteed that the objects are only needed in a nested
>>> scope. I like
>>> this better than garbage collection, because the
>>> deallocation occurs more consistently, when and where it is needed.
>>> I view this as being less error prone, if one is able to take
>>> advantage of this
>>> approach.
>>
>> Recent comments here in another thread indicate that this only works if
>> 'Storage_Size is defined for the access type.
>>
>
>
> That doesn't appear to be correct.
>
> In the latest GNAP GPL, the following program outputs;
>
> Finalizing
> Finalizing
> Exited Block

However, only Finalize is run, which apparently is
not triggering deallocation. The following makes me
observe a leak:

with Ada.Text_IO; use Ada.Text_IO;
with Ada.Finalization; use Ada;
procedure Test_Deallocation is

    procedure Block is
       type T is new Finalization.Limited_Controlled with null record;
       overriding procedure Finalize   (Object : in out T);
       overriding procedure Finalize (Object : in out T) is
       begin
          Put_Line ("Finalizing ");
       end Finalize;

       type T_Access is access T;
       T1, T2 : T_Access;
    begin
       T1 := new T;
       T2 := new T;
    end Block;

begin
    loop
       Block;
       Put_Line ("Exited Block");
       delay 0.2;
    end loop;
end Test_Deallocation;

Re: newbie: can't read fast enough... :-) memory leaks...

[Jeffrey Carter]

On 09/04/2014 07:23 AM, Brad Moore wrote:
> 
> In the latest GNAP GPL, the following program outputs;
> 
> Finalizing
> Finalizing
> Exited Block

Finalization is not deallocation. Wrap that in an infinite loop and observe the
effect on memory usage.

-- 
Jeff Carter
"C's solution to this [variable-sized array parameters] has real
problems, and people who are complaining about safety definitely
have a point."
Dennis Ritchie
25

Re: newbie: can't read fast enough... :-) memory leaks...

[Robert A Duff]

Brad Moore <brad.moore@shaw.ca> writes:

> I wonder how many implementations are out there that do not have a
> default storage pool that reclaims the storage of a collection of an
> nested access type when the master associated with of the access type
> declaration is exited.

I know of only one compiler that frees local collections automatically
-- Rational/IBM/Atego.  Unless you use a Storage_Size or Storage_Pool
clause, which works in all compilers, AFAIK.

> I'm pretty sure that GNAT does for Windows and Linux, but perhaps they
> have a minimalist runtime for certain targets where this is not needed
> or desired?

No, GNAT doesn't do it.  I don't think it should be automatic: if you
want that functionality, use a storage pool.

- Bob

Re: newbie: can't read fast enough... :-) memory leaks...

[anon]

For GNAT, you can copy to your local dir and modify 
"System.Pool_Global" (GNAT) and "System.Pool_Local" (GNAT) 
compile and link them you to your program.  The "Deallocation" 
routine is normally unfunctional so memory is "reclaimed" during 
program finalization only.  So the "Deallocate" routine that needs 
to be modified for GC collection.

But leaks may still appear from using outside non-Ada libraries or 
making some OS calls. It one reason why "binding" to these libraries 
or routines should be limited to testing phase only and then rewritten 
in Ada if possible.


In <wccfvg78ck5.fsf@shell01.TheWorld.com>, Robert A Duff <bobduff@shell01.TheWorld.com> writes:
>Brad Moore <brad.moore@shaw.ca> writes:
>
>> I wonder how many implementations are out there that do not have a
>> default storage pool that reclaims the storage of a collection of an
>> nested access type when the master associated with of the access type
>> declaration is exited.
>
>I know of only one compiler that frees local collections automatically
>-- Rational/IBM/Atego.  Unless you use a Storage_Size or Storage_Pool
>clause, which works in all compilers, AFAIK.
>
>> I'm pretty sure that GNAT does for Windows and Linux, but perhaps they
>> have a minimalist runtime for certain targets where this is not needed
>> or desired?
>
>No, GNAT doesn't do it.  I don't think it should be automatic: if you
>want that functionality, use a storage pool.
>
>- Bob

Re: newbie: can't read fast enough... :-) memory leaks...

[Simon Wright]

anon@att.net writes:

> The "Deallocation" routine is normally unfunctional so memory is
> "reclaimed" during program finalization only.

For GCC 4.9.0, Deallocate (both Pool_Global and Pool_Local) calls
Memory.Free which calls free().

Re: newbie: can't read fast enough... :-) memory leaks...

[anon]

But the "free()" from the C-library or OS links does not fully reclaim 
memory and no other program can use this memory until the program ends. 
Unless, that memory is setup as a OS share resource.

That's why I said the programmer needs to modify these routines to add 
Garage Collection (GC), but any GC algorithm will slow the program down.

All of GNAT's memory/storage pools packages and Memory package has its 
pluses and its minuses.  Now, the Global and Local default pools allow 
dynamic allocation without using the Ada attributes. So modifying these 
packages a programmers can add monitor all programs allocations including 
the creation of static variables. While the others pools in GNAT use static 
or stack allocated data and can only monitor it own data.


Now it seams that the map for pools is

  Runtime pools
    Memory
      Pool_Global    --  no automatic reclaim
                     --  default pool for all storage

      Pool_Local     -- handles unbounded reclaim pool
                     -- reclaimed at pool finalization
                     -- uses System.Pool_Global 
                     -- default pool

      Pool_Size      --  stack bounded pool
                     --  reclaimed by normally stack management


  User defined pools
    Memory
      Storage_Pools  --  user defined static pool


  Undefined pools 
    Memory
      Pool_Empty     --  GNAT compiler does reference package

      Remote_Pools   --  comments in GNAT compiler which reference the 
                     --  Implementation Permissions of the
                     --  RM: Remote Call Interface Library Units
                     --  So, package is not included or fully supported 
                     --  in GNAT runtime


  Special pools
    Memory
      Checked_Pools  -- Similar to Storage_Pools with Dereference monitoring
      Debug_Pools    -- Package to help track down memory corruption 
                     -- uses Checked_Pools


In <lyr3zqh03t.fsf@pushface.org>, Simon Wright <simon@pushface.org> writes:
>anon@att.net writes:
>
>> The "Deallocation" routine is normally unfunctional so memory is
>> "reclaimed" during program finalization only.
>
>For GCC 4.9.0, Deallocate (both Pool_Global and Pool_Local) calls
>Memory.Free which calls free().

Re: newbie: can't read fast enough... :-) memory leaks...

[Simon Wright]

anon@att.net writes:

> But the "free()" from the C-library or OS links does not fully reclaim
> memory and no other program can use this memory until the program
> ends.

And just how will reworking either of System.*_Pool make any difference
to the situation? If anything, you'd be targetting System.Memory. Or
free(). Or the way that the OS allocates memory to programs.