Problems with controlled types, gnatmem thinks handle is leaking memory (long)

Problems with controlled types, gnatmem thinks handle is leaking memory (long)

[Luke A. Guest]

Hi,

I need to implement a "smart pointer" in Ada and I did a few months ago
write one based on a few sources, but it had two levels of indirection in
it, so I looked at the Handles examples by Matthew Heaney on AdaPower.

I modified my code to have only one level, and an aliased item in my
"node" but according to gnatmem it leaks (now I remember running
gnatmem on the old one and it came out with similar results).

I tried this with GNAT-3.15p. I also copied over my test code to use with
Matthew's code, and it produces the same results, leaks. Is this a problem
with gnatmem?

Another thing is that there seems to be more calls to Adjust and Finalize
than I expect.

Can someone help me out? I'll post my code rather than Matthew's.

At the end, I've copied the output of the program, followed by an
annotated log where I've added comments and questions.

Thanks to everyone who helps,
Luke.


<code>
with Ada.Finalization;

generic

  -- Not an actual pointer, just the type that we will be creating an access type of.
  type Data_Type is limited private;
  
package Handle is

--  pragma Preelaborate;

  type Object is private;
  type Data_Access is access all Data_Type;

  function Create return Object;
  function Ref(Self : in Object) return Data_Access;
  
private

  type Handle_Type;
  type Handle_Access is access Handle_Type;

  type Object is new Ada.Finalization.Controlled with
    record
      Handle_Data : Handle_Access := null;
    end record;

  procedure Adjust(Self : in out Object);
  procedure Finalize(Self : in out Object);
  
end Handle;


with Text_IO;
with Unchecked_Deallocation;

package body Handle is

  type Handle_Type is
    record
      Data      : aliased Data_Type;
      Ref_Count : Natural;
    end record;
    
  -- Create.
  --
  -- Create a handle type with an access type.
  function Create return Object is
  
    Local : Handle_Access;
    
  begin
  
    Text_IO.Put_Line("Create: Enter");
    
    Local := new Handle_Type;
    
    Local.Ref_Count := 1;
  
    Text_IO.Put_Line("Create: Exit");
    
    return (Ada.Finalization.Controlled with Local);
    
  end Create;
  

  -- Ref.
  --
  -- Return the access type of the handle, so we can manipulate it.
  function Ref(Self : in Object) return Data_Access is
  
  begin
  
    Text_IO.Put_Line("Ref: Enter");
    
    return Self.Handle_Data.Data'Access;
    
  end Ref;
  
  
  -- Adjust
  procedure Adjust(Self : in out Object) is
  
  begin

    Text_IO.Put_Line("Adjust: Enter");
    
    Self.Handle_Data.Ref_Count := Self.Handle_Data.Ref_Count + 1;

    Text_IO.Put_Line("Adjust: Self.Handle_Data.Ref_Count = " & Natural'Image(Self.Handle_Data.Ref_Count));
      
    Text_IO.Put_Line("Adjust: Exit");
    
  end Adjust;
  
  
  -- Finalize
  procedure Finalize(Self : in out Object) is
  
    procedure Free_Handle is new Unchecked_Deallocation(Handle_Type, Handle_Access);
  
  begin

    Text_IO.Put_Line("Finalize: Enter");
    
    if Self.Handle_Data /= null then
    
      Self.Handle_Data.Ref_Count := Self.Handle_Data.Ref_Count - 1;
    
      Text_IO.Put_Line("Finalize: Self.Handle_Data.Ref_Count = " & Natural'Image(Self.Handle_Data.Ref_Count));
      
      if Self.Handle_Data.Ref_Count = 0 then
    
        Free_Handle(Self.Handle_Data);
          
        Self.Handle_Data := null;
      
        Text_IO.Put_Line("Finalize: Deallocated memory");
      
      end if;
      
    end if;
    
    Text_IO.Put_Line("Finalize: Exit");
    
  end Finalize;
  
end Handle;


with Handle;

package Data is

  type Object is
    record
      A : Integer;
      B : Float;
    end record;

  package Object_Handle is new Handle(Object);

end Data;


with Text_IO;
with Data;

procedure Test_Handle is

  package Integer_IO is new Text_IO.Integer_IO(Integer);
  package Float_IO is new Text_IO.Float_IO(Float);
  
  Test  : Data.Object_Handle.Object;-- := Data.Object_Handle.Create;
  Test2 : Data.Object_Handle.Object;
  
begin

  Test := Data.Object_Handle.Create;

  Data.Object_Handle.Ref(Test).A := 6;
  Data.Object_Handle.Ref(Test).B := 12.34;
  
  Text_IO.Put("A: ");
  Integer_IO.Put(Data.Object_Handle.Ref(Test).A);
  Text_IO.Put("  B: ");
  Float_IO.Put(Data.Object_Handle.Ref(Test).B, Fore => 2, Aft => 2, Exp => 0);
  Text_IO.New_Line;  
  
  Test2 := Test;
  
end Test_Handle;


-- Output.log
Create: Enter
Create: Exit
Adjust: Enter
Adjust: Self.Handle_Data.Ref_Count =  2
Adjust: Exit
Finalize: Enter
Finalize: Self.Handle_Data.Ref_Count =  1
Finalize: Exit
Finalize: Enter
Finalize: Exit
Adjust: Enter
Adjust: Self.Handle_Data.Ref_Count =  2
Adjust: Exit
Finalize: Enter
Finalize: Self.Handle_Data.Ref_Count =  1
Finalize: Exit
Ref: Enter
Ref: Enter
A: Ref: Enter
          6  B: Ref: Enter
12.34
Finalize: Enter
Finalize: Exit
Adjust: Enter
Adjust: Self.Handle_Data.Ref_Count =  2
Adjust: Exit
Finalize: Enter
Finalize: Self.Handle_Data.Ref_Count =  1
Finalize: Exit
Finalize: Enter
Finalize: Self.Handle_Data.Ref_Count =  0
Finalize: Deallocated memory
Finalize: Exit



-- Annotated Output.log
Create: Enter
Create: Exit

** This Adjust is the assignment on the line where the Create is called,
** inside the Create function the Ref_Count would be 1
Adjust: Enter
Adjust: Self.Handle_Data.Ref_Count =  2
Adjust: Exit

** This Finalize is obviously the local object inside Create
Finalize: Enter
Finalize: Self.Handle_Data.Ref_Count =  1
Finalize: Exit

** What is this Finalize, Adjust, Finalize?
Finalize: Enter
Finalize: Exit
Adjust: Enter
Adjust: Self.Handle_Data.Ref_Count =  2
Adjust: Exit
Finalize: Enter
Finalize: Self.Handle_Data.Ref_Count =  1
Finalize: Exit

** Now we're back at the two references, this is fine.
Ref: Enter
Ref: Enter
A: Ref: Enter
          6  B: Ref: Enter
12.34

** This Finalize is for Test2
Finalize: Enter
Finalize: Exit

** The object from Test has been copied to Test2 and has been Adjusted
Adjust: Enter
Adjust: Self.Handle_Data.Ref_Count =  2
Adjust: Exit

** This would be the Finalization of both Test & Test2 on exit
Finalize: Enter
Finalize: Self.Handle_Data.Ref_Count =  1
Finalize: Exit

** Notice this actually deletes the memory, so gnatmem is wrong here!
Finalize: Enter
Finalize: Self.Handle_Data.Ref_Count =  0
Finalize: Deallocated memory
Finalize: Exit

</code>

Re: Problems with controlled types, gnatmem thinks handle is leaking memory (long)

[Dmitry A. Kazakov]

On Sun, 20 Feb 2005 16:30:14 +0000, Luke A. Guest wrote:

> I need to implement a "smart pointer" in Ada and I did a few months ago
> write one based on a few sources, but it had two levels of indirection in
> it, so I looked at the Handles examples by Matthew Heaney on AdaPower.
> 
> I modified my code to have only one level, and an aliased item in my
> "node" but according to gnatmem it leaks (now I remember running
> gnatmem on the old one and it came out with similar results).
> 
> I tried this with GNAT-3.15p. I also copied over my test code to use with
> Matthew's code, and it produces the same results, leaks. Is this a problem
> with gnatmem?
> 
> Another thing is that there seems to be more calls to Adjust and Finalize
> than I expect.
> 
> Can someone help me out? I'll post my code rather than Matthew's.
> 
> At the end, I've copied the output of the program, followed by an
> annotated log where I've added comments and questions.
> 
> Thanks to everyone who helps,
> Luke.
> 
> <code>
[...]

The code does not much differ in implementation from mine
(http://www.dmitry-kazakov.de/ada/components.htm).

Minor comments:

In Finalize: I check if Ref_Count = 0 before decrementing and raise
Program_Error rather than Constraint_Error. When you call
Unchecked_Dellocation it sets the pointer to null. You don't need to do
that again. I also have the pointed objects limited controlled and check if
the Ref_Count is zero upon object's finalization. That helps a lot!
Especially if you mistakenly create circular dependencies in the objects.
(The compiler will finalize them anyway so you'll get an exception)

> -- Annotated Output.log
> Create: Enter
> Create: Exit
> 
> ** This Adjust is the assignment on the line where the Create is called,
> ** inside the Create function the Ref_Count would be 1
> Adjust: Enter
> Adjust: Self.Handle_Data.Ref_Count =  2
> Adjust: Exit
   -- The result of Create is adjusted
> 
> ** This Finalize is obviously the local object inside Create
> Finalize: Enter
> Finalize: Self.Handle_Data.Ref_Count =  1
> Finalize: Exit
   -- The aggregate in Create is finalized

> ** What is this Finalize, Adjust, Finalize?
> Finalize: Enter
> Finalize: Exit
   -- Test is finalized (it points to nothing)
> Adjust: Enter
> Adjust: Self.Handle_Data.Ref_Count =  2
> Adjust: Exit
   -- Test is adjusted
> Finalize: Enter
> Finalize: Self.Handle_Data.Ref_Count =  1
> Finalize: Exit
   -- Create's result is finalized
> 
> ** Now we're back at the two references, this is fine.
> Ref: Enter
> Ref: Enter
> A: Ref: Enter
>           6  B: Ref: Enter
> 12.34
> 
> ** This Finalize is for Test2
> Finalize: Enter
> Finalize: Exit
> 
> ** The object from Test has been copied to Test2 and has been Adjusted
> Adjust: Enter
> Adjust: Self.Handle_Data.Ref_Count =  2
> Adjust: Exit
> 
> ** This would be the Finalization of both Test & Test2 on exit
> Finalize: Enter
> Finalize: Self.Handle_Data.Ref_Count =  1
> Finalize: Exit
> 
> ** Notice this actually deletes the memory, so gnatmem is wrong here!
> Finalize: Enter
> Finalize: Self.Handle_Data.Ref_Count =  0
> Finalize: Deallocated memory
> Finalize: Exit

Everything seems OK to me.

You can make Handle_Type controlled add Initialize and Finalize to see when
they are called. Handle_Type.Finalize should be called *before* the text

Finalize: Deallocated memory

Otherwise, if you really had a leak, it would be called after that. (The
list of all controlled objects is maintained at run-time.)

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: Problems with controlled types, gnatmem thinks handle is leaking memory (long)

[Luke A. Guest]

On Sun, 20 Feb 2005 19:09:02 +0100, Dmitry A. Kazakov wrote:

> The code does not much differ in implementation from mine
> (http://www.dmitry-kazakov.de/ada/components.htm).

I'll have a look at this, thanks.
 
> Minor comments:
> 
> In Finalize: I check if Ref_Count = 0 before decrementing and raise

No need to check for the reference count to be 0, as it can never go
negative and if it does, there's a problem with the code.

> Program_Error rather than Constraint_Error. When you call

Ok,but I'm not raising any exceptions here.

> Unchecked_Dellocation it sets the pointer to null. You don't need to do

Fair enough, I can remove that then ;-)

> that again. I also have the pointed objects limited controlled and check

Well, my pointed to objects are limited controlled, so I only ever return
the reference to the object. I don't ever want to copy the pointer, just
the handle around it.

> if the Ref_Count is zero upon object's finalization. That helps a lot!

I suppose I could raise an exception if the reference count isn't zero on
finalisation, hmmm...but surely there would be an error in the code if
that were not the case? Surely it's another example of what I mentioned
above, it should never go wrong as long as the initialize, adjust &
finalize subprograms have been implemented correctly.

> Especially if you mistakenly create circular dependencies in the
> objects. (The compiler will finalize them anyway so you'll get an
> exception)

Well, considering Ada95 compilers have trouble with circular packages, I
can't see me having any trouble considering I have only one major type per
package.

[Missing extra annotations]

Thanks for the extra annotations, they were really helpful. I think it's a
problem with gnatmem, which is a problem considering that gcc-3.4.x and
above doesn't have a gnatmem at all and valgrind doesn't work with
anything I compile with gnat (I haven't tried gcc-3.4.x yet).

> Everything seems OK to me.
> 
> You can make Handle_Type controlled add Initialize and Finalize to see
> when they are called. Handle_Type.Finalize should be called *before* the
> text
> 
> Finalize: Deallocated memory
> 
> Otherwise, if you really had a leak, it would be called after that. (The
> list of all controlled objects is maintained at run-time.)

Good, point, I'll try it out and see what happens.

Thanks,
Luke.

Re: Problems with controlled types, gnatmem thinks handle is leaking memory (long)

[Dmitry A. Kazakov]

On Sun, 20 Feb 2005 23:09:15 +0000, Luke A. Guest wrote:

> On Sun, 20 Feb 2005 19:09:02 +0100, Dmitry A. Kazakov wrote:
> 
>> Minor comments:
>> 
>> In Finalize: I check if Ref_Count = 0 before decrementing and raise
> 
> No need to check for the reference count to be 0, as it can never go
> negative and if it does, there's a problem with the code.

That's the only reason why. (:-))
 
>> if the Ref_Count is zero upon object's finalization. That helps a lot!
> 
> I suppose I could raise an exception if the reference count isn't zero on
> finalisation, hmmm...but surely there would be an error in the code if
> that were not the case? Surely it's another example of what I mentioned
> above, it should never go wrong as long as the initialize, adjust &
> finalize subprograms have been implemented correctly.

Unfortunately smart pointers, even if implemented correctly, do not
warranty correct performance. A program still may [mis]use them in an
inappropriate way. A typical example is when an object directly or
indirectly holds a handle to itself, you could have a very long chain of
objects dependent on each other. It is highly desirable, but not always
possible to prevent this statically, per design.

A real-life example: I am using handles for object persistence. I need to
maintain a catalogue of all persistent objects resident in the memory. This
catalogue is itself an object that refers to a data base connection. Now
imagine, a persistent object, which is resident in that data base, so that
its memory counterpart is just a proxy to the data base. This object shall
refer to the data base connection object, otherwise that connection object
might vanish prematurely. But it also have to be in the catalogue. Boom! (I
added a complicated mechanism to break such dependencies in persistent
handles.)

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: Problems with controlled types, gnatmem thinks handle is leaking memory (long)

[Simon Wright]

"Luke A. Guest" <laguest@n_o_p_o_r_k_a_n_d_h_a_m.abyss2.demon.co.uk> writes:

> On Sun, 20 Feb 2005 19:09:02 +0100, Dmitry A. Kazakov wrote:

> > In Finalize: I check if Ref_Count = 0 before decrementing and raise
> 
> No need to check for the reference count to be 0, as it can never go
> negative and if it does, there's a problem with the code.

I believe it is possible for an object to be Finalized more than once.

> > if the Ref_Count is zero upon object's finalization. That helps a
> > lot!
> 
> I suppose I could raise an exception if the reference count isn't
> zero on finalisation, hmmm...but surely there would be an error in
> the code if that were not the case? Surely it's another example of
> what I mentioned above, it should never go wrong as long as the
> initialize, adjust & finalize subprograms have been implemented
> correctly.

It isn't a good idea to propagate an exception out of a Finalize
operation (it's a bounded error to do so, 7.6.1(14)).


-- 
Simon Wright                               100% Ada, no bugs.

Re: Problems with controlled types, gnatmem thinks handle is leaking memory (long)

[Dmitry A. Kazakov]

On 21 Feb 2005 20:12:34 +0000, Simon Wright wrote:

> "Luke A. Guest" <laguest@n_o_p_o_r_k_a_n_d_h_a_m.abyss2.demon.co.uk> writes:
> 
>> On Sun, 20 Feb 2005 19:09:02 +0100, Dmitry A. Kazakov wrote:
> 
>>> In Finalize: I check if Ref_Count = 0 before decrementing and raise
>> 
>> No need to check for the reference count to be 0, as it can never go
>> negative and if it does, there's a problem with the code.
> 
> I believe it is possible for an object to be Finalized more than once.

In the case of a handle it is not a problem because its Finalize is sort of

if pointer /= null then
   decrement reference count;
   if reference count = 0 then
      free object (and pointer is null);
   else
      pointer := null;
   end if;
end if;

Thus, when called again Finalize would do nothing.

>>> if the Ref_Count is zero upon object's finalization. That helps a
>>> lot!
>> 
>> I suppose I could raise an exception if the reference count isn't
>> zero on finalisation, hmmm...but surely there would be an error in
>> the code if that were not the case? Surely it's another example of
>> what I mentioned above, it should never go wrong as long as the
>> initialize, adjust & finalize subprograms have been implemented
>> correctly.
> 
> It isn't a good idea to propagate an exception out of a Finalize
> operation (it's a bounded error to do so, 7.6.1(14)).

Yes, but here (when the reference count is not zero) it is too late to
undertake anything. The exception just signals a fatal error. In this sense
it is already a bounded error to get here. It is as bad as Segmentation
Fault. The only thing one can do is to abort the program with all its tasks
as soon as possible. I raise Program_Error, though it would be nice to have
an exception that isn't propagated, but kills the partition at the spot...

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: Problems with controlled types, gnatmem thinks handle is leaking memory (long)

[Robert A Duff]

"Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> writes:

>...I raise Program_Error, though it would be nice to have
> an exception that isn't propagated, but kills the partition at the spot...

You can import "exit" from C.  Or you can abort the environment task.

- Bob