overriding in private part

overriding in private part

[Maxim Reznik]

Hi all

Trying to find memory leak in AWS I found strange bug.

Let my explain by example: http://pastebin.mozilla-russia.org/92280

There is a package hierarchy of three packages: A1, A1.A2 and A1.A3.
And type hierarchy: A1.Base, A2.Child derived from A1.Base, A3.Child
derived from A2.Child.

A1.Base has primitive procedure P2 in private part of A1.
A2.Child override P2 with its own P2 in private part of A2.
A3.Child is expected to override P2 with its own P2 in private part of
A3, but actually it can't!
As result A3.P2 is never called in this example. (And never free
memory in AWS)

My expectation about this code, that A3.P2 override A1.P2 because
private part of A1 is visible at this place, and A2.P2 invisible here,
so have no influence here. But A2.P2 hides A1.P2 indeed.

Every type in class A1.Base'Class has P2 primitive subprogram, but any
child of A2.Child can't override it, because P2 is hidden by A2.P2. It
seems unnatural to me.

Errors of such kind are very difficult to find.

Is there any way to prevent such errors? (Besides keyword
*overriding*)

--
Maxim Reznik

Re: overriding in private part

[Adam Beneschan]

On Oct 2, 8:49 am, Maxim Reznik <rezni...@gmail.com> wrote:
> Hi all
>
> Trying to find memory leak in AWS I found strange bug.
>
> Let my explain by example:http://pastebin.mozilla-russia.org/92280
>
> There is a package hierarchy of three packages: A1, A1.A2 and A1.A3.
> And type hierarchy: A1.Base, A2.Child derived from A1.Base, A3.Child
> derived from A2.Child.
>
> A1.Base has primitive procedure P2 in private part of A1.
> A2.Child override P2 with its own P2 in private part of A2.
> A3.Child is expected to override P2 with its own P2 in private part of
> A3, but actually it can't!
> As result A3.P2 is never called in this example. (And never free
> memory in AWS)
>
> My expectation about this code, that A3.P2 override A1.P2 because
> private part of A1 is visible at this place, and A2.P2 invisible here,
> so have no influence here. But A2.P2 hides A1.P2 indeed.
>
> Every type in class A1.Base'Class has P2 primitive subprogram, but any
> child of A2.Child can't override it, because P2 is hidden by A2.P2. It
> seems unnatural to me.
>
> Errors of such kind are very difficult to find.
>
> Is there any way to prevent such errors? (Besides keyword
> *overriding*)

That's one of the reasons "overriding" was added.  Taking a quick look
at AI-218, which proposed this feature, there's a lot of discussion
about private parts.

In general, the language doesn't want to take private declarations of
the parent into account when deciding whether something is
overriding.  E.g.:

package Pak1 is
   type T is tagged private;
private
   -- You're not supposed to care what is here
end Pak1;

package Pak2 is
   type T2 is new Pak1.T with record... end record;
   procedure New_Operation (Obj : in out T2);
end Pak2;

The intent was to define a *new* operation on the child type T2.  It
would be really bad if the behavior changed because the private part
of Pak1 happened to defined its own New_Operation on T, since you're
supposed to be able to write Pak2 and use T as a private type without
knowing anything about the private details of T.  So that's why the
language makes this last New_Operation a "not overriding" operation.

Your case is a little more confusing because, like the above example,
there's an invisible P2 operation of the parent type that shouldn't
affect the behavior, but this is inherited from a P2 operation of the
*grandparent* type that *is* visible in the private part of A1.A3.
Perhaps the compiler should generate a warning about the possible
confusion in this case.  If we didn't have an "overriding" keyword, I
might even go so far as to suggest the language should make this case
illegal, in order to prevent this kind of case from coming up.  But
since "overriding" has been added, you should just use it, and it will
solve the problem.  (Yes, I know, you're working with code written for
Ada 95.  Bummer.)

                                -- Adam

Re: overriding in private part

[Dmitry A. Kazakov]

On Thu, 2 Oct 2008 09:42:28 -0700 (PDT), Adam Beneschan wrote:

> In general, the language doesn't want to take private declarations of
> the parent into account when deciding whether something is
> overriding.

This is OK, but the problem is that the operation is formally considered
private when it is obviously (in common sense) not.

> Your case is a little more confusing because, like the above example,
> there's an invisible P2 operation of the parent type that shouldn't
> affect the behavior, but this is inherited from a P2 operation of the
> *grandparent* type that *is* visible in the private part of A1.A3.
> Perhaps the compiler should generate a warning about the possible
> confusion in this case.  If we didn't have an "overriding" keyword, I
> might even go so far as to suggest the language should make this case
> illegal, in order to prevent this kind of case from coming up.

It is not the overriding which must be illegal. The opposite should. I mean
it should be:

   overriding procedure P2 (X : Child); -- Legal (presently illegal)
   not overriding procedure P2 (X : Child); -- Illegal (presently legal)

> But
> since "overriding" has been added, you should just use it, and it will
> solve the problem.

Unfortunately it does not. The programmer desired to override P2, and this
is impossible to do, because there is a type in between which effectively
*hides* the operation in all packages, regardless their visibility.
Further, this behavior changes when the offending type is derived in A1
rather than in A1.A2. In that case A1.A3 would be able to override P2.

This is obviously broken to me.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: overriding in private part

[Adam Beneschan]

On Oct 3, 1:52 am, "Dmitry A. Kazakov" <mail...@dmitry-kazakov.de>
wrote:

> > But
> > since "overriding" has been added, you should just use it, and it will
> > solve the problem.
>
> Unfortunately it does not.

I shouldn't have said "solve".  It *partially* solves the problem, in
that if the programmers use "overriding" and "not overriding"
consistently on everything, it won't let you write a program that has
an unexpected result of the sort Maxim ran into.  Of course, having
the compiler tell you your program is illegal isn't really a full
solution when the language should be letting you do what you need to
do, but it's a heck of a lot better than accepting your program and
doing the wrong thing.


> The programmer desired to override P2, and this
> is impossible to do, because there is a type in between which effectively
> *hides* the operation in all packages, regardless their visibility.
> Further, this behavior changes when the offending type is derived in A1
> rather than in A1.A2. In that case A1.A3 would be able to override P2.
>
> This is obviously broken to me.

I think this is a good point, and I'm trying to think of a possible
solution.  The small stumbling block is that it's possible to declare
a private extension in a package Package_1

   type T1 is new T with private;

and then, in the private part of Package_1:

   type T1 is new T2 with ...

where T2 is a child of T.  Clearly, any *new* operations that were
declared for T2 but not for T shouldn't be visible, or overridable, by
any package that can't see into the private part of Package_1.  But
that just means that any proposed change would have to be written a
bit more carefully; it's not an insurmountable obstacle.

Anyway, I'm thinking about how this could be fixed.

                               -- Adam

Re: overriding in private part

[Robert A Duff]

Adam Beneschan <adam@irvine.com> writes:

> I shouldn't have said "solve".  It *partially* solves the problem, in
> that if the programmers use "overriding" and "not overriding"
> consistently on everything, it won't let you write a program that has
> an unexpected result of the sort Maxim ran into.

I think you should say "overriding" wherever it's legal,
but never say "not overriding".  And use a compiler that
warns on missing "overriding" -- such as a recent GNAT
with the -gnatyO switch.

I agree that in the OP's example, P2 should be overriding -- this is a
language design flaw.  But at least you won't get in trouble at
run time, if you follow the above convention.

- Bob

Re: overriding in private part

[Randy Brukardt]

"Robert A Duff" <bobduff@shell01.TheWorld.com> wrote in message 
news:wcc1vyxwhge.fsf@shell01.TheWorld.com...
> Adam Beneschan <adam@irvine.com> writes:
>
>> I shouldn't have said "solve".  It *partially* solves the problem, in
>> that if the programmers use "overriding" and "not overriding"
>> consistently on everything, it won't let you write a program that has
>> an unexpected result of the sort Maxim ran into.
>
> I think you should say "overriding" wherever it's legal,
> but never say "not overriding".  And use a compiler that
> warns on missing "overriding" -- such as a recent GNAT
> with the -gnatyO switch.

I agree with Adam; after all, there is a reason that "not overriding" 
exists. I'm curious as to why you think it shouldn't be used. If you write a 
routine that you do not expect to override something, and it does anyway, 
you have a problem (because you could be called from a dispatching routine 
with a completely different purpose, and conceivably different preconditions 
and postconditions). I think you'd like to know about that problem.

There are cases where you can't use either indicator (where the overriding 
happens "late"), but those usually indicate a program structuring issue 
(routines are being hidden in the root types that ought to be visible, or 
the child types are making routines visible that should be private). And in 
such cases I think it is important to document why no indicator was given 
with a comment.

I wanted indicators to be allowed on all subprograms in order that I could 
enforce a rule of "no indicator" -> style violation, but that got voted down 
(they're only allowed on primitive subprograms). I still think that was a 
serious mistake (it's weird to have to leave out indicators on class-wide 
routines, for instance).

                                 Randy.

Re: overriding in private part

[Robert A Duff]

"Randy Brukardt" <randy@rrsoftware.com> writes:

> "Robert A Duff" <bobduff@shell01.TheWorld.com> wrote in message 
> news:wcc1vyxwhge.fsf@shell01.TheWorld.com...
>> Adam Beneschan <adam@irvine.com> writes:
>>
>>> I shouldn't have said "solve".  It *partially* solves the problem, in
>>> that if the programmers use "overriding" and "not overriding"
>>> consistently on everything, it won't let you write a program that has
>>> an unexpected result of the sort Maxim ran into.
>>
>> I think you should say "overriding" wherever it's legal,
>> but never say "not overriding".  And use a compiler that
>> warns on missing "overriding" -- such as a recent GNAT
>> with the -gnatyO switch.
>
> I agree with Adam; after all, there is a reason that "not overriding" 
> exists. I'm curious as to why you think it shouldn't be used. If you write a 
> routine that you do not expect to override something, and it does anyway,
> you have a problem (because you could be called from a dispatching routine 
> with a completely different purpose, and conceivably different preconditions 
> and postconditions). I think you'd like to know about that problem.

If you use a compiler that warns on missing "overriding", then you don't
need to say "not overriding", because that's the default -- any
subprogram that doesn't say "overriding" is not overriding.
Saying "not overriding" is just noise.

There is no good reason for "not overriding" to exist.  If you "write a
routine that you do not expect to override something, and it does
anyway," then I don't agree you have a problem -- you get a warning.

The warning is crucial, of course.  Without that, IMHO these indicators
are nearly useless.  If I were designing the language from scratch,
I would make missing "overriding" an error.  The only reason not to do
that is for compatibility with Ada 95.

It's like parameter modes -- you should never explicitly say "in",
because that's the default.  You should explicitly say "in out" or
"out".  It's unfortunate that "in" is allowed, because now the language
has split into three dialects -- those that say "in", those that never
say "in", and those that say "in" for procedures but not functions.
That's not doing anyone any favors -- the folks who write in those three
dialects can't understand each other's code.

> There are cases where you can't use either indicator (where the overriding 
> happens "late"), but those usually indicate a program structuring issue 
> (routines are being hidden in the root types that ought to be visible, or 
> the child types are making routines visible that should be private). And in 
> such cases I think it is important to document why no indicator was given 
> with a comment.

And I think there are some generics-related cases, too.  But these are
all corner cases.  By and large, you can say "overriding" when you mean
it, and let "no indicator" implicitly mean "no overriding".

> I wanted indicators to be allowed on all subprograms in order that I could 
> enforce a rule of "no indicator" -> style violation, but that got voted down 
> (they're only allowed on primitive subprograms). I still think that was a 
> serious mistake (it's weird to have to leave out indicators on class-wide 
> routines, for instance).

I agree that if you want an indicator on every subprogram declaration,
then forbidding "no overriding" on some non-overriding subprograms is a
language design mistake.  But I don't want that -- I want a safe
(non-overriding) default.

- Bob

Re: overriding in private part

[Dmitry A. Kazakov]

On Sat, 04 Oct 2008 15:47:04 -0400, Robert A Duff wrote:

> If you use a compiler that warns on missing "overriding", then you don't
> need to say "not overriding", because that's the default -- any
> subprogram that doesn't say "overriding" is not overriding.
> Saying "not overriding" is just noise.

[...] 

> I agree that if you want an indicator on every subprogram declaration,
> then forbidding "no overriding" on some non-overriding subprograms is a
> language design mistake.  But I don't want that -- I want a safe
> (non-overriding) default.

But non-overriding is unsafe. Taking your example with in and in-out, when
the programmer uses in instead of in-out, that does not change the program
semantics, so long the program remains legal. Otherwise (if the body
actually changes the parameter) it will not compile. This is safe.

Now, if a procedure is not overriding where an overriding was meant, that
is a semantic change of an unpredictable damage, which slips undetected
through [*]. I think that actually the opposite case is safer. Let the
procedure is an occasional overriding. That does not influence any
type-specific code, because no dispatch happens there. It changes only the
semantics of class-wide code written in terms of parent classes.

There seems to be no safe default.

---------
* Unless the operation is abstract

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: overriding in private part

[Robert A Duff]

"Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> writes:

> On Sat, 04 Oct 2008 15:47:04 -0400, Robert A Duff wrote:
>
>> If you use a compiler that warns on missing "overriding", then you don't
>> need to say "not overriding", because that's the default -- any
>> subprogram that doesn't say "overriding" is not overriding.
>> Saying "not overriding" is just noise.
>
> [...] 
>
>> I agree that if you want an indicator on every subprogram declaration,
>> then forbidding "no overriding" on some non-overriding subprograms is a
>> language design mistake.  But I don't want that -- I want a safe
>> (non-overriding) default.
>
> But non-overriding is unsafe. Taking your example with in and in-out, when
> the programmer uses in instead of in-out, that does not change the program
> semantics, so long the program remains legal. Otherwise (if the body
> actually changes the parameter) it will not compile. This is safe.

Good point, but I'm only half convinced.  If you get in the habit of
always saying "overriding" when appropriate, and you use the warnings,
then you're unlikely to get into trouble.  And the idea of putting "not
overriding" all over the place seems awfully verbose, to me.

- Bob

Re: overriding in private part

[Dmitry A. Kazakov]

On Sun, 05 Oct 2008 15:57:48 -0400, Robert A Duff wrote:

> "Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> writes:
> 
>> On Sat, 04 Oct 2008 15:47:04 -0400, Robert A Duff wrote:
>>
>>> If you use a compiler that warns on missing "overriding", then you don't
>>> need to say "not overriding", because that's the default -- any
>>> subprogram that doesn't say "overriding" is not overriding.
>>> Saying "not overriding" is just noise.
>>
>> [...] 
>>
>>> I agree that if you want an indicator on every subprogram declaration,
>>> then forbidding "no overriding" on some non-overriding subprograms is a
>>> language design mistake.  But I don't want that -- I want a safe
>>> (non-overriding) default.
>>
>> But non-overriding is unsafe. Taking your example with in and in-out, when
>> the programmer uses in instead of in-out, that does not change the program
>> semantics, so long the program remains legal. Otherwise (if the body
>> actually changes the parameter) it will not compile. This is safe.
> 
> Good point, but I'm only half convinced.  If you get in the habit of
> always saying "overriding" when appropriate, and you use the warnings,
> then you're unlikely to get into trouble.  And the idea of putting "not
> overriding" all over the place seems awfully verbose, to me.

Both are unsafe. A felt verbosity comes from the strange decision to put
[not] overriding in front of the declaration. If it were:

   procedure Foo (X : Boo) is not overriding;

it would not be so offending.

As for me, I think that overriding could be a good default for all
subprograms with at least one controlling argument. Declarations of any new
primitive operation should then be explicit:

   procedure Foo (X : Boo) is [abstract] new; -- I hate "overriding"

Non-primitive operations should be made illegal if any of the arguments is
controlling:

   type T is tagged ...;
   procedure Foo (X : T) is new;

   package Bar is
      procedure Foo (X : T); -- You cannot do this!
      procedure Baz (X : T); -- Neither this!
      procedure Baz (X : T'Class); -- This is OK
   end Bar;

Maybe, that could be relaxed the bodies of the packages which
specifications declare the type:

   package A is
      type X is tagged ...;
   end A;

   package body A is
      procedure Some_Private_Stuff_Without_Redispatch (X : T) is not new;
   end A;

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: overriding in private part

[Randy Brukardt]

"Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> wrote in message 
news:16aosnc43o0l2.1snh410eman0v$.dlg@40tude.net...
...
> As for me, I think that overriding could be a good default for all
> subprograms with at least one controlling argument. Declarations of any 
> new
> primitive operation should then be explicit:
>
>   procedure Foo (X : Boo) is [abstract] new; -- I hate "overriding"
>
> Non-primitive operations should be made illegal if any of the arguments is
> controlling:

Yes, that would be the correct semantics if the language was being built 
from scratch. Unfortunately, the Ada 95 team decided to use the existing 
inheritance mechanism, and that isn't quite right. It's not so obvious until 
you try examples in various ways.

In any case, non-primitive routines with specific tagged types are pretty 
suspicious and clearly deserve a warning. (There are a couple of them in 
Claw - mostly functions returning some specific tagged type, but the 
majority are class-wide.)

                                Randy.

Re: overriding in private part

[stefan-lucks]

On Sat, 4 Oct 2008, Robert A Duff wrote:

> If you use a compiler that warns on missing "overriding", then you don't
> need to say "not overriding", because that's the default -- any
> subprogram that doesn't say "overriding" is not overriding.
> Saying "not overriding" is just noise.

"If you use a compiler that [does the right thing]" (or rather, if you use 
that compiler and the right set compiler switches) that works more or less 
fine. Except that you get only a warning on something that ought to be 
treated as an error. (Yes, I know that gnat has a switch to treat warnings 
as errors.)

But a compiler-agnostic way to enforce the proper behaviour (overriding 
indicators for all the subprograms which actually override another 
subprogram) would be preferable, IMHO. I wish there was an Ada 05
  "pragma Overriding_Indicators_For_All_Oriding_Subprograms"
or the like. Or did I overlook something like that in the standard?

So long

Stefan



-- 
------ Stefan Lucks   --  Bauhaus-University Weimar  --   Germany  ------
               Stefan dot Lucks at uni minus weimar dot de
------  I  love  the  taste  of  Cryptanalysis  in  the  morning!  ------

Re: overriding in private part

[Robert A Duff]

stefan-lucks@see-the.signature writes:

> On Sat, 4 Oct 2008, Robert A Duff wrote:
>
>> If you use a compiler that warns on missing "overriding", then you don't
>> need to say "not overriding", because that's the default -- any
>> subprogram that doesn't say "overriding" is not overriding.
>> Saying "not overriding" is just noise.
>
> "If you use a compiler that [does the right thing]" (or rather, if you use 
> that compiler and the right set compiler switches) that works more or less 
> fine. Except that you get only a warning on something that ought to be 
> treated as an error. (Yes, I know that gnat has a switch to treat warnings 
> as errors.)

Shrug.  To me, warnings and errors are pretty-much the same.
At AdaCore, we compile everything in warnings-as-errors mode.
Not always, but we have procedures in place that ensure no
Ada code can escape into customer's hands with warnings.

> But a compiler-agnostic way to enforce the proper behaviour (overriding 
> indicators for all the subprograms which actually override another 
> subprogram) would be preferable, IMHO. I wish there was an Ada 05
>   "pragma Overriding_Indicators_For_All_Oriding_Subprograms"
> or the like. Or did I overlook something like that in the standard?

I don't think you overlooked anything.  I agree portability is nice, so
it would nice to have a portable way to say this.  Pragma
Require_Overriding_Indicators might be a reaonable name.

But of course if you're worried about forgetting the warning switch, you
should be equally worried about forgetting that pragma.  Neither one
seems like a big problem -- it's something you do once, when setting up
your project-wide pragmas, or project-wide build scripts.  Much more
likely to forget "overriding" when declaring a procedure, which is
something you do every day.

Note that if you're writing code for several Ada compilers, you only
need one of them to give the warning.

- Bob

Re: overriding in private part

[Randy Brukardt]

"Robert A Duff" <bobduff@shell01.TheWorld.com> wrote in message 
news:wccfxna24b3.fsf@shell01.TheWorld.com...
> stefan-lucks@see-the.signature writes:
>
>> On Sat, 4 Oct 2008, Robert A Duff wrote:
>>
>>> If you use a compiler that warns on missing "overriding", then you don't
>>> need to say "not overriding", because that's the default -- any
>>> subprogram that doesn't say "overriding" is not overriding.
>>> Saying "not overriding" is just noise.
>>
>> "If you use a compiler that [does the right thing]" (or rather, if you 
>> use
>> that compiler and the right set compiler switches) that works more or 
>> less
>> fine. Except that you get only a warning on something that ought to be
>> treated as an error. (Yes, I know that gnat has a switch to treat 
>> warnings
>> as errors.)
>
> Shrug.  To me, warnings and errors are pretty-much the same.
> At AdaCore, we compile everything in warnings-as-errors mode.
> Not always, but we have procedures in place that ensure no
> Ada code can escape into customer's hands with warnings.

OK, I guess, but I disagree (and agree with Stefan); errors required by the 
language are always more powerful than something done by a specific 
implementation (unless you can be sure that you are never going to use 
another implementation!).

Besides, I don't see how your warning would work with Ada 95 code (such as 
Claw). There's little value to having warnings in such code (it isn't yours 
anyway), but you still would want the errors in your own code. So it seems 
likely that you would have to turn the warning off in large amounts of code; 
that is going to make it more likely that it is also omitted in code where 
you want it to be checked.

>> But a compiler-agnostic way to enforce the proper behaviour (overriding
>> indicators for all the subprograms which actually override another
>> subprogram) would be preferable, IMHO. I wish there was an Ada 05
>>   "pragma Overriding_Indicators_For_All_Oriding_Subprograms"
>> or the like. Or did I overlook something like that in the standard?
>
> I don't think you overlooked anything.  I agree portability is nice, so
> it would nice to have a portable way to say this.  Pragma
> Require_Overriding_Indicators might be a reaonable name.

We couldn't figure out how such a pragma would work with generics. That's 
the main reason this is a three-state switch: "overriding", "not 
overriding", and nothing (which is essentially "don't care"). This may be a 
case where "perfect" prevented "good enough" -- I surely wanted such a 
pragma, but it would have to have enough holes that it wasn't clear that it 
was worth anything.

                                            Randy.

Re: overriding in private part

[Randy Brukardt]

"Maxim Reznik" <reznikmm@gmail.com> wrote in message 
news:45b4a4cc-13f5-4175-9061-9c962e32d762@64g2000hsm.googlegroups.com...
...
> Every type in class A1.Base'Class has P2 primitive subprogram, but any
> child of A2.Child can't override it, because P2 is hidden by A2.P2. It
> seems unnatural to me.
>
> Errors of such kind are very difficult to find.
>
> Is there any way to prevent such errors? (Besides keyword
> *overriding*)

This happened to us frequently when we were building Claw. That's the reason 
I pushed so hard to add what became the keywords to the language. It's an 
error that cannot be avoided or detected in Ada 95 (short of avoiding any 
declarations in private parts, which is nasty). It is very hard to predict 
what really is going to happen, so the keywords allow telling the compiler 
what you meant (and then it will complain if it disagrees).

If you can't use new Ada features, you are SOL. Sorry.

                                               Randy.