Typical handling of packages through compilers

Typical handling of packages through compilers

[Hibou57 (Yannick Duchêne)]

Hi all,

I've noticed that the inclusion of a package, turns into the inclusion
of all of its content, despite just a few part of it is really used
(taking into account, internal dependencies). Using seperate data and
function sections and then the linker gc-section option, does not
change anything (the executable is even a bit bigger using these
options).

There is indeed no implementation requirement about it.

But I would like to know more on the handling of package through
various Ada compilers (including Janus if possible, if there are some
Janus users in the place).

By the way, on most compilers, is an inner package handled the same
way a child package is ? I mean... child packages are included only
when withed, but what about inner packages ? Are they tipically
included in whole as there are part of another package body or are
they tipically included only when explicitely referenced (althought
inner packages does not need to be withed) ?

Thanks for feed-backs


P.S. I know GNAT is very famous and widely used, but as it is also
very expensive (for most of people), it would be nice to have feed-
back about other compilers, not only GNAT.

Re: Typical handling of packages through compilers

[Gautier write-only]

Yannick:

> I've noticed that the inclusion of a package, turns into the inclusion
> of all of its content, despite just a few part of it is really used
> (taking into account, internal dependencies).

It's the lack of a proper smart linking in the GNU linker. This
technique is in use since more than 20 years in other systems - e.g.:
Turbo Pascal and, later, Delphi. It seems also that Janus has it (or
at least an old version had it).

> Using seperate data and
> function sections and then the linker gc-section option, does not
> change anything (the executable is even a bit bigger using these
> options).

Try with -ffunction-sections only. With separate data sections I get a
gain on most units, but unexpectedly one package turns into an
enormous .o . The GNU docs warns about these options:
"Only use these options when there are significant benefits from doing
so. When you specify these options, the assembler and linker will
create larger object and executable files and will also be slower. You
will not be able to use gprof on all systems if you specify this
option and you may have problems with debugging if you specify both
this option and -g."

You also have the gnatelim tool (a very dumb workaround, but it
works).

HTH
_________________________________________________________
Gautier's Ada programming -- http://sf.net/users/gdemont/
NB: For a direct answer, e-mail address on the Web site!

Re: Typical handling of packages through compilers

[Hibou57 (Yannick Duchêne)]

On 22 juil, 14:57, Gautier write-only <gautier_niou...@hotmail.com>
wrote:
> It's the lack of a proper smart linking in the GNU linker. This
> technique is in use since more than 20 years in other systems - e.g.:
> Turbo Pascal and, later, Delphi. It seems also that Janus has it (or
> at least an old version had it).
Oh, great! Thanks for the notice Gautier

(yes, the smart linking capability of TurboPascal was very efficient)

Re: Typical handling of packages through compilers

[Adam Beneschan]

On Jul 22, 5:21 am, Hibou57 (Yannick Duchêne)
<yannick_duch...@yahoo.fr> wrote:
> Hi all,
>
> I've noticed that the inclusion of a package, turns into the inclusion
> of all of its content, despite just a few part of it is really used
> (taking into account, internal dependencies). Using seperate data and
> function sections and then the linker gc-section option, does not
> change anything (the executable is even a bit bigger using these
> options).
>
> There is indeed no implementation requirement about it.
>
> But I would like to know more on the handling of package through
> various Ada compilers (including Janus if possible, if there are some
> Janus users in the place).
>
> By the way, on most compilers, is an inner package handled the same
> way a child package is ? I mean... child packages are included only
> when withed, but what about inner packages ? Are they tipically
> included in whole as there are part of another package body or are
> they tipically included only when explicitely referenced (althought
> inner packages does not need to be withed) ?

I think that typically, when a single source file is compiled, the
result is a single object file; and for many linkers, either that
object file gets included in the final executable or it doesn't, in an
all-or-nothing fashion.

One of our compilers (for a particular target processor) comes with a
linker that is able to eliminate unused sections.  The code for each
procedure, function, or other routine generated by the compiler is in
a separate section, and the linker doesn't include sections that
aren't referenced.  So if you have a procedure Proc1 in a package, but
it's never called (or referenced via 'Access, etc.) anywhere in the
program, then Proc1 won't be included in the executable.  Furthermore,
if Proc1 is the only procedure in the program that calls Proc2, then
if Proc1 isn't included then Proc2 won't be included either.  I think
the same applies to global variables---if there are no references to a
variable then by default it never gets allocated.  I'm not sure if
this is similar to what Gautier was referring to.  I know there are
other linkers out there that do a similar optimization (e.g. Analog
Devices' linker for 21000 series).

By the way, "inner packages" don't have any effect at all on this, and
I'd guess that that is true for other compilers as well.  Inner
packages are not at all reflected in the structure of the object
file.  Our linker considers each code or data item individually to see
if it can be eliminated; it doesn't care at all about whether they've
been grouped into another package.

                                           -- Adam

Re: Typical handling of packages through compilers

[tmoran]

> It's the lack of a proper smart linking in the GNU linker. This
> technique is in use since more than 20 years in other systems - e.g.:
> Turbo Pascal and, later, Delphi. It seems also that Janus has it (or
> at least an old version had it).
  Janus has smart linking.  Janus Ada started life on DOS machines, where
size mattered.  But Randy can tell you more.

Re: Typical handling of packages through compilers

[Hibou57 (Yannick Duchêne)]

On 22 juil, 16:29, Adam Beneschan <a...@irvine.com> wrote:
> By the way, "inner packages" don't have any effect at all on this, and
> I'd guess that that is true for other compilers as well.  Inner
> packages are not at all reflected in the structure of the object
> file.  Our linker considers each code or data item individually to see
> if it can be eliminated; it doesn't care at all about whether they've
> been grouped into another package.
>
>                                            -- Adam

So if I have

package P is
...
   package Q is
      ....
   end Q;
end P;

then with most compiler, Q will not be linked unless it is referenced

That's a good news

In the mean time, I know that a lot of Ada users don't bother about
plus or minus 150KB, so it's not a surprise if some compilers don't
bother about it.

Adam, are you a member of a compiler vendor team ?

Re: Typical handling of packages through compilers

[Hibou57 (Yannick Duchêne)]

On 22 juil, 22:06, tmo...@acm.org wrote:
>   Janus has smart linking.  Janus Ada started life on DOS machines, where
> size mattered.  But Randy can tell you more.

Yes, Randy and you are welcome to tell more about it. I already have
some french pages about GNAT, and if can get some technical details, I
would enjoy to write someting about Janus as well.

Re: Typical handling of packages through compilers

[Adam Beneschan]

On Jul 22, 2:25 pm, Hibou57 (Yannick Duchêne)
<yannick_duch...@yahoo.fr> wrote:
> On 22 juil, 16:29, Adam Beneschan <a...@irvine.com> wrote:
>
> > By the way, "inner packages" don't have any effect at all on this, and
> > I'd guess that that is true for other compilers as well.  Inner
> > packages are not at all reflected in the structure of the object
> > file.  Our linker considers each code or data item individually to see
> > if it can be eliminated; it doesn't care at all about whether they've
> > been grouped into another package.
>
> >                                            -- Adam
>
> So if I have
>
> package P is
> ...
>    package Q is
>       ....
>    end Q;
> end P;
>
> then with most compiler, Q will not be linked unless it is referenced

No, that isn't what I said, I think.  What I said is: some linkers
will link in all of package P (including P.Q) if any of it is
referenced; other linkers are able to include only those items in P
(including P.Q) that are needed.  For this purpose, though, the linker
won't know what items are in Q and what items aren't.  But I guess
this does mean that, for the second type of linker, if none of the
items in Q are referenced, then none of them will be linked in, but
this has nothing to do with whether they're inside Q or not.  The
linker will probably have no knowledge of "Q" as an entity or group of
entities.

> Adam, are you a member of a compiler vendor team ?

Yes.  www.irvine.com

                                   -- Adam

Re: Typical handling of packages through compilers

[Jeffrey Creem]

Hibou57 (Yannick Duch�ne) wrote:
> Hi all,
> 
> I've noticed that the inclusion of a package, turns into the inclusion
> of all of its content, despite just a few part of it is really used
> (taking into account, internal dependencies). Using seperate data and
> function sections and then the linker gc-section option, does not
> change anything (the executable is even a bit bigger using these
> options).
> 
> There is indeed no implementation requirement about it.
> 
> But I would like to know more on the handling of package through
> various Ada compilers (including Janus if possible, if there are some
> Janus users in the place).
> 
> By the way, on most compilers, is an inner package handled the same
> way a child package is ? I mean... child packages are included only
> when withed, but what about inner packages ? Are they tipically
> included in whole as there are part of another package body or are
> they tipically included only when explicitely referenced (althought
> inner packages does not need to be withed) ?
> 
> Thanks for feed-backs
> 
> 
> P.S. I know GNAT is very famous and widely used, but as it is also
> very expensive (for most of people), it would be nice to have feed-
> back about other compilers, not only GNAT.

A few data other non gnat data points

Most old Ada 83 Verdix (Then Rational) VADS compilers did smart linking 
(at least for vxWorks targets).

Greenhills Ada does it on some platforms (interestingly enough, does not 
  really seem to work for vxWorks targets but I also gave up a bit early)

Re: Typical handling of packages through compilers

[Randy Brukardt]

<tmoran@acm.org> wrote in message news:h47rgp$c9a$1@aioe.org...
>> It's the lack of a proper smart linking in the GNU linker. This
>> technique is in use since more than 20 years in other systems - e.g.:
>> Turbo Pascal and, later, Delphi. It seems also that Janus has it (or
>> at least an old version had it).
>  Janus has smart linking.  Janus Ada started life on DOS machines, where
> size mattered.  But Randy can tell you more.

There's not a lot more to say, it works rather as Adam had explained. Each 
top-level subprogram is included independently, depending on whether or not 
it is referenced. We do that recursively, so that subprograms that are 
referenced only by routines that are not referenced also are not included. 
We also have a mechanism for determining whether a routine present in a tag 
can actually be referenced from a dispatching call or not. These techniques 
can make a lot of difference in extreme cases: Claw example programs can be 
reduced by more than 500K in executable size.

                                   Randy.