Why is memory footprint smaller when compiled static?

Why is memory footprint smaller when compiled static?

[Preben Randhol]

I compiled my application (a GtkAda program) on Linux first as:

   gnatmake glosa -O3 `gtkada-config -static` -bargs -static

then as

   gnatmake glosa -O3 `gtkada-config`

I stripped both executables afterwards (as I forgot the -s flag)

The executable sizes becomes:

   2583572 bytes for the static version
    752060 bytes for the dynamic version

When I run the two programs and examine the memory usage of the two
programs I find :

   3296 kb for the static version
   4412 kb for the dynamic version

Why is there such a big difference? Is it because the compiler can do
better job at optimising the code when you build it statically.

I also did the same without the -O3 switch and then the memory usage
was:

   3412 kb for the static version
   4512 kb for the dynamic version


Preben who knows very little on how compilers work

Re: Why is memory footprint smaller when compiled static?

[Mark Johnson]

Preben Randhol wrote:
> [snip - static bigger on disk]
> When I run the two programs and examine the memory usage of the two
> programs I find :
> 
>    3296 kb for the static version
>    4412 kb for the dynamic version
> 
> Why is there such a big difference? Is it because the compiler can do
> better job at optimising the code when you build it statically.
> 
Hmm. I doubt the compiler (gcc) has anything to do with this. The linker
(ld) is the likely culprit and you should see two effects...
 - each static program has a "smaller footprint" because functions and
procedures in the library that are not used can be omitted from the
generated main program. However,...
 - your overall memory usage of the system can be higher when you run
several of these statically linked programs because they do not share
the run time library.
The second effect can be a pretty substantial by the way. I have seen
some systems implemented with a "server" and a separate main program for
each "display". With statically linked main programs, the memory usage
of the system is unacceptable when you get 5-10 displays up. With shared
libraries, the memory usage is OK. YMMV.

> I also did the same without the -O3 switch and then the memory usage
> was:
> 
>    3412 kb for the static version
>    4512 kb for the dynamic version
> 
It is somewhat interesting that the dynamic version went up by 100 kb
and the static version went up by 116 kb, but you might have just gotten
"lucky" with an extra segment or two going up by a 4k page in the static
version. Check the link map (something like - gnatmake ... -largs
-Wl,-Map,glosa.map) if you want to get more insight in how the linker is
putting the application together.
  --Mark

Re: Why is memory footprint smaller when compiled static?

[Florian Weimer]

Preben Randhol <randhol+abuse@pvv.org> writes:

> Why is there such a big difference?

Page granularity?

Re: Why is memory footprint smaller when compiled static?

[Florian Weimer]

Preben Randhol <randhol+abuse@pvv.org> writes:

> Why is there such a big difference?

Page granularity?  Space taken by the GOT?

Re: Why is memory footprint smaller when compiled static?

[Florian Weimer]

Preben Randhol <randhol+abuse@pvv.org> writes:

> Why is there such a big difference?

Page granularity?  Space taken by the GOT?  Overhead of
position-independent code?

Re: Why is memory footprint smaller when compiled static?

[Preben Randhol]

On Tue, 04 Jun 2002 23:03:34 +0200, Florian Weimer wrote:
> Preben Randhol <randhol+abuse@pvv.org> writes:
> 
>> Why is there such a big difference?
> 
> Page granularity?  Space taken by the GOT?  Overhead of
> position-independent code?

Can one have page granularity resulting in 1 Mb difference? The two
others I don't understand what is.

Preben

Re: Why is memory footprint smaller when compiled static?

[Grein, Christoph]

> > 
> >> Why is there such a big difference?
> > 
> > Page granularity?  Space taken by the GOT?  Overhead of
> > position-independent code?
> 
> Can one have page granularity resulting in 1 Mb difference? The two
> others I don't understand what is.

This is a qustion that appears every now and then. In my eyes, it's rather 
pointless to argue about Hello_World size. Real programs matter, and there the 
differences are minor if programs are really comparable (proper setting of 
options).

The difference in size normally can be attributed to differences in the IO parts 
of C and Ada: Ada does many more things than C, so IO is not comparable, and IO 
is all that matters in Hello_World.

Re: Why is memory footprint smaller when compiled static?

[Preben Randhol]

On Wed, 5 Jun 2002 13:37:36 +0200 (MET DST), Grein, Christoph wrote:
> 
> This is a qustion that appears every now and then. In my eyes, it's rather 
> pointless to argue about Hello_World size. Real programs matter, and
> there the differences are minor if programs are really comparable
> (proper setting of options).
> 
> The difference in size normally can be attributed to differences in
> the IO parts of C and Ada: Ada does many more things than C, so IO is
> not comparable, and IO is all that matters in Hello_World.

What on earth are you talking about? Why are you comparing C to Ada. I
haven't made any C code nor am I comparing C code. Neither is my program
a mere Hello_World program (ref : http://www.pvv.org/~randhol/Ada95/Glosa/).

I was only curious why the memory usage seems to drop 1 Mb after
compiling the program as static rather than shared. I don't care so much
how much memory my program takes, I just wanted to learn something about
this.

Preben Randhol

Re: Why is memory footprint smaller when compiled static?

[Mark Johnson]

Preben Randhol wrote:
> I was only curious why the memory usage seems to drop 1 Mb after
> compiling the program as static rather than shared. I don't care so much
> how much memory my program takes, I just wanted to learn something about
> this.
> 
Did you review the link map as I suggested in a previous reply?
  news:3CFCE726.38B15035%40raytheon.com
That should give you the information you need on what is included /
excluded from the static and dynamically linked application.
  --Mark

RE: Why is memory footprint smaller when compiled static?

[Steven Deller]

Preben,
When you link with a shared library, everything in that shared library
is loaded when it is loaded.  Everything, not just what you need.

When you link statically, just what you need/use is included.  One
caveat -- libraries, whether C or Ada, come in chunks (called object
files) that often include many routines, so you get all of them even
when you just want one.  Normally linkers can't do "dead code
elimination", though some are getting almost smart enough to do that.

In your case, there are lots of parts of I/O and Ada that you are not
using, so when you link statically, much of the library's contents are
not included in the executable.

One other caveat -- just looking at an executable, even a stripped
executable, does NOT tell you how much code you have.  The executable
file includes "text" which is read-only instructions and data, "data"
which is initialized read-write data, "bss" which defines how much
uninitialized data space the program has, as well as original detailed
object file symbols, debugging symbol information, and loading symbol
table information.  If you "strip" the file, that removes the debugging
and object file symbols.

There is also overhead in an executable file that defines the structure
of the file, but usually that is under 15KB.

Check the program "size" to look in more detail about the parts that
comprise an executable file.

Even after stripping there can be significant symbol table information
included, needed for relocatable loading on most machines.  That can
make the executable file much larger than what eventually runs in
memory.  On the other hand, "bss" only takes a few words to "reserve"
all data space -- if you have huge arrays that are statically defined,
but aren't initialized, then the image in memory can be much larger than
the file.

Finally, during execution, programs use stack space (usually not much)
and heap space (can often be a lot).  On the embedded system project I
am working on, file sizes reach 33MB or more, while text, data, and bss
only sum to about 14MB.  Even after stripping, the file size is about
19MB.  I have a super-strip that strips the loading symbol table as well
(since the target loader does not use that), and I get a 13MB file
image.  When the program runs, it takes about 24MB because it uses
almost 10MB of heap.

Regards,
Steve

> -----Original Message-----
> From: comp.lang.ada-admin@ada.eu.org 
> [mailto:comp.lang.ada-admin@ada.eu.org] On Behalf Of Preben Randhol
> Sent: Wednesday, June 05, 2002 7:37 AM
> To: comp.lang.ada@ada.eu.org
> Subject: Re: Why is memory footprint smaller when compiled static?

I was only curious why the memory usage seems to drop 1 Mb after
compiling the program as static rather than shared. I don't care so much
how much memory my program takes, I just wanted to learn something about
this.

Preben Randhol
_______________________________________________
comp.lang.ada mailing list
comp.lang.ada@ada.eu.org
http://ada.eu.org/mailman/listinfo/comp.lang.ada

Re: Why is memory footprint smaller when compiled static?

[Preben Randhol]

On Wed, 5 Jun 2002 19:02:30 -0500, Steven Deller wrote:
> Preben,
> When you link with a shared library, everything in that shared library
> is loaded when it is loaded.  Everything, not just what you need.

Ah I see. Thanks!

> When you link statically, just what you need/use is included.  One
> caveat -- libraries, whether C or Ada, come in chunks (called object
> files) that often include many routines, so you get all of them even
> when you just want one.  Normally linkers can't do "dead code
> elimination", though some are getting almost smart enough to do that.

Ok.

> In your case, there are lots of parts of I/O and Ada that you are not
> using, so when you link statically, much of the library's contents are
> not included in the executable.

Yes. Ok now I understand and I learned something new :-)

Again thanks.

Preben Randhol

Re: Why is memory footprint smaller when compiled static?

[Gautier]

Steven Deller:

# Normally linkers can't do "dead code elimination", though some are
# getting almost smart enough to do that.

May I correct ? I'd write:

  Normally linkers can do "dead code elimination", though some are
  not yet smart enough to do that.

:-)

________________________________________________________
Gautier  --  http://www.mysunrise.ch/users/gdm/gsoft.htm

NB: For a direct answer, address on the Web site!

Re: Why is memory footprint smaller when compiled static?

[Jean-Marc Bourguet]

gautier_niouzes@hotmail.com (Gautier) writes:

> Steven Deller:
> 
> # Normally linkers can't do "dead code elimination", though some are
> # getting almost smart enough to do that.
> 
> May I correct ? I'd write:
> 
>   Normally linkers can do "dead code elimination", though some are
>   not yet smart enough to do that.

The possibility of doing some "dead code elimination" at the linker
level depend also on the object file format and the way it is used by
the compiler.

-- 
Jean-Marc

Re: Why is memory footprint smaller when compiled static?

[Jean-Pierre Rosen]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 611 bytes --]


"Jean-Marc Bourguet" <jm@bourguet.org> a �crit dans le message news: 3cff7ae7@news.cadence.com...
> The possibility of doing some "dead code elimination" at the linker
> level depend also on the object file format and the way it is used by
> the compiler.
>
Back in 1978 (circa) I wrote a linker. It was able to do dead code elimination, as well as to resolve all references irrespectively
of the order of the modules given on the command line. Sigh....

--
---------------------------------------------------------
           J-P. Rosen (rosen@adalog.fr)
Visit Adalog's web site at http://www.adalog.fr

Re: Why is memory footprint smaller when compiled static?

[Jeffrey Carter]

Jean-Pierre Rosen wrote:
> 
> Back in 1978 (circa) I wrote a linker. It was able to do dead code elimination, as well as to resolve all references irrespectively
> of the order of the modules given on the command line. Sigh....

Yes, and I used to use a DOS-extender Ada-83 compiler from RR S/W that
did dead-code elimination in the 1980s. Also in the 1980s Tartan made an
Ada-83 compiler that produced faster code than their highly optimizing C
compilers. In 1969, the US was able to send people to the moon. In 1964,
the US could build an aircraft that could fly at Mach 4+ at 80,000+ ft.
But as we all know you can't do those things anymore.

-- 
Jeff Carter
"I fart in your general direction."
Monty Python & the Holy Grail

Re: Why is memory footprint smaller when compiled static?

[Randy Brukardt]

Steven Deller wrote in message ...
>When you link statically, just what you need/use is included.  One
>caveat -- libraries, whether C or Ada, come in chunks (called object
>files) that often include many routines, so you get all of them even
>when you just want one.  Normally linkers can't do "dead code
>elimination", though some are getting almost smart enough to do that.


Come now. Janus/Ada's linker has eliminated unused subprograms pretty
much since it was created in the mid-80s. I doubt very much that we were
15 years ahead of the curve there... :-)

I realize that a lot of systems don't do this, and I've never understood
why: it isn't particularly hard nor very expensive, and it can save a
lot of space (it saves nearly 1 megabyte of code size in the Claw
Builder, for example).

           Randy Brukardt

Re: Why is memory footprint smaller when compiled static?

[Robert A Duff]

"Randy Brukardt" <randy@rrsoftware.com> writes:

> Come now. Janus/Ada's linker has eliminated unused subprograms pretty
> much since it was created in the mid-80s. I doubt very much that we were
> 15 years ahead of the curve there... :-)
> 
> I realize that a lot of systems don't do this, and I've never understood
> why: it isn't particularly hard nor very expensive, and it can save a
> lot of space (it saves nearly 1 megabyte of code size in the Claw
> Builder, for example).

I agree -- linkers ought to do this.

But it's harder to do a good job with indirect calls (dispatching),
which you didn't have to worry about in Ada 83.

- Bob

Re: Why is memory footprint smaller when compiled static?

[Wes Groleau]

> > Come now. Janus/Ada's linker has eliminated unused subprograms pretty
> >
> > I realize that a lot of systems don't do this, and I've never understood
> > why: it isn't particularly hard nor very expensive, and it can save a
> 
> I agree -- linkers ought to do this.
> 
> But it's harder to do a good job with indirect calls (dispatching),

Or a package with'ed just so a task in it's body will be activated,
or to force implementation-specific code to be included in the RTS.

Plus, in Ada 95, intelligent use of child units can eliminate
most unused code from being linked.

-- 
Wes Groleau
http://freepages.rootsweb.com/~wgroleau

Re: Why is memory footprint smaller when compiled static?

[Preben Randhol]

Wes Groleau <wesgroleau@despammed.com> wrote on 11/06/2002 (18:52) :
> Plus, in Ada 95, intelligent use of child units can eliminate
> most unused code from being linked.

Can you give an example of this? It would be nice to learn. I probably
haven't done this so intelligently in my program :-)

-- 
Preben Randhol ------------------- http://www.pvv.org/~randhol/ --
                 �For me, Ada95 puts back the joy in programming.�

Re: Why is memory footprint smaller when compiled static?

[Wes Groleau]

> > Plus, in Ada 95, intelligent use of child units can eliminate
> > most unused code from being linked.
> 
> Can you give an example of this? It would be nice to learn. I probably
> haven't done this so intelligently in my program :-)

package Neato_Data_Type is

  type Object is ......

end Neato_Data_Type;


package Neato_Data_Type.Display is

  procedure Draw (Item : Object);

  procedure Highlight (Item : Object);

  ....

end Neato_Data_Type.Display;

----------------

with X11R6_Bindings;
package body Neato_Data_Type.Display is ....

----------------

package Neato_Data_Type.Storage is

   procedure Retrieve ( Item : Object;
                        Key  : String );

   procedure Store    ( Item : Object;
                        Key  : String );

   ....

end Neato_Data_Type.Storage;

----------------

with ODBC_Bindings;
package body Neato_Data_Type.Storage is ....

----------------

package Neato_Data_Type.Some_Other_Feature_Set is

   .....

end Neato_Data_Type.Some_Other_Feature_Set;

------------------

with Five_Gigabyte_Math_Library;
package body Neato_Data_Type.Some_Other_Feature_Set is ....

-------------

with Neato_Data_Type.Display;
procedure Display_Manager is
 ....
end Display_Manager;

-------------

with Neato_Data_Type.Storage;
procedure Database_Manager is
 ....
end Database_Manager;

-------------

with Neato_Data_Type.Some_Other_Feature_Set;
procedure Do_Things is
 ....
end Do_Things;

---=================

If the last three guys are main programs, then
Database_Manager does not link in the ODBC bindings
or the humongous math package.  Do_Things doesn't
link in the ODBC or X11 bindings.    etc.

In Ada 83, you could make them all independent packages,
but then you had to either give up private types or
write a heck of a lot of support code into the Neato_Data_Type
package.

-- 
Wes Groleau
http://freepages.rootsweb.com/~wgroleau

Re: Why is memory footprint smaller when compiled static?

[Randy Brukardt]

Robert A Duff wrote in message ...
>> I realize that a lot of systems don't do this, and I've never
understood
>> why: it isn't particularly hard nor very expensive, and it can save a
>> lot of space (it saves nearly 1 megabyte of code size in the Claw
>> Builder, for example).
>
>I agree -- linkers ought to do this.
>
>But it's harder to do a good job with indirect calls (dispatching),
>which you didn't have to worry about in Ada 83.


Not much harder. Janus/Ada 3.1.2 (unreleased as of now because of
optimization problems unrelated to trimming) has done this for more than
a year. It is very valuable with Claw. It does increase the amount of
stuff that has to be saved for the linker (an indication of dispatching
calls with their tag slot numbers), but it actually turned out to be
much easier to implement than I expected (took less than a week).

                Randy.

Re: Why is memory footprint smaller when compiled static?

[Randy Brukardt]

Wes Groleau wrote in message <3D064BAB.4040168B@despammed.com>...
>
>> > Plus, in Ada 95, intelligent use of child units can eliminate
>> > most unused code from being linked.
>>
>> Can you give an example of this? It would be nice to learn. I
probably
>> haven't done this so intelligently in my program :-)
>
>... (examples omitted)

>In Ada 83, you could make them all independent packages,
>but then you had to either give up private types or
>write a heck of a lot of support code into the Neato_Data_Type
>package.

But the problems reappear if these are intended to be extensible types
with overridable operations.

Claw is designed much as you suggest, so that you don't drag in control
and window types that you don't use. But you do drag in all of the
functionality for control and window types that you do use, as
everything has to be declared in a single package to support
extensibility and overridability. That means that unused subprogram
elimination is very valuable.

(Its also useful when using the standard Ada libraries, which are not
factored very well. The original reason we implemented it was the huge
size (by 1985 standards) of Text_IO.)

                 Randy.