Re: Use of "is separate" and compilatio

Re: Use of "is separate" and compilatio

[stt]

We have built a number of very large Ada systems at Intermetrics,
and our customers have built some even bigger ones.
As a result of this we have built up a very strong
prejudice *against* the use of subunits (aka "is separates").
Instead, we try to structure our system as a series
of medium size packages (about 500-2500 lines each),
using either a functional or object-oriented structuring.

Ideally, each of these packages represents a well-defined,
relatively self-contained component of the system.
When making a change or trying to understand a particular
feature of the system, it is generally only necessary to
look at the single medium-size package which implements it.
Furthermore, most of the candidates for low-level, small-chunk-of-code
reuse, are within a single package.

Subunits foul up this structure in a number of ways:
First and foremost, a single subunit is in no way "self-contained."
It has unlimited visibility into its parent unit, representing
a very "broad" interface (the worst kind).

Secondly, when editing a subunit, other related code is
not immediately available, so we have found that programmers
working on a heavily "subunited" program tend to reinvent the
wheel repeatedly, rather than noticing a similar sequence
of code in another subprogram of the same package, and
separating off that logic as a local shared subprogram
of the package.

Thirdly, subunits cause havoc with sophisticated optimizers,
especially subunits within subprograms.  When calling a separately-compiled
subunit the compiler must assume the worst: that the subunit
accesses and/or updates all objects it can see, meaning that
all local variables visible to the subunit must be stored back
into memory from registers before the call, and reloaded afterward.

Finally, they tend to allow single library units to grow
without bound.  Often when a subsystem grows too big, it
is wise to split it up into a number of packages, each with
a narrower purpose.  When using subunits, the tendency
is to instead just keep adding more and more functionality
into a single package.

As far as recompilation, subunits are also bad news, since
almost all compilers will recompile *all* subunits when
any change is made to the enclosing parent unit, including
just the addition of another subunit.  Of course, the total
amount of code is no greater than if the package had not
been "subunited" (presuming you resisted the growth-without-bound
phenomenon), but most compilers will compile a single 2500-line
package much faster than 50 50-line subunits.
Of course you win when the only change to be made was in a single
subunit, but in our experience, that incentive causes many
of the problems with subunits -- the avoidance of restructuring
of the package.  Many bugs can best be fixed by modest
restructuring of the implementation of a package.  
Any attempt at restructuring in the presence of subunits
causes them all to be recompiled again.

I should mention that reasonable people differ
on this whole "subunit" issue, and there are
selected places where a subunit can be extremely useful,
to, for instance, isolate a very small bit of host-dependent
code.  However, on balance, I would regard them almost
as bad as a goto -- to be used only when all other 
avenues have been exhausted.

As Lori Clarke, et al, said in their classic critique of
Ada subunits long ago, "nesting in Ada is for the birds."

S. Tucker Taft
Intermetrics, Inc.
Cambridge, MA  02138

Re: Use of "is separate" and compilatio

[Ed Matthews]

I pretty much agree with Tucker.  I used to really like separate subunits,
but now that I have switched to the compiler development side of the
world, it's easy to see that separate subunits is one of those issues
that gives compiler and optimizer writers nightmares.

In article <20600021@inmet> stt@inmet.inmet.com writes:

>When making a change or trying to understand a particular
>feature of the system, it is generally only necessary to
>look at the single medium-size package which implements it.

On the whole, Tuck makes some persuasive arguments, but I disagree here.
This argument is an artifact of the editors of the past.  With modern
editor technology and windowing, the entirety of a package, whether it
be one package of 2500 lines, or 25 subunits of 100 lines, can be 
easily viewed.  Further, in editors with syntactic and semantic
navigation, understanding a unit is equally easy no matter how the
program is structured.

>Furthermore, most of the candidates for low-level, small-chunk-of-code
>reuse, are within a single package.

I agree here -- reuse at the small-chunk-of-code level may be obscured
by subunits.  This is another another topic, but I wonder how much value
small-chunk-of-code reuse represents on the reuse continuum.  I do a lot
of reuse at this level.

>Thirdly, subunits cause havoc with sophisticated optimizers,
>especially subunits within subprograms.  When calling a separately-compiled
>subunit the compiler must assume the worst: that the subunit
>accesses and/or updates all objects it can see, meaning that
>all local variables visible to the subunit must be stored back
>into memory from registers before the call, and reloaded afterward.

Furthermore, the compiler has to assume that the subunit may contain tasks
and then it has to generate code to check for task termination etc.

Two guidelines to think about when using separate subunits:

1. Separate subunits should be declared in package bodies, not in
subprograms.

2. Nest subunits only one layer deep.  A reasonable relaxation might
allow separate package bodies inside separate package bodies.
-- 

Ed Matthews                                                edm@verdix.com
Verdix Corporation Headquarters                            (703) 378-7600
Chantilly, Virginia

Re: Use of "is separate" and compilatio

[arny.b.engelson]

In article <20600021@inmet> stt@inmet.inmet.com writes:
>We have built a number of very large Ada systems at Intermetrics,
>and our customers have built some even bigger ones.
>As a result of this we have built up a very strong
>prejudice *against* the use of subunits (aka "is separates").
>Instead, we try to structure our system as a series
>of medium size packages (about 500-2500 lines each),
>using either a functional or object-oriented structuring.
>
>Ideally, each of these packages represents a well-defined,
>relatively self-contained component of the system.
>When making a change or trying to understand a particular
>feature of the system, it is generally only necessary to
>look at the single medium-size package which implements it.
>Furthermore, most of the candidates for low-level, small-chunk-of-code
>reuse, are within a single package.

My personal experience has been working mostly on somewhat smaller
efforts (single person, 20000 - 30000 lines of code programs), and
I don't agree with this philosophy for those types of programs.

Generally, I have several "medium" size packages, but with most of
these having a few small (about 50-200 lines) subunits.  Each subunit
implements a single requirement (for lack of a better term).  Usually,
when a change is required, it is to a single subunit (not always, but
usually).  I believe this is due to a combination of the nature of the
application and a good design/program structure on my part.  Occasionally
it is necessary to restructure the parent package, but not often.

>Subunits foul up this structure in a number of ways:
>First and foremost, a single subunit is in no way "self-contained."
>It has unlimited visibility into its parent unit, representing
>a very "broad" interface (the worst kind).

The visibility of the subunit doesn't change by integrating it into
the parent.  If proper scoping of declarations is done (i.e. only what
really should be visible is visible), I don't see the problem.

>Secondly, when editing a subunit, other related code is
>not immediately available, so we have found that programmers
>working on a heavily "subunited" program tend to reinvent the
>wheel repeatedly, rather than noticing a similar sequence
>of code in another subprogram of the same package, and
>separating off that logic as a local shared subprogram
>of the package.

Isn't this a problem of the programmers not having enough knowledge of
the program, rather than a problem of the program structure itself?
If the programmer really knows the existing code, s/he will know enough
not to reinvent the wheel.  In my case, since I developed the entire
program, I know when I can reuse an existing piece of code.  Maybe your
anti-subunits stance should be limited to very large systems with
many programmers working on it.

>Thirdly, subunits cause havoc with sophisticated optimizers,
>especially subunits within subprograms.  When calling a separately-compiled
>subunit the compiler must assume the worst: that the subunit
>accesses and/or updates all objects it can see, meaning that
>all local variables visible to the subunit must be stored back
>into memory from registers before the call, and reloaded afterward.

Again, if proper scope is maintained for variables, there should be very
few that to which this would apply.

>Finally, they tend to allow single library units to grow
>without bound.  Often when a subsystem grows too big, it
>is wise to split it up into a number of packages, each with
>a narrower purpose.  When using subunits, the tendency
>is to instead just keep adding more and more functionality
>into a single package.

This is a very real risk, but is again a fault of the programmer.  A
good programmer should recognize that a subsystem needs to be split
regardless of whether there are subunits; a less qualified programmer
will let a single package grow to several thousand lines, containing
many unrelated subprograms, regardless of whether or not subunits are
employed.

>As far as recompilation, subunits are also bad news, since
>almost all compilers will recompile *all* subunits when
>any change is made to the enclosing parent unit, including

One of my goals in using subunits is to reduce the scope of changes.
Fewer changes should involve the parent.

>just the addition of another subunit.  Of course, the total
>amount of code is no greater than if the package had not
>been "subunited" (presuming you resisted the growth-without-bound

In other words (my words), the amount of code recompiled is the same,
and there is no real difference.

>phenomenon), but most compilers will compile a single 2500-line
>package much faster than 50 50-line subunits.

So what?  I don't think this amount of reduced compilation time is a
strong argument for changing your program structure.  But, if the change
is only to one subunit, compilation time can be reduced substantially.
This has actually saved me a very substantial amount of time when adding
a new feature as a subunit.  I can repeatedly recompile and debug this
very quickly.

>Of course you win when the only change to be made was in a single
>subunit, but in our experience, that incentive causes many
>of the problems with subunits -- the avoidance of restructuring
>of the package.  Many bugs can best be fixed by modest
>restructuring of the implementation of a package.  

Maybe and maybe not -- it depends on the bug.  As far as avoiding the
restructuring of a package because of subunits, I think this is again
a fault of the programmer.  I restructure when necessary, what's the
big deal?  Changing the documentation may be a different story, but that
depends on your documentation, and how it is maintained.

>Any attempt at restructuring in the presence of subunits
>causes them all to be recompiled again.

Even if they weren't subunits they would be getting recompiled anyway.
It's easier to see what areas are untouched by a change when the code
is in different files.

>I should mention that reasonable people differ
>on this whole "subunit" issue, and there are
>selected places where a subunit can be extremely useful,
>to, for instance, isolate a very small bit of host-dependent
>code.  However, on balance, I would regard them almost
>as bad as a goto -- to be used only when all other 
>avenues have been exhausted.

I think this view is a bit extreme, and its validity varies with each
type of application and with the size of the application.  Besides the
arguments I presented above, as a result of my work environment I have
saved a lot of time by using subunits.  I have saved time on recompilation
(on my sometimes overloaded system), on disk space (which we are always
running out of), on printouts (they are smaller), and on debugging time
(in many cases).  Your mileage may (rather, will) vary.

>As Lori Clarke, et al, said in their classic critique of
>Ada subunits long ago, "nesting in Ada is for the birds."

Tweet, tweet.  :-)

>S. Tucker Taft
>Intermetrics, Inc.
>Cambridge, MA  02138

Arny Engelson
AT&T Bell Laboratories
Whippany, NJ 07981
att!wayback!arny

Re: Use of "is separate" and compilatio

[Leonard Vanek]

In article <3076@cbnewsl.ATT.COM> arny@cbnewsl.ATT.COM (arny.b.engelson,wh,) writes:
>In article <20600021@inmet> stt@inmet.inmet.com writes:

>The visibility of the subunit doesn't change by integrating it into
>the parent.  If proper scoping of declarations is done (i.e. only what
>really should be visible is visible), I don't see the problem.

>>     subunits cause havoc with sophisticated optimizers,
>>especially subunits within subprograms.
>
>Again, if proper scope is maintained for variables, there should be very
>few that to which this would apply.

If "proper name scoping" is used to avoid the potential pitfalls of
subunits, how does the subunit differ substantially from a package?
I think that the alternative to subunits in this case is not to place
them in-line in the parent procedure or package, but to make them
packages in their own right. Why do we need to use the "is separate"
mechanism?
--------------------------------------------------------------------
Leonard Vanek                  UUCP: ... uunet!attcan!lsuc!array!len
Array Systems Computing Inc.    or ... utzoo!dciem!array!len
5000 Dufferin St. Suite 200     or lsuc!array!len@ai.toronto.edu
Downsview, Ont. M3H 5T5        Phone: (416) 736-0900
Canada                         FAX:   (416) 736-4715

Re: Use of "is separate" and compilatio

[Michael Schwartz]

> why do we need 'is separate'

This is just another tool to break up long files (and thus reduce
the burden of changes for recompilation).

Imagine a package with a task embedded in the body (easy to
imagine).  Imagine the task is substantial.  The task spec can
appear in the body, while the task body 'is separate'.
-- 
-----------------------
schwartz@pogo.den.mmc.com "Expect everything ... 
mschwartz@mmc.com                   and the unexpected never happens." 
ncar!dinl!schwartz                               --the phantom tollbooth

DISCLAIMER:  The opinions expressesed are not necessarily those of my 
	     employer or myself.

Re: Use of "is separate" and compilatio

[Mike Walsh]

In article <1501@dinl.mmc.UUCP> schwartz@dinl.UUCP (Michael Schwartz) writes:
>
>> why do we need 'is separate'
>
'is seperate' is useful for extracting certain procedures from a package.
I once wrote a package that had a classified algorithm in it.  By making the
one procedure separate, I could print out the file, and not have to worry
about having a classified document laying around.  Just an example ...


Mike Walsh
Star Technologies - Graphicon Products Division
Research Triangle Park, NC

walsh@stdc01    {...uunet!mcnc!rti!stdc01!walsh}

Re: Use of "is separate" and compilatio

[Fred Stluka]

In article <60@array.UUCP> len@array.UUCP (Leonard Vanek) writes:
> If "proper name scoping" is used to avoid the potential pitfalls of
> subunits, how does the subunit differ substantially from a package?
> I think that the alternative to subunits in this case is not to place
> them in-line in the parent procedure or package, but to make them
> packages in their own right. Why do we need to use the "is separate"
> mechanism?


Subunits are similar to library units in being allowed to have "with"
clauses.

However, they are different in not being able to be "with"ed by other 
units.  They are visible only to their parents.

Subunits are the only choice when you want to only a small portion of 
a large package to have visibility into other library units without 
allowing other units to have visibility into the small portion.  

--Fred
--
Fred Stluka                          CSNET:   stluka@software.org
Software Productivity Consortium     ARPANET: stluka%software.org@RELAY.CS.NET
2214 Rock Hill Rd, Herndon VA 22070  UUNET:   ...!uunet!sunny!stluka

Re: Use of "is separate" and compilatio

[stt]

Ed Matthews (edm@verdix.com) points out that fancy editors
can improve the situation w.r.t. subunits:
> 
> . . . With modern
> editor technology and windowing, the entirety of a package, whether it
> be one package of 2500 lines, or 25 subunits of 100 lines, can be 
> easily viewed.  Further, in editors with syntactic and semantic
> navigation, understanding a unit is equally easy no matter how the
> program is structured.
> 
I can't argue with this since I haven't got one of these fancy editors!
("Tags" with vi helps a little, but has real trouble with overloading
and the multi-level namespace.)

In any case, even though I do have a multi-window workstation on my desk,
I still find I like to look at good old hard-copy listings
now and then, and hyper-text just hasn't arrived on our line printer
(nor even our laser printer!) yet.  Lots of subunits make any
kind of paper and pencil work a royal pain.

S. Tucker Taft
Intermetrics, Inc.
Cambridge, MA  02138