Grouping data from different modules together

Grouping data from different modules together

[Andreas Hestermeyer]

We are designing software for small space borne control systems of scientific
experiments on spacecrafts. Because of the radiation environment, one
main goal is to keep the amount of memory needed, especially the ROM
memory, as small as possible.
  One possible contribution to small code is to group data from several
modules with the same functional semantics together in memory and let 
another, centralized, module use the data. Examples for such data groups 
are address lists of initialization functions, error counters, 
configuration data. 
  Whereas the data defining modules know the inidividual meaning of each
data item in a group, a system module may only know the structure and
basic semantics of each item and know nothing about the individual 
meaning. I.e. the system initialization code would run through the list
of initialization function addresses and call each initialization 
function. Or a reconfiguration module would take the block of 
configuration data and store it in a permanent powered RAM (without knowing
about each individual item). 
An error checking module could cyclically browse through the block 
of error counters and notify any changes in the outgoing datastream.

  So far for the basic concept. To be efficient, all items of a group
should be allocated an undivided space in memory. It is usually very
easy to define such GROUPS in assembler language and have the linker
taking care of grouping the data together. So each data item can be defined
in the source code file of the module it belongs to. No extra source
file, where all data items of a group would be defined together, is 
needed and this serves the aspects of modularization and independance 
very well.

  Is there any language construct in C,C++,PASCAL,Modula or ADA to 
support such data groups ?
  I haven't found any. If not : why didn't anybody invent such things ?

Comments to this are welcome.

   Andreas Hestermeyer
   internet : hestermeyer@ida.ing.tu-bs.de

Re: Grouping data from different modules together

[Mark A Biggar]

In article <1993Mar1.153217.3290@ibr.cs.tu-bs.de> hestermeyer@ida.ing.tu-bs.de (Andreas Hestermeyer) writes:
>  So far for the basic concept. To be efficient, all items of a group
>should be allocated an undivided space in memory. It is usually very
>easy to define such GROUPS in assembler language and have the linker
>taking care of grouping the data together. So each data item can be defined
>in the source code file of the module it belongs to. No extra source
>file, where all data items of a group would be defined together, is 
>needed and this serves the aspects of modularization and independance 
>very well.
>  Is there any language construct in C,C++,PASCAL,Modula or ADA to 
>support such data groups ?
>  I haven't found any. If not : why didn't anybody invent such things ?

There is a standard way to do this in all the above languages. It is called
the record stucture.  It is the standard way to group related data items
together.  This deos require that all the related data items be defined
in the same file, but if they are related enough to be grouped together, then
they are related enough to be defined together.

--
Mark Biggar
mab@wdl1.wdl.loral.com

Re: Grouping data from different modules together

[Robert Firth]

In article <1993Mar1.171045.17020@wdl.loral.com> mab@wdl1.wdl.loral.com (Mark A Biggar) writes:

>There is a standard way to do this in all the above languages. It is called
>the record stucture.  It is the standard way to group related data items
>together.  This deos require that all the related data items be defined
>in the same file, but if they are related enough to be grouped together, then
>they are related enough to be defined together.

I'm sorry, but I don't believe that.  Let me give a simple example from
the ugly past, which I'm quite prepared to believe applies also to the
ugly present.

The program was a transaction-processing executive, whose purpose was to
allow multiple "simultaneous" transactions, with different priorities, to
be processed by a single application.  This was before the days of re-entrant
code, and of course you couldn't run an application end-to-end without
frequent pauses to poll for interrupts, acknowledge high-priority events,
and so on.

The application was therefore structured as a sequence of small serially
reusable procedures, and the executive, or "driver" had this basic structure

	for i over fragment do
	    execute(fragment[i])
	    poll for anything urgent
	end

The table of code fragments had to be linear in memory, and of course was
constructed by the linker.  After all, when we were maintaining the
application - several hundred KLOC, a lotta code in those days - and we
changed a fragment, and it now ran for too long, we broke it in half, and
surely did *not* want to have to recompile the world after such a change.

It seems to me that this is close to the problem the original poster had,
and I for one don't think that fragment array is a global data structure
visible at the source language level.  And no, I don't know of any modern
high-level language that lets you do this - certainly not Ada, which can't
even build a table of procedure addresses at all, global or otherwise.

Re: Grouping data from different modules together

[agate!spool.mu.edu!caen!sol.ctr.columbia.edu!ira.uka.de!news.dfn.de!tubsi]

In article <1993Mar1.171045.17020@wdl.loral.com> mab@wdl1.wdl.loral.com (Mark A
 Biggar) writes:
>In article <1993Mar1.153217.3290@ibr.cs.tu-bs.de> hestermeyer@ida.ing.tu-bs.de
 (Andreas Hestermeyer) writes:
>>  So far for the basic concept. To be efficient, all items of a group
>>should be allocated an undivided space in memory. It is usually very
>>easy to define such GROUPS in assembler language and have the linker
>>taking care of grouping the data together. So each data item can be defined
>>in the source code file of the module it belongs to. No extra source
>>file, where all data items of a group would be defined together, is 
>>needed and this serves the aspects of modularization and independance 
>>very well.
>>  Is there any language construct in C,C++,PASCAL,Modula or ADA to 
>>support such data groups ?
>>  I haven't found any. If not : why didn't anybody invent such things ?
>
>There is a standard way to do this in all the above languages. It is called
>the record stucture.  It is the standard way to group related data items
>together.  This deos require that all the related data items be defined
>in the same file, but if they are related enough to be grouped together, then
>they are related enough to be defined together.
>
>--
>Mark Biggar
>mab@wdl1.wdl.loral.com
>
>
I disagree. This leads to a situtation where, if you exchange one module with
another, you would have to change (at least) to files. This really isn't necess
ary.
One should think of a portable way to implement this.

Andreas Hestermeyer

Re: Grouping data from different modules together

[throopw%sheol]

First of all, I note that this query was posted as four distinct
articles, <1993Mar1.153634.4146@ibr.cs.tu-bs.de>
and       <1993Mar1.153217.3290@ibr.cs.tu-bs.de>
and       <1993Mar1.153053.2961@ibr.cs.tu-bs.de>
and       <1993Mar1.153252.3446@ibr.cs.tu-bs.de>
with exactly the same text body in each.  (Well, there was a single
extra newline in the one in comp.lang.c)  I may have missed a posting
to comp.lang.pascal; perhaps it didn't arrive here yet.

In general, posting in this way is a Bad Idea, because it wastes storage
space on typical news systems, bandwidth on typical news connections,
and reader time for readers using typical newsreading packages.  A
better way is to post a single article to multiple groups, perhaps with
a Followup-To: setting to gather the discussion back into a single group.
The most discussion of this has occured in the comp.lang.ada group,
but it's relevant to many languages, so I'm issuing a followup-to
comp.lang.misc.  I hope nobody is offended by my attempt to hijack
the discussion in this way.

The original question as I understood it was how to get variables from
scattered modules grouped together contiguously in memory for
efficiency or other pragmatic reasons.

Using "pragmas" or equivalent notions in language implementations
which have them (the example I gave in <731046506@sheol.UUCP> being
taken from a C language system) seems the most practical way to go,
despite not being very portable.  More about portability below.

: From: mab@wdl1.wdl.loral.com (Mark A Biggar)
: Message-ID: <1993Mar1.171045.17020@wdl.loral.com>
: There is a standard way to do this in all the above languages. It is called
: the record stucture.  It is the standard way to group related data items
: together.  This deos require that all the related data items be defined
: in the same file, but if they are related enough to be grouped together, then
: they are related enough to be defined together.

This turns out not to be the case.  For example, items related only in
that they need to occur in specific address ranges because they need
to be placed in ROM vs RAM memory, or items related only in that they
happen to be consecutive IO register mappings, are NOT (I would claim)
related enough to be grouped together in source code.  In fact, I'll
even claim that such items are often UNrelated enough that they ought 
NOT to be grouped together in source code.

I think there are good reasons why such constraints should not
be stated to the compiler (because it has no need to know), but
should instead be stated to the linker/locator (because it does).

Another example of needing consequtivity of things essentially
unrelated needing consecutive storage:

: From: firth@sei.cmu.edu (Robert Firth)
: Message-ID: <1993Mar1.141751.17670@sei.cmu.edu>
: The application was therefore structured as a sequence of small serially
: reusable procedures, [...]
: The table of code fragments had to be linear in memory, and of course was
: constructed by the linker.  After all, when we were maintaining the
: application - several hundred KLOC, a lotta code in those days - and we
: changed a fragment, and it now ran for too long, we broke it in half, and
: surely did *not* want to have to recompile the world after such a change.

The original poster agrees with Robert, and adds:

: From: hestermeyer@ida.ing.tu-bs.de (Andreas Hestermeyer)
: Message-ID: <1993Mar2.073345.29349@ibr.cs.tu-bs.de>
: One should think of a portable way to implement this.

The difficulty with this is, the very notion of "contiguous memory"
involving separately declared variables is itself not portable across
all architectures.  It isn't even portable across all uses of ONE
architecture (the '286... consider that what counts as "contiguous"
depends upon whether one is programming in the "huge model" or
in other models of memory access).

This to me is strong reason why such directives belong in pragmas
and linker directives, and not in portable language constructs.

--
Wayne Throop   throopw%sheol@concert.net
               throop%aurgate@concert.net

Re: Grouping data from different modules together

[Andreas Hestermeyer]

In article <62368@aurs01.UUCP> throopw%sheol@concert.net writes:
>First of all, I note that this query was posted as four distinct
>articles, <1993Mar1.153634.4146@ibr.cs.tu-bs.de>
>and       <1993Mar1.153217.3290@ibr.cs.tu-bs.de>
>and       <1993Mar1.153053.2961@ibr.cs.tu-bs.de>
>and       <1993Mar1.153252.3446@ibr.cs.tu-bs.de>
>with exactly the same text body in each.  (Well, there was a single
>extra newline in the one in comp.lang.c)  I may have missed a posting
>to comp.lang.pascal; perhaps it didn't arrive here yet.
>
>In general, posting in this way is a Bad Idea, because it wastes storage
>space on typical news systems, bandwidth on typical news connections,
>and reader time for readers using typical newsreading packages.  A...

Sorry, I posted this to a couple of the lang.xxx groups to start a discussion.
I simply do not know how to send ONE post to more than one group other than
posting the article twice.

>
>I think there are good reasons why such constraints should not
>be stated to the compiler (because it has no need to know), but
>should instead be stated to the linker/locator (because it does).
>

But how should we tell the linker ? If we don't do that in the language
we again run into a situation where we would have to make changes in more 
than one file, if we change only one thing. 
And, looking at the way linkers presently get their instructions what to do :
it's all in the language by using keywords like 'extern' (C,C++). Why shouldn't
something like this be used here ?

>
>The difficulty with this is, the very notion of "contiguous memory"
>involving separately declared variables is itself not portable across
>all architectures.  It isn't even portable across all uses of ONE
>architecture (the '286... consider that what counts as "contiguous"
>depends upon whether one is programming in the "huge model" or
>in other models of memory access).
>

Hmm, I don't really see what you mean. If you're addressing segmented 
memory models like in the 80x86 architecture : right. But still, there
is physical contiguous memory and the compiler should be able to
map our variables where we want them. Well, I often do not want to know
'where' in memory (at which physical address) they are, simply the fact
that they reside in a contiguous portion of memory and that I have access 
to the start address and to the size of the block is sufficient. 
I agree that binding the variables to addresses might often be better
implemented as a pragma. Anyhow, other languages ( (Turbo)Pascal, ADA)
do provide this feature and they do a good job, I think.

Andreas Hestermeyer

Re: Grouping data from different modules together

[Richard A. O'Keefe]

In article <1993Mar4.080713.19045@ibr.cs.tu-bs.de>, hestermeyer@ida.ing.tu-bs.de (Andreas Hestermeyer) writes:
> But how should we tell the linker ?
    [ about grouping things ]
> If we don't do that in the language
> we again run into a situation where we would have to make changes in more 
> than one file, if we change only one thing. 
> And, looking at the way linkers presently get their instructions what to do :
> it's all in the language by using keywords like 'extern' (C,C++). Why shouldn't
> something like this be used here ?

Read the manual for the UNIX System V linker.  It will surprise you.
The on-line manual page says something like

	An input file [specified on the command line] that is not an
	object file is assumed to be an archive library (see ar(1))
	OR A TEXT FILE CONTAINING LINK EDITOR DIRECTIVES (see "The
	Link Editor" in the [System V] Programmer's Guide).

You can do amazing things with the System V linker.  And what's more,
you can do them with _all_ supported languages.  System dependent concepts
belong in system dependent tools.