Re: Rational Edge Design Papers

[Koni Buhrer <koni@Rational.Com>]

Al Christians wrote:
> 
> "Marc A. Criley" wrote:
> >  ... Lots of interesting stuff
> >
> 
> Thanks very much again.
> 
> I, too, like where this is going, ... except that in the business
> software world that I've inhabited,  the requirement for purely
> functional data transformations with no state could easily produce some
> monstrosity.

Well, you are right, but I am not saying data transformations can't have
state.
All I'm saying is that any data transformation state must be passed in
and out as the transformation server operation is invoked.
The rule ensures that transformation servers are deterministic and their
execution behavior predictable.
The data flow managers "own" the data transformation state.

>  This seems a common problem,  state-changing I/O and
> functional logic don't mix well.   It would be very ugly in a business
> program to have all the I/O at the top.  I guess I would change or
> interpret the rules to allow the  data entities to include database
> handles and let the primitive operations of the data entities do some
> database operations.

Yes, I understand your concern.
In the article I mention in a footnote that "files" (and databases are
just the same) are the exception to the rule that data entities must not
have embedded external interfaces.
Let me elaborate.

There are two ways in which we can look at a file (or a database):
A file can be viewed as an external interface from which we can read
records and to which we can write records.
Or a file can be viewed as a data object with somewhat funny and
convoluted rules on how its components are accessed.

If we consider the file to be a data object, it makes a fine data
entity.
A transformation server can read from it or write to it just like it can
access the public components of any other data entity.
Reading from a file or writing to a file thus becomes a data
transformation: the data is transformed from its file format to some
internal format, or from an internal format to the file format.
And the transformation server can read from or write to a file data
entity anywhere within its guts without intervention of data flow
managers and I/O servers.

Treating files and databases as data entities is very common.
I myself have written numerous programs doing just that.
And I believe that's ok, as long as the program is essentially
sequential and essentially free of real-time constraints.
If however a software system contains substantial concurrency or is
subject to substantial real-time constraints, data entities with hidden
external interfaces are very problematic.

External interfaces typically cause an execution thread to be delayed
(hardware latency), to synchronize with other threads (mutual
exclusion), and to block (data availability).
Sure, these issues are well known and a software design can deal with
them - usually.
What makes data entities with hidden external interfaces so dreadful is
the secrecy and subtlety by which they can introduce these problems into
a design.

Please consider:
To determine whether a software architecture is viable (for example
whether it can meet timing requirements), a software developer usually
looks at the high-level, architectural design.
At that level the details of data transformation operations and data
objects are rarely exposed.
It is therefore very easy for the software architect to overlook the
delay, synchronization, or blocking issues that are introduced by a data
entity with hidden external interfaces.

And even if the software architect is aware of the data entity, it may
be difficult for him to gage the impact the hidden external interface
has on overall system performance.
After all, we are looking at a data entity that can be used in any
number of places in any number of ways.
And even if the software architect takes all those uses and interactions
into account - during detailed design somebody may use the data entity
(and its hidden internal interface) in a new and unexpected way, thus
causing havoc on the system.

This is poison for a real time software project!

And it can be worse, because software developers - or should I say
humans? - have the tendency to defer difficult problems they don't know
how to deal with.
So it is very enticing for a software architect to hide a
hard-to-deal-with external interface inside a data entity to solve the
problem later, during detailed design.
That's dangerous, because the data entity might introduce delay,
synchronization, and blocking issues that affect overall system
viability.

A software project should deal with the hard issues - and concurrency is
definitely one of them - early during system design.
One of the benefits of the universal design pattern is that it exposes
the architecturally complex issues at the highest level.
The tedious but architecturally simple bulk of the software however is
encapsulated in data entities, transformation servers, and I/O servers.

> 
> Thoughts welcome again.
> 
> Al

Appreciate your comment.

Koni