* Re: Ada in Australia (was Ada ad in Embedded Systems Programming stinks)
1994-10-30 1:14 ` Andrew Lees
1994-10-30 2:41 ` Chris Perrott
@ 1994-11-04 21:46 ` Do-While Jones
1 sibling, 0 replies; 13+ messages in thread
From: Do-While Jones @ 1994-11-04 21:46 UTC (permalink / raw)
In article: 2009 of comp.lang.ada, Andrew Lees ( andrewl@zeta.org.au)
writes:
> bjw@f111.iassf.easams.com.au (Brendan WALKER) writes:
> [snip]
> >It is also worth noting that of three Ada projects you mentioned, I know for
> >a fact that at least two of these are in SERIOUS trouble, facing serious
> >architectual/design problems and the resulting cost and schedule slip (like 3+
> >years late!!).
> Hmm,
> I suspect that I should not really respond to this, but the implication here
> (that Ada is involved in any problems these projects may have ) is simply
> too wrong to leave alone.
I can't leave it alone, either. Two years ago, I attended a design
review for an Ada project that was in trouble. My observations were
published in the June 1993 issue of Ada User (reproduced below with
permission).
Last month I checked up on the project. It has not failed (yet), but
the software contractor came close to missing a payroll, and is flirting
with bankruptcy. The project is way over budget and hopelessly behind
schedule. They expect to deliver software that does not meet many
of the requirements. As predicted, Innocent Ada Gets the Blame.
+--------------------------------+
| Know Ada |
| [Ada's Portrait] |
| Will Travel |
| |
| wire do_while@ridgecrest.ca.us |
+--------------------------------+
------------------------------------------------------------
Reprinted from Ada User
Ada Language UK, Ltd.
with permission.
------------------------------------------------------------
1
+---------+
| Opinion |
+---------+
Innocent Ada Gets the Blame
Do-While Jones
I recently attended the software portion of a Critical
Design Review (CDR) for an important project being developed
jointly by two American armed services. One of the parties
involved asked me to attend and give my assessment of the
project status. It is my opinion that the project is in
trouble, and Ada will get the blame, even though it isn't
her fault.
Unfortunately, this project is typical of so many
software development projects. They are making the same
mistakes that are so common these days, but these mistakes
somehow remain unrecognized. Since Ada is supposed to save
us from all our problems, she will get the blame when the
project fails. We need to open our eyes to the real
problems and concentrate on solving them, rather than
blaming an innocent party.
Four major problems surfaced at that CDR, but nobody
seemed to notice. (1) The contractor and the U.S.
government are working against each other, rather than
working together. (2) The government-imposed software
development methodology encourages the contractor to produce
a bad design. (3) The contractor is trying to defeat Ada's
software engineering features. (4) The contractor's
configuration management system is badly flawed.
The project in question is a real-time multi-media
system that combines a graphical display of processed data
with audio (voice) data. The audio data is digitized,
compressed, and stored with the rest of the digital data so
that the audio will remain synchronized with the graphical
data when the event is replayed later for debriefing and
analysis. The software is written in Ada for a Silicon
Graphics workstation running X-windows on UNIX.
Lack of Teamwork
The attitudes of the people at the CDR quickly showed
that the project is in trouble. This was apparent from an
"us vs. them" relationship between the government and the
contractor. The contractor was late and defensive. The
government was angry and distrustful. There wasn't a team
spirit. They weren't working together. Each side was eager
to place the blame on the other.
Here is an example: This project is an improvement of
an existing system. One of the system requirements is the
new system should have all the capability of the old system,
and then some. The Silicon Graphics computers they are
using have audio capabilities that the old system didn't
2
have. The old system could digitize and record audio data,
but it had no way to play it back. It needed an external
audio amplification system. Since it could not reproduce
the audio data, it had a diagnostic feature that displayed
audio data as hexadecimal values that scrolled off the
screen faster than anyone could possibly read. Even if you
could read them, there isn't a person alive who could tell
what it sounded like. It was a totally useless feature.
The contractor wasted time and money by designing a
visual hexadecimal dump of audio data for the Silicon
Graphics system. At the CDR, the contractor reported that
they had done this, even though they knew the feature was
stupid and worthless. Although they didn't come right out
and say so, the tone of voice and body language said, "We
knew that if we didn't implement this feature, the
government would write us up as being deficient and penalize
us for it, so we beat you to the punch by wasting your time
and money and now there's nothing you can do about it
because our lawyers say there is an iron-clad requirement
for it. If you want us to take it out, it will cost you
even more money. Ha, ha, ha!"
The government has a long history of this kind of
relationship with contractors. It is a direct result of the
laws governing federal procurement. Corrupt people believe
everyone else is as corrupt as they are. Lawmakers see
trust and cooperation as unmistakable evidence of some
under-the-table kickbacks or other unfair labor practice.
Therefore, they have made it illegal to have any
relationship with a contractor that is based on trust or
cooperation. Contracts must always be awarded to the
contractor who underbids the most. Actual cost of the
contract is determined through litigation concurrent with
project development.
Methodology Maladies
MIL-STD-2167A describes "the one true way for good
software development." Unfortunately, that method is badly
flawed, and problems experienced by a software project are
generally proportional to the extent of its use. On this
particular project, the government is strictly enforcing
2167A, with the usual disastrous results.
Fundamental to 2167A is the "waterfall model" of
development. This model requires that there be a Critical
Design Review at the end of the design development phase.
The project must obtain approval at this review before the
implementation phase (i.e., writing code) may begin. The
primary document reviewed at a CDR is the Software Detailed
Design (SDD) document. In theory, this SDD contains all the
pertinent design information needed to make an intelligent
decision whether the project is on track or not. The
approved SDD theoretically serves as the foundation for the
coding effort.
3
Near the end of the presentation of the SDD at the CDR,
a man sitting next to me whispered that he had just figured
out what was wrong. Everyone was trying to guess what the
user's manual would be like from the description of the
software configuration items contained in the SDD. He
thought that there would be much less argument and much more
understanding if they simply wrote the user's manual first
and presented it at the CDR instead. He was on the right
track. Most people were trying to guess what the user's
manual would be like, but the rest of the people were trying
to guess what the Ada code would be like. Nobody really
cared about the SDD. They were just trying to use the SDD
as an awkward, indirect, unreliable way to find out what
they really wanted to know. A preliminary user's manual and
some carefully chosen code fragments from a prototype design
would have told them what they really wanted to know.
The questions that the government needed to ask the
contractor were: How is it going to work? How are you
doing it? How much have you done? When do you think it
will be finished? What is the biggest problem facing you at
the moment? What do you need us to do to help you? Do you
still think it will cost what we estimated it would?
Unfortunately, 2167A requires that contractors must cloak
their methods, intentions, and progress behind a vague and
ambiguous detailed design specification, and that they must
present these specters and shadows using smoke and mirrors
at a CDR.
Everything I really needed to know, I learned in the
hallway. In the hallway, the contractor was willing to
admit that they already had written code and were testing
it. In the hall, the contractor could tell the truth about
what was really going on. Only on rare occasions during the
formal presentations did the speakers slip and answer
questions in a careless way that betrayed that the SDD
hadn't quite kept up with their real design.
The format of the SDD makes it difficult to clearly
describe an existing design. Occasionally people in the
audience asked about things that weren't shown in the
detailed design document. The contractor's response was
always that their design really satisfies the requirement,
but the SDD just doesn't show it.
For example, the X-windows services don't appear
anywhere in the design specifications, yet it is X-windows
that holds the whole program together. More than half of
the code is X-windows and UNIX services, and they aren't
mentioned at all in the SDD! One could argue that X-windows
is an off-the-shelf product that isn't being designed by the
contractor, so there is no need to document it. But X-
windows controls the state transitions and largely
determines the program structure. The Motif widget
hierarchy is fundamental to the structure of the display
screens. One can't describe the design without talking
about these things. (Indeed, the reason I know how X-
windows and Motif are being used is because the presenters
4
TALKED about them in the hall, but these critical details
are entirely missing from the WRITTEN documentation.)
Meanwhile, back at the CDR, one view graph gave
estimates of memory usage and processing time. The
estimates were inconsistent and ridiculous, so this
naturally aroused some questions from the floor. After
extensive questioning, the contractor admitted that some of
these estimates came from "sound engineering judgment," and
the rest of the "estimates" were really measurements of
their prototype (which they didn't want to admit existed
because they weren't supposed to be writing code yet).
Several people in the audience wanted some indication of
which numbers were measured and which were guesses. The
contractor would not give that information. He argued that
all that mattered was that the estimates said there would be
at 30% spare time and memory. He even went so far as to say
that if the measurements differed too much from the
estimates, they would put more faith in the estimates than
the measurements!
The whole argument was really stupid. MIL-STD-2167A
requires that the software developer estimate there will be
30% spare capacity, and they did. What's the big fuss
about? Who cares if the estimate is bogus or not? There
isn't any requirement that the estimate be correct.
Of course it is necessary that the delivered system
have enough spare time and memory that minor modifications
can be made to the program and still run. If the program
doesn't have enough spare memory, then the contractor is
either going to have to add more memory, or do some space
optimizations, whichever makes more sense. If the program
doesn't run fast enough, the contractor will have to upgrade
to a faster processor, or make some time optimizations. If
nothing can be done to make the program small enough or fast
enough, then some functionality will have to be removed.
None of this is affected by the accuracy of the estimate of
resource usage.
Before any code is written or compiled, the estimate of
program size and speed can't be much more than a wild guess.
To allocate portions of this estimate to a preliminary guess
of what software configuration items there will be in the
final system is pure insanity. Yet, there is a requirement
to produce such a space/time budget and present it at the
CDR. Therefore, the contractor has to pull some numbers out
of the air (or some darker place) and put them on paper.
Furthermore, the numbers must add up to 30% less than the
total, so it should be no surprise that they do. After
wasting valuable time to produce this estimate, the
contractor must present these numbers to the government as
if they have some meaning or value. Any sharp government
representative is going to see immediately how foolish they
are, and challenge the contractor. The contractor is forced
to defend his numbers. Thus the process forces the
government and the contractor to be adversaries for no good
reason!
5
Ada Ignorance
In the hall, I asked the lead software engineer what he
would do differently if he had to do the project all over
from scratch. He immediately answered that he would do
everything the same except he would use C instead of Ada.
He blamed Ada for all their problems. Ada kept throwing up
"roadblocks" to keep them from doing what they wanted to do.
During a break I asked the program manager what she
would do differently. She said there were two things she
would do differently. First, she would do the documentation
differently. She said that 2167A documentation required was
badly formed and badly timed. The other change she would
make is that she wouldn't use Ada. All the programmers
complained about Ada, she said.
At lunch I asked another software engineer some pointed
questions about interfacing Ada with X-windows. He said
that Ada will interface to X-windows, but you have to be
careful to specify the proper link directives or the program
won't run at all. He was rather negative toward Ada's
strong typing, and said that they declared all their
variables to be either integer or float whenever they had
the opportunity. He thought it was unfortunate that the X-
windows interface they bought from Verdix declares special
integer types that required so many explicit type
conversions. He also didn't like declaring subtypes with
range constraints because he was afraid of CONSTRAINT_ERROR.
He preferred to write his own range checking whenever he
thought he needed it.
Ada was specifically designed to promote good software
engineering principles and discourage poor programming
techniques. Ada makes it very easy to write good programs,
and very hard to write bad ones. This contractor is
struggling hard against Ada because the programmers are
trying to do things Ada is designed to prevent them from
doing. They are trying to write C-style code using Ada
syntax. That's why it is so difficult.
The SDD contained many configuration items for shared
memory segments, semaphores, message queues, and pipes for
inter-process communication. These are loud alarm signals.
The proper way to do multi-tasking in Ada is to use Ada
tasks. Properly written Ada programs don't need extensive
use of shared memory, semaphores, queues, and pipes. They
were using C calls to UNIX services to do their own home-
brew inter-task communication! Mixing languages is tricky
and not very portable. They were doing an awful lot of
jumping back and forth between C and Ada, and every jump is
a chance for error.
They claim to be writing everything in Ada, but that
can't be true. For example, there is a software
configuration item design description that talks about
forking another process and the signal that will be
generated if the process fails (instead of an exception that
6
will be raised when a new task is elaborated). It must make
a C call to a UNIX service, completely bypassing Ada.
Another configuration item uses semaphores to control the
execution of the other modules. A real Ada program would
use parameterless entries into tasks for synchronization.
Even without seeing the actual code, it is apparent
that the contractor isn't designing an Ada program that
makes a few calls to C graphics library routines. They are
designing a C program that they are trying to write in Ada
syntax. All the Ada safety features have been intentionally
bypassed, and dirty tricks have been employed to try to keep
the Ada tasking from ruining everything.
Possible evidence that the contractor isn't proficient
at Ada surfaced when there was a digression on how many
times a person should be allowed to incorrectly enter a
password. There was no requirement that said that the
system should go into a catatonic state after the user
enters the wrong password after a certain number of times,
so the program lets the user try forever. When someone
suggested that they add a limit of five tries, the program
manager said that would not be an easy feature to add.
All they have to do is put their existing do-forever
loop inside a procedure and change it to a "for i in 1..5
loop" and return from inside the loop when the password is
successfully entered. After five unsuccessful attempts, it
drops out of the loop where the procedure could raise an
exception. This should be a trivial change. Yet the
contractor says it would be a major impact to the program,
much too difficult and costly to add now.
Why is that so difficult and costly? It might mean
that their code was so badly written that it can't easily be
changed. There is a good chance this is part of the reason,
but there is another reason, too. To make this change they
would have to add one more software configuration item.
That is very difficult and costly for them. Their
configuration management is killing them.
Configuration Mismanagement
The contractor briefly flashed many of the view graphs
on the screen because there was so much design redundancy.
Most of the display features in the live mode are identical
to display features in the playback mode. So, they reused
the design and made duplicate view graphs. They quite
properly hand-waved many configuration items away saying,
"This is just like something we've already talked about."
Such a high percentage of design reuse would imply a
high percentage of code reuse, but there isn't any real code
reuse at all! Most of their configuration items were
created by making a copy of the source code for another
configuration item and using a text editor to change its
name. They individually compiled, cataloged, and documented
each of these source code modules that are identical except
for their name. I asked why. The program manager and lead
7
software engineer both replied that their configuration
management can't handle reusable library components. (Each
software component needs to be associated with a particular
configuration item. Two different configuration items can't
share a software component.)
It is obviously wasteful to maintain and document
duplicate copies of source code. It is dangerous, too.
There is a very good chance that a bug will be found and
fixed in one of the components, but it won't be fixed in
other copies of that component. It is also very wasteful of
memory to have multiple copies of the same sequence of
statements. They have to do this, though, because their
configuration management system isn't compatible with Ada.
Ada does a lot of the configuration management work for
you. If you make changes to the specification of a
compilation unit, she forces you to recompile everything
that depended on the specification of that unit so she can
make sure the interfaces still match. If you change
something internal to the compilation unit that does not
affect the specification, she does not force you to
recompile anything else, but does automatically link the
latest version of the object code each time you link a main
program. So Ada makes sure that corrections are made in
every use of common routines. Making differently named
copies of common routines just defeats another of Ada's
safety features.
Even if their configuration management technique is so
inept that it requires unique names for every different use
of a common routine, there is a simple trick to keep
configuration management happy. The software designers
could do this:
generic
package BLAH_BLAH is
-- Some commonly used routines are in this package.
end BLAH_BLAH;
with BLAH_BLAH;
package LIVE_BLAH_BLAH is new BLAH_BLAH;
-- This makes a copy of BLAH_BLAH called
-- LIVE_BLAH_BLAH.
with BLAH_BLAH;
package PLAYBACK_BLAH_BLAH is new BLAH_BLAH;
-- This makes a copy of BLAH_BLAH called
-- PLAYBACK_BLAH_BLAH.
Any change to BLAH_BLAH automatically makes
LIVE_BLAH_BLAH and PLAYBACK_BLAH_BLAH obsolete. Any attempt
to link them to a main program will fail with a link-time
message saying that they are obsolete and must be
recompiled. As soon as they are recompiled, they become
current and can be linked.
8
(I don't really recommend this trick because
configuration management should be able to keep track of
reusable software components just as they keep track of
electronic components that are used on multiple projects. I
hope their hardware configuration management doesn't assign
unique part numbers to every 10 K-ohm resistor on every
different circuit board!)
Ada environments typically have tools that software
designers can use to answer the question, "If I make a
change to BLAH_BLAH, what other compilation units will be
affected?" The contractor is going out of his way to keep
Ada from being able to answer that question by using
different names for copies of the same thing.
Some of the contractor's configuration management
difficulties are symptoms of the Ada Ignorance already
mentioned. They aren't used to packages, tasks, and library
subprograms, so they just don't know how to keep track of
them.
People who use Ada correctly build up a library of
reusable, tested, software components. This library full of
building blocks is kept under configuration management.
When new designs use existing library components they simply
document them by reference to existing documentation.
Clearly, this contractor doesn't have a configuration
management system that allows them to do this.
Some of their configuration management problems are
symptoms of the Methodology Malady already mentioned. MIL-
STD-2167A forces the designers to identify configuration
items too early. It is not possible to guess in advance
exactly which features will be identical and which will
merely be similar. If you initially guess that two
different configuration items can share a component, and
later find out that they can't, configuration management
makes it difficult to change your mind. Therefore, the
apparently easy way out is to guess that all components will
be unique. This creates problems later when you discover
that some really are identical.
Don't micro-manage software components before they have
even been created. Unanticipated configuration items should
freely appear later in the design, and useless configuration
items should be allowed to disappear. Early configuration
management often makes that too difficult. The result is
that designers suffer with an awkward design just because it
is too much trouble to get configuration management to let
them do it right.
The "5-try password entry" modification I mentioned
earlier is an example. It would be a simple change to make,
except for the fact that they would have to create a new
software configuration item. The SDD would have to be
modified to include this new configuration item. The
requirements traceability matrix would have to be updated.
The requirements document would have to be revised. Some
drawings produced during the analysis phase would have to be
9
revised. All that work is just too much trouble. It is
easier to let the design remain wrong.
Conclusion
The project will be very late and over budget, but it
will eventually get done if the money holds out. It will
meet the written requirements, but it won't meet the user's
needs. In other words, it is a typical software development
project. It has the same old problems projects always have
because the MIL-STD-2167A software development process
forces them to make the same mistakes.
The only difference is that this time the government
and the contractor will both put the blame on Ada for the
added cost and delay, instead of trying to put the blame on
each other. Both parties will save face, and Ada will be
convicted because she has nobody to defend her.
^ permalink raw reply [flat|nested] 13+ messages in thread