Re: Quality systems (Was: Using Ada for device drivers? (Was: the Ada mandate, and why it collapsed and died))

["Robert I. Eachus" <rieachus@attbi.com>]

Shayne Wissler wrote (I'm quoting it in full because it deserves to be 
reread):

> I think what it indicates is that the development methods used are 
> brittle--which leads to the brittle code. If you see the secondary bug to 
> primary bug ratio consistently rising, the first thing to do is ask 
> questions about who is managing the project, not about the software itself. 
> Even the best software will have bad ratios if the people working on it are 
> sloppy.
> 
> Although it's not easy, I don't think it's rocket science to be able to make 
> consistent progress. If, for every bug you hit, you always investigate why 
> the bug happened, and then modify your approach accordingly, I don't think 
> you can get into the spiral you're referring to. I think it only happens 
> when people stop introspecting, and just blindly pull a bug off the top of 
> the list and proceed with "fixing" it.
> 
> A methodical approach can appear to be less efficient, and I'm sure it's the 
> pressure to deliver that leads to many breaches in integrity to a proper 
> engineering methodology. Of course, in the long run, such breaches always 
> end up costing you more.
> 
> A lot of software does deserve to be completely rewritten. But that should 
> be apparent by how slow you have to go when adding features, not by 
> ever-cascading bugs. On the contrary, if you see bugs spiraling out of 
> control, that's evidence that you don't want that particular team and/or 
> its management to undertake rewriting the system.

Everything you say is true, and good advice.  But you need to know where 
I am coming from.  At MITRE software engineers are often used by the 
government as firemen to go into an out-of-control project and determine 
what can be done to fix things.  Sometimes the advice has to be that the 
project is hopeless and can't be saved.  If the software is too brittle, 
it doesn't matter how it got that way, all that matters is that the only 
way to ever deliver a product is to fund a new project with a new team, 
and bury the original code so that it can't possibly be reused. 
Expensive?  Not really.  It is like a fireman deciding that they can 
save all the buildings around the one that is burning, or try to save 
that building and lose everything.  It is not a decision you take lightly.

Let me give two examples.  The first is one where MITRE was involved, 
called Cheyanne Mountain Upgrade.  At the time it was started, the 
Soviet Union was still a threat, and NORAD being off the air for even 
part of a day was considered unacceptable.  The overall project was a 
success, even though the reasons for that success involved spending 
extra money to put firewalls between each component of the upgrade. 
Some of the components were late and involved lots of nasty problems 
like changes in requirements.  (It doesn't matter if the system can deal 
with the original threat assessment.  If the threat changes, the real 
requirements must change.  Also one of the biggest requirements changes 
was caused by the Goldwater-Nichols reorganization of the DoD command 
structure.)  But because of the firewalling each component was of 
managable size and could be installed without depending on other parts 
of CMU.

The other example is a program where MITRE was not the FFRDC invovled. 
The A-12 attack aircraft program for the Navy did not have this type of 
firewalling.  By the time that budget overruns and schedule delays 
brought the program to the attention of Congress, the whole project was 
unsalvageable.  (And, yes, some of the people who let it get out of 
control were fired. See http://www.fas.org/man/dod-101/sys/ac/a-12.htm 
for a write up of the disaster.)

In any case, my reasons for wanting to develop a defect management tool 
was not for the people actually involved in the project.  It was so that 
the government and FFRDC people involved in managing the project for the 
government could detect the problem before it was too late.  The most 
important thing to take away from this discussion is that it is possible 
to get to the point where it really is too late to save the project.