AAS, was it Ada? Cleanroom?

AAS, was it Ada? Cleanroom?

[Sam Harbaugh (AQ)]

Nancy posted a brief statement on the incose (international council on
systems engineering) bb that AAS was an almost complete waste of 6 billion
dollars.  I asked her to elaborate and below is her reply.

I recall seeing an IBM exhibit at either a Tri-Ada or IITSC conference ( I
can't  remember which).  The person showing the large round display was very
zealous about the system.  I looked closely and saw that the display
contained many repetions of the same aircraft.  When I pointed it out he
mumbled and started talking to someone else.  It would have made a good
Dilbert cartoon.

So, was AAS Ada? It sounds like software was not the problem, I'm just
curious about the language.

Was AAS cleanroom design?  Again it doesn't sound like software design was
the problem, again I'm just curious.

Is Nancy off base with her comments?

sam harbaugh
--------------------------------------------------------------------------

>To: "Sam Harbaugh (AQ)" <harbaugh@acusys.com>
>cc: incose list <incose-list@xor.com>
>Subject: Re: Feasibility
>Date: Thu, 25 Apr 1996 18:09:31 PDT
>From: Nancy Leveson <leveson@cs.washington.edu>
>
>
>      Nancy
>      ---------------------------------------
>      Could you or anyone post a reference for the statements that
>
>      1. The system cost 6 billion dollars
>
>Sorry, I got the number a little bit wrong (but not much).  According to
>Business Week, April 26, 1993, the AAS project had cost (at that time)
>$5.1 billion, which was already $1.5 billion over its budget, and
>climbing.  I don't know what the final total is (or will be?).
>
>      2. Almost all of it has been thrown away.
>
>The process started in 1982, when the FAA started the system development
>and said that it would be introduced first into the Seattle area in 1992.
>In 1990, Congress was upset about projected delays (the FAA announced a
>19-month delay) and launched an investigation into cost overruns and
>mismanagement of AAS.  In 1992, more problems arose and a second 14 month
>delay was announced.  The FAA threatened IBM with cancellation of the
>program (they issued a "cure" letter, which is the first legal step in
>interminating a contract).  At the time of the Business Week article I
>noted above (April 1993), IBM announced that the project was at least 9
>years from completion and the new system would not be in place until well
>after 2000.
>
>One of the things in the article pertinent to this discussion is that
>the requirements were never carefully written (I was involved in the
>design competition for the system in the 1980s when I was consulting for
>Hughes and I told the FAA then that their requirements specifications
>were inadequate).  They don't seem to have asked the controllers what
>they thought of the design until late in the process (1990), when 500-700
>changes then had to be made in the specification.
>
>After that, I remember some official studies were made of the project for
>the FAA (I don't remember the names of who did them -- maybe someone else
>does) to determine whether they should continue.  I spoke privately to some
>people involved in these studies.  An official announcement was made
>sometime later that most of the system would be canceled but that some
>parts could be salvaged.
>
>      What are they using if they threw away the new system?
>
>The old system.
>
>      Who was the contractor(s)?  IBM for software?
>
>IBM for software and for the consoles as I understand it.  I don't know
>if anyone else was involved.
>
>      What was the problem?  Not understanding air traffic control?
>
>I tried to explain this above.  My take is that the primary problem was
>that nobody did a proper requirements analysis at the beginning.  But
>others may have a different opinion.
>
>Nancy
>
>

Re: AAS, was it Ada? Cleanroom?

[Brian Nettleton @pulsar]

I worked on AAS for over 3 years (the division of IBM which held the
contract was sold to Loral during that time) as a software engineer.  
The original AAS contract with IBM has been cancelled and replaced
with a new program called DSR (Display System Replacement).  DSR is
re-using most of the software (design and code) from AAS which is
applicable.  Ada was certainly not at fault for the problems on AAS.  I
can't think of anyone actually familiar with the project who blames
Ada.  In fact anyone who says "The problem on AAS was X" doesn't know
what they are talking about.  AAS had a multitude of problems ranging
from changing specifications to congressional meddling (the original
consolidation of certain facilities within the FAA proved to be politically
unacceptable) to grandiose expectations (it proved to be impractical to
build a system with only 3 seconds of downtime per year).

The new DSR has dramatically reduced the scope of the original AAS
contract.  Last I heard (January) the system was on schedule and maybe
even a week or so ahead of schedule.

Also a smaller version of the AAS project was sold and delivered to
Taiwan and is soon to be in operation (may already be in operation?).


In <199604260127.VAA10999@bb.iu.net> "Sam Harbaugh (AQ)" <harbaugh@ACUSYS.COM> writes:

>Nancy posted a brief statement on the incose (international council on
>systems engineering) bb that AAS was an almost complete waste of 6 billion
>dollars.  I asked her to elaborate and below is her reply.

>I recall seeing an IBM exhibit at either a Tri-Ada or IITSC conference ( I
>can't  remember which).  The person showing the large round display was very
>zealous about the system.  I looked closely and saw that the display
>contained many repetions of the same aircraft.  When I pointed it out he
>mumbled and started talking to someone else.  It would have made a good
>Dilbert cartoon.

Round display?  The existing system in use today uses round displays.  The
new AAS/DSR system uses large square displays.  Anyway most demo's of the 
system used canned data that may well have had the same aircraft duplicated.


>So, was AAS Ada? It sounds like software was not the problem, I'm just
>curious about the language.

AAS was written in Ada.  Again Ada was not a problem on AAS.  Certainly
the project was large enough to find compiler/tool bugs, but these were
never of a nature to justify the kind of schedule problems AAS ran into.


>Was AAS cleanroom design?  Again it doesn't sound like software design was
>the problem, again I'm just curious.

AAS did not use cleanroom.


>Is Nancy off base with her comments?

Most of Nancy's comments were are fairly accurate, except that the
system has not been thrown away, but rather scaled down.  While the
new system will provide much the same support as exists in the field
today, it will install a platform which provides for expansion in the
future.  Hindsight being 20/20 this is probably what should have been
done in the first place, instead of trying to bite everything off at
once.



>sam harbaugh
>--------------------------------------------------------------------------

-Brian Nettleton

Opinions are my own, not Thomson Software Products, IBM's, nor Loral's.

Re: AAS, was it Ada? Cleanroom?

[Dave Ceely]

In article <199604260127.VAA10999@bb.iu.net>,
 on Thu, 25 Apr 1996 21:24:15 +0600,
 AQ <harbaugh@ACUSYS.COM> writes:
>Nancy posted a brief statement on the incose (international council on
>systems engineering) bb that AAS was an almost complete waste of 6 billion
>dollars.  I asked her to elaborate and below is her reply.
[snip]
>So, was AAS Ada? It sounds like software was not the problem, I'm just
>curious about the language.

-- AAS was coded in Ada.
>Was AAS cleanroom design?

-- AAS was NOT Cleanroom (specification, design, certification, or any
-- other aspect of Cleanroom Software Engineering
>Is Nancy off base with her comments?

-- Nancy is somewhat off base in asserting that the entire effort
-- was a waste.  Much of the existing design, code and integration
-- was reused in the renogiated Display System Replacement Contract
-- that Lockheed Martin is performing on (formerly Loral.)  That contract
-- is for somewhat less than $1B, and is firm fixed price.
-- Btw, the original contract with IBM had grown to about $6B, but
-- nowhere near that much was actually spent.

>sam harbaugh
>--------------------------------------------------------------------------

Re: AAS, was it Ada? Cleanroom?

[guest]

In article <199604260127.VAA10999@bb.iu.net>, harbaugh@ACUSYS.COM 
says...
>
>Nancy posted a brief statement on the incose (international council on
>systems engineering) bb that AAS was an almost complete waste of 6 
billion
>dollars.  I asked her to elaborate and below is her reply.
>
>I recall seeing an IBM exhibit at either a Tri-Ada or IITSC conference 
( I
>can't  remember which).  The person showing the large round display was 
very
>zealous about the system.  I looked closely and saw that the display
>contained many repetions of the same aircraft.  When I pointed it out 
he
>mumbled and started talking to someone else.  It would have made a good
>Dilbert cartoon.
>
>So, was AAS Ada? It sounds like software was not the problem, I'm just
>curious about the language.
>
>Was AAS cleanroom design?  Again it doesn't sound like software design 
was
>the problem, again I'm just curious.
>
>Is Nancy off base with her comments?
>
>sam harbaugh
>-----------------------------------------------------------------------
---
>
>>To: "Sam Harbaugh (AQ)" <harbaugh@acusys.com>
>>cc: incose list <incose-list@xor.com>
>>Subject: Re: Feasibility
>>Date: Thu, 25 Apr 1996 18:09:31 PDT
>>From: Nancy Leveson <leveson@cs.washington.edu>
>>
>>
>>      Nancy
>>      ---------------------------------------
>>      Could you or anyone post a reference for the statements that
>>
>>      1. The system cost 6 billion dollars
>>
>>Sorry, I got the number a little bit wrong (but not much).  According 
to
>>Business Week, April 26, 1993, the AAS project had cost (at that time)
>>$5.1 billion, which was already $1.5 billion over its budget, and
>>climbing.  I don't know what the final total is (or will be?).
>>
>>      2. Almost all of it has been thrown away.
>>
>>The process started in 1982, when the FAA started the system 
development
>>and said that it would be introduced first into the Seattle area in 
1992.
>>In 1990, Congress was upset about projected delays (the FAA announced 
a
>>19-month delay) and launched an investigation into cost overruns and
>>mismanagement of AAS.  In 1992, more problems arose and a second 14 
month
>>delay was announced.  The FAA threatened IBM with cancellation of the
>>program (they issued a "cure" letter, which is the first legal step in
>>interminating a contract).  At the time of the Business Week article I
>>noted above (April 1993), IBM announced that the project was at least 
9
>>years from completion and the new system would not be in place until 
well
>>after 2000.
>>
>>One of the things in the article pertinent to this discussion is that
>>the requirements were never carefully written (I was involved in the
>>design competition for the system in the 1980s when I was consulting 
for
>>Hughes and I told the FAA then that their requirements specifications
>>were inadequate).  They don't seem to have asked the controllers what
>>they thought of the design until late in the process (1990), when 
500-700
>>changes then had to be made in the specification.
>>
>>After that, I remember some official studies were made of the project 
for
>>the FAA (I don't remember the names of who did them -- maybe someone 
else
>>does) to determine whether they should continue.  I spoke privately to 
some
>>people involved in these studies.  An official announcement was made
>>sometime later that most of the system would be canceled but that some
>>parts could be salvaged.
>>
>>      What are they using if they threw away the new system?
>>
>>The old system.
>>
>>      Who was the contractor(s)?  IBM for software?
>>
>>IBM for software and for the consoles as I understand it.  I don't 
know
>>if anyone else was involved.
>>
>>      What was the problem?  Not understanding air traffic control?
>>
>>I tried to explain this above.  My take is that the primary problem 
was
>>that nobody did a proper requirements analysis at the beginning.  But
>>others may have a different opinion.
>>
>>Nancy
>>
>>

As a developer for a sub, then an independent, and finally placing 
developers myself; this should give you an idea of how this project was 
considered the premiere cash cow of the 90's.

My recollections were of 4000 developers, in at least a dozen locations, 
divided into four separate deliveries, to be implemented in sliding 
schedules, and development staff growing in double digits.

My previous experience was three years of Ada in the military and two 
years commercial.  I thought these were some of the brightest Ada minds 
I had ever worked with.  QA/QI was very thorough, perhaps even limiting 
at times, this was an issue and should be.  I didn't think the design 
documentation was lacking (maybe I've just had to produce with much less 
in my time) or below the standards I received in the military.

I recall working long and hard hours, just to get decent Ada work.  I 
felt the focus was not on producing results, but, extending the work and 
expanding the staff.  The three things I remember as obstacles were 
change control, project turnover, and tool automation (sounds like most 
projects I've worked on).  

I remember helping support and update an Ada source code generator 
written in REXX using Bachman case tool design.  Our designs came off of 
a mainframe environment.  There was an IBMer tech lead and we talked 
about the systems engineers as if they were Elvis sightings.

Calling like I remember...



-- 
Troy E. Swallow
(405) 942-3327 or (405) 943-1408
-- hochunk@ionet.net