Re: Critique of SEI's Ada DARK project

[hinton@gca.UUCP (Edward Hinton)]

In article <SRCTRAN.91May21140221@world.std.com> srctran@world.std.com (Gregory Aharonian) writes:
>
>	I am giving a talk during the first week of June on cost/benefit
>analysis and economics as applied to software reuse and engineering.  One
>example I plan to use is the Software Engineering Institute (SEI) Distributed
>Ada Real-Time Kernel (DARK). My contention is that the costs in terms of tax
>dollars far outweighed the benefits the country received in terms of any new
>original information from the project' results. I am interested in what others
>think about the cost and benefits of this project.

First question:  What is the scope of your talk?  If it is limited to
software and reuse and engineering in low-risk projects, then DARK is
at best borderline as an example.  If it includes high risk, how do you propose
that cost/benefit analysis be applied?  Are you assuming predictability
and high success rates for high-risk projects?  This would be
contradictory.

>	After developing and experimenting with the code, the SEI tried
>licensing the kernel to industry in 1990. I assume that this effort failed,
>because in 1991, the Department of Commerce's National Technical Information
>Service (NTIS) announced it was selling the DARK system (the cost is a few
>hundred dollars for a magnetic tape and documentation). To the best of my
>knowledge, this is the current official status of DARK. I have not examined
>the code, but I assume that like other Ada code developed at SEI, the quality
>of the Ada source code is very high and reliable.

Second Question:  Is SEI applying what it learned to other SEI projects?
If so, failure may not be an appropriate word.

>        An alternate goal might have been to have some SEI staff gain
>experience in this area. However the DoD could have sent them to one of these
>companies for training, at a much lower cost than what the DoD funded the SEI.

Third question:  What kind of experience is the best teacher, experience
doing, or experience watching and listening?

>I bring this up because DARK is along the same model of projects we might
>consider taking on: DARK probably LOOKED like
>    * something worth doing (if nothing else, as a proof of concept that 
>        such things could be done in Ada.)
>    * a somewhat risky project (people probably weren't very confident that
>        it could be done.)
>    * something that industrial teams might not do on their own (because
>        of the risk).
>

Last question:  Would inductry have teamed up on this?

My point is, looking at cost/benefit analysis before the fact on
high risk technological projects must take into account the
POTENTIAL advancements, be they new technology, or wider
acceptance of existing technology.  The goal of proving something could
be done seems reasonable compared to the cost (in my opinion).
In fact, any software project of sufficient technical complexity
to warrant government funds at all probably would cost at least
the estimated $300,000 cited.  Industry can and should do the 
easy cases if they are worth doing.  In fact, I would fault SEI for
not doing more on this project if it was worth doing in the
first place.  Their sights were not set high enough.

As an analogy, getting to the moon probably provided little
benefit to the country for the cost, but such high sights resulted
in tremendous technological efforts which were well worth the cost
in return for the other uses the technology was applied to.

I don't know much about DARK itself, but I think SEI research
apart from industry is too lightly being criticized.

(Aside:  I'm not generally in favor of rampant government
spending, but I do think we can learn from the Japanese
about the benefits to the country as a whole of government
spending in high-risk technological areas.)