Re: Please do not start a language war (was Re: Papers on the Ariane-5 crash and Design by Contract

[Ken Garlington <GarlingtonKE@lmtas.lmco.com>]

Jean-Marc Jezequel wrote:
> 
> In article <332ED8AB.21E7@lmtas.lmco.com>, Ken Garlington <GarlingtonKE@lmtas.lmco.com> writes:
> 
> >   2. No one ran a full integration test with realistic flight data,
> >which would have
> >      alerted them to the mistake made in #1. Particularly for a
> >distributed mission-
> >      critical system, this should be considered an absolute
> >requirement.
> 
> Yes, this is true. But you have to understand that the mere design of the SRI
> made it very difficult to test it in an other way that performing a launch.
> This is because of the tight integration of hard-to-fool hardware with software
> in a black box functional unit.

We have exactly the same coupling of inertials to flight controls on a
current project, and
we are able to test the coupled system in a black box environment in our
labs, with pilots in the loop, performing the same flight profiles we
expect to see in operation.

Interestingly enough, the IRS for the this project is also "reused" from
a different platform, and there was a proposal to minimize testing of
the system because it had already
been "proven." We were able to quash that "cost-savings" measure before
it gained much
support. However, in an environment with Cost As an Independent Variable
(CAIV), I can
certainly see how easy it would be to adopt such ideas.

> What can be your test strategy for a black box
> containing an inertial central? If the software had been designed with less coupling
> on this particular hardware, you could have test the software vs. a simulation of the
> hardware and its environment. Here, the launch was the test.

Is this what Aerospatiale told you?

> 
> >Although I like some of Mr. Meyer's columns, I found this particular
> >column not only wrong,
> >but dangerous, since it continues the trend of denigrating sound test
> >strategy and common
> >sense system engineering in favor of a software-centric approach.
> 
> I don't see where it denigrates sound testing. IMHO, sound testing *is* needed.
> And testing is effective only if you have a sound test strategy.
> Even more when you use programming by contract. In the paper,
> we just recall that you cannot rely on tests only.

Then why in your message on comp.lang.ada, did you say that to a "lesser
extent"
this was an integration test problem? Not doing this testing was a
*critical*
part of the problem!

Although I think programming by contract is worthwhile, I still contend
that there
is no compelling reason to believe it would have avoided *this
particular problem*, for
two reasons:

1. There is strong evidence to believe that the Ariane IV SRI team did
not see this
   as a particularly important assertion to make (otherwise, they would
not have
   suppressed the check). In the context of what they knew (the Ariane
IV flight
   profile), their opinion was at least understandable if not
justifiable.

2. There is some evidence to indicate that the Ariane V team would not
have considered
   themselves in violation of the assertion, even if it had been
explictly coded. Apparently,
   they did not challenge the decision of the Ariane IV team to suppress
the exception
   checks for the unit. Based on my experience in similar situations, it
is sometimes
   difficult to translate such high-level information as a flight
profile to the specific
   range of parameters to be passed to a particular unit. Additionally,
there is no
   evidence that the software team, reading the source code, had
sufficient systems
   knowledge and experience to detect the violation of the assertion
strictly from reading
   the code.

Based on my experience with aerospace systems, and in particular
integrated inertial and
flight control systems (from the flight controls side), I see this
scenario as entirely
possible. In this case, no amount of documentation at the code level
would have solved
the problem, and no amount of embedded run-time checks would solve the
problem in the
absence of realistic testing.
> 
> --
> Jean-Marc Jezequel               Tel : +33 2 99847192
> IRISA/CNRS                       Fax : +33 2 99847171
> Campus de Beaulieu               e-mail : jezequel@irisa.fr
> F-35042 RENNES (FRANCE)          http://www.irisa.fr/pampa/PROF/jmj.html

--
LMTAS - The Fighter Enterprise - "Our Brand Means Quality"
For job listings, other info: http://www.lmtas.com or
http://www.lmco.com