From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on polar.synack.me X-Spam-Level: X-Spam-Status: No, score=-1.3 required=5.0 tests=BAYES_00,INVALID_MSGID autolearn=no autolearn_force=no version=3.4.4 X-Google-Language: ENGLISH,ASCII-7-bit X-Google-Thread: fac41,a48e5b99425d742a X-Google-Attributes: gidfac41,public X-Google-Thread: 103376,a48e5b99425d742a X-Google-Attributes: gid103376,public X-Google-Thread: f43e6,a48e5b99425d742a X-Google-Attributes: gidf43e6,public X-Google-Thread: 1108a1,5da92b52f6784b63 X-Google-Attributes: gid1108a1,public From: Ken Garlington Subject: Re: Please do not start a language war (was Re: Papers on the Ariane-5 crash and Design by Contract Date: 1997/03/19 Message-ID: <33302A36.7434@lmtas.lmco.com>#1/1 X-Deja-AN: 226750502 References: <332B5495.167EB0E7@eiffel.com> <5giu3p$beb$1@news.irisa.fr> <332ED8AB.21E7@lmtas.lmco.com> <199703190839.JAA02652@stormbringer.irisa.fr> To: Jean-Marc Jezequel Organization: Lockheed Martin Tactical Aircraft Systems Newsgroups: comp.lang.eiffel,comp.object,comp.software-eng,comp.lang.ada Date: 1997-03-19T00:00:00+00:00 List-Id: Jean-Marc Jezequel wrote: > > In article <332ED8AB.21E7@lmtas.lmco.com>, Ken Garlington writes: > > > 2. No one ran a full integration test with realistic flight data, > >which would have > > alerted them to the mistake made in #1. Particularly for a > >distributed mission- > > critical system, this should be considered an absolute > >requirement. > > Yes, this is true. But you have to understand that the mere design of the SRI > made it very difficult to test it in an other way that performing a launch. > This is because of the tight integration of hard-to-fool hardware with software > in a black box functional unit. We have exactly the same coupling of inertials to flight controls on a current project, and we are able to test the coupled system in a black box environment in our labs, with pilots in the loop, performing the same flight profiles we expect to see in operation. Interestingly enough, the IRS for the this project is also "reused" from a different platform, and there was a proposal to minimize testing of the system because it had already been "proven." We were able to quash that "cost-savings" measure before it gained much support. However, in an environment with Cost As an Independent Variable (CAIV), I can certainly see how easy it would be to adopt such ideas. > What can be your test strategy for a black box > containing an inertial central? If the software had been designed with less coupling > on this particular hardware, you could have test the software vs. a simulation of the > hardware and its environment. Here, the launch was the test. Is this what Aerospatiale told you? > > >Although I like some of Mr. Meyer's columns, I found this particular > >column not only wrong, > >but dangerous, since it continues the trend of denigrating sound test > >strategy and common > >sense system engineering in favor of a software-centric approach. > > I don't see where it denigrates sound testing. IMHO, sound testing *is* needed. > And testing is effective only if you have a sound test strategy. > Even more when you use programming by contract. In the paper, > we just recall that you cannot rely on tests only. Then why in your message on comp.lang.ada, did you say that to a "lesser extent" this was an integration test problem? Not doing this testing was a *critical* part of the problem! Although I think programming by contract is worthwhile, I still contend that there is no compelling reason to believe it would have avoided *this particular problem*, for two reasons: 1. There is strong evidence to believe that the Ariane IV SRI team did not see this as a particularly important assertion to make (otherwise, they would not have suppressed the check). In the context of what they knew (the Ariane IV flight profile), their opinion was at least understandable if not justifiable. 2. There is some evidence to indicate that the Ariane V team would not have considered themselves in violation of the assertion, even if it had been explictly coded. Apparently, they did not challenge the decision of the Ariane IV team to suppress the exception checks for the unit. Based on my experience in similar situations, it is sometimes difficult to translate such high-level information as a flight profile to the specific range of parameters to be passed to a particular unit. Additionally, there is no evidence that the software team, reading the source code, had sufficient systems knowledge and experience to detect the violation of the assertion strictly from reading the code. Based on my experience with aerospace systems, and in particular integrated inertial and flight control systems (from the flight controls side), I see this scenario as entirely possible. In this case, no amount of documentation at the code level would have solved the problem, and no amount of embedded run-time checks would solve the problem in the absence of realistic testing. > > -- > Jean-Marc Jezequel Tel : +33 2 99847192 > IRISA/CNRS Fax : +33 2 99847171 > Campus de Beaulieu e-mail : jezequel@irisa.fr > F-35042 RENNES (FRANCE) http://www.irisa.fr/pampa/PROF/jmj.html -- LMTAS - The Fighter Enterprise - "Our Brand Means Quality" For job listings, other info: http://www.lmtas.com or http://www.lmco.com