From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on polar.synack.me X-Spam-Level: X-Spam-Status: No, score=-1.3 required=5.0 tests=BAYES_00,INVALID_MSGID autolearn=no autolearn_force=no version=3.4.4 X-Google-Language: ENGLISH,ASCII-7-bit X-Google-Thread: fac41,a48e5b99425d742a X-Google-Attributes: gidfac41,public X-Google-Thread: 1108a1,5da92b52f6784b63 X-Google-Attributes: gid1108a1,public X-Google-Thread: f43e6,a48e5b99425d742a X-Google-Attributes: gidf43e6,public X-Google-Thread: 103376,a48e5b99425d742a X-Google-Attributes: gid103376,public From: eachus@spectre.mitre.org (Robert I. Eachus) Subject: Re: Trust but verify (was Re: Papers on the Ariane-5 crash and Design by Contract Date: 1997/03/27 Message-ID: #1/1 X-Deja-AN: 228830258 References: <332B5495.167EB0E7@eiffel.com> Organization: The Mitre Corp., Bedford, MA. Newsgroups: comp.lang.eiffel,comp.object,comp.software-eng,comp.lang.ada Date: 1997-03-27T00:00:00+00:00 List-Id: In article <3338AA0A.7A79CB24@eiffel.com> Bertrand Meyer writes: > But of course this is not a reason to say there should not be > simulation, testing, inspections, independent reviews etc. etc. > I don't recall anyone even coming close to suggesting this in the > present thread. And our paper certainly does not say anything of > the sort. But the Final Report on the Ariane 5 failure does say that there were NO simulation, testing, inspections, independent reviews etc. etc. for the Ariane 5. Not minimal, not inadequate, none. And this is where your paper misleads. Any "reasonable" methods would be better than no testing, no inspections, and no reviews. > It would be as absurd to use Design by Contract or other a priori > relability techniques as an excuse not to do a posteriori V & V > (testing, simulation etc.), as it would be to develop in a sloppy > way with the expectation that testing will uncover any problems. But in this case, effectively both excuses were used. No requirements review was done because there would be full up testing, then the full-up test was cancelled because the software had not changed! > But the need for a posteriori verification is too often an excuse > for not applying the proven a priori reliability techniques, such > as Design by Contract. This is not only silly but a recipe for > disaster of Ariane-5 proportions. Yep, and doing neither is not a recipe, it is a guarantee. > I think that its arguments and conclusion are perfectly in line with > many useful comments made by people who thought they disagreed with it. > Please check what it really has to say. What Bertrand Meyer continues to miss, probably because it is so difficult to believe, is that the software (and hardware) were reused without any requirements review or testing. None. Not inadequate review or incomplete testing, none. His paper assumes that review and testing occured but missed a subtle point. There were several glaringly obvious problems that would not have been missed by any competent engineer--they probably wouldn't have been missed by a teenager whose only experience was in model rocketry: 1) The alignment software continued running for 40 seconds after launch, for reasons documented in the code but totally irrelevant to Ariane 5. 2) The permissible engine deflection angles were based on Ariane 4 requirements, and could (and did) result in destruction of an Ariane 5. (Would lesser max deflection values have met Ariane 5 requirements for crosswind tolerance and wind shear? No clue, the analysis wasn't done.) 3) There were several other analyses that depended on knowledge of Ariane 4 flight paths and physical limits. Again these were very invalid but not redone. (This includes the failure that triggered the entire chain of events.) 4) The assumption--well documented--in the code that any Ada exception not locally handled was due to hardware not software failure. In Ariane 4, this assumption may be correct. But in Ariane 5, the decision should have been reconsidered, at least for the first test flight. I was determined that I had made my last post on this subject, but I got sucked in again. I want to make sure that no one misunderstands the nature of this failure--it was a failure of management, not of software engineering. And it wasn't even software management that failed, the politics that caused the failure occured at a much higher management level. In this it was very similar to the Apollo fire and the Challenger disaster. In the Apollo fire, two tests with very different requirements were merged to "speed things up." Three psi oxygen and three psi above ambient atmosphere are two very different things. But the tests were merged after the safety engineers had signed off on them separately. George Low was brought in to take over the Apollo program after the fire, and he instituted procedures to prevent anything similar from ever happening again on a manned test or manned flight. These procedures were followed until STS 25. (Although there were a number of leaks of "pressure to maintain the launch schedule" heard before then, the final flight worthiness decision for the shuttle and all subsystems was always made by an engineer.) -- Robert I. Eachus with Standard_Disclaimer; use Standard_Disclaimer; function Message (Text: in Clever_Ideas) return Better_Ideas is...