From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on polar.synack.me X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham autolearn_force=no version=3.4.4 X-Google-Language: ENGLISH,ASCII-7-bit X-Google-Thread: fac41,f66d11aeda114c52 X-Google-Attributes: gidfac41,public X-Google-Thread: 103376,f66d11aeda114c52 X-Google-Attributes: gid103376,public From: Bertrand Meyer Subject: Re: Critique of Ariane 5 paper (finally!) Date: 1997/08/17 Message-ID: <33F7C3C0.446B9B3D@eiffel.com> X-Deja-AN: 265586486 References: <33E503B3.3278@flash.net> <33E8FC54.41C67EA6@eiffel.com> <33E9B217.39DA@flash.net> <33EA5592.5855@flash.net> <33EB4935.167EB0E7@eiffel.com> <33EB754E.446B9B3D@eiffel.com> <33EBE46D.2149@flash.net> <33EF9487.41C67EA6@eiffel.com> <33F22B91.167EB0E7@eiffel.com> Organization: Interactive Software Engineering Inc. Newsgroups: comp.lang.ada,comp.lang.eiffel Date: 1997-08-17T00:00:00+00:00 List-Id: Quoting my message that said among other things: >>> The "necessary" condition is simple: you won't get reliable >>> software unless you take care to associate specifications -- >>> contracts -- with software elements, especially reusable >>> components. You can then use these contracts to build >>> the software, to document it, to validate it statically, >>> to test it (if you have support for run-time monitoring, >>> as in Eiffel), to control the proper use of inheritance, >>> to discuss the software within the software team as well >>> as with management, customers, users and regulatory agencies. >>> Anyone who has used the approach knows that it dramatically >>> improves the quality of the product and the process. >>> And it's not even an expensive technique.>>>, Robert Dewar writes: > This is demonstrably false. There are lots of examples of highly reliable > software written by people who don't even know what a specification is, > let alone how to carefully associate them with software elements. > > If you want details on this, I can send you hundreds of thousands of > lines of COBOL code. This code is completely inpenetrable in places, > and I consider it pretty horrible, but it is from a completely reliable > system, where reliability is measured in the terms that matter, i.e. > it does what it is supposed to do in a highly reliable manner. This is eloquently said, but incorrect all the same. The definition of reliability which this implies is that a system is "highly reliable" if it has been working satisfactorily for, say, 30 {seconds | minutes | hours | days | weeks | months | years} -- pick one. This is one possible definition of reliability, which gets more and more interesting as it moves to the right of the list of choices; but it is by no means the only "terms that matter". True, it may be pragmatically sufficent in some cases. For example if you tell me my car insurance is being computed by a program that has worked "satisfactorily" for 10 years, I may be reasonably pacified. After all, even though I know that processing my case may hit a bug that no one encountered before, the (damage * likelihood) factor is small enough for me to accept the risk (although, being a computing scientist, I might ask for evidence of what exactly you mean by "satisfactorily" or even, using Prof. Dewar's terms, "highly reliably"). But for a mission-critical system of the kind that started this discussion -- one, such as Ariane, whose non-high-reliability may imply loss of life, or destruction of huge amounts of property, or some other catastrophic event -- that is not good enough. "The software has passed the acceptance tests" or even "the software has worked properly in the first two missions" is interesting, but not sufficient to allay the concerns of, e.g., regulatory agencies. I have seen requirement specifications for mission-critical systems which include formal conditions such as "at most one failure for each 10^n passengers", where n is a very large number -- corresponding, say, to 500-year continuous operation. Leaving aside the question of whether we can honestly make such promises (since the scientific debate should not prevent us to try our best as engineers), we are NOT going to reassure, let alone satisfy our censors through pragmatic reliability arguments of the form "it has worked well so far, ergo it is reliable". They will listen with interest but they will demand more. Part of what they will demand will be guarantee about our software process -- something in the line of ISO 900x, Capability Maturity Model, quality assurance plan and the like. But they will also want to know what software techniques we use and, in particular, why we believe that each software component does its job -- especially a reusable software component pulled out from somewhere else. This is where there is simply no substitute for Design by Contract: a necessary condition, as I wrote. By no means sufficient (it makes sense as part of an array of reliability mechanisms, some technical, some organizational); but, until someone comes with a better idea to guarantee reliability, necessary. The recently deceased Harlan Mills, then of IBM, published in 1975 (Proc.Intl. Conf. on Software Reliability, see reference in OOSC-2) a paper entitled "How to write correct programs, and know it." This title truly capture the essence of the problem. It is not sufficient, as Prof. Dewar seems to imply, to write reliable programs (where reliability includes correctness) which we think are reliable because they have worked well enough for long enough. To convince others (and ourselves) that they are indeed reliable, we need better arguments. These arguments can only come from a precise description of what the elements are supposed to do -- the contracts -- and some evidence that the implementation does honor the contracts. Without these mechanisms in place, we cannot even discuss properly the reliability of our software, since we haven't even defined what we are talking about. So when Robert Dewar writes > "Associating specifications with software elements" is an important tool > that will aid in the production of reliable software. > Of course no one will disagree with that, what people might disagree with > is the huge leap to say > "Therefore any system that does not use DBC is not reliable" I believe he is wrong on the second point, at least if we replace "reliable", in the last line, by "demonstrably reliable" (using "demonstrably" not in the absolute mathematical sense but in the practical sense of being able to convince competent colleagues). My view here is, apparently at least, the exact reverse of his. There is no claim whatsoever that Design by Contract is "sufficient". I would be happy to know a magical recipe to guarantee software reliability, but I don't. I do know, however, techniques that tremendously improve reliability (techniques that have saved me and many other people countless bugs, including some potentially spectacular ones), and without which it is not possible, in the state of software technology as I understand it, to achieve similar results. In short, necessary conditions. I understand that not everyone is convinced, and that some may think the claims are too grandiose even though they are really very modest and down-to-earth; but that does not diminish the relevance of these techniques to software reliability, the importance for every software engineer of learning them, and, yes, their necessity. -- Bertrand Meyer, President, ISE Inc. ISE Building, 2nd floor, 270 Storke Road, Goleta CA 93117 805-685-1006, fax 805-685-6869, http://www.eiffel.com, with instructions for free download == ISE Eiffel 4: Eiffel from those who invented it ==