* Re: The stupidity of all the Ariane 5 analysts. @ 1997-08-18 0:00 Joerg Rieling 1997-08-19 0:00 ` Robert S. White 0 siblings, 1 reply; 26+ messages in thread From: Joerg Rieling @ 1997-08-18 0:00 UTC (permalink / raw) hi all, ok, i perfectly agree with some of the points you made concerning inherent robustness etc etc etc. but, a minimum requirement for any software is to test it under sufficiently realistic conditions, isn't it ? if the responsible software engineers would have fed _simulated_ data containing some information about the trajectory into their programs, they could (would) have found the bug, d'accord ? sometimes experimental programming is not the worst choice ... bye j"org. -- <----> (- PGP key available.-) ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: The stupidity of all the Ariane 5 analysts. 1997-08-18 0:00 The stupidity of all the Ariane 5 analysts Joerg Rieling @ 1997-08-19 0:00 ` Robert S. White 0 siblings, 0 replies; 26+ messages in thread From: Robert S. White @ 1997-08-19 0:00 UTC (permalink / raw) In article <5ta2e6$itn$1@enterprise.desy.de>, joerg@hera-b.desy.de says... > >but, a minimum requirement for any software is to test it under >sufficiently realistic conditions, isn't it ? if the responsible >software engineers would have fed _simulated_ data containing some >information about the trajectory into their programs, they could >(would) have found the bug, d'accord ? That has been _exactly_ one of my positions as an INS (IRS) practitioner! Who exactly are you addressing your comments to? BM, JY, or DH? The other problem, as pointed out, has been that there was software re-use without a correct analysis of its suitability by problem domain experts. A system requirements review with the involvement of the right problem domain experts _could_ have determined by analysis whether the Ariane 4 IRS software should have been reused as is. The failsafe for this review _should_ have been proper simulation testing as you describe. _____________________________________________________________________ Robert S. White -- An embedded systems software engineer ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: The stupidity of all the Ariane 5 analysts.
@ 1997-08-19 0:00 Marin David Condic, 561.796.8997, M/S 731-96
0 siblings, 0 replies; 26+ messages in thread
From: Marin David Condic, 561.796.8997, M/S 731-96 @ 1997-08-19 0:00 UTC (permalink / raw)
Joerg Rieling <joerg@HERA-B.DESY.DE> writes:
>but, a minimum requirement for any software is to test it under
>sufficiently realistic conditions, isn't it ? if the responsible
>software engineers would have fed _simulated_ data containing some
>information about the trajectory into their programs, they could
>(would) have found the bug, d'accord ?
>sometimes experimental programming is not the worst choice ...
>
This is exactly the point that has so many people torqued off
about the whole debate. (Especially those of us who actually build
this sort of software for a living.) The claim seems to be on the
table that a) Existing software development practices are
insufficient for developing safe software and b) the only way to
be sure that you don't blow up a rocket is to use Design By
Contract as exemplified by Eiffel.
The problem is - as those of us in the business know - existing
practices, HAD THEY BEEN FOLLOWED, would have caught the problem.
(Your example of testing the software across the expected flight
envelope.) So would lots of other existing practices. The problem
is that the human beings in charge of the process didn't follow
the usual and customary procedures (probably under cost and
schedule pressure). Hence, no technology would likely have helped
keep them out of the Gulag they are now most likely consigned to.
(It's sad too for those of us who have ever experienced the "Hurry
up and get it done no matter what the risk..." pressure that can
be brought onto a project.)
DBC and Assertions, in my not so humble opinion, would have proven
to make no significant difference in the situation. DBC is a kind
of process and apparently the people in charge were into
circumventing processes. So DBC could have as easily been
circumvented for the same reasons as they circumvented existing,
industry accepted and used processes. (which, IMNSHO, would
probably have caught the problem had they been followed.) If you
walk a tightrope without a net often enough, eventually you'll
splat up against the pavement.
Assertions are just another kind of runtime check which - although
useful - would probably have made no difference either. The reason
being that the existing checks were disabled (for good reason)
and so assertions would also have been disabled. Disabled checks
don't change anything. The notion that they would have served as
documentation implies some kind of review/testing and go back to
the above paragraph on DBC. They had already circumvented that
part of the game, so forget that argument.
The seriously flawed notion that the runtime checks be turned
on during testing and then disabled for production only displays
lack of knowledge of how flight qualified software is really
built. Nobody is going to go through any sort of serious
verification of a software package only to allow you to change
some of the bits afterwards and claim "well, it's really the same
thing so let's go fly with it..." That might be O.K. for testing
something like Microsoft Word, but it will never pass muster for
safety critical software. Hence, you run with them in or you run
with them out. Running with them in might have done something to
save the day - but then so would the existing range checks that
were taken out.
MDC
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: Safety-critical development in Ada and Eiffel @ 1997-07-12 0:00 Ken Garlington 1997-07-15 0:00 ` Don Harrison 0 siblings, 1 reply; 26+ messages in thread From: Ken Garlington @ 1997-07-12 0:00 UTC (permalink / raw) Don Harrison wrote: > > If even this small overhead is unacceptable, you can force static binding > where necessary by freezing routines. > Although you forgo the flexibility of > polymorphism, you can still take advantage of inheritance by virtue of Eiffel > synonyms - these allow you to declare a routine under different names. But I can do this in Ada, as well, so I don't see the benefit. :Does this "pre-allocated memory pool" mean that you will know exactly > :what address > :each object's data will reside at, or merely that you dynamically > :allocate memory > :at initialization and do not dynamically allocate after initialization > :is complete? > > The latter. You don't actually need to know *where* objects are allocated, > merely that there is sufficient memory to allocate them. Actually, in my systems, I do need to know where they are allocated, so that I can examine them from the hideously crude display in the cockpit during troubleshooting. > Maybe general assertions are more common than you imagine. However, I agree > that range constraints are roughly covered by Ada subtypes. I use them liberally > for just that purpose. Maybe, but again, I would like to see how these complex assertions are used. Again, the code I've seen to date doesn't seem to use them that much. > Correct me I misunderstand you.. Are you wondering whether contracting > helps you to identify more bugs? I can offer my own experience which has > firmly convinced me of this. Contracting definitely identifies more bugs. However, my concern with Eiffel assertions (and Ada assertions, for that matter) is that, once raised, it's not always clear what to do about them. In a safety-critical system such as we build, you can't just generate a core dump, or ask the user what to do next. You have to generate a correct response to the assertion failure, and you have to do it very quickly. Static assertion checks, of course, can be detected and corrected during development, which is why I like Ada's strong static type checking. There is a school of thought that says to add in assertions, test the system, and then remove/suppress the exceptions. I have seen too many cases of code that works with the assertions active, and then breaks (due to timing differences, or more likely code generation errors) to have much confidence in this approach. Another consequence of using assertions is that you have to develop and test the assertions. As a result, you need confidence that the benefits of the assertions outweigh the dilution of your test effort. One of the advantages of Ada's simpler approach (at least, it seems simpler to me) to simple pre/post-conditions (range checks in particular) is that the cost of adding them is somewhat reduced. However, in both languages, overuse of assertions may be just as bad as underuse. (As you can tell, I'm somewhat of an assertions heretic. However, I will say that for non-real-time, non-safety-critical systems, they are outstanding. My tool code is lousy with them.) > Consequently, when we (a team of 5) recently > reached the end of a 5 month long development effort of incremental coding > and integration testing, the parts of the system that used contracting work > flawlessly apart from problems resulting from misinterpretation of requirements. Most of the problems we see in formal testing are from misinterpretation of requirements - which is another issue I have with assertions as a magic safety wand. Of course, Bertand Meyer's Eiffel website insists that even misinterpretation of requirements (Ariane V) will be a problem no longer when Eiffel is used! :) ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: Safety-critical development in Ada and Eiffel 1997-07-12 0:00 Safety-critical development in Ada and Eiffel Ken Garlington @ 1997-07-15 0:00 ` Don Harrison 1997-07-15 0:00 ` Ken Garlington 0 siblings, 1 reply; 26+ messages in thread From: Don Harrison @ 1997-07-15 0:00 UTC (permalink / raw) Ken Garlington wrote: :Don Harrison wrote: :> :> If even this small overhead is unacceptable, you can force static binding :> where necessary by freezing routines. :> Although you forgo the flexibility of :> polymorphism, you can still take advantage of inheritance by virtue of Eiffel :> synonyms - these allow you to declare a routine under different names. : :But I can do this in Ada, as well, so I don't see the benefit. I'm not saying there is any extra benefit - merely that you don't have to wear the overhead of dynamic binding if you're desperate to maximise efficiency. Tucker mentioned that overhead. I'm saying it isn't an issue. BTW, how do call a procedure under different names in Ada? : :Does this "pre-allocated memory pool" mean that you will know exactly :> :what address :> :each object's data will reside at, or merely that you dynamically :> :allocate memory :> :at initialization and do not dynamically allocate after initialization :> :is complete? :> :> The latter. You don't actually need to know *where* objects are allocated, :> merely that there is sufficient memory to allocate them. : :Actually, in my systems, I do need to know where they are allocated, so :that :I can examine them from the hideously crude display in the cockpit :during :troubleshooting. That *is* hideously crude (and brings back some bad memories). Depending on how crude you mean, maybe a primitive tool could give you a map of where objects get put in memory. If you mean incredibly crude, use non-reference (expanded in Eiffel) types and use a linker-generated memory map. :> Maybe general assertions are more common than you imagine. However, I agree :> that range constraints are roughly covered by Ada subtypes. I use them liberally :> for just that purpose. : :Maybe, but again, I would like to see how these complex assertions are :used. You would use them anywhere that a piece of code makes assumptions. For example, to help avoid the Ariane fiasco, include contracts in the INS(?) that specify Ariane 4 dynamics. Then, in testing, you will get an assertion violation when you apply Ariane 5 dynamics to it. :Again, the code I've seen to date doesn't seem to use them that much. If you're looking in Ada code, you're looking in the wrong place because few Ada developers recognise there are additional benefits from using them in addition to static typing and the predefined contracts already in Ada (the conditions under which predefined exceptions - eg. Constraint_Error - are raised). Static typing gives you static contract checking. Assertions (predefined and user-defined give you dynamic contract checking *during development*. What most Ada software engineers miss out on is the user-defined variety which bring a non-trivial benefit to reliability and reuse. :> Correct me I misunderstand you.. Are you wondering whether contracting :> helps you to identify more bugs? I can offer my own experience which has :> firmly convinced me of this. : :Contracting definitely identifies more bugs. However, my concern with :Eiffel :assertions (and Ada assertions, for that matter) is that, once raised, :it's :not always clear what to do about them. They are only turned on during development, so what you do is fix the underlying problem which is usually not difficult to find if you've used them systematically. Even if you haven't (I was the only one of a team of 5 using them in our recent development cycle), they still tell you more than you would otherwise be told. Good design complemented by static typing and Design by Contract is unbeatable, IME. :In a safety-critical system such :as :we build, you can't just generate a core dump, or ask the user what to :do next. :You have to generate a correct response to the assertion failure, and :you have :to do it very quickly. Right. However, only an incorrect program will raise assertion violations at runtime. As you systematically identify and fix the errors causing them during development, you can turn assertions off with confidence when the system goes live. Then, any errors you may have missed will be handled by exception handlers in the usual way. :Static assertion checks, of course, can be :detected and :corrected during development, which is why I like Ada's strong static :type checking. I agree completely. Static typing is great but it isn't the *whole* story. :There is a school of thought that says to add in assertions, test the :system, and :then remove/suppress the exceptions. I have seen too many cases of code :that works :with the assertions active, and then breaks (due to timing differences, Yes, timing *is* something to be concerned about when applied to realtime systems. That's why I suggested not using assertions in the hard realtime threads of a process. BTW, software designed using DBC may actually run *faster* than code without it because you get to strip out all the defensive validity checks. You have already ascertained during development that operations are called in valid contexts so you don't have to check the context. :or more :likely code generation errors) to have much confidence in this approach. If that's a problem, you need to get your vendor to clean their act up. :Another consequence of using assertions is that you have to develop and :test the :assertions. Correct. My initial reaction when I started using them was "Gee, I've got to do this extra work on top of writing the "real" code! However, I found I was spending about a third of the time integrating compared with my colleagues which meant I was saving time overall and producing more reliable code to boot. :As a result, you need confidence that the benefits of the :assertions :outweigh the dilution of your test effort. IME, they do. :One of the advantages of :Ada's simpler :approach (at least, it seems simpler to me) to simple :pre/post-conditions (range :checks in particular) is that the cost of adding them is somewhat :reduced. I disagree. It takes me about the same amount of time to specify an Ada subtype as it does to specify a more general assertion. Even if the general assertion is a function, I would more than likely need to write it anyway so would have expended the effort anyway. :However, :in both languages, overuse of assertions may be just as bad as underuse. It is possible to overuse them but 99% of developers err on the othger side. :) :(As you can tell, I'm somewhat of an assertions heretic. That's fine but have a go at using them where appropriate. I'm sure you'll become convinced. :However, I will :say that :for non-real-time, non-safety-critical systems, they are outstanding. My :tool code :is lousy with them.) : :> Consequently, when we (a team of 5) recently :> reached the end of a 5 month long development effort of incremental coding :> and integration testing, the parts of the system that used contracting work :> flawlessly apart from problems resulting from misinterpretation of requirements. : :Most of the problems we see in formal testing are from misinterpretation :of :requirements - which is another issue I have with assertions as a magic :safety wand. No use of assertions will stop you misinterpreting requirements. They're not a magic wand but they *will* help you remove a class of errors you would otherwise miss. :Of course, Bertand Meyer's Eiffel website insists that even :misinterpretation of :requirements (Ariane V) will be a problem no longer when Eiffel is used! Come on. He's not saying that. Don. =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Don Harrison donh@syd.csa.com.au ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: Safety-critical development in Ada and Eiffel 1997-07-15 0:00 ` Don Harrison @ 1997-07-15 0:00 ` Ken Garlington 1997-07-16 0:00 ` Jean-Marc Jezequel 0 siblings, 1 reply; 26+ messages in thread From: Ken Garlington @ 1997-07-15 0:00 UTC (permalink / raw) Don Harrison wrote: > > Ken Garlington wrote: > > :Don Harrison wrote: > :> > :> If even this small overhead is unacceptable, you can force static binding > :> where necessary by freezing routines. > :> Although you forgo the flexibility of > :> polymorphism, you can still take advantage of inheritance by virtue of Eiffel > :> synonyms - these allow you to declare a routine under different names. > : > :But I can do this in Ada, as well, so I don't see the benefit. > > I'm not saying there is any extra benefit - merely that you don't have to > wear the overhead of dynamic binding if you're desperate to maximise efficiency. > Tucker mentioned that overhead. I'm saying it isn't an issue. But you also said that it was required to use that part of Eiffel inheritance not available in Ada. In this statement, you seem to be saying that if I want an efficient safety-critical system (which I always do), I have to give us some of that extra inheritance. Do I need the full Eiffel inheritance model, or don't I? > > BTW, how do call a procedure under different names in Ada? Via the "rename" statement. > > : :Does this "pre-allocated memory pool" mean that you will know exactly > :> :what address > :> :each object's data will reside at, or merely that you dynamically > :> :allocate memory > :> :at initialization and do not dynamically allocate after initialization > :> :is complete? > :> > :> The latter. You don't actually need to know *where* objects are allocated, > :> merely that there is sufficient memory to allocate them. > : > :Actually, in my systems, I do need to know where they are allocated, so > :that > :I can examine them from the hideously crude display in the cockpit > :during > :troubleshooting. > > That *is* hideously crude (and brings back some bad memories). Depending on how > crude you mean, maybe a primitive tool could give you a map of where objects get > put in memory. If you mean incredibly crude, use non-reference (expanded in > Eiffel) types and use a linker-generated memory map. Both approaches require that the memory address be known before execution time, however, so I still care where objects are located. I also care for other reasons, such as memory-mapped I/O, of course. > > :> Maybe general assertions are more common than you imagine. However, I agree > :> that range constraints are roughly covered by Ada subtypes. I use them liberally > :> for just that purpose. > : > :Maybe, but again, I would like to see how these complex assertions are > :used. > > You would use them anywhere that a piece of code makes assumptions. For example, > to help avoid the Ariane fiasco, include contracts in the INS(?) that specify > Ariane 4 dynamics. Then, in testing, you will get an assertion violation when > you apply Ariane 5 dynamics to it. This is why my blood pressure goes up! The _Ada_ implementation did generate an assertion violation the very first time Ariane 5 dynamics were applied to it. Unfortunately, the very first time Ariane 5 dynamics were applied to it was at LANUCH! They were not applied earlier, because the contractor assumed that the Ariane 5 dynamics were the same as the Ariane 4, so why spend the money testing to the same conditions already tested earlier? (They were wrong, of course, but they didn't know that at the time). The other part of the problem was that the _response_ to the assertion violation was wrong. The designer assumed the assertion would occur due to a hardware failure, not a software requirements failure, and so the handler shut down the "offending" hardware, rather than attempting some other action (e.g. replacing the out-of-range value with a "safe" value). Frankly, I don't know that I would have written the handler differently myself. This is all clearly documented in the final report, available on the Web. According to Meyer, et. al. Eiffel programmers would have written the handler differently, and also they would have known to do the testing differently (even though the requirements specification said that the tests need not change). However, their explanation as to why this is true is far from convincing (basically, that "Eiffel programmers believe in quality" or some such nonsense). Their "analysis" is also on the Web. I've asked them to post my rebuttal to their paper, but I don't know if they ever did. This is why I dislike people with no background in safety-critical flight systems discussing such issues. They believe that "software is software." and apply the expertise in their domain to mine. They usually do about as good a job as when I try to discuss compiler design :) > > :Again, the code I've seen to date doesn't seem to use them that much. > > If you're looking in Ada code, you're looking in the wrong place because few > Ada developers recognise there are additional benefits from using them in > addition to static typing and the predefined contracts already in Ada > (the conditions under which predefined exceptions - eg. Constraint_Error - > are raised). No - I'm looking at Eiffel code. (Why would I look at Ada code to determine how Eiffel is used?) > Static typing gives you static contract checking. Assertions (predefined and > user-defined give you dynamic contract checking *during development*. What > most Ada software engineers miss out on is the user-defined variety which > bring a non-trivial benefit to reliability and reuse. Not this Ada engineer! I have dynamic assertions all over the place in my tools! My problem, again, is in the fundamental nature of dynamic assertions for safety-critical embedded systems, which have some unique risks, whether you (a) leave them in after development, or (b) take them out after developmental testing, but before usage. > > :> Correct me I misunderstand you.. Are you wondering whether contracting > :> helps you to identify more bugs? I can offer my own experience which has > :> firmly convinced me of this. > : > :Contracting definitely identifies more bugs. However, my concern with > :Eiffel > :assertions (and Ada assertions, for that matter) is that, once raised, > :it's > :not always clear what to do about them. > > They are only turned on during development, so what you do is fix the underlying > problem which is usually not difficult to find if you've used them systematically. > Even if you haven't (I was the only one of a team of 5 using them in our recent > development cycle), they still tell you more than you would otherwise be told. I am aware of at least one project developing an embedded missile controller, that did exactly what you describe (in Ada). Here's what happened: 1. Their timings changed, which introduced some subtle differences in the relationships between tasks and external interrupts. Some of these subtle differences invalidated their original testing. 2. Their object code changed (obviously), which required reanalysis of their object-code branch coverage. 3. At least one compiler bug was introduced, which was definitely unsettling. What safety-critical system did you develop where you turned off assertions afer development? What standard (DO-178?) did you use for certification? The other prong of this is that, while assertions are keen for catching design problems, I haven't seen much evidence that they catch high-level requirements problems (not surprising, since the code was presumably written to match the requirements). In 13 years of developing safety-critical embedded systems, our process tends to handle design problems fairly early. It's those high-level requirements problems where we need the most help. > > Good design complemented by static typing and Design by Contract is unbeatable, > IME. > > :In a safety-critical system such > :as > :we build, you can't just generate a core dump, or ask the user what to > :do next. > :You have to generate a correct response to the assertion failure, and > :you have > :to do it very quickly. > > Right. However, only an incorrect program will raise assertion violations at > runtime. As you systematically identify and fix the errors causing them during > development, you can turn assertions off with confidence when the system > goes live. Then, any errors you may have missed will be handled by exception > handlers in the usual way. > > :Static assertion checks, of course, can be > :detected and > :corrected during development, which is why I like Ada's strong static > :type checking. > > I agree completely. Static typing is great but it isn't the *whole* story. > > :There is a school of thought that says to add in assertions, test the > :system, and > :then remove/suppress the exceptions. I have seen too many cases of code > :that works > :with the assertions active, and then breaks (due to timing differences, > > Yes, timing *is* something to be concerned about when applied to realtime > systems. That's why I suggested not using assertions in the hard realtime > threads of a process. BTW, software designed using DBC may actually run *faster* > than code without it because you get to strip out all the defensive validity > checks. You have already ascertained during development that operations are > called in valid contexts so you don't have to check the context. But almost all of our safety-critical threads are hard realtime threads! That's a HUGE limitation for our systems! > > :or more > :likely code generation errors) to have much confidence in this approach. > > If that's a problem, you need to get your vendor to clean their act up. Easier said than done! :) What is the process to certify Eiffel compilers as being acceptably mature (code generation-wise) for safety-critical systems? Is all testing performed with assertions on and assertions off in all test cases? > > :Another consequence of using assertions is that you have to develop and > :test the > :assertions. > > Correct. My initial reaction when I started using them was "Gee, I've got to > do this extra work on top of writing the "real" code! However, I found I was > spending about a third of the time integrating compared with my colleagues > which meant I was saving time overall and producing more reliable code to boot. > > :As a result, you need confidence that the benefits of the > :assertions > :outweigh the dilution of your test effort. > > IME, they do. However, it sounds like you're talking about a non-safety-critical system, and certainly not a hard real-time embedded safety-critical systems. In that case, I agree. For tools, simulations, etc. I would use all of the assertions I could. However, what about the topic of this thread (safety-critical systems)? It sounds like there's a lot of limitations as to using (and inheriting) assertions for this environment. > > :One of the advantages of > :Ada's simpler > :approach (at least, it seems simpler to me) to simple > :pre/post-conditions (range > :checks in particular) is that the cost of adding them is somewhat > :reduced. > > I disagree. It takes me about the same amount of time to specify an Ada > subtype as it does to specify a more general assertion. Even if the general > assertion is a function, I would more than likely need to write it anyway so > would have expended the effort anyway. > > :However, > :in both languages, overuse of assertions may be just as bad as underuse. > > It is possible to overuse them but 99% of developers err on the othger side. :) > > :(As you can tell, I'm somewhat of an assertions heretic. > > That's fine but have a go at using them where appropriate. I'm sure you'll > become convinced. > > :However, I will > :say that > :for non-real-time, non-safety-critical systems, they are outstanding. My > :tool code > :is lousy with them.) > : > :> Consequently, when we (a team of 5) recently > :> reached the end of a 5 month long development effort of incremental coding > :> and integration testing, the parts of the system that used contracting work > :> flawlessly apart from problems resulting from misinterpretation of requirements. > : > :Most of the problems we see in formal testing are from misinterpretation > :of > :requirements - which is another issue I have with assertions as a magic > :safety wand. > > No use of assertions will stop you misinterpreting requirements. They're not > a magic wand but they *will* help you remove a class of errors you would > otherwise miss. > > :Of course, Bertand Meyer's Eiffel website insists that even > :misinterpretation of > :requirements (Ariane V) will be a problem no longer when Eiffel is used! > > Come on. He's not saying that. Read his analysis of Ariane V. He says _exactly_ that, and has defended saying that in multiple newsgroups. A smart person, but not someone who knows my domain particularly well. > > Don. > =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- > Don Harrison donh@syd.csa.com.au ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: Safety-critical development in Ada and Eiffel 1997-07-15 0:00 ` Ken Garlington @ 1997-07-16 0:00 ` Jean-Marc Jezequel 1997-07-16 0:00 ` Ken Garlington 0 siblings, 1 reply; 26+ messages in thread From: Jean-Marc Jezequel @ 1997-07-16 0:00 UTC (permalink / raw) In article <33CC0548.4099@flash.net>, Ken Garlington <keg0@flash.net> writes: >Don Harrison wrote: >> You would use them anywhere that a piece of code makes assumptions. For example, >> to help avoid the Ariane fiasco, include contracts in the INS(?) that specify >> Ariane 4 dynamics. Then, in testing, you will get an assertion violation when >> you apply Ariane 5 dynamics to it. > >This is why my blood pressure goes up! > >The _Ada_ implementation did generate an assertion violation the very >first time Ariane 5 dynamics were applied to it. Unfortunately, the very first time >Ariane 5 dynamics were applied to it was at LANUCH! >They were not applied earlier, because the contractor assumed that the >Ariane 5 dynamics were the same as the Ariane 4, so why spend the money testing >to the same conditions already tested earlier? (They were wrong, of course, but they >didn't know that at the time). At the risk of repeating myself, and reopening a thread beaten to death, the all point of design by contract (DBC) is to make this kind of assumptions explicit. Ariane 5 is just a nice striking example of working with assumptions that are true at a point in time (Ariane 4) and no longer later on (Ariane 5). I think we agreed on this previously. To sum up your point, you think that DBC, i.e. expressing hidden assumptions with Eiffel-like assertions would not have been practicable in this case. Others think it would have... You seem to be a specialist in flight software. We are not. But specialists might, from time to time, learn something from people with a broader perspective (this is a general statement, I do not claim in particular that I have a broader perspective than you, or anyone else for that matter;-). Basically, our arguments failed to convinced you. Yours failed to convince us. That's life, and I hope readers can make their mind themselves instead of relying on the one of us who shouts louder. >The other part of the problem was that the _response_ to the assertion >violation was wrong. The designer assumed the assertion would occur due to a hardware >failure, not a software requirements failure, and so the handler shut down the >"offending" hardware, rather than attempting some other action (e.g. replacing the >out-of-range value with a "safe" value). Frankly, I don't know that I would have written the >handler differently myself. Agreed, that's a different problem altogether. >According to Meyer, et. al. Eiffel programmers would have written the >handler differently, and also they would have known to do the testing >differently (even though the requirements specification said that the tests need not change). >However, their explanation as to why this is true is far from convincing (basically, >that "Eiffel programmers believe in quality" or some such nonsense). Their "analysis" >is also on the Web. I've asked them to post my rebuttal to their paper, but I >don't know if they ever did. Just for the record, I'm still waiting for your rebuttal paper to put it on my web alongside with the Computer article. PS: Ken's post was brought to my attention by a colleague: I don't have time to check the news very often these days, and anyway I won't follow-up on this thread to avoid reopening the interminable Ariane thread. -- Jean-Marc Jezequel Tel : +33 299 847 192 IRISA/CNRS Fax : +33 299 847 171 Campus de Beaulieu e-mail : jezequel@irisa.fr F-35042 RENNES (FRANCE) http://www.irisa.fr/pampa/PROF/jmj.html ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: Safety-critical development in Ada and Eiffel 1997-07-16 0:00 ` Jean-Marc Jezequel @ 1997-07-16 0:00 ` Ken Garlington 1997-07-17 0:00 ` The stupidity of all the Ariane 5 analysts Thaddeus L. Olczyk 0 siblings, 1 reply; 26+ messages in thread From: Ken Garlington @ 1997-07-16 0:00 UTC (permalink / raw) Jean-Marc Jezequel wrote: > > In article <33CC0548.4099@flash.net>, Ken Garlington <keg0@flash.net> writes: > >Don Harrison wrote: > > >> You would use them anywhere that a piece of code makes assumptions. For example, > >> to help avoid the Ariane fiasco, include contracts in the INS(?) that specify > >> Ariane 4 dynamics. Then, in testing, you will get an assertion violation when > >> you apply Ariane 5 dynamics to it. > > > >This is why my blood pressure goes up! > > > >The _Ada_ implementation did generate an assertion violation the very > >first time Ariane 5 dynamics were applied to it. Unfortunately, the very first time > >Ariane 5 dynamics were applied to it was at LANUCH! > > >They were not applied earlier, because the contractor assumed that the > >Ariane 5 dynamics were the same as the Ariane 4, so why spend the money testing > >to the same conditions already tested earlier? (They were wrong, of course, but they > >didn't know that at the time). > > At the risk of repeating myself, and reopening a thread beaten to death, > the all point of design by contract (DBC) is to > make this kind of assumptions explicit. Which would not have happened in the Ariane V case. > Ariane 5 is just a nice striking example > of working with assumptions that are true at a point in time (Ariane 4) and no longer > later on (Ariane 5). I think we agreed on this previously. Yes. However, the assumption would not have been documented via an Eiffel assertion, as I claimed and you/Meyer never refuted (except to exclaim, "casuistry!"). > > To sum up your point, you think that DBC, i.e. expressing hidden assumptions with > Eiffel-like assertions would not have been practicable in this case. > Others think it would have... The distinction, of course, is that I gavce specific reasons why it would not have happened. Please post the evidence stating that the assertion would have been added (beyond the bizarre "Eiffel programmers are more careful" argument). > You seem to be a specialist in flight software. We are not. But specialists might, > from time to time, learn something from people with a broader perspective > (this is a general statement, I do not claim in particular that I have a broader > perspective than you, or anyone else for that matter;-). Unfortunately, the reverse is never true. Generalists never listen to or learn from specialists. Consider the work of "generalists" that is incorporated into my code today: Ada (Ichbiah et. al.), object-based design principles (Gomaa/SPC, which in turn is based on a lot of previous work), safety-critical design (Levison and many others), testing practices (Beizer, et. al.)... there are many examples. In your work on Ariane V, what specialists in the flight software field (from Arianespace or elsewhere) did you use to review your paper prior to its publication? After its publication, how many specialists in this field publicly (or privately) supported your position? I certainly was not the only one to object publicly! > Basically, our arguments failed to convinced you. Unfortunately, you failed to even discuss some of the points I (and others) made. > Yours failed to convince us. > That's life, and I hope readers can make their mind themselves instead of > relying on the one of us who shouts louder. Yes, perhaps other criteria would be better. What would you suggest? Company revenues? I would have thought practical experience would be a strong determinant... > >The other part of the problem was that the _response_ to the assertion > >violation was wrong. The designer assumed the assertion would occur due to a hardware > >failure, not a software requirements failure, and so the handler shut down the > >"offending" hardware, rather than attempting some other action (e.g. replacing the > >out-of-range value with a "safe" value). Frankly, I don't know that I would have written the > >handler differently myself. > > Agreed, that's a different problem altogether. But an important one, if the assertion is left in the code. If the assertion is removed from the code, then other risks emerge. > > >According to Meyer, et. al. Eiffel programmers would have written the > >handler differently, and also they would have known to do the testing > >differently (even though the requirements specification said that the tests need not change). > >However, their explanation as to why this is true is far from convincing (basically, > >that "Eiffel programmers believe in quality" or some such nonsense). Their "analysis" > >is also on the Web. I've asked them to post my rebuttal to their paper, but I > >don't know if they ever did. > > Just for the record, I'm still waiting for your rebuttal paper to put it on my web alongside > with the Computer article. I thought I sent you the summary from the previous thread. I apologize. I will go into DejaNews and re-send it this weekend. > > PS: Ken's post was brought to my attention by a colleague: I don't have time to check the news > very often these days, and anyway I won't follow-up on this thread to avoid reopening the interminable > Ariane thread. Unfortunately, it's already been re-opened. As I feared, people are using your paper to "prove" the worth of DBC/Eiffel in safety-critical systems, although no one has yet published a list of such systems that have been written in Eiffel. > > -- > Jean-Marc Jezequel Tel : +33 299 847 192 > IRISA/CNRS Fax : +33 299 847 171 > Campus de Beaulieu e-mail : jezequel@irisa.fr > F-35042 RENNES (FRANCE) http://www.irisa.fr/pampa/PROF/jmj.html ^ permalink raw reply [flat|nested] 26+ messages in thread
* The stupidity of all the Ariane 5 analysts. 1997-07-16 0:00 ` Ken Garlington @ 1997-07-17 0:00 ` Thaddeus L. Olczyk [not found] ` <33CEAF05.6389@flash.net> 0 siblings, 1 reply; 26+ messages in thread From: Thaddeus L. Olczyk @ 1997-07-17 0:00 UTC (permalink / raw) Frankly I'm getting a little tired of all the stupidity of the language enthusiasts and methodologists who are out there saying "Ariane 5 crashed because they didn't use my language or my methodology". They are all ignoring one basic fact: the backup and primary systems were identical. Effectively this meant that most bugs or generic hardware defects were likely to be duplicated, eliminating most of the safety that a backup system provides. While not strongly conversant in the design of fault tolerant systems, I do know that some systems use backups which are the same and others use backups which are entirely different ( different hardware, software, and methodology ). I believe that it doesn't take a rocket scientist to figure out that among the later are thingfs like: missle launching systems, remote facilities and rockets. ^ permalink raw reply [flat|nested] 26+ messages in thread
[parent not found: <33CEAF05.6389@flash.net>]
* Re: The stupidity of all the Ariane 5 analysts. [not found] ` <33CEAF05.6389@flash.net> @ 1997-07-20 0:00 ` Bertrand Meyer 1997-07-21 0:00 ` Ian Begg ` (3 more replies) 0 siblings, 4 replies; 26+ messages in thread From: Bertrand Meyer @ 1997-07-20 0:00 UTC (permalink / raw) When someone resorts to changing the subject header of a thread to "The stupidity of all" those who have analyzed an issue and come to different views, it's a pretty good sign that he has run out of technical arguments. To repeat once again the basic point made in the paper by Jean-Marc Jezequel and myself: it is dangerous and unjustifiable, especially in a mission-critical setting, to reuse a software element without a specification. From the beginning to the end of the software lifecycle, Design by Contract encourages associating such specifications with everything that you write. The method applies on at least four levels: getting the stuff right in the first place, by writing the software elements (the description of *how* things are done) together with their contracts (*what* they are supposed to achieve); documenting them, through automatic extraction of contracts as the basic information on a class; using the contracts as one of the principal guides to reuse; and applying them to debugging, which becomes less of a blind chase and more of a focused analysis of possible discrepancies between intents (the contracts) and reality (the implementations). None of this automatically guarantees perfection, but it sure helps, as reported in these threads by every poster who had actually used Eiffel, where the ideas of Design by Contract are most directly realized. It is true that to someone who has not really tried it some of the benefits may appear exaggerated, but they're real. Just one case in which an incorrect call to a routine produces a violated precondition (caught right away at testing time, whereas it could in another approach have gone undetected for ages, and caused painful bugs) justifies the investment. In Eiffel development, this happens all the time. The Ariane case provides a textbook example of the dangers of not using contracts. Of course it is easy to dismiss this example through below-the-belt arguments, such as - "Jezequel and Meyer are not aerospace industry experts." True, but it doesn't seem the experts did so well in that case (and this is not the first major example of aerospace software failure either), so perhaps it's time to listen to others. Besides, the official report is clear and detailed enough to enable software professionals, aerospace experts or not, to form their own judgment. - "This is only a pitch for Eiffel". The paper by Jezequel and myself says explicitly that the Ariane disaster was not due to a language problem. In spite of this clear and obvious statement some of the Ada enthusiasts in these newsgroups have mistakenly taken the mention of Ariane as a personal affront on their language. The pitch, if any, is for the method of Design by Contract. That this method is directly reflected in Eiffel as a language, and less easy to apply in Ada, is a fact (and presumably in the case of Ada 95 a conscious design decision), not something to be held against the Eiffel community. - "Design by Contract is being promoted as a cure-all." It's not (not a cure-all, and not promoted as such). But it makes a major contribution. All this is rhetorics and cannot succeed to obscure the basic claim that systematic use of Design by Contract would probably have avoided the crash. Of course like any reconstruction of the past this conjecture cannot be proved, but there is enough evidence in the official report to support it. One can also object that other techniques would also have achieved the same goal, such as heavy-artillery a posteriori quality assurance, but they seem far more difficult and costly than integrating Design by Contract, a simple and easy-to-apply idea, into the design, implementation, documentation, reuse and validation process. Some of the negative reactions seem to recall arguments that were used against the proponents of structured programming in the 70s, and against those of object technology in the 80s. They are probably inevitable (and in part healthy, as they force the methodologists to clarify their ideas, refine them, and avoid overselling). But when a useful methodological principle surfaces, all the rhetoric in the world will not prevent its diffusion. -- Bertrand Meyer, President, ISE Inc., Santa Barbara (California) 805-685-1006, fax 805-685-6869, <Bertrand.Meyer@eiffel.com> Web: http://www.eiffel.com, with instructions for free download == ISE Eiffel 4: Eiffel straight from those who invented it == ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: The stupidity of all the Ariane 5 analysts. 1997-07-20 0:00 ` Bertrand Meyer @ 1997-07-21 0:00 ` Ian Begg 1997-07-21 0:00 ` Robert S. White ` (2 subsequent siblings) 3 siblings, 0 replies; 26+ messages in thread From: Ian Begg @ 1997-07-21 0:00 UTC (permalink / raw) On Sun, 20 Jul 1997 14:26:19 -0700, Bertrand Meyer <Bertrand.Meyer@eiffel.com> wrote: <snip> >To repeat once again the basic point made in the >paper by Jean-Marc Jezequel and myself: it is dangerous >and unjustifiable, especially in a mission-critical setting, >to reuse a software element without a specification. My personal opinion, I would add ".... or, probably more importantly, without fully testing that software element in the environment it is being re-used into, i.e. under the conditions it is expected to be used in." My understanding is that the expected flight profile was not flown as part of the integration or acceptance testing, and that had this been done the problem would (probably) have been found. This is what I mean by testing in the environment it is being re-used into. <snip> > - "This is only a pitch for Eiffel". The paper by > Jezequel and myself says explicitly that the > Ariane disaster was not due to a language problem. > In spite of this clear and obvious statement ... Even clear and obvious statements have to be assessed to decide on the intent of the author/speaker, for instance is he saying this because he means it or because he's trying to pull the wool over our eyes. [I have not read your report so have no idea how to answer in this case, everyone must make their own decision. The author must ensure they don't leave themselves open to the wrong interpretation.] > ... some of > the Ada enthusiasts in these newsgroups have > mistakenly taken the mention of Ariane as a personal > affront on their language. The pitch, if any, is for > the method of Design by Contract. How many languages support Design by Contract as part of the language as in the case of Eiffel? I don't have experience of a broad range of languages so I can't answer this. However, if the answer is only Eiffel, (plus perhaps some other non relevant langauges) then it is not difficult to see how people might interpret your motives as "OK we can't be so obvious as to say Eiffel is the answer, but if we say Design by Contract is the answer, people will need to find a language to support it, hence Eiffel." You are the only person who knows your real motives, but the discussions going on suggest some people are interpreting your motives in this way. <snip> > One can also object that other techniques would also >have achieved the same goal, such as heavy-artillery >a posteriori quality assurance, but they seem far more difficult >and costly than integrating Design by Contract, a simple >and easy-to-apply idea, into the design, implementation, >documentation, reuse and validation process. How easy is this to apply in non Eiffel implementations? I guess it needs extra work by the engineers, in an area most are probably not familiar with and will hence need training, and success is dependent on setting up good procedures to enforce its use etc. Compare the cost of that with doing integration and acceptance testing under anticipated use conditions which should be done anyway and I come to the conclusion use of Design by Contract was not the best way of preventing this accident. This is all of course my own personal opinion, and none of it should be read as an 'attack' on anyone elses opinions, only a difference of opinion. Sorry if this is repeating stuff from the original thread(s) which I have not been tracking closely. Ian Begg ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: The stupidity of all the Ariane 5 analysts. 1997-07-20 0:00 ` Bertrand Meyer 1997-07-21 0:00 ` Ian Begg @ 1997-07-21 0:00 ` Robert S. White 1997-07-21 0:00 ` Ken Garlington ` (2 more replies) 1997-07-21 0:00 ` Ken Garlington 1997-07-23 0:00 ` Joerg Rodemann 3 siblings, 3 replies; 26+ messages in thread From: Robert S. White @ 1997-07-21 0:00 UTC (permalink / raw) In article <33D2827B.41C67EA6@eiffel.com>, Bertrand.Meyer@eiffel.com says... > >To repeat once again the basic point made in the >paper by Jean-Marc Jezequel and myself: it is dangerous >and unjustifiable, especially in a mission-critical setting, >to reuse a software element without a specification. We "aerospace" people agree. >All this is rhetorics and cannot succeed to obscure the basic >claim that systematic use of Design by Contract would probably >have avoided the crash. The problem with this is that you are preaching to the choir. Long long ago, before Eiffel, the general concept of capturing system requirements in a spec document, flowing down to a software requirements document, conducting detailed design reviews with the requirment documents as resources/authorities (or getting them updated) and finally having software qualification tests done to prove performance to the software requirements spec has been practiced by the "aerospace" industry. The "crash" resulted because of a failure to fully follow this existing practice. Your papers about Eiffel and Design By Contract are just, IMO, another way to implement the concept of developing software that must comply to requirements. _____________________________________________________________________ Robert S. White -- An embedded systems software engineer ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: The stupidity of all the Ariane 5 analysts. 1997-07-21 0:00 ` Robert S. White @ 1997-07-21 0:00 ` Ken Garlington 1997-07-23 0:00 ` Robert S. White 1997-07-21 0:00 ` Robert Dewar 1997-07-22 0:00 ` W. Wesley Groleau x4923 2 siblings, 1 reply; 26+ messages in thread From: Ken Garlington @ 1997-07-21 0:00 UTC (permalink / raw) Robert S. White wrote: > > The "crash" resulted > because of a failure to fully follow this existing practice. Yes, I agree. (I decided not to address the statement in the Eiffel paper that says the opposite, it seemed to be piling on.) Hope you get a chance to review my draft critique. Geoff Eldridge <geldridg@progsoc.uts.edu.au> was kind enough to create an HTML version at: http://www.progsoc.uts.edu.au/~geldridg/eiffel/ariane/ BTW, I just noticed the flaws in the example Eiffel code at the last minute (midnight Saturday, in fact). Did you and everyone else know it was wrong, and just not say anything, or did we all overlook the errors? _____________________________________________________________________ > Robert S. White -- An embedded systems software engineer ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: The stupidity of all the Ariane 5 analysts. 1997-07-21 0:00 ` Ken Garlington @ 1997-07-23 0:00 ` Robert S. White 1997-07-23 0:00 ` Ken Garlington 1997-07-23 0:00 ` Robert Dewar 0 siblings, 2 replies; 26+ messages in thread From: Robert S. White @ 1997-07-23 0:00 UTC (permalink / raw) In article <33D3F2CB.61DC@flash.net>, kennieg@flash.net says... >BTW, I just noticed the flaws in the example Eiffel code at the >last minute (midnight Saturday, in fact). Did you and everyone >else know it was wrong, and just not say anything, or did we >all overlook the errors? Do you mean what you addres in your Critique's paragraph 3.1.1 about Meyer's Eiffel paper's solution code: "...any software element that has such a fundamental constraint should state it explicitly, as part of a mechanism present in the programming language, as in the Eiffel construct convert (horizontal_bias: INTEGER): INTEGER is require horizontal_bias <= Maximum_bias do ... ensure ... end where the precondition states clearly and precisely what the input must satisfy to be acceptable." Yes it looks like the above code should do a floating point absolute value on horizontal_bias before it checks against Maximum_bias. But I consider that a _typical_ coding error that is normally found by software code inspections or unit level tests - whoops - DBC/Eiffel does _not_ find this error! And yes, the test _must_ be done while the value is still expressed in floating point _before_ conversion to integer...details, details. Stuff we encounter all of the time in our problem domain. One could argue that the above is just a nit (flawed example) and the overall premise of DBC/Eiffel as a "silver bullet" panacea versus conventional "aerospace industry software engineering process" is the real issue. _____________________________________________________________________ Robert S. White -- An embedded systems software engineer (newsgroups trimmed to contain the "language wars") ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: The stupidity of all the Ariane 5 analysts. 1997-07-23 0:00 ` Robert S. White @ 1997-07-23 0:00 ` Ken Garlington 1997-07-25 0:00 ` Robert S. White 1997-07-23 0:00 ` Robert Dewar 1 sibling, 1 reply; 26+ messages in thread From: Ken Garlington @ 1997-07-23 0:00 UTC (permalink / raw) Robert S. White wrote: > > In article <33D3F2CB.61DC@flash.net>, kennieg@flash.net says... > > >BTW, I just noticed the flaws in the example Eiffel code at the > >last minute (midnight Saturday, in fact). Did you and everyone > >else know it was wrong, and just not say anything, or did we > >all overlook the errors? > > Do you mean what you addres in your Critique's paragraph 3.1.1 about > Meyer's Eiffel paper's solution code: > > "...any software element that has such a fundamental constraint should > state it explicitly, as part of a mechanism present in the programming > language, as in the Eiffel construct > > convert (horizontal_bias: INTEGER): INTEGER is > require > horizontal_bias <= Maximum_bias > do > ... > > ensure > ... > > end > > where the precondition states clearly and precisely what the input must > satisfy to be acceptable." > > Yes it looks like the above code should do a floating point absolute > value on horizontal_bias before it checks against Maximum_bias. Yes, this is what I mean. > But > I consider that a _typical_ coding error that is normally found by software > code inspections or unit level tests - whoops - DBC/Eiffel does _not_ > find this error! And yes, the test _must_ be done while the value is > still expressed in floating point _before_ conversion to integer...details, > details. Stuff we encounter all of the time in our problem domain. It would have been found if the design made it clear a different solution was necessary, or "common sense" was used in this example. > One could argue that the above is just a nit (flawed example) and > the overall premise of DBC/Eiffel as a "silver bullet" panacea versus > conventional "aerospace industry software engineering process" is > the real issue. However, I think there's a fundamental point made by this "nit." Several people read the paper; in fact, I saw one example in DejaNews last year where someone posted the Ada version of this code... incorporating the same mistakes! No one, as far as I could tell, saw this "obvious" error in three lines of code despite many months of opportunity. What does this say about the credibility of manual inspections at the source code level in picking up errors of this type during the reuse of software? > _____________________________________________________________________ > Robert S. White -- An embedded systems software engineer > (newsgroups trimmed to contain the "language wars") ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: The stupidity of all the Ariane 5 analysts. 1997-07-23 0:00 ` Ken Garlington @ 1997-07-25 0:00 ` Robert S. White 0 siblings, 0 replies; 26+ messages in thread From: Robert S. White @ 1997-07-25 0:00 UTC (permalink / raw) In article <33D69CAF.19F1@flash.net>, kennieg@flash.net says... > Robert S. White wrote: >> I consider that a _typical_ coding error that is normally found by software >> code inspections or unit level tests >It would have been found if the design made it clear a different >solution >was necessary, or "common sense" was used in this example. Perhaps I should have provided more detail about our industry software engineering process. During the detail design review of the software module, Test Cases, for doing the module unit level tests are also presented. One of the checklist items for the test cases is to ensure that the module is tested at boundry conditions and beyond boundry conditions - both for the positive and negative extremes in the case of a signed value. This type of software design process with inspections using checklists even against the test cases is a way to spread out that very rare/precious commondity called "common sense". _____________________________________________________________________ Robert S. White -- An embedded systems software engineer ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: The stupidity of all the Ariane 5 analysts. 1997-07-23 0:00 ` Robert S. White 1997-07-23 0:00 ` Ken Garlington @ 1997-07-23 0:00 ` Robert Dewar 1997-07-24 0:00 ` Ken Garlington 1997-07-29 0:00 ` Shmuel (Seymour J.) Metz 1 sibling, 2 replies; 26+ messages in thread From: Robert Dewar @ 1997-07-23 0:00 UTC (permalink / raw) Robert White said << One could argue that the above is just a nit (flawed example) and the overall premise of DBC/Eiffel as a "silver bullet" panacea versus conventional "aerospace industry software engineering process" is the real issue. >> Unfortunately the Ariane crash was the result of just such a nit! I would think experience would teach us to be very suspicious of any claims by anyone to have a silver bullet panacea wjhen it comes to software engineering for large systems. However tools and techniques that help, even a little bit, are welcome even if they are not a panacea (I consider Ada to be in this category, and I always find it annoying when people advertise Ada as being a silver bullet panacea). ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: The stupidity of all the Ariane 5 analysts. 1997-07-23 0:00 ` Robert Dewar @ 1997-07-24 0:00 ` Ken Garlington 1997-07-29 0:00 ` Shmuel (Seymour J.) Metz 1 sibling, 0 replies; 26+ messages in thread From: Ken Garlington @ 1997-07-24 0:00 UTC (permalink / raw) Robert Dewar wrote: > > Robert White said > > << One could argue that the above is just a nit (flawed example) and > the overall premise of DBC/Eiffel as a "silver bullet" panacea versus > conventional "aerospace industry software engineering process" is > the real issue. > >> > > Unfortunately the Ariane crash was the result of just such a nit! > > I would think experience would teach us to be very suspicious of any > claims by anyone to have a silver bullet panacea wjhen it comes to > software engineering for large systems. > > However tools and techniques that help, even a little bit, are welcome > even if they are not a panacea (I consider Ada to be in this category, > and I always find it annoying when people advertise Ada as being > a silver bullet panacea). I agree with this, but I also think every tool and technique has both advantages _and risks_. Failing to understand the risks, and where they are (and aren't) important, can lead to extremely dangerous choices made with the best of intentions. I view executable assertions (Ada, Eiffel, etc.) in this category. In some environments, they are clear wins. In other environments, I think it is at least debatable whether their advantages outweigh their risks. Complicating this factor is that many people confuse the abstract Good Thing (e.g. well-documented assumptions) with a specific implementation of the Good Thing. They consider the Good Thing so obvious that they don't consider (a) what Bad Things may also be associated with the _implementation_ -- the proponents of the implementation aren't always candid about this part (or are sometimes just blind to it) and (b) whether, for a given project, there are better ways to achieve the Good Thing. As you've said in the past, Computer Science is in part a misnomer in that there doesn't seem to be much in the way of controlled experiments. So, a lot of these risk/reward analyses are based on incomplete data, at best. This doesn't mean that they shouldn't be done, however. Ignoring the risks (or worse, dismisssing them as "they just haven't seen the light yet," as Mr. Meyer did recently) is not smart in any environment, and certainly not in safety-critical systems. ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: The stupidity of all the Ariane 5 analysts. 1997-07-23 0:00 ` Robert Dewar 1997-07-24 0:00 ` Ken Garlington @ 1997-07-29 0:00 ` Shmuel (Seymour J.) Metz 1997-07-31 0:00 ` Robert Dewar 1997-08-08 0:00 ` Don Harrison 1 sibling, 2 replies; 26+ messages in thread From: Shmuel (Seymour J.) Metz @ 1997-07-29 0:00 UTC (permalink / raw) Robert Dewar wrote: > > I would think experience would teach us to be very suspicious of any > claims by anyone to have a silver bullet panacea wjhen it comes to > software engineering for large systems. > > However tools and techniques that help, even a little bit, are welcome > even if they are not a panacea (I consider Ada to be in this category, > and I always find it annoying when people advertise Ada as being > a silver bullet panacea). I would go farther; I am suspicious of anyone who advocates a particular languages and can't tell me anything in the language that he dislikes. If he can't identify flaws, either he's too inexperienced to rely on or isn't perceptive enough. *any* programming language will have warts, and a real professional will recognize them and have strategies for dealing with them. -- Shmuel (Seymour J.) Metz Senior Software SE The values in from and reply-to are for the benefit of spammers: reply to domain eds.com, user msustys1.smetz or to domain gsg.eds.com, user smetz. ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: The stupidity of all the Ariane 5 analysts. 1997-07-29 0:00 ` Shmuel (Seymour J.) Metz @ 1997-07-31 0:00 ` Robert Dewar 1997-08-01 0:00 ` Shmuel (Seymour J.) Metz 1997-08-04 0:00 ` Larry Kilgallen 1997-08-08 0:00 ` Don Harrison 1 sibling, 2 replies; 26+ messages in thread From: Robert Dewar @ 1997-07-31 0:00 UTC (permalink / raw) Shmuel says <<I would go farther; I am suspicious of anyone who advocates a particular languages and can't tell me anything in the language that he dislikes. If he can't identify flaws, either he's too inexperienced to rely on or isn't perceptive enough. *any* programming language will have warts, and a real professional will recognize them and have strategies for dealing with them. >> Of course, that goes without saying, but did you ever meet anyone who was in that category whose ideas you would otherwise respect. I know some people who display this kind of ignorance, but they are not the kind of people I would take seriously. You can say the same thing about operating systems of course, for example anyone who cannot point out the flaws in XXX, simply indicates that they do not know the system very well, or that they lack the experience to know that improvements are possible (XXX = NT, Unix, Win95, OS/2, VMS etc etc) ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: The stupidity of all the Ariane 5 analysts. 1997-07-31 0:00 ` Robert Dewar @ 1997-08-01 0:00 ` Shmuel (Seymour J.) Metz 1997-08-04 0:00 ` Larry Kilgallen 1 sibling, 0 replies; 26+ messages in thread From: Shmuel (Seymour J.) Metz @ 1997-08-01 0:00 UTC (permalink / raw) Robert Dewar wrote: > > You can say the same thing about operating systems of course, In fact, I *do* say the same sort of thing about operating systems. ;-) -- Shmuel (Seymour J.) Metz Senior Software SE The values in from and reply-to are for the benefit of spammers: reply to domain eds.com, user msustys1.smetz or to domain gsg.eds.com, user smetz. ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: The stupidity of all the Ariane 5 analysts. 1997-07-31 0:00 ` Robert Dewar 1997-08-01 0:00 ` Shmuel (Seymour J.) Metz @ 1997-08-04 0:00 ` Larry Kilgallen 1 sibling, 0 replies; 26+ messages in thread From: Larry Kilgallen @ 1997-08-04 0:00 UTC (permalink / raw) In article <dewar.870362889@merv>, dewar@merv.cs.nyu.edu (Robert Dewar) writes: > You can say the same thing about operating systems of course, for example > anyone who cannot point out the flaws in XXX, simply indicates that they > do not know the system very well, or that they lack the experience to > know that improvements are possible > > (XXX = NT, Unix, Win95, OS/2, VMS etc etc) I think blindness to flaws has more to do with inexperience in _other_ environments than in the one under discussion. If my experience is limited to languages without uplevel addressing (e.g., Bliss, C) I will not see that as a shortcoming the way I would if I had used other languages (Ada, Pascal). Larry Kilgallen ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: The stupidity of all the Ariane 5 analysts. 1997-07-29 0:00 ` Shmuel (Seymour J.) Metz 1997-07-31 0:00 ` Robert Dewar @ 1997-08-08 0:00 ` Don Harrison 1997-08-11 0:00 ` Shmuel (Seymour J.) Metz 1 sibling, 1 reply; 26+ messages in thread From: Don Harrison @ 1997-08-08 0:00 UTC (permalink / raw) This is a response to something posted to comp.lang.ada but not comp.lang.eiffel.. "Shmuel (Seymour J.) Metz" wrote: :Robert Dewar wrote: :> :> I would think experience would teach us to be very suspicious of any :> claims by anyone to have a silver bullet panacea wjhen it comes to :> software engineering for large systems. :> :> However tools and techniques that help, even a little bit, are welcome :> even if they are not a panacea (I consider Ada to be in this category, :> and I always find it annoying when people advertise Ada as being :> a silver bullet panacea). Me too. :) :I would go farther; I am suspicious of anyone who advocates a particular :languages and can't tell me anything in the language that he dislikes. Presumably, you're talking about me. :) If you think I never have any criticisms of Eiffel, you're mistaken. What may surprise you even more is that I sometimes have positive things to say about Ada! - for example, two recent posts to comp.lang.eiffel under the subject "Ennumerations in Eiffel" (31.7.97 and 1.8.97). These included two criticisms of Eiffel and one positive comment about Ada. Also, was my agreement with criticisms of Eiffel WRT use of assertions and GC in hard realtime software a glowing endorsement of Eiffel? I think not. :If he can't identify flaws, either he's too inexperienced to rely on or :isn't perceptive enough. Thank you. :) :*any* programming language will have warts, Eiffel has its flaws; it's just that Ada has about 50 times more. :) :and a real professional will recognize them and have strategies for dealing :with them. Agree. BTW, one the few areas where Ada is streets ahead of Eiffel (and most languages), IMO, is standardisation. While Eiffel does have a standard language and libraries, not all vendors comply with them. Part of the problem is that there is no official certification of Eiffel compilers and, hence, no validation suite as there is in Ada. Obviously, this causes portability problems. Just because someone is biased, doesn't mean they can't also be honest and objective. Also, one can be biased with or without justification. P.S. It shows considerably more intestinal fortitude to criticise someone to their face rather than where you don't think they'll hear you. Even if they don't agree with what you have to say, they may at least respect you for not saying it behind their back. :( Don. =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Don Harrison donh@syd.csa.com.au ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: The stupidity of all the Ariane 5 analysts. 1997-08-08 0:00 ` Don Harrison @ 1997-08-11 0:00 ` Shmuel (Seymour J.) Metz 0 siblings, 0 replies; 26+ messages in thread From: Shmuel (Seymour J.) Metz @ 1997-08-11 0:00 UTC (permalink / raw) To: donh Don Harrison wrote: > > Presumably, you're talking about me. :) Assuming that your name is legion ;-) > P.S. It shows considerably more intestinal fortitude to criticise someone to > their face rather than where you don't think they'll hear you. Even if they > don't agree with what you have to say, they may at least respect you for not > saying it behind their back. :( Let's see; I post a public message in a forum in which you have been participating, and that constitutes saying it behind your back? If you think that I should have sent you an E-mail, plenty of people have been flamed for doing just that, on the grounds that is was an additional copy of a message that would already have been read by virtue of having been posted. I am generally very sparing of cc headers in Usenet, and see nothing in my message to justify ending you an extra copy. Further, since the comment applied to multiple messages in the thread, I would have had to search out all of the others to get their addresses and include them on the cc header. Or did you think that you were more important than the others? > Don. > =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- > Don Harrison donh@syd.csa.com.au BTW, why on earth would I include a signature block if I were attempting to criticize someone behind their back? -- Shmuel (Seymour J.) Metz Senior Software SE The values in from and reply-to are for the benefit of spammers: reply to domain eds.com, user msustys1.smetz or to domain gsg.eds.com, user smetz. ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: The stupidity of all the Ariane 5 analysts. 1997-07-21 0:00 ` Robert S. White 1997-07-21 0:00 ` Ken Garlington @ 1997-07-21 0:00 ` Robert Dewar 1997-07-22 0:00 ` W. Wesley Groleau x4923 2 siblings, 0 replies; 26+ messages in thread From: Robert Dewar @ 1997-07-21 0:00 UTC (permalink / raw) Bertrand said <<>All this is rhetorics and cannot succeed to obscure the basic >claim that systematic use of Design by Contract would probably >have avoided the crash. >> Well all sorts of things would have avoided the crash. One can also say that systematic proof of correctness, or systematic code review, or in fact almost any steps to be a bit more careful in this particular area, would have avoided the crash. As usual, a disaster of this type is a chain of occurrences, and fixing any one of them would have avoided the problem. Certainly the notion of clear interfaces and interface requirements (I find the use of the capitalized "Design by Contract" a little pretentious, since I see no new concept there that justifies a new technical term) is one which we can all agree on. The argument that the use of Eiffel would have prevented the problem is entirely unconvincing, and has the unfortunate effect of encouraging people to think that the entire argument is just empty language advocacy. When there is a major screwup in software (some examples, the long lines failure at AT&T, the Denver baggage handling, the overnight funds transfer disaster in an NY bank etc), there is a natural tendency for over enthusiastic pushers of a particular language to try to argue that their particular pet language would have guaranteed freedom from the problem. Such arguments are never convincing, because even if it *were* the case that the particular problem at hand might *possibly* have had a better chance of being avoided writing in language X, extending this to the allegation that the use of language X would in general have been beneficial stretches credibility. In particular, in a large complex system of this type, the language and all other components have to meet all kinds of complex requirements, and people who know nothing at all about the whole application have no idea whatsoever if their pet language would in fact meet the criteria. If Bertrand is saying: (a) The Ariane problem (at least little piece of it) is a good example of why designing interfaces that are complete with respect to establishing a requirements contract for clients is a good idea. (b) One of the attractive features of Eiffel is that it supports this concept directly and encourages programmers to use this approach. Then the argument is entirely reasonable, and one that I cannot see anyone objecting to. But the way the argument is stated, it reads as though he is saying: If Eiffel had been used for the Ariane project, it would not have crashed. And I find that claim entirely unjustified, and it detracts from the important underlying point. No language can force people to do things right, and to a large extent no language prevents people from doing things right. What a language can do is to encoruage a style of programming and thinking that helps people find their way to doing things right a little more easily. This is indeed the principle behind the Ada design, and a lot of the reason that Ada is successful in practice is precisely because it encourages "good thinking". But when people said "if only the Denver baggae system had been written in Ada, I would be flying to the new airport today", this crossed the line into non-credible language advocacy. Similarly, a lot of the thinking behind the Eiffel design is also to encourage "good thinking", and in that sense Ada and Eiffel are kindred spirits in their approach to language design, though they make different choices about what is important to encourage. However, when the Eiffel advocacy crosses the line to saying that a large complex system, if written in Eiffel, would have succeeeded where otherwise it failed, again this crosses the line into non-credible language advocacy. I am, as everyone knows, a strong advocate of the use of Ada. However, I take a lot of care NOT to make exaggerated claims, since I don't think this helps at all. What Ada (or any other language) can do, is to push programmers up a little bit on the quality and productivity scales. A little bit may not sound like much, but these scales are logarithmic scales in which the productivity of programmers and the quality of the code they right varies over orders of magnitudes, so a "little bit" can add up to very substantial savings in costs, and very substantial improvements in quality in practice. ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: The stupidity of all the Ariane 5 analysts. 1997-07-21 0:00 ` Robert S. White 1997-07-21 0:00 ` Ken Garlington 1997-07-21 0:00 ` Robert Dewar @ 1997-07-22 0:00 ` W. Wesley Groleau x4923 2 siblings, 0 replies; 26+ messages in thread From: W. Wesley Groleau x4923 @ 1997-07-22 0:00 UTC (permalink / raw) > Your papers about Eiffel and Design By Contract are just, IMO, > another way to implement the concept of developing software that > must comply to requirements. He/they start from the (obviously debatable) belief that "'ideas of Design by Contract are most directly realized' in Eiffel" and then make an unjustified jump to "if it doesn't look like Eiffel, it doesn't do Design By Contract." -- ---------------------------------------------------------------------- Wes Groleau, Hughes Defense Communications, Fort Wayne, IN USA Senior Software Engineer - AFATDS Tool-smith Wanna-be Don't send advertisements to this domain unless asked! All disk space on fw.hac.com hosts belongs to either Hughes Defense Communications or the United States government. Using email to store YOUR advertising on them is trespassing! ---------------------------------------------------------------------- ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: The stupidity of all the Ariane 5 analysts. 1997-07-20 0:00 ` Bertrand Meyer 1997-07-21 0:00 ` Ian Begg 1997-07-21 0:00 ` Robert S. White @ 1997-07-21 0:00 ` Ken Garlington 1997-07-31 0:00 ` Al Christians 1997-07-23 0:00 ` Joerg Rodemann 3 siblings, 1 reply; 26+ messages in thread From: Ken Garlington @ 1997-07-21 0:00 UTC (permalink / raw) Bertrand Meyer wrote: > > To repeat once again the basic point made in the > paper by Jean-Marc Jezequel and myself: it is dangerous > and unjustifiable, especially in a mission-critical setting, > to reuse a software element without a specification. Geoff Eldridge <geldridg@progsoc.uts.edu.au> was kind enough to post my draft critique of your paper in HTML format at: http://www.progsoc.uts.edu.au/~geldridg/eiffel/ariane/ Let me know if you have any comments before I submit the final version to you at the end of the month. Thank you. ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: The stupidity of all the Ariane 5 analysts. 1997-07-21 0:00 ` Ken Garlington @ 1997-07-31 0:00 ` Al Christians 1997-08-01 0:00 ` "Paul E. Bennett" 0 siblings, 1 reply; 26+ messages in thread From: Al Christians @ 1997-07-31 0:00 UTC (permalink / raw) It occurs to me that DBC is less than 100% applicable in this case. As I read it, contracts constrain the terms of interaction between clients and servers, and as I recall, the server should expose to the client all the methods it uses to evaluate preconditions, so that the client can verify its compliance before requesting the service. In the case of the Ariane, however, the client was hardware, some kind of dumb sensor that took physical measurements and sent them into the machine, with no real interest in what came out later, and not enough neurons to figure out if it was in compliance with any set of preconditions or not. So why should the server bother to check preconditions in this case -- after all, the client doesn't give a rusty, so why should the server? Who's he gonna call? If that's a contract, then my roof is a contract between my house and the sky. The guys who built the roof didn't know what wind speed it was supposed to withstand, they just built it the way they always build them, and that's good enough. And if the roof is spec'd to handle winds up to 80 mph, I'm figuring that if one comes along at 85 mph, the roof isn't going to shut itself down on account of a contract violation, it's going to try its best to hold together anyway, which is what I'd want. OTOH, the software on the Ariane did shut itself down while the thing was in flight, didn't it? It would be nice if there could be a clear spec that includes everything that might happen in the real world, but when the real world does something that the spec didn't anticipate, do we want the software to just curl up and die? Al ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: The stupidity of all the Ariane 5 analysts. 1997-07-31 0:00 ` Al Christians @ 1997-08-01 0:00 ` "Paul E. Bennett" 1997-08-01 0:00 ` Ken Garlington 0 siblings, 1 reply; 26+ messages in thread From: "Paul E. Bennett" @ 1997-08-01 0:00 UTC (permalink / raw) In article <33E06929.59F6@easystreet.com> achrist@easystreet.com "Al Christians" writes: > > It would be nice if there could be a clear spec that includes everything > that might happen in the real world, but when the real world does > something that the spec didn't anticipate, do we want the software to > just curl up and die? If such a spec could be produced it would probably be too large to understand in reasonable time. Instead, we need to construct our systems so that out-of-the-ordinary stimulii do not cause un-expected activity of the system. This involves knowing what happens in the system when the stimulii exceed the design limitations. This takes some effort in FMECA and the designing in of "Inherent Robustness" for the system to become dependable. If we are going to re-use components, we need to be more certain about what the effects are for component failure. In the A5 flight systems situation, there were other factors in the organisation which blindly accepted the decision for non-provision of flight profiles. A risk assessment for the decision should have been conducted to determine if this was a reasonable decision. -- Paul E. Bennett ................... <peb@transcontech.co.uk> Transport Control Technology Ltd. <http://www.tcontec.demon.co.uk/> +44 (0)117-9499861 <enquiry@transcontech.co.uk> Going Forth Safely ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: The stupidity of all the Ariane 5 analysts. 1997-08-01 0:00 ` "Paul E. Bennett" @ 1997-08-01 0:00 ` Ken Garlington 0 siblings, 0 replies; 26+ messages in thread From: Ken Garlington @ 1997-08-01 0:00 UTC (permalink / raw) Paul E. Bennett wrote: > > In article <33E06929.59F6@easystreet.com> > achrist@easystreet.com "Al Christians" writes: > > > > > It would be nice if there could be a clear spec that includes everything > > that might happen in the real world, but when the real world does > > something that the spec didn't anticipate, do we want the software to > > just curl up and die? > > If such a spec could be produced it would probably be too large to > understand in reasonable time. Instead, we need to construct our systems > so that out-of-the-ordinary stimulii do not cause un-expected activity of > the system. This involves knowing what happens in the system when the > stimulii exceed the design limitations. This takes some effort in FMECA > and the designing in of "Inherent Robustness" for the system to become > dependable. Also, FMECA/FMET is quite valuable in that it usually is not solely driven by the specification, but also includes past experience driven from failure records. We've found FMET in particular to be quite useful in finding holes in the specification. > If we are going to re-use components, we need to be more certain about > what the effects are for component failure. In the A5 flight systems > situation, there were other factors in the organisation which blindly > accepted the decision for non-provision of flight profiles. A risk > assessment for the decision should have been conducted to determine if > this was a reasonable decision. > > -- > Paul E. Bennett ................... <peb@transcontech.co.uk> > Transport Control Technology Ltd. <http://www.tcontec.demon.co.uk/> > +44 (0)117-9499861 <enquiry@transcontech.co.uk> > Going Forth Safely ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: The stupidity of all the Ariane 5 analysts. 1997-07-20 0:00 ` Bertrand Meyer ` (2 preceding siblings ...) 1997-07-21 0:00 ` Ken Garlington @ 1997-07-23 0:00 ` Joerg Rodemann 3 siblings, 0 replies; 26+ messages in thread From: Joerg Rodemann @ 1997-07-23 0:00 UTC (permalink / raw) Bertrand Meyer (Bertrand.Meyer@eiffel.com) wrote: > To repeat once again the basic point made in the > paper by Jean-Marc Jezequel and myself: it is dangerous > and unjustifiable, especially in a mission-critical setting, > to reuse a software element without a specification. Agreed: it should be clearly stated what something is intended for. Although the idea of preconditions sound very good to me at first sight I am very sceptical if this is very useful at all since one has to think of EVERY possibility the routine might be used for. E. g. assume the implementation of some mathematical function that is not well defined for the complete complex plane or not continously defined for real numbers. On the other hand you might have an algorithm at hand that is suitable for every value the function is defined. (Maybe the Gamma, Beta and Zeta functions are some examples that meet my assumptions --- I can't remember them exactly right now.) On the other hand you know that for your actual problem this function is indeed defined for each possible value since you are --- from your requirements --- quite sure that some area will not be left. So you could enter this as a precondition. But now guess you like to use this component for some other work --- and you are not dealing with complex numbers of different areas of the plane. It seems to me that you list of preconditions will grow very fast --- leaving the code quite unreadable. (Kind of a " i++; // increments i" type comment.) Code is the more difficult to comprehend the large it is. And especially emphasizing details may obscure the essential parts. Was not this abstraction was all about? From this point of view I would think that easy checkable preconditions should be checked (e. g. using assertions). Others should be kept in the documentation. As far as Ariane V is concerned the report clearly states that the problem was know and discussed during develepment but due to cpu time limits not every suspected variable was covered. It seems to me that a lack of knowledge about the actual application was one reason for the error to occur. (Among the devastating assumption that exceptions are only to occur when hardware fails!) No pre-/postcondition theology would have cured that, IMHO. (I believe no methodology could cure that. and this might be a problem that is not easy if at all to solve for systems that complex.) Well, just my humble opinion...and I have to admit that I have no experions within realtime or safety critical applications development. Yours sincerly Joerg -- rodemann@mathematik.uni-ulm.de | Dipl.-Phys. Joerg S. Rodemann Phone: ++49-(0)711-5090670 | Flurstrasse 21, D-70372 Stuttgart, Germany -------------------------------+--------------------------------------------- rodemann@rus.uni-stuttgart.de | University of Stuttgart, Computing Center Phone: ++49-(0)711-685-5815 | Visualization Department, Office: 0.304 Fax: ++49-(0)711-678-7626 | Allmandring 30a, D-70550 Stuttgart, Germany ^ permalink raw reply [flat|nested] 26+ messages in thread
end of thread, other threads:[~1997-08-19 0:00 UTC | newest] Thread overview: 26+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 1997-08-18 0:00 The stupidity of all the Ariane 5 analysts Joerg Rieling 1997-08-19 0:00 ` Robert S. White -- strict thread matches above, loose matches on Subject: below -- 1997-08-19 0:00 Marin David Condic, 561.796.8997, M/S 731-96 1997-07-12 0:00 Safety-critical development in Ada and Eiffel Ken Garlington 1997-07-15 0:00 ` Don Harrison 1997-07-15 0:00 ` Ken Garlington 1997-07-16 0:00 ` Jean-Marc Jezequel 1997-07-16 0:00 ` Ken Garlington 1997-07-17 0:00 ` The stupidity of all the Ariane 5 analysts Thaddeus L. Olczyk [not found] ` <33CEAF05.6389@flash.net> 1997-07-20 0:00 ` Bertrand Meyer 1997-07-21 0:00 ` Ian Begg 1997-07-21 0:00 ` Robert S. White 1997-07-21 0:00 ` Ken Garlington 1997-07-23 0:00 ` Robert S. White 1997-07-23 0:00 ` Ken Garlington 1997-07-25 0:00 ` Robert S. White 1997-07-23 0:00 ` Robert Dewar 1997-07-24 0:00 ` Ken Garlington 1997-07-29 0:00 ` Shmuel (Seymour J.) Metz 1997-07-31 0:00 ` Robert Dewar 1997-08-01 0:00 ` Shmuel (Seymour J.) Metz 1997-08-04 0:00 ` Larry Kilgallen 1997-08-08 0:00 ` Don Harrison 1997-08-11 0:00 ` Shmuel (Seymour J.) Metz 1997-07-21 0:00 ` Robert Dewar 1997-07-22 0:00 ` W. Wesley Groleau x4923 1997-07-21 0:00 ` Ken Garlington 1997-07-31 0:00 ` Al Christians 1997-08-01 0:00 ` "Paul E. Bennett" 1997-08-01 0:00 ` Ken Garlington 1997-07-23 0:00 ` Joerg Rodemann
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox