Re: The stupidity of all the Ariane 5 analysts.

Re: The stupidity of all the Ariane 5 analysts.

[Joerg Rieling]

hi all,

ok, i perfectly agree with some of the points you made concerning
inherent robustness etc etc etc.

but, a minimum requirement for any software is to test it under
sufficiently realistic conditions, isn't it ?   if the responsible
software engineers would have fed _simulated_ data containing some
information about the trajectory into their programs, they could
(would) have found the  bug, d'accord ?
sometimes experimental programming is not the worst choice ...

bye

j"org.

--
<---->


(- PGP key available.-)

Re: The stupidity of all the Ariane 5 analysts.

[Robert S. White]

In article <5ta2e6$itn$1@enterprise.desy.de>, joerg@hera-b.desy.de says...
>
>but, a minimum requirement for any software is to test it under
>sufficiently realistic conditions, isn't it ?   if the responsible
>software engineers would have fed _simulated_ data containing some
>information about the trajectory into their programs, they could
>(would) have found the  bug, d'accord ?

  That has been _exactly_ one of my positions as an INS (IRS) 
practitioner!  Who exactly are you addressing your comments to?  BM, 
JY, or DH?  The other problem, as pointed out, has been that there 
was software re-use without a correct analysis of its suitability 
by problem domain experts.  A system requirements review with the 
involvement of the right problem domain experts _could_ have 
determined by analysis whether the Ariane 4 IRS software should 
have been reused as is.  The failsafe for this review _should_ have
been proper simulation testing as you describe.
_____________________________________________________________________
Robert S. White         -- An embedded systems software engineer

Re: The stupidity of all the Ariane 5 analysts.

[Marin David Condic, 561.796.8997, M/S 731-96]

Joerg Rieling <joerg@HERA-B.DESY.DE> writes:
>but, a minimum requirement for any software is to test it under
>sufficiently realistic conditions, isn't it ?   if the responsible
>software engineers would have fed _simulated_ data containing some
>information about the trajectory into their programs, they could
>(would) have found the  bug, d'accord ?
>sometimes experimental programming is not the worst choice ...
>
    This is exactly the point that has so many people torqued off
    about the whole debate. (Especially those of us who actually build
    this sort of software for a living.) The claim seems to be on the
    table that a) Existing software development practices are
    insufficient for developing safe software and b) the only way to
    be sure that you don't blow up a rocket is to use Design By
    Contract as exemplified by Eiffel.

    The problem is - as those of us in the business know - existing
    practices, HAD THEY BEEN FOLLOWED, would have caught the problem.
    (Your example of testing the software across the expected flight
    envelope.) So would lots of other existing practices. The problem
    is that the human beings in charge of the process didn't follow
    the usual and customary procedures (probably under cost and
    schedule pressure). Hence, no technology would likely have helped
    keep them out of the Gulag they are now most likely consigned to.
    (It's sad too for those of us who have ever experienced the "Hurry
    up and get it done no matter what the risk..." pressure that can
    be brought onto a project.)

    DBC and Assertions, in my not so humble opinion, would have proven
    to make no significant difference in the situation. DBC is a kind
    of process and apparently the people in charge were into
    circumventing processes. So DBC could have as easily been
    circumvented for the same reasons as they circumvented existing,
    industry accepted and used processes. (which, IMNSHO, would
    probably have caught the problem had they been followed.) If you
    walk a tightrope without a net often enough, eventually you'll
    splat up against the pavement.

    Assertions are just another kind of runtime check which - although
    useful - would probably have made no difference either. The reason
    being that the existing checks were disabled (for good reason)
    and so assertions would also have been disabled. Disabled checks
    don't change anything. The notion that they would have served as
    documentation implies some kind of review/testing and go back to
    the above paragraph on DBC. They had already circumvented that
    part of the game, so forget that argument.

    The seriously flawed notion that the runtime checks be turned
    on during testing and then disabled for production only displays
    lack of knowledge of how flight qualified software is really
    built. Nobody is going to go through any sort of serious
    verification of a software package only to allow you to change
    some of the bits afterwards and claim "well, it's really the same
    thing so let's go fly with it..." That might be O.K. for testing
    something like Microsoft Word, but it will never pass muster for
    safety critical software. Hence, you run with them in or you run
    with them out. Running with them in might have done something to
    save the day - but then so would the existing range checks that
    were taken out.

    MDC

Re: Safety-critical development in Ada and Eiffel

[Ken Garlington]

Don Harrison wrote:
> 
> If even this small overhead is unacceptable, you can force static binding
> where necessary by freezing routines.
> Although you forgo the flexibility of
> polymorphism, you can still take advantage of inheritance by virtue of Eiffel
> synonyms - these allow you to declare a routine under different names.

But I can do this in Ada, as well, so I don't see the benefit.

 :Does this "pre-allocated memory pool" mean that you will know exactly
> :what address
> :each object's data will reside at, or merely that you dynamically
> :allocate memory
> :at initialization and do not dynamically allocate after initialization
> :is complete?
> 
> The latter. You don't actually need to know *where* objects are allocated,
> merely that there is sufficient memory to allocate them.

Actually, in my systems, I do need to know where they are allocated, so
that
I can examine them from the hideously crude display in the cockpit
during
troubleshooting.

> Maybe general assertions are more common than you imagine. However, I agree
> that range constraints are roughly covered by Ada subtypes. I use them liberally
> for just that purpose.

Maybe, but again, I would like to see how these complex assertions are
used. Again,
the code I've seen to date doesn't seem to use them that much.

> Correct me I misunderstand you.. Are you wondering whether contracting
> helps you to identify more bugs? I can offer my own experience which has
> firmly convinced me of this.

Contracting definitely identifies more bugs. However, my concern with
Eiffel
assertions (and Ada assertions, for that matter) is that, once raised,
it's
not always clear what to do about them. In a safety-critical system such
as
we build, you can't just generate a core dump, or ask the user what to
do next.
You have to generate a correct response to the assertion failure, and
you have
to do it very quickly. Static assertion checks, of course, can be
detected and
corrected during development, which is why I like Ada's strong static
type checking.

There is a school of thought that says to add in assertions, test the
system, and
then remove/suppress the exceptions. I have seen too many cases of code
that works
with the assertions active, and then breaks (due to timing differences,
or more
likely code generation errors) to have much confidence in this approach.

Another consequence of using assertions is that you have to develop and
test the
assertions. As a result, you need confidence that the benefits of the
assertions
outweigh the dilution of your test effort. One of the advantages of
Ada's simpler
approach (at least, it seems simpler to me) to simple
pre/post-conditions (range
checks in particular) is that the cost of adding them is somewhat
reduced. However,
in both languages, overuse of assertions may be just as bad as underuse.

(As you can tell, I'm somewhat of an assertions heretic. However, I will
say that
for non-real-time, non-safety-critical systems, they are outstanding. My
tool code
is lousy with them.)

> Consequently, when we (a team of 5) recently
> reached the end of a 5 month long development effort of incremental coding
> and integration testing, the parts of the system that used contracting work
> flawlessly apart from problems resulting from misinterpretation of requirements.

Most of the problems we see in formal testing are from misinterpretation
of
requirements - which is another issue I have with assertions as a magic
safety wand.
Of course, Bertand Meyer's Eiffel website insists that even
misinterpretation of
requirements (Ariane V) will be a problem no longer when Eiffel is used!
:)

Re: Safety-critical development in Ada and Eiffel

[Don Harrison]

Ken Garlington wrote:

:Don Harrison wrote:
:> 
:> If even this small overhead is unacceptable, you can force static binding
:> where necessary by freezing routines.
:> Although you forgo the flexibility of
:> polymorphism, you can still take advantage of inheritance by virtue of Eiffel
:> synonyms - these allow you to declare a routine under different names.
:
:But I can do this in Ada, as well, so I don't see the benefit.

I'm not saying there is any extra benefit - merely that you don't have to 
wear the overhead of dynamic binding if you're desperate to maximise efficiency.
Tucker mentioned that overhead. I'm saying it isn't an issue.

BTW, how do call a procedure under different names in Ada?


: :Does this "pre-allocated memory pool" mean that you will know exactly
:> :what address
:> :each object's data will reside at, or merely that you dynamically
:> :allocate memory
:> :at initialization and do not dynamically allocate after initialization
:> :is complete?
:> 
:> The latter. You don't actually need to know *where* objects are allocated,
:> merely that there is sufficient memory to allocate them.
:
:Actually, in my systems, I do need to know where they are allocated, so
:that
:I can examine them from the hideously crude display in the cockpit
:during
:troubleshooting.

That *is* hideously crude (and brings back some bad memories). Depending on how
crude you mean, maybe a primitive tool could give you a map of where objects get 
put in memory. If you mean incredibly crude, use non-reference (expanded in 
Eiffel) types and use a linker-generated memory map.


:> Maybe general assertions are more common than you imagine. However, I agree
:> that range constraints are roughly covered by Ada subtypes. I use them liberally
:> for just that purpose.
:
:Maybe, but again, I would like to see how these complex assertions are
:used. 

You would use them anywhere that a piece of code makes assumptions. For example,
to help avoid the Ariane fiasco, include contracts in the INS(?) that specify 
Ariane 4 dynamics. Then, in testing, you will get an assertion violation when
you apply Ariane 5 dynamics to it.

:Again, the code I've seen to date doesn't seem to use them that much.

If you're looking in Ada code, you're looking in the wrong place because few
Ada developers recognise there are additional benefits from using them in 
addition to static typing and the predefined contracts already in Ada 
(the conditions under which predefined exceptions - eg. Constraint_Error - 
are raised).

Static typing gives you static contract checking. Assertions (predefined and
user-defined give you dynamic contract checking *during development*. What 
most Ada software engineers miss out on is the user-defined variety which 
bring a non-trivial benefit to reliability and reuse.


:> Correct me I misunderstand you.. Are you wondering whether contracting
:> helps you to identify more bugs? I can offer my own experience which has
:> firmly convinced me of this.
:
:Contracting definitely identifies more bugs. However, my concern with
:Eiffel
:assertions (and Ada assertions, for that matter) is that, once raised,
:it's
:not always clear what to do about them. 

They are only turned on during development, so what you do is fix the underlying
problem which is usually not difficult to find if you've used them systematically.
Even if you haven't (I was the only one of a team of 5 using them in our recent
development cycle), they still tell you more than you would otherwise be told.

Good design complemented by static typing and Design by Contract is unbeatable, 
IME.


:In a safety-critical system such
:as
:we build, you can't just generate a core dump, or ask the user what to
:do next.
:You have to generate a correct response to the assertion failure, and
:you have
:to do it very quickly. 

Right. However, only an incorrect program will raise assertion violations at
runtime. As you systematically identify and fix the errors causing them during
development, you can turn assertions off with confidence when the system 
goes live. Then, any errors you may have missed will be handled by exception
handlers in the usual way.
 
:Static assertion checks, of course, can be
:detected and
:corrected during development, which is why I like Ada's strong static
:type checking.

I agree completely. Static typing is great but it isn't the *whole* story.


:There is a school of thought that says to add in assertions, test the
:system, and
:then remove/suppress the exceptions. I have seen too many cases of code
:that works
:with the assertions active, and then breaks (due to timing differences,

Yes, timing *is* something to be concerned about when applied to realtime 
systems. That's why I suggested not using assertions in the hard realtime
threads of a process. BTW, software designed using DBC may actually run *faster*
than code without it because you get to strip out all the defensive validity
checks. You have already ascertained during development that operations are
called in valid contexts so you don't have to check the context.

:or more
:likely code generation errors) to have much confidence in this approach.

If that's a problem, you need to get your vendor to clean their act up.


:Another consequence of using assertions is that you have to develop and
:test the
:assertions. 

Correct. My initial reaction when I started using them was "Gee, I've got to
do this extra work on top of writing the "real" code! However, I found I was 
spending about a third of the time integrating compared with my colleagues
which meant I was saving time overall and producing more reliable code to boot.


:As a result, you need confidence that the benefits of the
:assertions
:outweigh the dilution of your test effort. 

IME, they do.

:One of the advantages of
:Ada's simpler
:approach (at least, it seems simpler to me) to simple
:pre/post-conditions (range
:checks in particular) is that the cost of adding them is somewhat
:reduced. 

I disagree. It takes me about the same amount of time to specify an Ada
subtype as it does to specify a more general assertion. Even if the general 
assertion is a function, I would more than likely need to write it anyway so
would have expended the effort anyway.


:However,
:in both languages, overuse of assertions may be just as bad as underuse.

It is possible to overuse them but 99% of developers err on the othger side. :)


:(As you can tell, I'm somewhat of an assertions heretic. 

That's fine but have a go at using them where appropriate. I'm sure you'll 
become convinced.

:However, I will
:say that
:for non-real-time, non-safety-critical systems, they are outstanding. My
:tool code
:is lousy with them.)
:
:> Consequently, when we (a team of 5) recently
:> reached the end of a 5 month long development effort of incremental coding
:> and integration testing, the parts of the system that used contracting work
:> flawlessly apart from problems resulting from misinterpretation of requirements.
:
:Most of the problems we see in formal testing are from misinterpretation
:of
:requirements - which is another issue I have with assertions as a magic
:safety wand.

No use of assertions will stop you misinterpreting requirements. They're not 
a magic wand but they *will* help you remove a class of errors you would 
otherwise miss.


:Of course, Bertand Meyer's Eiffel website insists that even
:misinterpretation of
:requirements (Ariane V) will be a problem no longer when Eiffel is used!

Come on. He's not saying that.


Don.
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Don Harrison             donh@syd.csa.com.au

Re: Safety-critical development in Ada and Eiffel

[Ken Garlington]

Don Harrison wrote:
> 
> Ken Garlington wrote:
> 
> :Don Harrison wrote:
> :>
> :> If even this small overhead is unacceptable, you can force static binding
> :> where necessary by freezing routines.
> :> Although you forgo the flexibility of
> :> polymorphism, you can still take advantage of inheritance by virtue of Eiffel
> :> synonyms - these allow you to declare a routine under different names.
> :
> :But I can do this in Ada, as well, so I don't see the benefit.
> 
> I'm not saying there is any extra benefit - merely that you don't have to
> wear the overhead of dynamic binding if you're desperate to maximise efficiency.
> Tucker mentioned that overhead. I'm saying it isn't an issue.

But you also said that it was required to use that part of Eiffel
inheritance not
available in Ada. In this statement, you seem to be saying that if I
want an
efficient safety-critical system (which I always do), I have to give us
some of
that extra inheritance. Do I need the full Eiffel inheritance model, or
don't I?

> 
> BTW, how do call a procedure under different names in Ada?

Via the "rename" statement.

> 
> : :Does this "pre-allocated memory pool" mean that you will know exactly
> :> :what address
> :> :each object's data will reside at, or merely that you dynamically
> :> :allocate memory
> :> :at initialization and do not dynamically allocate after initialization
> :> :is complete?
> :>
> :> The latter. You don't actually need to know *where* objects are allocated,
> :> merely that there is sufficient memory to allocate them.
> :
> :Actually, in my systems, I do need to know where they are allocated, so
> :that
> :I can examine them from the hideously crude display in the cockpit
> :during
> :troubleshooting.
> 
> That *is* hideously crude (and brings back some bad memories). Depending on how
> crude you mean, maybe a primitive tool could give you a map of where objects get
> put in memory. If you mean incredibly crude, use non-reference (expanded in
> Eiffel) types and use a linker-generated memory map.

Both approaches require that the memory address be known before
execution time,
however, so I still care where objects are located. I also care for
other reasons,
such as memory-mapped I/O, of course.

> 
> :> Maybe general assertions are more common than you imagine. However, I agree
> :> that range constraints are roughly covered by Ada subtypes. I use them liberally
> :> for just that purpose.
> :
> :Maybe, but again, I would like to see how these complex assertions are
> :used.
> 
> You would use them anywhere that a piece of code makes assumptions. For example,
> to help avoid the Ariane fiasco, include contracts in the INS(?) that specify
> Ariane 4 dynamics. Then, in testing, you will get an assertion violation when
> you apply Ariane 5 dynamics to it.

This is why my blood pressure goes up!

The _Ada_ implementation did generate an assertion violation the very
first time
Ariane 5 dynamics were applied to it. Unfortunately, the very first time
Ariane 5
dynamics were applied to it was at LANUCH!

They were not applied earlier, because the contractor assumed that the
Ariane 5
dynamics were the same as the Ariane 4, so why spend the money testing
to the same
conditions already tested earlier? (They were wrong, of course, but they
didn't know
that at the time).

The other part of the problem was that the _response_ to the assertion
violation was
wrong. The designer assumed the assertion would occur due to a hardware
failure, not
a software requirements failure, and so the handler shut down the
"offending" hardware,
rather than attempting some other action (e.g. replacing the
out-of-range value with
a "safe" value). Frankly, I don't know that I would have written the
handler differently
myself.

This is all clearly documented in the final report, available on the
Web.

According to Meyer, et. al. Eiffel programmers would have written the
handler
differently, and also they would have known to do the testing
differently (even though
the requirements specification said that the tests need not change).
However, their
explanation as to why this is true is far from convincing (basically,
that "Eiffel
programmers believe in quality" or some such nonsense). Their "analysis"
is also
on the Web. I've asked them to post my rebuttal to their paper, but I
don't know
if they ever did.

This is why I dislike people with no background in safety-critical
flight systems discussing
such issues. They believe that "software is software." and apply the
expertise in their
domain to mine. They usually do about as good a job as when I try to
discuss compiler
design :)

> 
> :Again, the code I've seen to date doesn't seem to use them that much.
> 
> If you're looking in Ada code, you're looking in the wrong place because few
> Ada developers recognise there are additional benefits from using them in
> addition to static typing and the predefined contracts already in Ada
> (the conditions under which predefined exceptions - eg. Constraint_Error -
> are raised).

No - I'm looking at Eiffel code. (Why would I look at Ada code to
determine how
Eiffel is used?)

> Static typing gives you static contract checking. Assertions (predefined and
> user-defined give you dynamic contract checking *during development*. What
> most Ada software engineers miss out on is the user-defined variety which
> bring a non-trivial benefit to reliability and reuse.

Not this Ada engineer! I have dynamic assertions all over the place in
my
tools!

My problem, again, is in the fundamental nature of dynamic assertions
for safety-critical
embedded systems, which have some unique risks, whether you (a) leave
them in after
development, or (b) take them out after developmental testing, but
before usage.

> 
> :> Correct me I misunderstand you.. Are you wondering whether contracting
> :> helps you to identify more bugs? I can offer my own experience which has
> :> firmly convinced me of this.
> :
> :Contracting definitely identifies more bugs. However, my concern with
> :Eiffel
> :assertions (and Ada assertions, for that matter) is that, once raised,
> :it's
> :not always clear what to do about them.
> 
> They are only turned on during development, so what you do is fix the underlying
> problem which is usually not difficult to find if you've used them systematically.
> Even if you haven't (I was the only one of a team of 5 using them in our recent
> development cycle), they still tell you more than you would otherwise be told.

I am aware of at least one project developing an embedded missile
controller,
that did exactly what you describe (in Ada). Here's what happened:

1. Their timings changed, which introduced some subtle differences in
the
relationships between tasks and external interrupts. Some of these
subtle
differences invalidated their original testing.

2. Their object code changed (obviously), which required reanalysis of
their
object-code branch coverage.

3. At least one compiler bug was introduced, which was definitely
unsettling.

What safety-critical system did you develop where you turned off
assertions
afer development? What standard (DO-178?) did you use for certification?

The other prong of this is that, while assertions are keen for catching
design problems, I haven't seen much evidence that they catch high-level
requirements problems (not surprising, since the code was presumably
written
to match the requirements). In 13 years of developing safety-critical
embedded systems, our process tends to handle design problems fairly
early.
It's those high-level requirements problems where we need the most help.

> 
> Good design complemented by static typing and Design by Contract is unbeatable,
> IME.
> 
> :In a safety-critical system such
> :as
> :we build, you can't just generate a core dump, or ask the user what to
> :do next.
> :You have to generate a correct response to the assertion failure, and
> :you have
> :to do it very quickly.
> 
> Right. However, only an incorrect program will raise assertion violations at
> runtime. As you systematically identify and fix the errors causing them during
> development, you can turn assertions off with confidence when the system
> goes live. Then, any errors you may have missed will be handled by exception
> handlers in the usual way.
> 
> :Static assertion checks, of course, can be
> :detected and
> :corrected during development, which is why I like Ada's strong static
> :type checking.
> 
> I agree completely. Static typing is great but it isn't the *whole* story.
> 
> :There is a school of thought that says to add in assertions, test the
> :system, and
> :then remove/suppress the exceptions. I have seen too many cases of code
> :that works
> :with the assertions active, and then breaks (due to timing differences,
> 
> Yes, timing *is* something to be concerned about when applied to realtime
> systems. That's why I suggested not using assertions in the hard realtime
> threads of a process. BTW, software designed using DBC may actually run *faster*
> than code without it because you get to strip out all the defensive validity
> checks. You have already ascertained during development that operations are
> called in valid contexts so you don't have to check the context.

But almost all of our safety-critical threads are hard realtime threads!
That's a HUGE limitation for our systems!

> 
> :or more
> :likely code generation errors) to have much confidence in this approach.
> 
> If that's a problem, you need to get your vendor to clean their act up.

Easier said than done! :)

What is the process to certify Eiffel compilers as being acceptably
mature
(code generation-wise) for safety-critical systems? Is all testing
performed
with assertions on and assertions off in all test cases?

> 
> :Another consequence of using assertions is that you have to develop and
> :test the
> :assertions.
> 
> Correct. My initial reaction when I started using them was "Gee, I've got to
> do this extra work on top of writing the "real" code! However, I found I was
> spending about a third of the time integrating compared with my colleagues
> which meant I was saving time overall and producing more reliable code to boot.
> 
> :As a result, you need confidence that the benefits of the
> :assertions
> :outweigh the dilution of your test effort.
> 
> IME, they do.

However, it sounds like you're talking about a non-safety-critical
system, and
certainly not a hard real-time embedded safety-critical systems. In that
case,
I agree. For tools, simulations, etc. I would use all of the assertions
I could.

However, what about the topic of this thread (safety-critical systems)?
It sounds
like there's a lot of limitations as to using (and inheriting)
assertions for
this environment.

> 
> :One of the advantages of
> :Ada's simpler
> :approach (at least, it seems simpler to me) to simple
> :pre/post-conditions (range
> :checks in particular) is that the cost of adding them is somewhat
> :reduced.
> 
> I disagree. It takes me about the same amount of time to specify an Ada
> subtype as it does to specify a more general assertion. Even if the general
> assertion is a function, I would more than likely need to write it anyway so
> would have expended the effort anyway.
> 
> :However,
> :in both languages, overuse of assertions may be just as bad as underuse.
> 
> It is possible to overuse them but 99% of developers err on the othger side. :)
> 
> :(As you can tell, I'm somewhat of an assertions heretic.
> 
> That's fine but have a go at using them where appropriate. I'm sure you'll
> become convinced.
> 
> :However, I will
> :say that
> :for non-real-time, non-safety-critical systems, they are outstanding. My
> :tool code
> :is lousy with them.)
> :
> :> Consequently, when we (a team of 5) recently
> :> reached the end of a 5 month long development effort of incremental coding
> :> and integration testing, the parts of the system that used contracting work
> :> flawlessly apart from problems resulting from misinterpretation of requirements.
> :
> :Most of the problems we see in formal testing are from misinterpretation
> :of
> :requirements - which is another issue I have with assertions as a magic
> :safety wand.
> 
> No use of assertions will stop you misinterpreting requirements. They're not
> a magic wand but they *will* help you remove a class of errors you would
> otherwise miss.
> 
> :Of course, Bertand Meyer's Eiffel website insists that even
> :misinterpretation of
> :requirements (Ariane V) will be a problem no longer when Eiffel is used!
> 
> Come on. He's not saying that.

Read his analysis of Ariane V. He says _exactly_ that, and has defended
saying that in multiple newsgroups. A smart person, but not someone
who knows my domain particularly well.

> 
> Don.
> =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
> Don Harrison             donh@syd.csa.com.au

Re: Safety-critical development in Ada and Eiffel

[Jean-Marc Jezequel]

In article <33CC0548.4099@flash.net>, Ken Garlington <keg0@flash.net> writes:
>Don Harrison wrote:

>> You would use them anywhere that a piece of code makes assumptions. For example,
>> to help avoid the Ariane fiasco, include contracts in the INS(?) that specify
>> Ariane 4 dynamics. Then, in testing, you will get an assertion violation when
>> you apply Ariane 5 dynamics to it.
>
>This is why my blood pressure goes up!
>
>The _Ada_ implementation did generate an assertion violation the very
>first time Ariane 5 dynamics were applied to it. Unfortunately, the very first time
>Ariane 5 dynamics were applied to it was at LANUCH!

>They were not applied earlier, because the contractor assumed that the
>Ariane 5 dynamics were the same as the Ariane 4, so why spend the money testing
>to the same conditions already tested earlier? (They were wrong, of course, but they
>didn't know that at the time).

At the risk of repeating myself, and reopening a thread beaten to death,
the all point of design by contract (DBC) is to
make this kind of assumptions explicit. Ariane 5 is just a nice striking example
of working with assumptions that are true at a point in time (Ariane 4) and no longer
later on (Ariane 5). I think we agreed on this previously.

To sum up your point, you think that DBC, i.e. expressing hidden assumptions with 
Eiffel-like assertions would not have been practicable in this case.
Others think it would have...

You seem to be a specialist in flight software. We are not. But specialists might,
from time to time, learn something from people with a broader perspective
(this is a general statement, I do not claim in particular that I have a broader
perspective than you, or anyone else for that matter;-).

Basically, our arguments failed to convinced you. Yours failed to convince us. 
That's life, and I hope readers can make their mind themselves instead of
relying on the one of us who shouts louder.

>The other part of the problem was that the _response_ to the assertion
>violation was wrong. The designer assumed the assertion would occur due to a hardware
>failure, not a software requirements failure, and so the handler shut down the
>"offending" hardware, rather than attempting some other action (e.g. replacing the
>out-of-range value with a "safe" value). Frankly, I don't know that I would have written the
>handler differently myself.

Agreed, that's a different problem altogether.

>According to Meyer, et. al. Eiffel programmers would have written the
>handler differently, and also they would have known to do the testing
>differently (even though the requirements specification said that the tests need not change).
>However, their explanation as to why this is true is far from convincing (basically,
>that "Eiffel programmers believe in quality" or some such nonsense). Their "analysis"
>is also on the Web. I've asked them to post my rebuttal to their paper, but I
>don't know if they ever did.

Just for the record, I'm still waiting for your rebuttal paper to put it on my web alongside
with the Computer article. 

PS: Ken's post was brought to my attention by a colleague: I don't have time to check the news
very often these days, and anyway I won't follow-up on this thread to avoid reopening the interminable
Ariane thread.

-- 
Jean-Marc Jezequel               Tel : +33 299 847 192         
IRISA/CNRS                       Fax : +33 299 847 171         
Campus de Beaulieu               e-mail : jezequel@irisa.fr 
F-35042 RENNES (FRANCE)          http://www.irisa.fr/pampa/PROF/jmj.html

Re: Safety-critical development in Ada and Eiffel

[Ken Garlington]

Jean-Marc Jezequel wrote:
> 
> In article <33CC0548.4099@flash.net>, Ken Garlington <keg0@flash.net> writes:
> >Don Harrison wrote:
> 
> >> You would use them anywhere that a piece of code makes assumptions. For example,
> >> to help avoid the Ariane fiasco, include contracts in the INS(?) that specify
> >> Ariane 4 dynamics. Then, in testing, you will get an assertion violation when
> >> you apply Ariane 5 dynamics to it.
> >
> >This is why my blood pressure goes up!
> >
> >The _Ada_ implementation did generate an assertion violation the very
> >first time Ariane 5 dynamics were applied to it. Unfortunately, the very first time
> >Ariane 5 dynamics were applied to it was at LANUCH!
> 
> >They were not applied earlier, because the contractor assumed that the
> >Ariane 5 dynamics were the same as the Ariane 4, so why spend the money testing
> >to the same conditions already tested earlier? (They were wrong, of course, but they
> >didn't know that at the time).
> 
> At the risk of repeating myself, and reopening a thread beaten to death,
> the all point of design by contract (DBC) is to
> make this kind of assumptions explicit.

Which would not have happened in the Ariane V case.

> Ariane 5 is just a nice striking example
> of working with assumptions that are true at a point in time (Ariane 4) and no longer
> later on (Ariane 5). I think we agreed on this previously.

Yes. However, the assumption would not have been documented via an
Eiffel assertion,
as I claimed and you/Meyer never refuted (except to exclaim,
"casuistry!").

> 
> To sum up your point, you think that DBC, i.e. expressing hidden assumptions with
> Eiffel-like assertions would not have been practicable in this case.
> Others think it would have...

The distinction, of course, is that I gavce specific reasons why it
would not
have happened. Please post the evidence stating that the assertion would
have been
added (beyond the bizarre "Eiffel programmers are more careful"
argument).


> You seem to be a specialist in flight software. We are not. But specialists might,
> from time to time, learn something from people with a broader perspective
> (this is a general statement, I do not claim in particular that I have a broader
> perspective than you, or anyone else for that matter;-).

Unfortunately, the reverse is never true. Generalists never listen to or
learn from
specialists.

Consider the work of "generalists" that is incorporated into my code
today: Ada (Ichbiah
et. al.), object-based design principles (Gomaa/SPC, which in turn is
based on a lot
of previous work), safety-critical design (Levison and many others),
testing practices
(Beizer, et. al.)... there are many examples.

In your work on Ariane V, what specialists in the flight software field
(from
Arianespace or elsewhere) did you use to review your paper prior to its
publication?

After its publication, how many specialists in this field publicly (or
privately)
supported your position? I certainly was not the only one to object
publicly!

> Basically, our arguments failed to convinced you. 

Unfortunately, you failed to even discuss some of the points I (and
others) made.

> Yours failed to convince us.
> That's life, and I hope readers can make their mind themselves instead of
> relying on the one of us who shouts louder.

Yes, perhaps other criteria would be better. What would you suggest?
Company
revenues?

I would have thought practical experience would be a strong
determinant...

> >The other part of the problem was that the _response_ to the assertion
> >violation was wrong. The designer assumed the assertion would occur due to a hardware
> >failure, not a software requirements failure, and so the handler shut down the
> >"offending" hardware, rather than attempting some other action (e.g. replacing the
> >out-of-range value with a "safe" value). Frankly, I don't know that I would have written the
> >handler differently myself.
> 
> Agreed, that's a different problem altogether.

But an important one, if the assertion is left in the code. If the
assertion is removed
from the code, then other risks emerge.

> 
> >According to Meyer, et. al. Eiffel programmers would have written the
> >handler differently, and also they would have known to do the testing
> >differently (even though the requirements specification said that the tests need not change).
> >However, their explanation as to why this is true is far from convincing (basically,
> >that "Eiffel programmers believe in quality" or some such nonsense). Their "analysis"
> >is also on the Web. I've asked them to post my rebuttal to their paper, but I
> >don't know if they ever did.
> 
> Just for the record, I'm still waiting for your rebuttal paper to put it on my web alongside
> with the Computer article.

I thought I sent you the summary from the previous thread. I apologize.
I will go into
DejaNews and re-send it this weekend.

> 
> PS: Ken's post was brought to my attention by a colleague: I don't have time to check the news
> very often these days, and anyway I won't follow-up on this thread to avoid reopening the interminable
> Ariane thread.

Unfortunately, it's already been re-opened. As I feared, people are
using your paper to
"prove" the worth of DBC/Eiffel in safety-critical systems, although no
one has yet
published a list of such systems that have been written in Eiffel.

> 
> --
> Jean-Marc Jezequel               Tel : +33 299 847 192
> IRISA/CNRS                       Fax : +33 299 847 171
> Campus de Beaulieu               e-mail : jezequel@irisa.fr
> F-35042 RENNES (FRANCE)          http://www.irisa.fr/pampa/PROF/jmj.html

The stupidity of all the Ariane 5 analysts.

[Thaddeus L. Olczyk]

Frankly I'm getting a little tired of all the stupidity of the language
enthusiasts
and methodologists who are out there saying "Ariane 5 crashed because they
didn't use my language or my methodology". They are all ignoring one basic 
fact: the backup and primary systems were identical. Effectively this meant
that most bugs or generic hardware defects were likely to be duplicated,
eliminating
most of the safety that a backup system provides.

While not strongly conversant in the design of fault tolerant systems, I do
know
that some systems use backups which are the same and others use backups
which are entirely different ( different hardware, software, and
methodology ).
I believe that it doesn't take a rocket scientist to figure out that among
the later
are thingfs like: missle launching systems, remote facilities and rockets.

Re: The stupidity of all the Ariane 5 analysts.

[Bertrand Meyer]

When someone resorts to changing the subject header of
a thread to "The stupidity of all" those who have analyzed
an issue and come to different views, it's a pretty
good sign that he has run out of technical arguments.

To repeat once again the basic point made in the
paper by Jean-Marc Jezequel and myself: it is dangerous
and unjustifiable, especially in a mission-critical setting,
to reuse a software element without a specification.

From the beginning to the end of the software lifecycle,
Design by Contract encourages associating
such specifications with everything that you write.  
The method applies on at least four levels:
getting the stuff right in the first place, by
writing the software elements (the description of
*how* things are done) together with their contracts
(*what* they are supposed to achieve); documenting
them, through automatic extraction of contracts
as the basic information on a class; using the contracts
as one of the principal guides to reuse; and applying
them to debugging, which becomes less of a blind chase
and more of a focused analysis of possible discrepancies
between intents (the contracts) and reality (the
implementations).

None of this automatically guarantees perfection, but it
sure helps, as reported in these threads by
every poster who had actually used Eiffel, where the
ideas of Design by Contract are most directly realized.
It is true that to someone who has not really tried it
some of the benefits may appear exaggerated, but they're
real. Just one case in which an incorrect call to a
routine produces a violated precondition (caught
right away at testing time, whereas it could in
another approach have gone undetected for ages,
and caused painful bugs) justifies the investment.
In Eiffel development, this happens all the time. 

The Ariane case provides a textbook example of the dangers
of not using contracts. Of course it is easy to dismiss
this example through below-the-belt arguments, such as
 
	- "Jezequel and Meyer are not aerospace industry
	  experts." True, but it doesn't seem the experts
	  did so well in that case (and this is not the
	  first major example of aerospace software failure
	  either), so perhaps it's time to listen to
	  others. Besides, the official report
	  is clear and detailed enough to enable 
	  software professionals, aerospace experts or
	  not, to form their own judgment.

	- "This is only a pitch for Eiffel". The paper by
	  Jezequel and myself says explicitly that the
	  Ariane disaster was not due to a language problem.
	  In spite of this clear and obvious statement some of
	  the Ada enthusiasts in these newsgroups have
	  mistakenly taken the mention of Ariane as a personal
	  affront on their language. The pitch, if any, is for
	  the method of Design by Contract. That this method is
	  directly reflected in Eiffel as a language, and less
	  easy to apply in Ada, is a fact (and presumably
	  in the case of Ada 95 a conscious design decision),
	  not something to be held against the Eiffel community.

	- "Design by Contract is being promoted as a cure-all."
	  It's not (not a cure-all, and not promoted as such).
	  But it makes a major contribution.

All this is rhetorics and cannot succeed to obscure the basic
claim that systematic use of Design by Contract would probably
have avoided the crash. Of course like any reconstruction of
the past this conjecture cannot be proved, but there
is enough evidence in the official report to support
it. One can also object that other techniques would also
have achieved the same goal, such as heavy-artillery 
a posteriori quality assurance, but they seem far more difficult
and costly than integrating Design by Contract, a simple
and easy-to-apply idea, into the design, implementation,
documentation, reuse and validation process.

Some of the negative reactions seem to recall arguments
that were used against the proponents of structured programming
in the 70s, and against those of object technology in the 80s.
They are probably inevitable (and in part healthy, as they
force the methodologists to clarify their ideas, refine them,
and avoid overselling). But when a useful methodological
principle surfaces, all the rhetoric in the world will
not prevent its diffusion.

-- 
Bertrand Meyer, President, ISE Inc., Santa Barbara (California)
805-685-1006, fax 805-685-6869, <Bertrand.Meyer@eiffel.com>
Web: http://www.eiffel.com, with instructions for free download
== ISE Eiffel 4: Eiffel straight from those who invented it ==

Re: The stupidity of all the Ariane 5 analysts.

[Ken Garlington]

Bertrand Meyer wrote:
> 
> To repeat once again the basic point made in the
> paper by Jean-Marc Jezequel and myself: it is dangerous
> and unjustifiable, especially in a mission-critical setting,
> to reuse a software element without a specification.

Geoff Eldridge <geldridg@progsoc.uts.edu.au> was kind enough
to post my draft critique of your paper in HTML format at:

   http://www.progsoc.uts.edu.au/~geldridg/eiffel/ariane/

Let me know if you have any comments before I submit the final
version to you at the end of the month. Thank you.

Re: The stupidity of all the Ariane 5 analysts.

[Al Christians]

It occurs to me that DBC is less than 100% applicable in this case.  As
I read it, contracts constrain the terms of interaction between clients
and servers, and as I recall, the server should expose to the client all
the methods it uses to evaluate preconditions, so that the client can
verify its compliance before requesting the service.  In the case of the
Ariane, however, the client was hardware, some kind of dumb sensor that
took physical measurements and sent them into the machine, with no real
interest in what came out later, and not enough neurons to figure out if
it was in compliance with any set of preconditions or not.  So why
should the server bother to check preconditions in this case -- after
all, the client doesn't give a rusty, so why should the server?  Who's
he gonna call? If that's a contract, then my roof is a contract between
my house and the sky.   The guys who built the roof didn't know what
wind speed it was supposed to withstand, they just built it the way they
always build them, and that's good enough.  And if the roof is spec'd to
handle winds up to 80 mph, I'm figuring that if one comes along at 85
mph, the roof isn't going to shut itself down on account of a contract
violation, it's going to try its best to hold together anyway, which is
what I'd want. OTOH, the software on the Ariane did shut itself down
while the thing was in flight, didn't it?

It would be nice if there could be a clear spec that includes everything
that might happen in the real world,  but when the real world does
something that the spec didn't anticipate, do we want the software to
just curl up and die?

Al

Re: The stupidity of all the Ariane 5 analysts.

["Paul E. Bennett"]

In article <33E06929.59F6@easystreet.com>
           achrist@easystreet.com "Al Christians" writes:

> 
> It would be nice if there could be a clear spec that includes everything
> that might happen in the real world,  but when the real world does
> something that the spec didn't anticipate, do we want the software to
> just curl up and die?

If such a spec could be produced it would probably be too large to 
understand in reasonable time. Instead, we need to construct our systems 
so that out-of-the-ordinary stimulii do not cause un-expected activity of 
the system. This involves knowing what happens in the system when the 
stimulii exceed the design limitations. This takes some effort in FMECA
and the designing in of "Inherent Robustness" for the system to become
dependable.

If we are going to re-use components, we need to be more certain about
what the effects are for component failure. In the A5 flight systems
situation, there were other factors in the organisation which blindly 
accepted the decision for non-provision of flight profiles. A risk 
assessment for the decision should have been conducted to determine if 
this was a reasonable decision.


-- 
Paul E. Bennett ................... <peb@transcontech.co.uk>
Transport Control Technology Ltd.   <http://www.tcontec.demon.co.uk/>
+44 (0)117-9499861                  <enquiry@transcontech.co.uk>
Going Forth Safely

Re: The stupidity of all the Ariane 5 analysts.

[Ken Garlington]

Paul E. Bennett wrote:
> 
> In article <33E06929.59F6@easystreet.com>
>            achrist@easystreet.com "Al Christians" writes:
> 
> >
> > It would be nice if there could be a clear spec that includes everything
> > that might happen in the real world,  but when the real world does
> > something that the spec didn't anticipate, do we want the software to
> > just curl up and die?
> 
> If such a spec could be produced it would probably be too large to
> understand in reasonable time. Instead, we need to construct our systems
> so that out-of-the-ordinary stimulii do not cause un-expected activity of
> the system. This involves knowing what happens in the system when the
> stimulii exceed the design limitations. This takes some effort in FMECA
> and the designing in of "Inherent Robustness" for the system to become
> dependable.

Also, FMECA/FMET is quite valuable in that it usually is not solely
driven
by the specification, but also includes past experience driven from
failure
records. We've found FMET in particular to be quite useful in finding
holes in the specification.

> If we are going to re-use components, we need to be more certain about
> what the effects are for component failure. In the A5 flight systems
> situation, there were other factors in the organisation which blindly
> accepted the decision for non-provision of flight profiles. A risk
> assessment for the decision should have been conducted to determine if
> this was a reasonable decision.
> 
> --
> Paul E. Bennett ................... <peb@transcontech.co.uk>
> Transport Control Technology Ltd.   <http://www.tcontec.demon.co.uk/>
> +44 (0)117-9499861                  <enquiry@transcontech.co.uk>
> Going Forth Safely

Re: The stupidity of all the Ariane 5 analysts.

[Ian Begg]

On Sun, 20 Jul 1997 14:26:19 -0700, Bertrand Meyer
<Bertrand.Meyer@eiffel.com> wrote:

<snip>

>To repeat once again the basic point made in the
>paper by Jean-Marc Jezequel and myself: it is dangerous
>and unjustifiable, especially in a mission-critical setting,
>to reuse a software element without a specification.

My personal opinion, I would add ".... or, probably more importantly,
without fully testing that software element in the environment it is
being re-used into, i.e. under the conditions it is expected to be
used in."  My understanding is that the expected flight profile was
not flown as part of the integration or acceptance testing, and that
had this been done the problem would (probably) have been found.  This
is what I mean by testing in the environment it is being re-used into.

<snip>

>	- "This is only a pitch for Eiffel". The paper by
>	  Jezequel and myself says explicitly that the
>	  Ariane disaster was not due to a language problem.
>	  In spite of this clear and obvious statement ...

Even clear and obvious statements have to be assessed to decide on the
intent of the author/speaker, for instance is he saying this because
he means it or because he's trying to pull the wool over our eyes.  [I
have not read your report so have no idea how to answer in this case,
everyone must make their own decision.  The author must ensure they
don't leave themselves open to the wrong interpretation.]

>                                                                                ... some of
>	  the Ada enthusiasts in these newsgroups have
>	  mistakenly taken the mention of Ariane as a personal
>	  affront on their language. The pitch, if any, is for
>	  the method of Design by Contract. 

How many languages support Design by Contract as part of the language
as in the case of Eiffel?  I don't have experience of a broad range of
languages so I can't answer this.  However, if the answer is only
Eiffel, (plus perhaps some other non relevant langauges) then it is
not difficult to see how people might interpret your motives as "OK we
can't be so obvious as to say Eiffel is the answer, but if we say
Design by Contract is the answer, people will need to find a language
to support it, hence Eiffel."  You are the only person who knows your
real motives, but the discussions going on suggest some people are
interpreting your motives in this way.

<snip>

>    One can also object that other techniques would also
>have achieved the same goal, such as heavy-artillery 
>a posteriori quality assurance, but they seem far more difficult
>and costly than integrating Design by Contract, a simple
>and easy-to-apply idea, into the design, implementation,
>documentation, reuse and validation process.

How easy is this to apply in non Eiffel implementations?  I guess it
needs extra work by the engineers, in an area most are probably not
familiar with and will hence need training, and success is dependent
on setting up good procedures to enforce its use etc.  Compare the
cost of that with doing integration and acceptance testing under
anticipated use conditions which should be done anyway and I come to
the conclusion use of Design by Contract was not the best way of
preventing this accident.

This is all of course my own personal opinion, and none of it should
be read as an 'attack' on anyone elses opinions, only a difference of
opinion.  Sorry if this is repeating stuff from the original thread(s)
which I have not been tracking closely.

Ian Begg

Re: The stupidity of all the Ariane 5 analysts.

[Robert S. White]

In article <33D2827B.41C67EA6@eiffel.com>, Bertrand.Meyer@eiffel.com says...
>
>To repeat once again the basic point made in the
>paper by Jean-Marc Jezequel and myself: it is dangerous
>and unjustifiable, especially in a mission-critical setting,
>to reuse a software element without a specification.

  We "aerospace" people agree.


>All this is rhetorics and cannot succeed to obscure the basic
>claim that systematic use of Design by Contract would probably
>have avoided the crash.

  The problem with this is that you are preaching to the choir.
Long long ago, before Eiffel, the general concept of capturing
system requirements in a spec document, flowing down to a software
requirements document, conducting detailed design reviews with the
requirment documents as resources/authorities (or getting them 
updated) and finally having software qualification tests done to 
prove performance to the software requirements spec has been 
practiced by the "aerospace" industry.  The "crash" resulted 
because of a failure to fully follow this existing practice.  
Your papers about Eiffel and Design By Contract are just, IMO,
another way to implement the concept of developing software that
must comply to requirements.

_____________________________________________________________________
Robert S. White         -- An embedded systems software engineer

Re: The stupidity of all the Ariane 5 analysts.

[Robert Dewar]

Bertrand said

<<>All this is rhetorics and cannot succeed to obscure the basic
>claim that systematic use of Design by Contract would probably
>have avoided the crash.
>>

Well all sorts of things would have avoided the crash. One can also say
that systematic proof of correctness, or systematic code review, or
in fact almost any steps to be a bit more careful in this particular
area, would have avoided the crash. As usual, a disaster of this type
is a chain of occurrences, and fixing any one of them would have avoided
the problem.

Certainly the notion of clear interfaces and interface requirements
(I find the use of the capitalized "Design by Contract" a little 
pretentious, since I see no new concept there that justifies a new
technical term) is one which we can all agree on.

The argument that the use of Eiffel would have prevented the problem is
entirely unconvincing, and has the unfortunate effect of encouraging
people to think that the entire argument is just empty language advocacy.


When there is a major screwup in software (some examples, the long lines
failure at AT&T, the Denver baggage handling, the overnight funds transfer
disaster in an NY bank etc), there is a natural tendency for over
enthusiastic pushers of a particular language to try to argue that
their particular pet language would have guaranteed freedom from the
problem.

Such arguments are never convincing, because even if it *were* the case
that the particular problem at hand might *possibly* have had a better
chance of being avoided writing in language X, extending this to the
allegation that the use of language X would in general have been beneficial
stretches credibility. In particular, in a large complex system of this
type, the language and all other components have to meet all kinds of
complex requirements, and people who know nothing at all about the whole
application have no idea whatsoever if their pet language would in fact
meet the criteria.

If Bertrand is saying:

(a) The Ariane problem (at least little piece of it) is a good example of
why designing interfaces that are complete with respect to establishing
a requirements contract for clients is a good idea.

(b) One of the attractive features of Eiffel is that it supports this
concept directly and encourages programmers to use this approach.

Then the argument is entirely reasonable, and one that I cannot see
anyone objecting to. But the way the argument is stated, it reads as
though he is saying:

If Eiffel had been used for the Ariane project, it would not have crashed.

And I find that claim entirely unjustified, and it detracts from the
important underlying point.

No language can force people to do things right, and to a large extent
no language prevents people from doing things right. What a language
can do is to encoruage a style of programming and thinking that helps
people find their way to doing things right a little more easily.


This is indeed the principle behind the Ada design, and a lot of the
reason that Ada is successful in practice is precisely because it
encourages "good thinking". 

But when people said "if only the Denver baggae system had been written
in Ada, I would be flying to the new airport today", this crossed the
line into non-credible language advocacy.

Similarly, a lot of the thinking behind the Eiffel design is also
to encourage "good thinking", and in that sense Ada and Eiffel are
kindred spirits in their approach to language design, though they
make different choices about what is important to encourage.

However, when the Eiffel advocacy crosses the line to saying that a large
complex system, if written in Eiffel, would have succeeeded where otherwise
it failed, again this crosses the line into non-credible language advocacy.

I am, as everyone knows, a strong advocate of the use of Ada. However, I
take a lot of care NOT to make exaggerated claims, since I don't think
this helps at all. What Ada (or any other language) can do, is to push
programmers up a little bit on the quality and productivity scales.

A little bit may not sound like much, but these scales are logarithmic
scales in which the productivity of programmers and the quality of the
code they right varies over orders of magnitudes, so a "little bit" can
add up to very substantial savings in costs, and very substantial
improvements in quality in practice. 

Re: The stupidity of all the Ariane 5 analysts.

[Ken Garlington]

Robert S. White wrote:
> 
> The "crash" resulted
> because of a failure to fully follow this existing practice.

Yes, I agree. (I decided not to address the statement in the Eiffel
paper that says the opposite, it seemed to be piling on.)

Hope you get a chance to review my draft critique. Geoff Eldridge
<geldridg@progsoc.uts.edu.au> was kind enough to create an HTML
version at:

   http://www.progsoc.uts.edu.au/~geldridg/eiffel/ariane/

BTW, I just noticed the flaws in the example Eiffel code at the
last minute (midnight Saturday, in fact). Did you and everyone
else know it was wrong, and just not say anything, or did we
all overlook the errors?

 _____________________________________________________________________
> Robert S. White         -- An embedded systems software engineer

Re: The stupidity of all the Ariane 5 analysts.

[Robert S. White]

In article <33D3F2CB.61DC@flash.net>, kennieg@flash.net says...

>BTW, I just noticed the flaws in the example Eiffel code at the
>last minute (midnight Saturday, in fact). Did you and everyone
>else know it was wrong, and just not say anything, or did we
>all overlook the errors?


Do you mean what you addres in your Critique's paragraph 3.1.1 about
Meyer's Eiffel paper's solution code:

"...any software element that has such a fundamental constraint should 
state it explicitly, as part of a mechanism present in the programming 
language, as in the Eiffel construct

      convert (horizontal_bias: INTEGER): INTEGER is
            require
                  horizontal_bias <= Maximum_bias
            do
                  ...

            ensure
                  ...

            end

where the precondition states clearly and precisely what the input must 
satisfy to be acceptable."

  Yes it looks like the above code should do a floating point absolute
value on horizontal_bias before it checks against Maximum_bias.  But
I consider that a _typical_ coding error that is normally found by software
code inspections or unit level tests - whoops - DBC/Eiffel does _not_
find this error!  And yes, the test _must_ be done while the value is
still expressed in floating point _before_ conversion to integer...details,
details.  Stuff we encounter all of the time in our problem domain.

  One could argue that the above is just a nit (flawed example) and 
the overall premise of DBC/Eiffel as a "silver bullet" panacea versus
conventional "aerospace industry software engineering process" is 
the real issue.
_____________________________________________________________________
Robert S. White         -- An embedded systems software engineer
                  (newsgroups trimmed to contain the "language wars")

Re: The stupidity of all the Ariane 5 analysts.

[Robert Dewar]

Robert White said

<<  One could argue that the above is just a nit (flawed example) and
the overall premise of DBC/Eiffel as a "silver bullet" panacea versus
conventional "aerospace industry software engineering process" is
the real issue.
>>

Unfortunately the Ariane crash was the result of just such a nit!

I would think experience would teach us to be very suspicious of any
claims by anyone to have a silver bullet panacea wjhen it comes to
software engineering for large systems.

However tools and techniques that help, even a little bit, are welcome
even if they are not a panacea (I consider Ada to be in this category,
and I always find it annoying when people advertise Ada as being
a silver bullet panacea).

Re: The stupidity of all the Ariane 5 analysts.

[Ken Garlington]

Robert Dewar wrote:
> 
> Robert White said
> 
> <<  One could argue that the above is just a nit (flawed example) and
> the overall premise of DBC/Eiffel as a "silver bullet" panacea versus
> conventional "aerospace industry software engineering process" is
> the real issue.
> >>
> 
> Unfortunately the Ariane crash was the result of just such a nit!
> 
> I would think experience would teach us to be very suspicious of any
> claims by anyone to have a silver bullet panacea wjhen it comes to
> software engineering for large systems.
> 
> However tools and techniques that help, even a little bit, are welcome
> even if they are not a panacea (I consider Ada to be in this category,
> and I always find it annoying when people advertise Ada as being
> a silver bullet panacea).

I agree with this, but I also think every tool and technique has both
advantages _and risks_. Failing to understand the risks, and where they
are (and aren't) important, can lead to extremely dangerous choices made
with the best of intentions. I view executable assertions (Ada, Eiffel,
etc.) in this category. In some environments, they are clear wins. In
other
environments, I think it is at least debatable whether their advantages
outweigh their risks.

Complicating this factor is that many people confuse the abstract Good
Thing
(e.g. well-documented assumptions) with a specific implementation of the
Good Thing. They consider the Good Thing so obvious that they don't
consider
(a) what Bad Things may also be associated with the _implementation_ --
the
proponents of the implementation aren't always candid about this part
(or are
sometimes just blind to it) and (b) whether, for a given project, there
are better
ways to achieve the Good Thing.

As you've said in the past, Computer Science is in part a misnomer in
that
there doesn't seem to be much in the way of controlled experiments. So,
a
lot of these risk/reward analyses are based on incomplete data, at best.
This doesn't mean that they shouldn't be done, however. Ignoring the
risks
(or worse, dismisssing them as "they just haven't seen the light yet,"
as
Mr. Meyer did recently) is not smart in any environment, and certainly
not
in safety-critical systems.

Re: The stupidity of all the Ariane 5 analysts.

[Shmuel (Seymour J.) Metz]

Robert Dewar wrote:
> 
> I would think experience would teach us to be very suspicious of any
> claims by anyone to have a silver bullet panacea wjhen it comes to
> software engineering for large systems.
> 
> However tools and techniques that help, even a little bit, are welcome
> even if they are not a panacea (I consider Ada to be in this category,
> and I always find it annoying when people advertise Ada as being
> a silver bullet panacea).

I would go farther; I am suspicious of anyone who advocates a particular
languages and can't tell me anything in the language that he dislikes.
If
he can't identify flaws, either he's too inexperienced to rely on or 
isn't perceptive enough. *any* programming language will have warts, and
a real professional will recognize them and have strategies for dealing 
with them.

-- 

                        Shmuel (Seymour J.) Metz
                        Senior Software SE

The values in from and reply-to are for the benefit of spammers:
reply to domain eds.com, user msustys1.smetz or to domain gsg.eds.com,
user smetz.

Re: The stupidity of all the Ariane 5 analysts.

[Robert Dewar]

Shmuel says

<<I would go farther; I am suspicious of anyone who advocates a particular
languages and can't tell me anything in the language that he dislikes.
If
he can't identify flaws, either he's too inexperienced to rely on or
isn't perceptive enough. *any* programming language will have warts, and
a real professional will recognize them and have strategies for dealing
with them.
>>

Of course, that goes without saying, but did you ever meet anyone who was
in that category whose ideas you would otherwise respect. I know some
people who display this kind of ignorance, but they are not the kind of
people I would take seriously.

You can say the same thing about operating systems of course, for example
anyone who cannot point out the flaws in XXX, simply indicates that they
do not know the system very well, or that they lack the experience to
know that improvements are possible

(XXX = NT, Unix, Win95, OS/2, VMS etc etc)

Re: The stupidity of all the Ariane 5 analysts.

[Shmuel (Seymour J.) Metz]

Robert Dewar wrote:
> 
> You can say the same thing about operating systems of course, 

In fact, I *do* say the same sort of thing about operating systems. ;-)

-- 

                        Shmuel (Seymour J.) Metz
                        Senior Software SE

The values in from and reply-to are for the benefit of spammers:
reply to domain eds.com, user msustys1.smetz or to domain gsg.eds.com,
user smetz.

Re: The stupidity of all the Ariane 5 analysts.

[Larry Kilgallen]

In article <dewar.870362889@merv>, dewar@merv.cs.nyu.edu (Robert Dewar) writes:

> You can say the same thing about operating systems of course, for example
> anyone who cannot point out the flaws in XXX, simply indicates that they
> do not know the system very well, or that they lack the experience to
> know that improvements are possible
> 
> (XXX = NT, Unix, Win95, OS/2, VMS etc etc)

I think blindness to flaws has more to do with inexperience in
_other_ environments than in the one under discussion.

If my experience is limited to languages without uplevel addressing
(e.g., Bliss, C) I will not see that as a shortcoming the way I
would if I had used other languages (Ada, Pascal).

Larry Kilgallen

Re: The stupidity of all the Ariane 5 analysts.

[Don Harrison]

This is a response to something posted to comp.lang.ada but not comp.lang.eiffel..

"Shmuel (Seymour J.) Metz" wrote:

:Robert Dewar wrote:
:> 
:> I would think experience would teach us to be very suspicious of any
:> claims by anyone to have a silver bullet panacea wjhen it comes to
:> software engineering for large systems.
:> 
:> However tools and techniques that help, even a little bit, are welcome
:> even if they are not a panacea (I consider Ada to be in this category,
:> and I always find it annoying when people advertise Ada as being
:> a silver bullet panacea).

Me too. :)

:I would go farther; I am suspicious of anyone who advocates a particular
:languages and can't tell me anything in the language that he dislikes.

Presumably, you're talking about me. :)

If you think I never have any criticisms of Eiffel, you're mistaken. What may 
surprise you even more is that I sometimes have positive things to say about
Ada! - for example, two recent posts to comp.lang.eiffel under the subject
"Ennumerations in Eiffel" (31.7.97 and 1.8.97). These included two criticisms
of Eiffel and one positive comment about Ada.

Also, was my agreement with criticisms of Eiffel WRT use of assertions and GC
in hard realtime software a glowing endorsement of Eiffel? I think not.

:If he can't identify flaws, either he's too inexperienced to rely on or 
:isn't perceptive enough. 

Thank you. :)

:*any* programming language will have warts, 

Eiffel has its flaws; it's just that Ada has about 50 times more. :)

:and a real professional will recognize them and have strategies for dealing 
:with them.

Agree.


BTW, one the few areas where Ada is streets ahead of Eiffel (and most languages), 
IMO, is standardisation. While Eiffel does have a standard language and libraries, 
not all vendors comply with them. Part of the problem is that there is no 
official certification of Eiffel compilers and, hence, no validation suite as 
there is in Ada. Obviously, this causes portability problems. 

Just because someone is biased, doesn't mean they can't also be honest and 
objective. Also, one can be biased with or without justification.


P.S. It shows considerably more intestinal fortitude to criticise someone to 
their face rather than where you don't think they'll hear you. Even if they 
don't agree with what you have to say, they may at least respect you for not
saying it behind their back. :(


Don.
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Don Harrison             donh@syd.csa.com.au

Re: The stupidity of all the Ariane 5 analysts.

[Shmuel (Seymour J.) Metz]

Don Harrison wrote:
>
> Presumably, you're talking about me. :)

Assuming that your name is legion ;-)
 
> P.S. It shows considerably more intestinal fortitude to criticise someone to
> their face rather than where you don't think they'll hear you. Even if they
> don't agree with what you have to say, they may at least respect you for not
> saying it behind their back. :(

Let's see; I post a public message in a forum in which you have been 
participating, and that constitutes saying it behind your back? If you
think 
that I should have sent you an E-mail, plenty of people have been flamed
for 
doing just that, on the grounds that is was an additional copy of a
message that
would already have been read by virtue of having been posted. I am
generally 
very sparing of cc headers in Usenet, and see nothing in my message to
justify ending you an extra copy.

Further, since the comment applied to multiple messages in the thread, I
would 
have had to search out all of the others to get their addresses and
include them 
on the cc header. Or did you think that you were more important than the
others?
 
> Don.
> =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
> Don Harrison             donh@syd.csa.com.au

BTW, why on earth would I include a signature block if I were attempting
to 
criticize someone behind their back?

-- 

                        Shmuel (Seymour J.) Metz
                        Senior Software SE

The values in from and reply-to are for the benefit of spammers:
reply to domain eds.com, user msustys1.smetz or to domain gsg.eds.com,
user smetz.

Re: The stupidity of all the Ariane 5 analysts.

[Ken Garlington]

Robert S. White wrote:
> 
> In article <33D3F2CB.61DC@flash.net>, kennieg@flash.net says...
> 
> >BTW, I just noticed the flaws in the example Eiffel code at the
> >last minute (midnight Saturday, in fact). Did you and everyone
> >else know it was wrong, and just not say anything, or did we
> >all overlook the errors?
> 
> Do you mean what you addres in your Critique's paragraph 3.1.1 about
> Meyer's Eiffel paper's solution code:
> 
> "...any software element that has such a fundamental constraint should
> state it explicitly, as part of a mechanism present in the programming
> language, as in the Eiffel construct
> 
>       convert (horizontal_bias: INTEGER): INTEGER is
>             require
>                   horizontal_bias <= Maximum_bias
>             do
>                   ...
> 
>             ensure
>                   ...
> 
>             end
> 
> where the precondition states clearly and precisely what the input must
> satisfy to be acceptable."
> 
>   Yes it looks like the above code should do a floating point absolute
> value on horizontal_bias before it checks against Maximum_bias. 

Yes, this is what I mean.

> But
> I consider that a _typical_ coding error that is normally found by software
> code inspections or unit level tests - whoops - DBC/Eiffel does _not_
> find this error!  And yes, the test _must_ be done while the value is
> still expressed in floating point _before_ conversion to integer...details,
> details.  Stuff we encounter all of the time in our problem domain.

It would have been found if the design made it clear a different
solution
was necessary, or "common sense" was used in this example.

>   One could argue that the above is just a nit (flawed example) and
> the overall premise of DBC/Eiffel as a "silver bullet" panacea versus
> conventional "aerospace industry software engineering process" is
> the real issue.

However, I think there's a fundamental point made by this "nit." Several
people read the paper; in fact, I saw one example in DejaNews last year
where someone posted the Ada version of this code... incorporating the
same mistakes! No one, as far as I could tell, saw this "obvious" error
in three lines of code despite many months of opportunity. What does
this
say about the credibility of manual inspections at the source code level
in picking up errors of this type during the reuse of software?

> _____________________________________________________________________
> Robert S. White         -- An embedded systems software engineer
>                   (newsgroups trimmed to contain the "language wars")

Re: The stupidity of all the Ariane 5 analysts.

[Robert S. White]

In article <33D69CAF.19F1@flash.net>, kennieg@flash.net says...

> Robert S. White wrote:
>> I consider that a _typical_ coding error that is normally found by software
>> code inspections or unit level tests

>It would have been found if the design made it clear a different
>solution
>was necessary, or "common sense" was used in this example.

  Perhaps I should have provided more detail about our industry
software engineering process.  During the detail design review of 
the software module, Test Cases, for doing the module unit level
tests are also presented.  One of the checklist items for the test
cases is to ensure that the module is tested at boundry conditions
and beyond boundry conditions - both for the positive and negative
extremes in the case of a signed value.  This type of software
design process with inspections using checklists even against the
test cases is a way to spread out that very rare/precious commondity
called "common sense".
_____________________________________________________________________
Robert S. White         -- An embedded systems software engineer

Re: The stupidity of all the Ariane 5 analysts.

[W. Wesley Groleau x4923]

> Your papers about Eiffel and Design By Contract are just, IMO,
> another way to implement the concept of developing software that
> must comply to requirements.

He/they start from the (obviously debatable) belief that 
"'ideas of Design by Contract are most directly realized' in Eiffel" 
and then make an unjustified jump to 
"if it doesn't look like Eiffel, it doesn't do Design By Contract."

-- 
----------------------------------------------------------------------
    Wes Groleau, Hughes Defense Communications, Fort Wayne, IN USA
Senior Software Engineer - AFATDS                  Tool-smith Wanna-be

Don't send advertisements to this domain unless asked!  All disk space
on fw.hac.com hosts belongs to either Hughes Defense Communications or 
the United States government.  Using email to store YOUR advertising 
on them is trespassing!
----------------------------------------------------------------------

Re: The stupidity of all the Ariane 5 analysts.

[Joerg Rodemann]

Bertrand Meyer (Bertrand.Meyer@eiffel.com) wrote:
> To repeat once again the basic point made in the
> paper by Jean-Marc Jezequel and myself: it is dangerous
> and unjustifiable, especially in a mission-critical setting,
> to reuse a software element without a specification.

Agreed: it should be clearly stated what something is intended for. Although
the idea of preconditions sound very good to me at first sight I am very
sceptical if this is very useful at all since one has to think of EVERY
possibility the routine might be used for. E. g. assume the implementation
of some mathematical function that is not well defined for the complete
complex plane or not continously defined for real numbers. On the other
hand you might have an algorithm at hand that is suitable for every value
the function is defined. (Maybe the Gamma, Beta and Zeta functions are
some examples that meet my assumptions --- I can't remember them exactly right
now.) On the other hand you know that for your actual problem this function
is indeed defined for each possible value since you are --- from your
requirements --- quite sure that some area will not be left. So you could
enter this as a precondition. 

But now guess you like to use this component for some other work --- and 
you are not dealing with complex numbers of different areas of the plane.
It seems to me that you list of preconditions will grow very fast ---
leaving the code quite unreadable. (Kind of a " i++;  // increments i" type
comment.) Code is the more difficult to comprehend the large it is. And
especially emphasizing details may obscure the essential parts. Was not
this abstraction was all about?

From this point of view I would think that easy checkable preconditions
should be checked (e. g. using assertions). Others should be kept in the
documentation. As far as Ariane V is concerned the report clearly states
that the problem was know and discussed during develepment but due to
cpu time limits not every suspected variable was covered. It seems to me
that a lack of knowledge about the actual application was one reason for
the error to occur. (Among the devastating assumption that exceptions are
only to occur when hardware fails!) No pre-/postcondition theology would have
cured that, IMHO. (I believe no methodology could cure that. and this might
be a problem that is not easy if at all to solve for systems that complex.)

Well, just my humble opinion...and I have to admit that I have no experions
within realtime or safety critical applications development.

Yours sincerly

Joerg


--
rodemann@mathematik.uni-ulm.de | Dipl.-Phys. Joerg S. Rodemann
Phone: ++49-(0)711-5090670     | Flurstrasse 21, D-70372 Stuttgart, Germany
-------------------------------+---------------------------------------------
rodemann@rus.uni-stuttgart.de  | University of Stuttgart, Computing Center
Phone: ++49-(0)711-685-5815    | Visualization Department, Office: 0.304
Fax:   ++49-(0)711-678-7626    | Allmandring 30a, D-70550 Stuttgart, Germany