Critique of Ariane 5 paper (finally!)

Critique of Ariane 5 paper (finally!)

[Ken Garlington]

I've put what I hope is my final version of my critique of the Eiffel
Ariane 5 paper at:

  http://www.flash.net/~kennieg/ariane.html

I hope that this gets referenced from the Eiffel web site, to provide
some balance to the original article. Thanks to everyone who helped out.

Readers may also be interested in the Meyer article on Java at

  http://www.cm.cf.ac.uk/CLE//volume97/18338

The discussion is interesting in that Meyer (a) criticizes Java for not
being used on large projects (whatever happened to unfair criticism of
new languages? :), (b) uses the Ariane 5 paper as justification for the
need for assertions, and (c) notes that Eiffel supports C++ interfaces,
with Java VM support coming soon (why wait when Intermetrics already
supports Ada applets running on the JVM? :)

Re: Critique of Ariane 5 paper (finally!)

[Juergen Schlegelmilch]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 2002 bytes --]


On Wed, 06 Aug 1997, Bertrand Meyer <Bertrand.Meyer@eiffel.com> wrote:
>I am "willing to guess" anything but guesses are not very
>enlightening. More to the point, however, you may want to read
>the article by Alan Radding in Software Magazine (not IEEE
>Software), July 1997, pages 51-54. It is entitled
>"Tool Immaturity Tempers Java" and begins: "With the pack of
>vendors offering Java-based development tools and developers
>clamoring for them, you'd think everyone have Java projects
>underway. They don't. The lack of mature tools and standards
>is keeping most companies in exploration mode." The article
>quotes Judith Hurwitz from the Hurwitz Group as "counsel[ing]
>developers not to expect a robust, industrial-strength Java
>development environment for about three years". It adds
>"In addition, Java compilers are slow", quoting Mitch Kramer,
>a senior analyst with Patricia Seybold Group. Jeffry Borror,
>director of information technology at Daiwa Securities America
>is quoted as saying "Java hasn't stabilized as a language, and
>the tools still lack many features". Etc.

If you have a look at Eiffel, the situation is similar: Today
we see the third version of the language, and there are still
subtle points under discussion in this group. Also, it took
Eiffel vendors about 10 years to offer mature tool support, and 
the compilers are still not the fastest (although much improved).

So, don't disqualify a language because it is young.

  J�rgen Schlegelmilch
-- 
+-----------------------------------------------------------------------------+
 Dipl.-Inf. Juergen Schlegelmilch                 University of Rostock
 email: schlegel@Informatik.Uni-Rostock.de        Computer Science Department
 http://www.informatik.uni-rostock.de/~schlegel   Database Research Group 
 Tel: ++49 381 498 3402                           18051 Rostock 
 Fax: ++49 381 498 3426                           Germany
+-----------------------------------------------------------------------------+

Re: Critique of Ariane 5 paper (finally!)

[Ken Garlington]

Bertrand Meyer wrote:
> 
> Robert Dewar wrote:
> 
> [Quoting Ken Garlington]
> 
> !!! The discussion is interesting in that Meyer (a) criticizes
> !!! Java for not being used on large projects
> !!! (whatever happened to unfair criticism of new languages?
> 
> [Robert Dewar]
> 
> > Hmmm! I guess he does not consider the Corel office suite large!!!
> > Or perhaps simply does not know about it
> 
> It would be difficult not to know about it, as it gets hammered
> over and again by Java proponents (along with Java
> tools themselves) as the example of completed Java development,
> to the extent that one may wonder whether there is any other.

How many large-scale projects need to be performed before evidence
of scalability is established?

What was the standard set for Eiffel, for example? How long after
Eiffel's introduction was this standard met?

> 
> > (actually a surprising number of
> > large projects are being done in Java, I would be willing
> > to guess that at any program size, the use of Java eclipses
> > the use of Eiffel by a very wide margin).
> 
> I am "willing to guess" anything but guesses are not very
> enlightening. More to the point, however, you may want to read
> the article by Alan Radding in Software Magazine (not IEEE
> Software), July 1997, pages 51-54. It is entitled
> "Tool Immaturity Tempers Java" and begins: "With the pack of
> vendors offering Java-based development tools and developers
> clamoring for them, you'd think everyone have Java projects
> underway. They don't. The lack of mature tools and standards
> is keeping most companies in exploration mode." The article
> quotes Judith Hurwitz from the Hurwitz Group as "counsel[ing]
> developers not to expect a robust, industrial-strength Java
> development environment for about three years". It adds
> "In addition, Java compilers are slow", quoting Mitch Kramer,
> a senior analyst with Patricia Seybold Group. Jeffry Borror,
> director of information technology at Daiwa Securities America
> is quoted as saying "Java hasn't stabilized as a language, and
> the tools still lack many features". Etc.

Weren't some of these same complaints made about Eiffel in it's
early years? (I seem to remember recently seeing some posts to
this effect, in fact.) I know they were made about Ada (I made
some of them :), and Ada has certainly shown that it can be
used in some very large-scale projects (1M+ SLOCs), that
it can be reused/ported successfully, etc.

It just seems a little premature to bash Java simply because
of lack of use. As I said about Eiffel:

"There does not appear to be any evidence that DBC/Eiffel
has been successfully used in a system like the Ariane IRS,
much less that it produced significantly improved safety
or reliability. By itself, this is not particularly
damaging: All new ideas have to have a first user."

> 
> On portability see also the article extracts (from
> ComputerWorld and IEEE Computer) at
> http://www.eiffel.com/general/news.
> 
> --
> Bertrand Meyer, President, ISE Inc., Santa Barbara (California)
> 805-685-1006, fax 805-685-6869, <Bertrand.Meyer@eiffel.com>
> Web: http://www.eiffel.com, with instructions for free download
> == ISE Eiffel 4: Eiffel straight from those who invented it ==

Re: Critique of Ariane 5 paper (finally!)

[Ken Garlington]

Another note regarding the Meyer attack on Java:

The portability of Java is questioned, primarily by reference
to a trade article where Microsoft has "declared war" on Sun
over Java -- the implication being that Java will not be available
on Microsoft OSs (e.g. Windows). I'm not sure that the cause and
effect is particularly well established, but coincidentally I
read that, in the new Apple-Microsoft collaboration, the two
parties have agreed to work to provide more Java support on
Apple products. Apparently, the "war" is more Microsoft vs.
Sun than Microsoft vs. Java.

Re: Critique of Ariane 5 paper (finally!)

[Bertrand Meyer]

It's impossible to have a meaningful discussion
if one's arguments are distorted and misquoted.

Ken Garlington wrote:

> Another note regarding the Meyer attack on Java:

There is no such thing as a "Meyer attack on Java".
Read my postings and articles and you will see that
if they "attack" something it is the idea that Java
is the language to replace all languages. That's
rather different.

> The portability of Java is questioned [in an article
> at http://www.eiffel.com/general/news] primarily by reference
> to a trade article where Microsoft has "declared war" on Sun
> over Java -- the implication being that Java will not be available
> on Microsoft OSs (e.g. Windows).

There is no such implication, which would be foolish.
Microsoft has said they would not support the Java Foundation
Class libraries. This does not mean they are not making
Java itself available; but it does break the myth of total and
automatic Java portability.

There is enough room for controversy in what people actually
write not to create artificial disputes about what someone
else believe they think. This is just a waste of everyone's
time.
-- 
Bertrand Meyer, President, ISE Inc., Santa Barbara (California)
805-685-1006, fax 805-685-6869, <Bertrand.Meyer@eiffel.com>
Web: http://www.eiffel.com, with instructions for free download
== ISE Eiffel 4: Eiffel straight from those who invented it ==

Re: Critique of Ariane 5 paper (finally!)

[Ken Garlington]

Ken Garlington wrote:
> 
> The article goes on to discuss Microsoft and Apple collaborating
> on Java implementations. I note that the Eiffel comments on this
> article mentions that Eiffel is available on "Windows NT, Windows 95,
> Unix (a dozen different platforms), Linux and VMS...." Is an Apple
> implementation in the works?

After sending this, I realized that again I may have oversimplified
Mr. Meyer's statement in this area. This is the full quote:

"Last week's story showed the limits of Java's promise of openness.
With the rift between Microsoft and Sun, the promise of
portability also evaporates. 

"Eiffel, and in particular ISE Eiffel, provides one of the very few
truly portable development environment available in the software
industry. The same set of tools and libraries runs across Windows
NT, Windows 95, Unix (a dozen different platforms), Linux and
VMS with no need for source code change. In particular: EiffelVision
is a fully portable graphical library; and the EiffelNet library
supports multi-platform client-server development, with fully
transparent object exchange. The recently released DAIS-Eiffel
CORBA implementation from ICL Inc. supports interoperability
with other platforms and languages. ISE's C and C++ interface is
one more tool supporting openness and portability.

"For corporate customers who are looking for an open and portable
solution available today, rather than promises subject to the
conflicting requirements of companies that fiercely compete with
each other, the choice is clear."

I hope this corrects any misconceptions that might have occured
from the earlier summarized quote.

Re: Critique of Ariane 5 paper (finally!)

[Ken Garlington]

Bertrand Meyer wrote:
> 
> It's impossible to have a meaningful discussion
> if one's arguments are distorted and misquoted.
> 
> Ken Garlington wrote:
> 
> > Another note regarding the Meyer attack on Java:
> 
> There is no such thing as a "Meyer attack on Java".
> Read my postings and articles and you will see that
> if they "attack" something it is the idea that Java
> is the language to replace all languages. That's
> rather different.

I stand corrected. I should have said: "Another note
regarding the Meyer 'attack' on Java being the language
to replace all other languages..."

This "attack" was predicated (in part) on the following
points, which I challenged:

1. That there was not enough evidence that Java could
scale to large projects. I questioned whether this
was really a fundamental obstacle, since there are
also concerns about the scalability of Eiffel. Furthermore,
your own statements about the lack of language use not
being a sufficient argument for rejecting it (e.g.
Eiffel for real-time safety-critical avionics) seems
to shed doubt on the value of your contention.

2. That using the Ariane 5 example to "prove" that Java
was flawed due to a lack of built-in assertions was,
itself, flawed. This is discussed in more detail in
http://www.flash.net/~kennieg/ariane.html.

I assume there is no response to these rebuttals?

> 
> > The portability of Java is questioned [in an article
> > at http://www.eiffel.com/general/news] primarily by reference
> > to a trade article where Microsoft has "declared war" on Sun
> > over Java -- the implication being that Java will not be available
> > on Microsoft OSs (e.g. Windows).
> 
> There is no such implication, which would be foolish.
> Microsoft has said they would not support the Java Foundation
> Class libraries. This does not mean they are not making
> Java itself available; but it does break the myth of total and
> automatic Java portability.

Again, you are correct: I should have said "the implication being
that Java Foundation Class libraries will not be available on
Microsoft OSs (e.g. Windows)."

Today's paper had more information about Microsoft and Java
(in the context of the Apple deal):

"Sun, along with IBM, Oracle, and other industry players, has
been promoting Java as a new operating system that would
succeed Microsoft's Windows as the future industry standard
as more computing power is expected to occur over networks,
instead of the PC desktop.

"Not surprisingly, Microsoft has been resisting those efforts.
It has embraced Java as a programming language that software
companies can use to write programs that run on Microsoft's
Windows PC operating system software."

So, with respect to Java as a portable OS, I believe you are
correct. However, with respect to Java as a programming
language, it's unclear to me whether there will be a single
internationally accepted standard or not. Are all
Eiffel implementations from all vendors standardized? How is this
determined?

The article goes on to discuss Microsoft and Apple collaborating
on Java implementations. I note that the Eiffel comments on this
article mentions that Eiffel is available on "Windows NT, Windows 95,
Unix (a dozen different platforms), Linux and VMS...." Is an Apple
implementation in the works?

> 
> There is enough room for controversy in what people actually
> write not to create artificial disputes about what someone
> else believe they think. This is just a waste of everyone's
> time.

I apologize if my overly-simplified summary misled anyone.

> --
> Bertrand Meyer, President, ISE Inc., Santa Barbara (California)
> 805-685-1006, fax 805-685-6869, <Bertrand.Meyer@eiffel.com>
> Web: http://www.eiffel.com, with instructions for free download
> == ISE Eiffel 4: Eiffel straight from those who invented it ==

Re: Critique of Ariane 5 paper (finally!)

[Marinos J. Yannikos]

In article <33EB754E.446B9B3D@eiffel.com>, Bertrand Meyer wrote:
>[...]
>There is no such implication, which would be foolish.
>Microsoft has said they would not support the Java Foundation
>Class libraries. This does not mean they are not making
>Java itself available; but it does break the myth of total and
>automatic Java portability.

Since the JFC are written in 100% Java, Microsoft couldn't prevent them
from working unless they didn't offer "Java" (but something resembling it
only, instead). However, to my knowledge, the dispute between Microsoft
and Sun is focused on Microsoft's announcement to make sure that Java
"works best or only" on their operating systems, thus intentionally
breaking its portability. The article on news.com about this was 
strange, since the actual wording went similar to this: "we, Microsoft
will make sure blah blah... but the death of Java's portability is
inevitable anyway, i.e. they seemed to justify their plans with the
inevitability of their effects (even though noone else had announced 
such intentions at that time).

In any case, "total and automatic portability" is a bit vague, since it
doesn't clarify whether you mean the design or the implementation. The
design itself is mostly (sadly, not entirely, as some of the details
in the Core API are implementation-dependent as far as I can tell)
portable. I don't think that intentionally non-portable implementations
can change that. (besides, in Microsoft's case, there's always the
JDK and JWS for Windows from Sun to run JFC programs on)

-nino
-- 
Please change the last part of my address to "at" if you're replying by mail.

Re: Critique of Ariane 5 paper (finally!)

[Don Harrison]

Ken Garlington wrote:

:This "attack" was predicated (in part) on the following
:points, which I challenged:
:
:1. That there was not enough evidence that Java could
:scale to large projects. I questioned whether this
:was really a fundamental obstacle, since there are
:also concerns about the scalability of Eiffel. 

It's also said that Goody Proctor is a witch. :)  However, even though the 
inhabitants of Salem readily believed it, saying so didn't make her one. 
(Apologies to Arthur Miller, author of "The Crucible". BTW, the recent film 
by the same name is well worth watching for any student of mass hysteria.)

I'm still waiting to see a valid argument why Eiffel software is not scalable. 
I don't have the time or desire to argue why it *is* scalable (and there are 
others better qualified to do so), but two contributing factors stand out:

1) The existence of configuration "modules" which support framework (cluster)
   hierarchies - this makes configuration scalable.

   See the section titled "Scalability" in Walden and Nerson - "Seamless 
   Object-oriented Software Architecture" P. 18-19.

2) The ability to encapsulate high-level abstractions in high-level classes 
   using standard design patterns - this makes abstraction and contracting 
   scalable.


Don.
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Don Harrison             donh@syd.csa.com.au

Re: Critique of Ariane 5 paper (finally!)

[Bertrand Meyer]

I am deeply appreciative to  Ken Garlington for his
willingness to moderate his earlier restatements
of my positions. This greatly facilitates the discussion
and shows openness of mind. Thanks.

On the crux of the issue, I have looked at his messages
and really don't have much to add to what has been written
before, especially the Jezequel-Meyer paper on Ariane
accessible from http://www.eiffel.com, the ISE
"news items of the week" at the same address, and
of course "Object-Oriented Software Construction,
2nd edition" where the Design by Contract ideas
are developed in detail.

As convincingly as I could, my colleagues and I
have explained: why in our view
software technology crucially requires the systematic use of
Design by Contract; why Design by Contract is a
necessary condition to avoid more Ariane-like failures;
and what is missing in this respect in such approaches
as Java, Ada, C++, IDL.

Ken Garlington, and others such as Robert Dewar, disagree
at least in part. I find it hard to understand their position,
but there is now enough published arguments on both sides to
enable every reader to make up his mind.


-- 
Bertrand Meyer, President, ISE Inc., Santa Barbara (California)
805-685-1006, fax 805-685-6869, <Bertrand.Meyer@eiffel.com>
Web: http://www.eiffel.com, with instructions for free download
== ISE Eiffel 4: Eiffel straight from those who invented it ==

Re: Critique of Ariane 5 paper (finally!)

[Robert Dewar]

Bertrand says

<<As convincingly as I could, my colleagues and I
have explained: why in our view
software technology crucially requires the systematic use of
Design by Contract; why Design by Contract is a
necessary condition to avoid more Ariane-like failures;
and what is missing in this respect in such approaches
as Java, Ada, C++, IDL.>>

Your argument at *best* says that DBC might have been a *sufficient*
condition for avoiding the Ariane failure. Even there, it seems over-facile
and rather academic, and does not seem to understand fully the exact nature
of the Ariane problem.

But to make the jump from sufficient to necessary is completely without basis,
and can only be regarded as advertising puffery. To prove this you would have
to show that no other method can succeed, an impossible burden.

You have to be very careful in not making excessive claims, especially
when it is clear that you are not disinterested in the claim!

Re: Critique of Ariane 5 paper (finally!)

[Bertrand Meyer]

Professor Robert Dewar wrote that my argument

> seems over-facile and rather academic 

Wow! But I'll choose to take "academic", coming from
a .edu address, as a compliment. And "over-facile"
might refer to how easy the basic ideas of Design
by Contract are to teach and implement.

He adds that what he calls my "jump from sufficient to
necessary" 
 
> can only be regarded as advertising puffery.

Advertising puffery! Delightful choice of words.
Now this is a solid technical rebuttal. But here is the
explanation:

> [I]t is clear that you [i.e. me, BM] are not disinterested
> in the claim!

OK, finally things are clear. When everything else fails, resort
to questioning the other person's integrity: I have a vested
interest in seeing Eiffel succeed -- that explains it all.
(Doesn't explain, though, why so many other contributors have
expressed agreement with the analysis of the Jezequel-Meyer
Ariane paper and support for Design by Contract.)
 
This is a slippery path to follow since very few people
can claim to be "disinterested". So are Robert Dewar and
I  going to spend the next three months arguing who is the most
unbiased? No, no and no. I won't be dragged down to that level
and will just assume that the line quoted above was a
regrettable slip of the pen. 

The "necessary" condition is simple: you won't get reliable
software unless you take care to associate specifications --
contracts -- with software elements, especially reusable
components. You can then use these contracts to build
the software, to document it, to validate it statically,
to test it (if you have support for run-time monitoring,
as in Eiffel), to control the proper use of inheritance,
to discuss the software within the software team as well
as with management, customers, users and regulatory agencies.
Anyone who has used the approach knows that it dramatically
improves the quality of the product and the process.
And it's not even an expensive technique.

No one ever said Design by Contract was "sufficient".
My "Object-Oriented Software Construction, 2nd Edition"
(Prentice Hall) has a 9-page section (11.14, pp. 398-406)
discussing its limitations. But to encourage developers of
mission-critical software to ignore this powerful tool is
irresponsible. How many more Ariane-like failures do we
need?


-- 
Bertrand Meyer, President, ISE Inc., Santa Barbara (California)
805-685-1006, fax 805-685-6869, <Bertrand.Meyer@eiffel.com>
Web: http://www.eiffel.com, with instructions for free download
== ISE Eiffel 4: Eiffel straight from those who invented it ==

Re: Critique of Ariane 5 paper (finally!)

[Bertrand Meyer]

Samuel Mize wrote:
 
> > But to make the jump from sufficient to necessary is completely
> > without basis, and can only be regarded as advertising puffery.
> 
> Which appeared in a column in IEEE Computer magazine, positioning
> it deceptively as a technical item instead of an ad.

This is outrageous. The article in question suggested a technical
principle (Design by Contract) to avoid new occurrences
of a documented major disaster. It did not mention any
software product. IEEE Computer and other magazines
publish thousand of pages suggesting technical solutions
to problems -- often, in fact, advocating products
and companies. Is any argument for CORBA, or OLE, or UML, or
Unix, or Windows to be dismissed as as "deceptive",
an "ad" and not a "technical item"?

For example the June issue of IEEE Computer included a 6-page
eulogy of Java by its creator, James Gosling. Is this a
deceptive ad too? After all, unlike the Ariane paper, it does
talk throughout about its author's company (Sun) and products
(Solaris etc.) But if we barred technical advocacy, IEEE
Computer and other magazines would soon become rather boring
places. Fortunately I don't think that's going to happen.

The underlying issue is what we do to improve software
reliability, and that's not going to be resolved by 
gratuitous attacks.
 
-- 
Bertrand Meyer, President, ISE Inc.
ISE Building, 2nd floor, 270 Storke Road, Goleta CA 93117
805-685-1006, fax 805-685-6869, <Bertrand.Meyer@eiffel.com>
http://www.eiffel.com, with instructions for free download

Re: Critique of Ariane 5 paper (finally!)

[Ken Garlington]

Bertrand Meyer wrote:
> 
> (Doesn't explain, though, why so many other contributors have
> expressed agreement with the analysis of the Jezequel-Meyer
> Ariane paper and support for Design by Contract.)

How many of them build safety-critical real-time systems? None
who support DBC have indicated experience in this field. Thus,
my explanation is simple: they have insufficient experience
to judge the likely pitfalls in the application of DBC to a domain
they (and, I assume, you) don't understand.

Your case seems to be somewhat less strong among the actual
practitioners in the field. Coupled with the apparent lack
of Eiffel applications in the domain under discussion, your
appeals to authority (random Eiffel users on Internet?) aren't
all that compelling.

> This is a slippery path to follow since very few people
> can claim to be "disinterested". So are Robert Dewar and
> I  going to spend the next three months arguing who is the most
> unbiased? No, no and no. I won't be dragged down to that level
> and will just assume that the line quoted above was a
> regrettable slip of the pen.

On the other hand, _I_ have absolutely no vested interest in
either promoting Ada or attacking Eiffel; nor do I have any
interest in defending the Ariane 5 team. I do have an interest
in using new technologies in the domain in which you have
taken a sudden interest, and would like to understand how
it could help. However, my concerns have been uniformly
unaddressed through any sort of technical argument.

I notice that both you and Mr. Jezequel have a strong desire
to uncover "vested interests." When I first joined this
discussion, he assured me that my job as a software test
engineer would not be eliminated by a switch to DBC. Since
I'm not a software test specialist, this was a strange
statement to make, to be sure. :)

So, other than some perceived bias of practitioners against
"the next big technology." as I believe you indicated in
a previous post, what is the bias that practitioners such
as myself, Mr. Kohl, and Mr. White have against DBC (other
than the obvious one -- we won't use a technology whose
authors are so emotionally involved in their quest that
they are insulted by the mere suggestion of contrary views)?

> The "necessary" condition is simple: you won't get reliable
> software unless you take care to associate specifications --
> contracts -- with software elements, especially reusable
> components. You can then use these contracts to build
> the software, to document it, to validate it statically,
> to test it (if you have support for run-time monitoring,
> as in Eiffel), to control the proper use of inheritance,
> to discuss the software within the software team as well
> as with management, customers, users and regulatory agencies.
> Anyone who has used the approach knows that it dramatically
> improves the quality of the product and the process.
> And it's not even an expensive technique.
> 
> No one ever said Design by Contract was "sufficient".

Other than youself, in your Ariane 5 paper.

Perhaps it's a problem of the English language; but "probably yes"
certainly translates as "sufficient" in my understanding.

> My "Object-Oriented Software Construction, 2nd Edition"
> (Prentice Hall) has a 9-page section (11.14, pp. 398-406)
> discussing its limitations. But to encourage developers of
> mission-critical software to ignore this powerful tool is
> irresponsible. 

This is really the true irony. Several practitioners have
invested a great deal of Internet time _not_ ignoring this
technology. We have tried to understand why it would help in
our domain; have tried to draw the Eiffel experts into a
technical discussion of its strengths and weaknesses, and
have generally been told

  -- we don't understand our own products
  -- we don't understand software engineering
  -- we are biases against new ideas

I wonder why there aren't more Eiffel projects in this
domain? :)

Mr. Jezequel and Mr. Meyer have both indicated they
have no interest in pursuing any discussions on this
subject. Isn't this the same as irresponsibly discouraging
the use of DBC?

>How many more Ariane-like failures do we
> need?

No more. This is why we need a balanced budget immediately!
(Why not? If I don't have to show that a balanced budget
would have prevented the problem, it's an equally important
claim!)

> 
> --
> Bertrand Meyer, President, ISE Inc., Santa Barbara (California)
> 805-685-1006, fax 805-685-6869, <Bertrand.Meyer@eiffel.com>
> Web: http://www.eiffel.com, with instructions for free download
> == ISE Eiffel 4: Eiffel straight from those who invented it ==

Re: Critique of Ariane 5 paper (finally!)

[Ken Garlington]

Samuel Mize wrote:
> 
> Robert Dewar wrote:
> >
> > Bertrand says
> >
> > <<As convincingly as I could, my colleagues and I
> > have explained: why in our view
> > software technology crucially requires the systematic use of
> > Design by Contract; why Design by Contract is a
> > necessary condition to avoid more Ariane-like failures;
> > and what is missing in this respect in such approaches
> > as Java, Ada, C++, IDL.>>
> >
> > Your argument at *best* says that DBC might have been a *sufficient*
> > condition for avoiding the Ariane failure. Even there, it seems
> > over-facile and rather academic, and does not seem to understand
> > fully the exact nature of the Ariane problem.
> 
> DBC proponents have said that using DBC IMPLIES review of
> requirements, review of design, and testing the component.
> In this case, the claim that DBC would "probably" have prevented
> the crash is nugatory but true.

Note that Mr. Jezequel, one of the authors of the DBC Ariane paper,
argued for some time that the tests described in the inquiry's
report were infeasible, and that DBC would substitute for them.
Therefore, it does not seem valid to assume that DBC equates to
full testing. 

See also my arguments in section 3.2 (and subsections) of
  http://www.flash.net/~kennieg/ariane.html

> However, the claim that "widely accepted industry practices" would
> not have done so is false.  Requirements review, design review, and
> in-situ test are standard for a mission-critical component.  Not
> ONE of these was done for the Ariane 5 INS.  To claim that this is
> "widely accepted industry practice" is disingenuous at best, and
> appears intentionally misleading to a lot of us.

I'm not sure that a requirements and/or design review was left out
of the process. I think it is more clear to say that insufficient
review by _external parties_ was performed; parties with sufficient
independence and knowledge to identify the invalid assumption.

> It is this false claim that destroys the argument that DBC is
> "a necessary condition to avoid more Ariane-like failures."
> 
> It's rather like saying that a new method of navigation would
> "probably" have prevented the Exxon Valdez crash.  True, but
> only because ANY navigation would have prevented it!
> 
> > But to make the jump from sufficient to necessary is completely
> > without basis, and can only be regarded as advertising puffery.
> 
> Which appeared in a column in IEEE Computer magazine, positioning
> it deceptively as a technical item instead of an ad.

I suppose for the sake of fairness, that it should be pointed out that
Mr. Meyer may not consciously be attempting promotion of his products
through this means. How would we know? I do get the feeling that
much of his defense of his paper is based on emotion rather than
logic. He has yet to engage in a discussion of the issues; most of
his responses have either been in terms of his "wounded pride," or
oblique personal attacks ("if only practitioners would listen to the
voice of reason"), or mere filler ("we'll just have to agree to
disagree," as in this latest post). Mr. Jezequel was at least willing
to discuss some issues (e.g. the potential for testing), and did
soften his stance in some areas as a result (as did I). However,
he soon gave up the discussion, which was unfortunate.

I also agree that it's unfortunate that IEEE Computer doesn't publish
more critical letters. In particular, it's difficult to explain a
contrary postion adequately in the space provided. It also is much
less likely to be read.

> 
> Samuel Mize

Re: Critique of Ariane 5 paper (finally!)

[Ken Garlington]

Bertrand Meyer wrote:
> 
> Samuel Mize wrote:
> 
> > > But to make the jump from sufficient to necessary is completely
> > > without basis, and can only be regarded as advertising puffery.
> >
> > Which appeared in a column in IEEE Computer magazine, positioning
> > it deceptively as a technical item instead of an ad.
> 
> This is outrageous. The article in question suggested a technical
> principle (Design by Contract) to avoid new occurrences
> of a documented major disaster. It did not mention any
> software product.

Except Eiffel, of course. It went on to contrast Eiffel with other
competing product lines. The fact that you were identified as
"Bertrand Meyer, EiffelSoft" didn't help either. (Not your fault,
of course, but it adds to the perception).

Contrast this, for example, to your outstanding article on object
taxonomies. Although you used Eiffel for your examples, nowhere
did you say, "and Eiffel is much better than Java, Ada, etc. at
expressing these taxonomies."

You have to agree that someone who makes money selling books and
products for both DBC and Eiffel should be both conservative and
balanced in their claims, to avoid the _appearance_ (and I stress
appearance, because I'm not suggesting you intentionally meant to
advertise your products) of a conflict of interest.

> IEEE Computer and other magazines
> publish thousand of pages suggesting technical solutions
> to problems -- often, in fact, advocating products
> and companies. Is any argument for CORBA, or OLE, or UML, or
> Unix, or Windows to be dismissed as as "deceptive",
> an "ad" and not a "technical item"?
> 
> For example the June issue of IEEE Computer included a 6-page
> eulogy of Java by its creator, James Gosling. Is this a
> deceptive ad too? After all, unlike the Ariane paper, it does
> talk throughout about its author's company (Sun) and products
> (Solaris etc.) 

Given that you have described his argument as a "eulogy," then
I guess the answer is yes, you can dismiss (or at least question)
an argument where there is a perception of strong bias
(particularly financial gain). I know I question some of the
claims made by Java, as do you (see, for example):

            NEWS ITEM OF THE WEEK: 
   MICROSOFT DECLARES WAR ON JAVA PORTABILITY

at http://www.eiffel.com, "NEW" index item.

> But if we barred technical advocacy, IEEE
> Computer and other magazines would soon become rather boring
> places. Fortunately I don't think that's going to happen.
> 
> The underlying issue is what we do to improve software
> reliability, and that's not going to be resolved by
> gratuitous attacks.

Well said!

Having put the gratuitous attacks aside, I would appreciate
your review and comments on my paper at:

  http://www.flash.net/~kennieg/ariane.html

which, to the best of my ability, contains no gratuitous
attacks of any kind.

> 
> --
> Bertrand Meyer, President, ISE Inc.
> ISE Building, 2nd floor, 270 Storke Road, Goleta CA 93117
> 805-685-1006, fax 805-685-6869, <Bertrand.Meyer@eiffel.com>
> http://www.eiffel.com, with instructions for free download

Re: Critique of Ariane 5 paper (finally!)

[Bertrand Meyer]

Ken Garlington wrote:

> Bertrand Meyer wrote:

>>>> The [Jezequel-Meyer IEEE Computer] article did not mention
>>>> any software product.

> Except Eiffel, of course. It went on to contrast Eiffel with other
> competing product lines.

Absolutely wrong.  Eiffel is not a product.
It is a non-proprietary method and language for the
construction of quality software.

The article did not mention any products. It was about
principles of software design and how they can help avoid
Ariane-like catastrophes.
 
-- BM

Re: Critique of Ariane 5 paper (finally!)

[Samuel Mize]

Robert Dewar wrote:
> 
> Bertrand says
> 
> <<As convincingly as I could, my colleagues and I
> have explained: why in our view
> software technology crucially requires the systematic use of
> Design by Contract; why Design by Contract is a
> necessary condition to avoid more Ariane-like failures;
> and what is missing in this respect in such approaches
> as Java, Ada, C++, IDL.>>
> 
> Your argument at *best* says that DBC might have been a *sufficient*
> condition for avoiding the Ariane failure. Even there, it seems
> over-facile and rather academic, and does not seem to understand
> fully the exact nature of the Ariane problem.

DBC proponents have said that using DBC IMPLIES review of
requirements, review of design, and testing the component.
In this case, the claim that DBC would "probably" have prevented
the crash is nugatory but true.

However, the claim that "widely accepted industry practices" would
not have done so is false.  Requirements review, design review, and
in-situ test are standard for a mission-critical component.  Not
ONE of these was done for the Ariane 5 INS.  To claim that this is
"widely accepted industry practice" is disingenuous at best, and
appears intentionally misleading to a lot of us.

It is this false claim that destroys the argument that DBC is
"a necessary condition to avoid more Ariane-like failures."

It's rather like saying that a new method of navigation would
"probably" have prevented the Exxon Valdez crash.  True, but
only because ANY navigation would have prevented it!


> But to make the jump from sufficient to necessary is completely
> without basis, and can only be regarded as advertising puffery.

Which appeared in a column in IEEE Computer magazine, positioning
it deceptively as a technical item instead of an ad.

Samuel Mize

Re: Critique of Ariane 5 paper (finally!)

[Samuel Mize]

Bertrand Meyer wrote:
> 
> Samuel Mize wrote:
> 
> > > But to make the jump from sufficient to necessary is completely
> > > without basis, and can only be regarded as advertising puffery.
> >
> > Which appeared in a column in IEEE Computer magazine, positioning
> > it deceptively as a technical item instead of an ad.

As stated this was too strong, and I apologize.

> This is outrageous. ...
> Is any argument for CORBA, or OLE, or UML, or
> Unix, or Windows to be dismissed as as "deceptive",
> an "ad" and not a "technical item"?

In the part of my message you deleted, I stated my opinion that
your paper's conclusion is based on a severely and plainly false
premise.  It was on the basis of that that I dismiss it as being
an "ad" and not a "technical item."

I should have said that its appearance in IEEE Computer gives it
a cachet which may mislead people about its quality and objectivity.

I did not intend to suggest an intent to deceive on your part.
This is why I said the placement, not the author, was deceptive.
Again, I apologize.


> It seems that the idea of using modern techniques to
> improve software reliability shocks some people so much
> that they will find no argument too low in their effort
> to suppress it. Won't work.

Fortunately true.

Samuel Mize

Re: Critique of Ariane 5 paper (finally!)

[Jon S Anthony]

In article <33F22AD8.41C67EA6@eiffel.com> Bertrand Meyer <Bertrand.Meyer@eiffel.com> writes:

> > Which appeared in a column in IEEE Computer magazine, positioning
> > it deceptively as a technical item instead of an ad.
> 
> This is outrageous. The article in question suggested a technical

Perhaps, but since this is a pretty wide spread impression, it may be
worth considering which side of the "desk" it originates.

> solution (Design by Contract) to avoid new occurrences
  ^^^^^^^^

Interesting that you changed this in your own followup to your own
post here.


> For example the June issue of IEEE Computer included a 6-page eulogy
> of Java by its creator, James Gosling. Is this a deceptive ad too?

Possibly.  Wouldn't surprise me in the least.

> technical advocacy, IEEE Computer and other magazines will soon
> become rather boring places. Fortunately I don't think that's going
> to happen.

Possibly - or more reasonable and cogent.

> It seems that the idea of using modern techniques to
> improve software reliability shocks some people so much

There is nothing new about assertions or the notion behind DBC.  The
basic ideas have been floated in various forms for a looonnggg time
now.

/Jon
-- 
Jon Anthony
OMI, Belmont, MA 02178, 617.484.3383 
"Nightmares - Ha!  The way my life's been going lately,
 Who'd notice?"  -- Londo Mollari

Re: Critique of Ariane 5 paper (finally!)

[geldridg]

In article <JSA.97Aug14131421@alexandria.organon.com>,
jsa@alexandria.organon.com says...
>
>In article <33F22AD8.41C67EA6@eiffel.com> Bertrand Meyer
>><Bertrand.Meyer@eiffel.com> writes:

[...]

>> It seems that the idea of using modern techniques to
>> improve software reliability shocks some people so much
>
>There is nothing new about assertions or the notion behind DBC.  The
>basic ideas have been floated in various forms for a looonnggg time
>now.

.. and Bertrand Meyer will be the first to acknownledge such.

I think the following beta-release `Eiffel Liberty Journal' (elj)
article by Bertrand Meyer, titled:

 `Eiffel's Design by Contract: Predecessors and Original Contributions'

at:

 http://www.progsoc.uts.edu.au/~geldridg/eiffel/liberty/v1/n1/bm/bmdbc.html

might shed some more light on your comments.

Hope this helps.

Geoff Eldridge

-- geldridg@progsoc.uts.edu.au

Re: Critique of Ariane 5 paper (finally!)

[Bertrand Meyer]

Jon S Anthony wrote:
 
> There is nothing new about assertions or the notion behind
> [Design by Contract].  The basic ideas have been floated in
> various forms for a looonnggg time now.

If this is what the discussion finally amounts to, it is hard
to resist self-quoting from the second paragraph of the
preface to "Object-Oriented Software Construction"
(http://www.eiffel.com/doc/manuals/technology/oosc/preface):

 	just as inevitable is the well-known three-step sequence
	of reactions that meets the introduction of a new
	methodological principle: (1) "it's trivial"; (2) "it
	cannot work"; (3) "that's how I did it all along anyway".
	(The order may vary.)

I guess we have reached the third step now.

More seriously, it is absolutely correct that Design by
Contract is based on ideas that have been around for a long
time. It is also true that it includes original ideas too,
including some which in my experience shock most people
who encounter them for the first time, such as the
idea that to obtain more reliable software you should
usually *remove* checks. See also the connection with
inheritance (weakening/strengthening), the connection
with exceptions etc.

The question originality came up not long ago on comp.lang.eiffel;
Geoff Eldridge has collected at

http://www.progsoc.uts.edu.au/~geldridg/eiffel/liberty/v1/n1/bm/bmdbc.html

two messages posted in response, one by James McKim and
one by me, which tried to ascertain what is new and what
comes from earlier work. Also, the final sections
of the chapters on Design by Contract and exceptions in
"Object-Oriented Software Construction" have detailed
bibliographical discussions of earlier publications on related
topics.	

-- 
Bertrand Meyer, President, ISE Inc.
ISE Building, 2nd floor, 270 Storke Road, Goleta CA 93117
805-685-1006, fax 805-685-6869, <Bertrand.Meyer@eiffel.com>
http://www.eiffel.com, with instructions for download
== ISE Eiffel 4: Eiffel from those who invented it ==

Re: Critique of Ariane 5 paper (finally!)

[Robert S. White]

In article <33F22AD8.41C67EA6@eiffel.com>, Bertrand.Meyer@eiffel.com says...
>
>It seems that the idea of using modern techniques to
>improve software reliability shocks some people so much
>that they will find no argument too low in their effort
>to suppress it. Won't work.
> 
  This seems to me to be stooping a bit low.  I am more than willing
to use new "validated" methods of improving/doing software engineering.
That is why the company I work for decided it was very worthwhile to
implement the CMM (now independently certified to level 3) and 
ISO-900x (again independently certified).  I have been able to take
company funded courses in OOA, OOD and software inspections taught
by outside contractors brought it just for the course.  I am using 
the UML to document the high level software design of the hard
real time project that I have been working on for the last few
months.  Convince me of the techinical merits of this new engineering
approach/tool/method for my problem domain and I will most willingly
use it.  All(most?) of us engineers are inherently lazy and _want_ 
to use the most error free, efficient, high quality design 
techniques that gain us satisfaction of a job well done.  A job 
poorly done will come back to haunt you unless you change jobs often.
Some of us don't.  We have been taking a very thorough look at JAVA 
technologies and are already committed to making the JVM and JAVA byte
code work well for our applications.

  Argue your position on its technical merits and please do not make
broad generalizations about the motives of authors of critiques of
your statements.
_____________________________________________________________________
Robert S. White         -- An embedded systems software engineer

Re: Critique of Ariane 5 paper (finally!)

[Matthew Heaney]

In article <JSA.97Aug14131421@alexandria.organon.com>,
jsa@alexandria.organon.com (Jon S Anthony) wrote:

>> It seems that the idea of using modern techniques to
>> improve software reliability shocks some people so much
>
>There is nothing new about assertions or the notion behind DBC.  The
>basic ideas have been floated in various forms for a looonnggg time
>now.

Perhaps that's true, but I myself didn't learn this until I read Bertrand's
book back in '89.   You can trace axiomatic specification back to the Floyd
article, the Hoare article (and to the "Little Black Book"), and the later
papers about ADTs by Guttag, Liskov, and Reynolds, but I don't think it's
inappropriate to give Bertrand credit for popularizing the idea of a
contract between client and server.

Had he not brought the contract model to the masses (it rode the wave of
popularity of the object movement), I'm not sure I or anyone else would be
arguing about it now.  I read his book with the intention of learning about
object-oriented programming, but came away with much more than that.  To
me, the concept of design by contract is at least as important as object
technology.  (Indeed, as our discipline matures, we see that there are
myriad "architectural styles," object-oriented being only one of them.  But
the contract idea will always be applicable, no matter how you build your
house.)

Bertrand and I have plenty to disagree about (Ada is AOK with me, thank you
very much), but the efficacy of design by contract is not one of them.  He
would be wise to not claim too much credit, to avoid the mistakes of his
peers (What exactly is the "Booch method"?  Or the "Object Modeling
Technique"?  Are Grady's objects any different from mine or yours?), but
the effect of Bertrand's cogent enunciation of the contract philosophy is
undeniable.

I can say without hyperbole that Object-Oriented Software Construction
changed my whole way of thinking about programming, and I consider that
book to be among the finest software engineering texts ever written.

So sally forth, Bertrand, and thank you!

--------------------------------------------------------------------
Matthew Heaney
Software Development Consultant
<mailto:matthew_heaney@acm.org>
(818) 985-1271

Re: Critique of Ariane 5 paper (finally!)

[Ken Garlington]

Bertrand Meyer wrote:
> 
> Ken Garlington wrote:
> 
> > Bertrand Meyer wrote:
> 
> >>>> The [Jezequel-Meyer IEEE Computer] article did not mention
> >>>> any software product.
> 
> > Except Eiffel, of course. It went on to contrast Eiffel with other
> > competing product lines.
> 
> Absolutely wrong.  Eiffel is not a product.
> It is a non-proprietary method and language for the
> construction of quality software.

We'll just have to agree to disagree on whether Eiffel is
_also_ a product line.

"Personal Eiffel for Windows: $69.95 (electronic version)."
          ------
from the page titled:

"Prices for ISE Eiffel products and support"
                ---------------------------
on the server

http://www.eiffel.com.
           ----------

> The article did not mention any products. It was about
> principles of software design and how they can help avoid
> Ariane-like catastrophes.

I believe you. However, as I tried to explain in very
precise terms, many people were left with the _impression_
that it was also a sales brochure. With a slightly different
approach, you could have still made your point wihtout making
such an impression.

I suppose you could say that impressions are irrelevant.
However, this is certainly an odd position to take for
someone advocating a method and language, not to mention
someone in the business world.

Overall, I'd have to say your responses to legitimate questions
are so emotional and acidic as to make it difficult for
some people to take you seriously. Given that these are the
same practitioners you're trying to convince...

> 
> -- BM

Re: Critique of Ariane 5 paper (finally!)

[Bertrand Meyer]

Samuel Mize wrote as part of his response to Thomas Beale:
 
> The paper claims DBC/Eiffel is THE ONE method that could have
> found the Ariane 5 flaw.

The paper in question (Jean-Marc Jezequel and Bertrand Meyer,
"Design by Contract: The Lessons of Ariane", IEEE Computer,
January 1997, pages 129-130, slightly revised version at
http://www.eiffel.com) says no such thing, which would be silly.
Once you have discovered a bug, you can usually think ex post
facto of many "methods" that "could" have found it --
some technical (such as Design by Contract), some 
organizational (more design/code reviews, more testing
etc.).

What the paper says (going into the details, and drawing
the lessons) is that reuse without a contract is folly,
and that such improper reuse was the direct cause of the
Ariane failure.

I don't see the point of criticizing people for something
else than what they wrote, especially when the text is
available for everyone to read. 
 
-- 
Bertrand Meyer, President, ISE Inc.
ISE Building, 2nd floor, 270 Storke Road, Goleta CA 93117
805-685-1006, fax 805-685-6869, <Bertrand.Meyer@eiffel.com>
http://www.eiffel.com, with downloading instructions
== ISE Eiffel 4: Eiffel from those who invented it ==

Re: Critique of Ariane 5 paper (finally!)

[Jon S Anthony]

In article <33F440F5.5A6@flash.net> Ken Garlington <kennieg@flash.net> writes:

[...speaking about some more BM "statements"...]

> Overall, I'd have to say your responses to legitimate questions are
> so emotional and acidic as to make it difficult for some people to
> take you seriously. Given that these are the same practitioners
> you're trying to convince...

IMO, Meyer should print this out, blow it up, and glue it to his
computer screen.

/Jon

-- 
Jon Anthony
OMI, Belmont, MA 02178, 617.484.3383 
"Nightmares - Ha!  The way my life's been going lately,
 Who'd notice?"  -- Londo Mollari

Re: Critique of Ariane 5 paper (finally!)

[Jon S Anthony]

In article <33F48C0B.41C67EA6@eiffel.com> Bertrand Meyer <Bertrand.Meyer@eiffel.com> writes:

> What the paper says (going into the details, and drawing the
> lessons) is that reuse without a contract is folly, and that such
> improper reuse was the direct cause of the Ariane failure.

That's about as interesting as saying "the sky is blue".  And about as
controversial.  So, given that the "paper" has set off (unfortunately)
a controversy, it would appear that the paper is either a) rather
opaque or b) says something more (or less, depending on you POV) than
this.

/Jon
-- 
Jon Anthony
OMI, Belmont, MA 02178, 617.484.3383 
"Nightmares - Ha!  The way my life's been going lately,
 Who'd notice?"  -- Londo Mollari

Re: Critique of Ariane 5 paper (finally!)

[Ken Garlington]

Robert S. White wrote:
> 
> In article <33F22AD8.41C67EA6@eiffel.com>, Bertrand.Meyer@eiffel.com says...
> >
> >It seems that the idea of using modern techniques to
> >improve software reliability shocks some people so much
> >that they will find no argument too low in their effort
> >to suppress it. Won't work.
> >
>   This seems to me to be stooping a bit low. 

More to the point, it evades the issue.

>   Argue your position on its technical merits and please do not make
> broad generalizations about the motives of authors of critiques of
> your statements.

This appears to be a lost cause. I've never received a response to
any requests of this type (beyond the "you silly engineers don't
understand the Truth if you see it" pap).

Hell of a way to run a business -- if there is such a thing as an
Eiffel business; after the last message from On High, I can't say
for sure :)

> _____________________________________________________________________
> Robert S. White         -- An embedded systems software engineer

Re: Critique of Ariane 5 paper (finally!)

[Thomas Beale]

Samuel Mize wrote:
> 
> Bertrand Meyer wrote:
> > This is outrageous. ...
> > Is any argument for CORBA, or OLE, or UML, or
> > Unix, or Windows to be dismissed as as "deceptive",
> > an "ad" and not a "technical item"?
> 
> In the part of my message you deleted, I stated my opinion that
> your paper's conclusion is based on a severely and plainly false
> premise.  It was on the basis of that that I dismiss it as being
> an "ad" and not a "technical item."

With this point of view you would consign centuries of vigorous
intellectual debate in all domains of knowledge to the dustbin.
I wonder what Plato would have thought, if you, rather than including
your judgements of his theories (no matter how extreme your 
disagreement) in the debate at the time, tried to paint them
as product advertisements?

If you don't like Bertrand Meyer's opinions, fine. We respect your
right to contribute to the debate also. But don't try and pervert
it by calling the bits you don't like "non-debate". For one thing,
it damages any hope you have as being heard as a credible 
contributor yourself.

- thomas beale

Re: Critique of Ariane 5 paper (finally!)

[Samuel Mize]

Thomas Beale wrote:
> With this point of view you would consign centuries of vigorous
> intellectual debate in all domains of knowledge to the dustbin.
> I wonder what Plato would have thought, if you, rather than including
> your judgements of his theories (no matter how extreme your
> disagreement) in the debate at the time, tried to paint them
> as product advertisements?

I'm not talking about his theories (DBC/Eiffel).  I'm talking about
the quality and content of one specific paper.  In that context it
IS appropriate to characterize the paper.

Specifically, I'm talking about its misstatement of OTHER
PEOPLES' practices.

The paper claims DBC/Eiffel is THE ONE method that could have
found the Ariane 5 flaw, and that current methods did not.  But
the appropriate methods WERE NOT USED.  The Ariane 5 crash does
NOT demonstrate the inadequacy of current practice.

The paper raises and kills a straw man.  It thus fails to qualify
as a technical argument.

It seems to me, this should have been apparent to the authors:

* They have apparently read the ESA analysis, which lists several
  accepted and common methods that WOULD have uncovered the flaw.

* To publish a paper critical of current practice in the
  development of mission-critical and life-critical software,
  it would be appropriate to investigate just what the current
  practice IS in that field.  Postings by people in that field
  have made clear that the Ariane 5 INS reuse was NOT an example
  of "widely accepted industry practices" in this field.

Had he claimed that DBC/Eiffel was one of several methods that
could have averted the disaster, I would consider his paper weak
but technically competent.  Instead he asserts: "To attempt to
reuse software without Eiffel-like assertions is to invite failures
of potentially disastrous consequences."  "For reuse to be
effective, Design By Contract is a requirement."

Since, in my opinion, these absolutist claims are supported ONLY
by raising and killing a straw man, and the authors should have
realized this, I cannot consider the paper a valid technical
contribution.  It seems to be either advertising fluffery or
evidence of complete ignorance about technical debate.  It seems
less insulting to consider it the former.

Both you and Mr. Meyer are responding to a perceived attack on
DBC/Eiffel.  I have made no such attack.  I have stated my
opinion that DBC/Eiffel is ONE method that might have caught
the Ariane 5 flaw.  However, the paper claims that ONLY Eiffel
could have caught the flaw, and this is (in my opinion) an
irresponsible denigration of the standard practices of a lot
of my colleagues.

Samuel Mize

Re: Critique of Ariane 5 paper (finally!)

[Jon S Anthony]

In article <33F3C21D.ABD322C@eiffel.com> Bertrand Meyer <Bertrand.Meyer@eiffel.com> writes:

> Jon S Anthony wrote:
>  
> > There is nothing new about assertions or the notion behind
> > [Design by Contract].  The basic ideas have been floated in
> > various forms for a looonnggg time now.
> 
> If this is what the discussion finally amounts to, it is hard

It is simply a statement of fact.  Do you have a problem with that?  I
was NOT making a value judgement - that seems to be your province.


>  	just as inevitable is the well-known three-step sequence
> 	of reactions that meets the introduction of a new
> 	methodological principle: (1) "it's trivial"; (2) "it
> 	cannot work"; (3) "that's how I did it all along anyway".

This, of course, is completely irrelevant.


> I guess we have reached the third step now.

One would have to go through 1 and 2 first - for which you'd have to
have a "what".  But, generally speaking, I've claimed it is more
difficult than you.  Of course, to be fair, the "what" has been
typically ill defined in this "debate" and so it is not clear what any
of 1, 2, or 3 even refer to in this context.  Typical.


> More seriously, it is absolutely correct that Design by
> Contract is based on ideas that have been around for a long

Which is ALL I said.  Where's your evidence that I said anything
else??  It would be nice if you were to start following your own
advice and stop blindly impugning the statements of others.  Or do you
feel that you are above such advice?

/Jon
-- 
Jon Anthony
OMI, Belmont, MA 02178, 617.484.3383 
"Nightmares - Ha!  The way my life's been going lately,
 Who'd notice?"  -- Londo Mollari

Re: Critique of Ariane 5 paper (finally!)

[Robert Dewar]

Bertrand said

<<The "necessary" condition is simple: you won't get reliable
software unless you take care to associate specifications --
contracts -- with software elements, especially reusable
components. You can then use these contracts to build
the software, to document it, to validate it statically,
to test it (if you have support for run-time monitoring,
as in Eiffel), to control the proper use of inheritance,
to discuss the software within the software team as well
as with management, customers, users and regulatory agencies.
Anyone who has used the approach knows that it dramatically
improves the quality of the product and the process.
And it's not even an expensive technique.>>


This is demonstrably false. There are lots of examples of highly reliable
software written by people who don't even know what a specification is,
let alone how to carefully associate them with software elements.

If you want details on this, I can send you hundreds of thousands of
lines of COBOL code. This code is completely inpenetrable in places,
and I consider it pretty horrible, but it is from a completely reliable
system, where reliability is measured in the terms that matter, i.e.
it does what it is supposed to do in a highly reliable manner.

The supportable statement is something like

"Associating specifications with software elements" is an important tool
that will aid in the production of reliable software.

Of course no one will disagree with that, what people might disagree with
is the huge leap to say

"Therefore any system that does not use DBC is not reliable"

But that is what the contrapositive of your first sentence above says.

As I said in my previous message, I don't think exaggerations of this
kind are helpful at all, even in a heated advocacy argument. They tend
to weaken a position, not strengthen it.

Robert

Re: Critique of Ariane 5 paper (finally!)

[Robert Dewar]

Bertrand said

<<OK, finally things are clear. When everything else fails, resort
to questioning the other person's integrity: I have a vested
interest in seeing Eiffel succeed.>>

I think you missed my point. Everyone knows that you have a vested
interest in seeing Eiffel succeed. Everyone knows that I have a vested
interest in seeing Ada succeed. Nothing wrong with either situation,
and it does not damage our integrity that we have these vested interests.

What I was saying is that in this situation you have to be very careful
that claims you make are solidly based, and objectively supportable.
Personally I find people who make puffed claims for Ada a real menace,
and I don't think it helps Ada one bit.

In this case, I agree that DBC would have been sufficient, if employed
in certain ways, to avoid the Ariane problem, but then so would a zillion
other reasonable approaches.

Arguing the benefits of DBC with regard to Ariane is perfectly legitimate.
To say, or even vaguely imply, that it is a necessary part of any approach
that could have solved this problem is so obviously "advertising puffery"
[i.e. advocacy for a technology without any basis in fact] that it should
be dismissed as such.

If you never made a such a claim, then obviously what I say is
inapplicable, and the person quoting this claim was in error (I really
don't remember who it was that introduced this claim into the thread).

I really think that your position is stronger if you are very careful
to avoid over-inflated claims. That's because otherwise people will
dissmiss them on the basis of self-interest. That's not a matter of
questioning your integrity at all. It's a matter of effective tactics.

I try to avoid making *any* positive statements about Ada that I cannot
back up, precisely because I figure that people will just dismiss them
on the grounds that I am trying to get rich (if it were true, which it
is not, then I guess it just shows I don't succeed at everything I try :-)

Anyway, I certainly did not mean to offend here, and I apologize if I did.

Robert

Re: Critique of Ariane 5 paper (finally!)

[Robert Dewar]

In article <33F22AD8.41C67EA6@eiffel.com>, Bertrand.Meyer@eiffel.com says...
>
>It seems that the idea of using modern techniques to
>improve software reliability shocks some people so much
>that they will find no argument too low in their effort
>to suppress it. Won't work.


Obviously this is a red herring, since no one acts that way. But I suspect
what we have here is

(a) I have a wonderful method, look here ...
(b) My goodness, you guys aren't using my method ...
(c) The only explanation I can think of is the above paragraph

(a) and (b) OK, but (c) is hardly a clear conclusion :-)

No one is trying to suppress anything, they are just being skeptical of
the claims you are making. If you do not expect such skepticism, then
you really have eebn too walled up in the ivory tower! But really you
perfectly well know enough of the real world to know that new techniques
are a hard sell, if nothing else because most of them are over-hyped and
either not nearly so useful as advertised, or useless, or even actively
unhelpful. It is no surprise for people to be skeptical!

Re: Critique of Ariane 5 paper (finally!)

[Ken Garlington]

Jon S Anthony wrote:
> 
> In article <33F440F5.5A6@flash.net> Ken Garlington <kennieg@flash.net> writes:
> 
> [...speaking about some more BM "statements"...]
> 
> > Overall, I'd have to say your responses to legitimate questions are
> > so emotional and acidic as to make it difficult for some people to
> > take you seriously. Given that these are the same practitioners
> > you're trying to convince...
> 
> IMO, Meyer should print this out, blow it up, and glue it to his
> computer screen.

Unfortunately, I'm pretty sure my messages are marked in Mr. Meyer's
"kill" file (his mental one, if not his computer one :). However, if
someone _did_ want to have a technical discussion about the merits
of Eiffel as applied to the Ariane 5 case, my starting position is at

  http://www.flash.net/~kennieg/ariane.html

(I know it's a bore to see this repeated, but since it hasn't appeared
on any of the "official" Eiffel web sites to my knowledge, it's pretty
much the only way people can find it...)

------------------------------------------------------------

Subject: Rebuttal to Ariane paper
   Date: Mon, 04 Aug 1997 18:48:49 -0500
   From: Ken Garlington <kennieg@flash.net>
    To:  jezequel@irisa.fr
    CC:  Bertrand.Meyer@eiffel.com

In article <5qitoi$fdv$1@news.irisa.fr> on 1997/07/16, you wrote:

> Just for the record, I'm still waiting for your rebuttal paper
> to put it on my web alongside with the Computer article.

It's now available at:

  http://www.flash.net/~kennieg/ariane.html

Will the "master" version on www.eiffel.com also be changed to point to
this, or just a copy at www.irisa.fr?

Re: Critique of Ariane 5 paper (finally!)

[Ken Garlington]

Bertrand Meyer wrote:
> 
> Samuel Mize wrote as part of his response to Thomas Beale:
> 
> > The paper claims DBC/Eiffel is THE ONE method that could have
> > found the Ariane 5 flaw.
> 
> I don't see the point of criticizing people for something
> else than what they wrote, especially when the text is
> available for everyone to read.

Very true. See section 2 of

  http://www.flash.net/~kennieg/ariane.html

for the specific quote from the Eiffel paper that substantiates
Mr. Mize's position.

(By the way, has anyone seen if this link has been placed on
www.irisa.fr or www.eiffel.com?)

> 
> --
> Bertrand Meyer, President, ISE Inc.
> ISE Building, 2nd floor, 270 Storke Road, Goleta CA 93117
> 805-685-1006, fax 805-685-6869, <Bertrand.Meyer@eiffel.com>
> http://www.eiffel.com, with downloading instructions
> == ISE Eiffel 4: Eiffel from those who invented it ==

Re: Critique of Ariane 5 paper (finally!)

[Bertrand Meyer]

Quoting my message that said among other things:
 
>>> The "necessary" condition is simple: you won't get reliable
>>> software unless you take care to associate specifications --
>>> contracts -- with software elements, especially reusable
>>> components. You can then use these contracts to build
>>> the software, to document it, to validate it statically,
>>> to test it (if you have support for run-time monitoring,
>>> as in Eiffel), to control the proper use of inheritance,
>>> to discuss the software within the software team as well
>>> as with management, customers, users and regulatory agencies.
>>> Anyone who has used the approach knows that it dramatically
>>> improves the quality of the product and the process.
>>> And it's not even an expensive technique.>>>,

Robert Dewar writes:

> This is demonstrably false. There are lots of examples of highly reliable
> software written by people who don't even know what a specification is,
> let alone how to carefully associate them with software elements.
> 
> If you want details on this, I can send you hundreds of thousands of
> lines of COBOL code. This code is completely inpenetrable in places,
> and I consider it pretty horrible, but it is from a completely reliable
> system, where reliability is measured in the terms that matter, i.e.
> it does what it is supposed to do in a highly reliable manner.

This is eloquently said, but incorrect all the same.

The definition of reliability which this implies is that a system
is "highly reliable" if it has been working satisfactorily for,
say, 30 {seconds | minutes | hours | days | weeks | months | years}
-- pick one. This is one possible definition of reliability, which gets
more and more interesting as it moves to the right of the list
of choices; but it is by no means the only "terms that matter".

True, it may be pragmatically sufficent in some cases.
For example if you tell me my car insurance is being computed
by a program that has worked "satisfactorily" for 10 years, I
may be reasonably pacified. After all, even though I know that
processing my case may hit a bug that no one encountered
before, the (damage * likelihood) factor is small enough for
me to accept the risk (although, being a computing scientist,
I might ask for evidence of what exactly you mean by
"satisfactorily" or even, using Prof. Dewar's terms,
"highly reliably").

But for a mission-critical system of the kind that started
this discussion -- one, such as Ariane, whose non-high-reliability
may imply loss of life, or destruction of huge amounts of
property, or some other catastrophic event -- that is not
good enough. "The software has passed the acceptance tests" or
even "the software has worked properly in the first two missions"
is interesting, but not sufficient to allay the concerns
of, e.g., regulatory agencies. I have seen requirement
specifications for mission-critical systems which
include formal conditions such as "at most one failure for each
10^n passengers", where n is a very large number  --
corresponding, say, to 500-year continuous operation.
Leaving aside the question of whether we can honestly make such
promises (since the scientific debate should not prevent us
to try our best as engineers), we are NOT going to reassure,
let alone satisfy our censors through pragmatic reliability
arguments of the form "it has worked well so far, ergo it is
reliable".  They will listen with interest but they will demand
more.
 

Part of what they will demand will be guarantee about our software
process -- something in the line of ISO 900x, Capability Maturity
Model, quality assurance plan and the like. But they will also
want to know what software techniques we use and, in particular,
why we believe that each software component does its
job -- especially a reusable software component pulled out
from somewhere else. This is where there is simply no substitute
for Design by Contract: a necessary condition, as I wrote.
By no means sufficient (it makes sense as part of an array
of reliability mechanisms, some technical, some organizational);
but, until someone comes with a better idea to guarantee
reliability, necessary.

The recently deceased Harlan Mills, then of IBM, published
in 1975 (Proc.Intl. Conf. on Software Reliability,
see reference in OOSC-2) a paper entitled

	"How to write correct programs, and know it."

This title truly capture the essence of the problem. It is
not sufficient, as Prof. Dewar seems to imply, to write
reliable programs (where reliability includes correctness)
which we think are reliable because they have worked well
enough for long enough. To convince others (and ourselves)
that they are indeed reliable, we need better arguments.
These arguments can only come from a precise description
of what the elements are supposed to do -- the contracts --
and some evidence that the implementation does honor the
contracts. Without these mechanisms in place, we cannot
even discuss properly the reliability of our software,
since we haven't even defined what we are talking about.

So when Robert Dewar writes

> "Associating specifications with software elements" is an important tool
> that will aid in the production of reliable software.

> Of course no one will disagree with that, what people might disagree with
> is the huge leap to say

> "Therefore any system that does not use DBC is not reliable"

I believe he is wrong on the second point, at least if we
replace "reliable", in the last line, by "demonstrably
reliable" (using "demonstrably" not in the absolute
mathematical sense but in the practical sense of being able
to convince competent colleagues).

My view here is, apparently at least, the exact reverse of
his. There is no claim whatsoever that Design by Contract
is "sufficient".  I would be happy to know a magical recipe
to guarantee software reliability, but I don't. I do know,
however, techniques that tremendously improve reliability
(techniques that have saved me and many other people
countless bugs, including some potentially spectacular
ones), and without which it is not possible, in the
state of software technology as I understand it, to achieve
similar results. In short, necessary conditions.

I understand that not everyone is convinced, and that some may
think the claims are too grandiose even though they are really
very modest and down-to-earth; but that does not diminish the
relevance of these techniques to software reliability, the
importance for every software engineer of learning them,
and, yes, their necessity.
 
-- 
Bertrand Meyer, President, ISE Inc.
ISE Building, 2nd floor, 270 Storke Road, Goleta CA 93117
805-685-1006, fax 805-685-6869, <Bertrand.Meyer@eiffel.com>
http://www.eiffel.com, with instructions for free download
== ISE Eiffel 4: Eiffel from those who invented it ==

Re: Critique of Ariane 5 paper (finally!)

[Ken Garlington]

Bertrand Meyer wrote:
> 
> Robert Dewar writes:
> 
> > This is demonstrably false. There are lots of examples of highly reliable
> > software written by people who don't even know what a specification is,
> > let alone how to carefully associate them with software elements.
> >
> > If you want details on this, I can send you hundreds of thousands of
> > lines of COBOL code. This code is completely inpenetrable in places,
> > and I consider it pretty horrible, but it is from a completely reliable
> > system, where reliability is measured in the terms that matter, i.e.
> > it does what it is supposed to do in a highly reliable manner.
> 
> This is eloquently said, but incorrect all the same.
> 
> The definition of reliability which this implies is that a system
> is "highly reliable" if it has been working satisfactorily for,
> say, 30 {seconds | minutes | hours | days | weeks | months | years}
> -- pick one. This is one possible definition of reliability, which gets
> more and more interesting as it moves to the right of the list
> of choices; but it is by no means the only "terms that matter".
> 
> True, it may be pragmatically sufficent in some cases.
> For example if you tell me my car insurance is being computed
> by a program that has worked "satisfactorily" for 10 years, I
> may be reasonably pacified. After all, even though I know that
> processing my case may hit a bug that no one encountered
> before, the (damage * likelihood) factor is small enough for
> me to accept the risk (although, being a computing scientist,
> I might ask for evidence of what exactly you mean by
> "satisfactorily" or even, using Prof. Dewar's terms,
> "highly reliably").
> 
> But for a mission-critical system of the kind that started
> this discussion -- one, such as Ariane, whose non-high-reliability
> may imply loss of life, or destruction of huge amounts of
> property, or some other catastrophic event -- that is not
> good enough. "The software has passed the acceptance tests" or
> even "the software has worked properly in the first two missions"
> is interesting, but not sufficient to allay the concerns
> of, e.g., regulatory agencies. I have seen requirement
> specifications for mission-critical systems which
> include formal conditions such as "at most one failure for each
> 10^n passengers", where n is a very large number  --
> corresponding, say, to 500-year continuous operation.
> Leaving aside the question of whether we can honestly make such
> promises (since the scientific debate should not prevent us
> to try our best as engineers), we are NOT going to reassure,
> let alone satisfy our censors through pragmatic reliability
> arguments of the form "it has worked well so far, ergo it is
> reliable".  They will listen with interest but they will demand
> more.
> 
> Part of what they will demand will be guarantee about our software
> process -- something in the line of ISO 900x, Capability Maturity
> Model, quality assurance plan and the like. But they will also
> want to know what software techniques we use and, in particular,
> why we believe that each software component does its
> job -- especially a reusable software component pulled out
> from somewhere else. This is where there is simply no substitute
> for Design by Contract: a necessary condition, as I wrote.

I have _safety-critical_ software systems in operation today that
were developed with a process that has been independently evaluated
to both ISO 9001 and SEI III. The products have been validated using
MIL-STD-882B and DoD-STD-2167, and also meet all validation requirements
for RTCA/DO-178B (although no customer has asked for this yet). No
agency responsible for acceptance of my products has ever raised the
issue of Design by Contract in the _specific_ sense used in the Ariane
paper.

I am also familiar with systems accepted under MoD DEF STANs (C-130, in
particular), and they did not require Design by Contract (although
SPARK was used on that system).

Can you identify a regulatory software safety standard that specifically
calls out Design by Contract? If so, please provide the citation; I
would be interested in reviewing it.

> By no means sufficient (it makes sense as part of an array
> of reliability mechanisms, some technical, some organizational);
> but, until someone comes with a better idea to guarantee
> reliability, necessary.

There are quantitative software reliability models; none, to my
knowledge, call out Design by Contract as specifically described
in your Ariane paper.

> The recently deceased Harlan Mills, then of IBM, published
> in 1975 (Proc.Intl. Conf. on Software Reliability,
> see reference in OOSC-2) a paper entitled
> 
>         "How to write correct programs, and know it."
> 
> This title truly capture the essence of the problem. It is
> not sufficient, as Prof. Dewar seems to imply, to write
> reliable programs (where reliability includes correctness)
> which we think are reliable because they have worked well
> enough for long enough. To convince others (and ourselves)
> that they are indeed reliable, we need better arguments.
> These arguments can only come from a precise description
> of what the elements are supposed to do -- the contracts --
> and some evidence that the implementation does honor the
> contracts. Without these mechanisms in place, we cannot
> even discuss properly the reliability of our software,
> since we haven't even defined what we are talking about.

See the work of Levison or Musa to alternative ways to
make these "arguments."

> 
> So when Robert Dewar writes
> 
> > "Associating specifications with software elements" is an important tool
> > that will aid in the production of reliable software.
> 
> > Of course no one will disagree with that, what people might disagree with
> > is the huge leap to say
> 
> > "Therefore any system that does not use DBC is not reliable"
> 
> I believe he is wrong on the second point, at least if we
> replace "reliable", in the last line, by "demonstrably
> reliable" (using "demonstrably" not in the absolute
> mathematical sense but in the practical sense of being able
> to convince competent colleagues).

I can provide specific factual evidence to the contrary.
Competent colleagues can be convinced through alternative means.

> 
> My view here is, apparently at least, the exact reverse of
> his. There is no claim whatsoever that Design by Contract
> is "sufficient".  I would be happy to know a magical recipe
> to guarantee software reliability, but I don't. I do know,
> however, techniques that tremendously improve reliability
> (techniques that have saved me and many other people
> countless bugs, including some potentially spectacular
> ones), and without which it is not possible, in the
> state of software technology as I understand it, to achieve
> similar results. In short, necessary conditions.
> 
> I understand that not everyone is convinced, and that some may
> think the claims are too grandiose even though they are really
> very modest and down-to-earth; but that does not diminish the
> relevance of these techniques to software reliability, the
> importance for every software engineer of learning them,
> and, yes, their necessity.

Can this necessity be correlated to a safety-critical system
in production? In what regulatory context was DBC used to
certify the system?

> 
> --
> Bertrand Meyer, President, ISE Inc.
> ISE Building, 2nd floor, 270 Storke Road, Goleta CA 93117
> 805-685-1006, fax 805-685-6869, <Bertrand.Meyer@eiffel.com>
> http://www.eiffel.com, with instructions for free download
> == ISE Eiffel 4: Eiffel from those who invented it ==

Re: Critique of Ariane 5 paper (finally!)

[Robert Dewar]

Bertrand Meyer wrote:
>
> Robert Dewar writes:
>
> > This is demonstrably false. There are lots of examples of highly reliable
> > software written by people who don't even know what a specification is,
> > let alone how to carefully associate them with software elements.
> >
> > If you want details on this, I can send you hundreds of thousands of
> > lines of COBOL code. This code is completely inpenetrable in places,
> > and I consider it pretty horrible, but it is from a completely reliable
> > system, where reliability is measured in the terms that matter, i.e.
> > it does what it is supposed to do in a highly reliable manner.
>
> This is eloquently said, but incorrect all the same.
>
> The definition of reliability which this implies is that a system
> is "highly reliable" if it has been working satisfactorily for,
> say, 30 {seconds | minutes | hours | days | weeks | months | years}
> -- pick one. This is one possible definition of reliability, which gets
> more and more interesting as it moves to the right of the list
> of choices; but it is by no means the only "terms that matter".
>


This is complete nonsense. I am talking about systems which are reliabale
by any conceivable measure.

Now of course if your measure of reliability includes that it must explicitly
use DBC, then you reduce your argument to a meaningless tautology.

Personally I find obviously bloated claims like this (my method is the
only one that can generate reliable code, and it is impossible to 
generate reliable code any other way) to be highly counter-productive.

I have occasionally heard people make similar bogus absolute claims for
Ada -- and in my opinion nothing is more damaging, since it causes people
who know better to simply ignore not only the obviously incorrect claim,
but also more reasonable claims.

In this particular case, the very reasonable point that DBC may be a useful
tool in helping to achieve reliability in some circumstances is getting
submerged by the more absurd claim that it is the only way to achieve this
goal.

Re: Critique of Ariane 5 paper (finally!)

[Bertrand Meyer]

Note: Invectives ("complete nonsense", "bogus", "absurd" etc.)
serve no useful purpose. In particular they are not a
substitute for arguments. We are all very passionate
about these things, which is good, but I think we should
all continue to accept that the other side is not
completely incompetent. I certainly respect Robert
Dewar's views, even those (not all) with which I
disagree. (End of note.)

Robert Dewar initially wrote

> > reliability is measured in the terms that matter, i.e.
> > it does what it is supposed to do in a highly reliable manner.

which says little more than: a system is reliable if it is
highly reliable. It would be hard to quarrel with his statement,
but as a definition it's not very useful. He then refined
this definition into

> I said that the
> only measure of reliability that made any sense was that a program
> behaved in a realiable manner and did what it was supposed to do

Again, probably true but not sufficient as a definition. The
first part is a tautology (a program is reliable if it behaves
in a [reliable] manner -- sure). The second part, "[does] what is
supposed to do", is broadly correct as a general, informal
definition (although even so one can do better, in particular
by distinguishing between two components of reliability,
correctness and robustness), but is not enforceable in any way.
How do you know that a system "does what it is supposed to do"?
The most you can expect (in the absence of formal techniques) is,
as I wrote in my earlier message, that the program has "done
what it is supposed to do" for a certain period of time,
be it 30 seconds or 30 years.

        (And even that is subject to doubt,
        as you can hardly be sure that everything was perfectly
        all right. For example if we take one of the pragmatically
        reliable COBOL programs that Prof. Dewar cites, assuming
        it has been computing payroll taxes for the past 10 years,
        few people would bet $50,000 of their own money that no one
        will sue and win damages on the basis of wrongful operation
        of the program during that past period.) 

These are difficult issues and no one has a silver bullet. But
it turns out that it is possible to do better at least
at the component level. As Prof. Dewar has correctly pointed
out, specifying the behavior of a complete system -- i.e.
"what it is supposed to do" -- is hard. But for a component
of that system it is often possible to write such a specification,
partially or totally formal. This is what Design by Contract is
about.

The claim that it is necessary has an obvious basis: you can't
get the system right if you can't get the components
right. To get the components rights, you need to convince enough
people, beginning with yourself, that they are
"doing what they are supposed to do", and you can't do
that without stating what they are supposed to do,
as precisely as possible. The good news is that for a
component you can realistically be more precise than for
a whole system.

The more fundamental issue is that we need better ways
to achieve the reliability of mission-critical systems.
Not just better organization and management, but better
techniques. Design by Contract is one such technique. It
is not by itself the answer, but I think it is a required part
of the answer.


-- 
Bertrand Meyer

Re: Critique of Ariane 5 paper (finally!)

[Robert Dewar]

Bertrand said

<<> The definition of reliability which this implies is that a system
> is "highly reliable" if it has been working satisfactorily for,
> say, 30 {seconds | minutes | hours | days | weeks | months | years}
> -- pick one. This is one possible definition of reliability, which gets
> more and more interesting as it moves to the right of the list
> of choices; but it is by no means the only "terms that matter".>>


Incidentally I said nothing of the kind in my message, I said that the
only measure of reliability that made any sense was that a program
behaved in a realiable manner and did what it was supposed to do.

The idea that this implies that I think the only possible test of this
is to run the system and see if it works is an idea created by the
reader not the writer!

There are many ways that one can use to determine if a program is reliable.
For example, in some limited small cases, total proof of correctness may
be a useful tool. Certainly the contstruction of such proofs can be aided
by accurate assertions about the code, but such proofs do not (and cannot)
depend critically on the presence of such assertions.

Similarly, systematic coverage and branch testing are pretty much 
independent of whether DBC was or was not used.

Showing that a program is realiable is a complex process that involves
the use of many tools and techniques. Yes, in some cases, DBC may aid
this process, but the suggestion that no program can be realiable which
does not use DBC is plain unreasonable.

Re: Critique of Ariane 5 paper (finally!)

[Thomas Beale]

Robert Dewar wrote:
> 
> There are many ways that one can use to determine if a program is reliable.
> For example, in some limited small cases, total proof of correctness may
> be a useful tool. Certainly the contstruction of such proofs can be aided
> by accurate assertions about the code, but such proofs do not (and cannot)
> depend critically on the presence of such assertions.

As an observer to this long-winded thread, can I suggest that you define
what
you mean by reliable? Software engineers such as myself often use the
term 
"correctness" to mean what you seem to be talking about above. I would 
define "reliability" as the ability of a system to execute correctly
over 
some time, and with "normal" inputs ("robustness" would be a measure of
a 
system handling pathological inputs). Reliability in my experience has 
often been specified by an mtbf - mean time between failures - figure.
So 
the aspect of quality we are interested in here is longevity of correct 
operation, not just correct function. So, when you say "reliability",
exactly
what do you mean?

- thomas beale

Re: Critique of Ariane 5 paper (finally!)

[Robert Dewar]

Thomas said

<<As an observer to this long-winded thread, can I suggest that you define
what
you mean by reliable? Software engineers such as myself often use the
term
"correctness" to mean what you seem to be talking about above. I would
define "reliability" as the ability of a system to execute correctly
over
some time, and with "normal" inputs ("robustness" would be a measure of
a
system handling pathological inputs). Reliability in my experience has
often been specified by an mtbf - mean time between failures - figure.
So
the aspect of quality we are interested in here is longevity of correct
operation, not just correct function. So, when you say "reliability",
exactly
what do you mean?>>


Sure I think we all accept your definition of reliability (the ability of
a system to execute  correctly ....) although we might want to add something
about ease of modification and maintenance (for me these are also aspects
of reliability).

But that definition is not one we can use directly to ensure reliability.
Sure we can judge in retrospect whether we succeeded in generating a
realiable application, but we can't use this criterion directly to ensure
reliability in advance.

If you look at my earlier postings, you will see that the point you are
making is *precisely* the one that I emaphasized in my replies to Bertrand.

Now what Bertrand claims is that any methodology which results in reliable
programs MUST ALWAYS use an approach that includes an explicit use of DBC.
What I am saying is that there are many methods and techniques that we
use in practice for judging reliability.

Correctness is a different property from reliability as you point out (there
are unreliable correct programs, and reliable incorrect programs). However,
that does not mean that demonstration of correctness is not a useful tool
in the quest after reliability. The effort to demonstrate correctness may
well show up flaws that would indeed effect reliability. Of course even if
you prove total correctness, you have not demonstrated reliability, but then
that is true of other techniques (such as formal testing, etc). In practide,
we ensure reliability by using a variety of techniques and tools.

DBC in the sense in which Bertrand means it is a possible tool. It is
neither necessary nor sufficient, but it is one more useful tool (I cannot
imagine anyone contesting this point). However, use of DBC does not ensure
reliability, and failure to use it does not guarantee unreliability!

Re: Critique of Ariane 5 paper (finally!)

[W. Wesley Groleau x4923]

> The supportable statement is something like
> "Associating specifications with software elements" is an important 
> tool that will aid in the production of reliable software.

what some people disagree with is the use of two definitions of
DBC by the same advocates.

Def'n 1 - Associating specifications with software elements" and 
related practices.
Def'n 2 - The Eiffel implementation of a subset of these practices.

Now before anyone denies having used definition 2,  please explain
what definition of DBC satisfies the claim "only Eiffel directly
supports DBC"

-- 
----------------------------------------------------------------------
    Wes Groleau, Hughes Defense Communications, Fort Wayne, IN USA
Senior Software Engineer - AFATDS                  Tool-smith Wanna-be

Don't send advertisements to this domain unless asked!  All disk space
on fw.hac.com hosts belongs to either Hughes Defense Communications or 
the United States government.  Using email to store YOUR advertising 
on them is trespassing!
----------------------------------------------------------------------

Re: Critique of Ariane 5 paper (finally!)

[aek]

In <dewar.872088939@merv> dewar@merv.cs.nyu.edu (Robert Dewar) wrote:

>In this particular case, the very reasonable point that DBC may be a
useful
>tool in helping to achieve reliability in some circumstances
                                        ^^^^^^^^^^^^^^^^^^^^^
This is the point. When one claims that some new method or tool may be
useful
in some circumstances he seems to be obliged to describe those
circumstances
more or less precisely. But if one claims that this new method or tool is
very
useful universally then he frees himself from this trouble and invites
other
people to do this job.


Alexander Kopilovitch                      aek@vib.usr.pu.ru
Saint-Petersburg
Russia


\x1a

-------------------==== Posted via Deja News ====-----------------------
      http://www.dejanews.com/     Search, Read, Post to Usenet

Re: Critique of Ariane 5 paper (finally!)

[Bertrand Meyer]

W. Wesley Groleau wrote:
 
> what some people disagree with is the use of two definitions of
> DBC by the same advocates.
> 
> Def'n 1 - Associating specifications with software elements" and
> related practices.
> Def'n 2 - The Eiffel implementation of a subset of these practices.
> 
> Now before anyone denies having used definition 2,  please explain
> what definition of DBC satisfies the claim "only Eiffel directly
> supports DBC"

This has been explained many times. One can to some degree
apply the principles of Design by Contract in any language,
but among commercially commercially available languages Eiffel
is the only one to offer direct support for "associating
specifications with software elements" as part of the software
elements themselves, i.e. using language constructs --
preconditions, postconditions, class invariants, loop invariants,
loop variants.

In other languages you have to resort to purely methodological
guidelines or to comment conventions etc.
It's better than nothing, but does not go as far, because
the presence of the assertion constructs provides Eiffel
users  with major benefits such as: 

	- Documentation tools (in the supporting environments)
	  that extract the specification from the software.
	  Such tools provide a standard form of documentation
	  for reusable components and are essential to the success
	  of reusability efforts.
	  
	- Better reusable libraries (this too is a property
	  of the environments, made possible by the language)
	  thanks to their extensive
	  use of assertions (e.g. EiffelBase). Also,
	  these libraries naturally serve as models and
	  guides for users learning the approach, so they
	  indirectly lead to  more reliable and clearer
	  application software. 

	- Runtime assertion monitoring (again in the
	  environments) providing a particularly effective
	  tool for testing, debugging and quality assurance.

	- Language rules that associate assertions with
	  inheritance: precondition weakening and postcondition
	  strengthening in routine redefinitions, invariant
	  accumulation (a class inherits its parents'
	  integrity constraints). This is essential to
	  control the power of polymorphism and dynamic
	  binding, further decreasing bug sources.

	- Close connection with the exception handling
	  mechanism.

	- Simplification of the software's structure, since
	  supplier classes can avoid redundant checking
	  of special cases in the presence of preconditions.

	- General improvements in software quality: having
	  a construct in the language itself encourages
	  developers to use it -- especially, as noted, thanks to
	  the influence of reusable components --,  more so than
	  a purely methodological injunction to "be good"
	  and specify. In other words, use of Eiffel
	  helps promote a quality-oriented and
	  reliability-conscious mindset. This is a less
	  directly palpable benefit (and please do not
	  construe it as a claim that "Eiffel users make
	  no errors", no one ever said that) but significant
	  all the same.

Most of this is unique to Eiffel among widely used commercial
languages. Again it does not mean that you cannot get any benefit
of Design by Contract in other languages; you can (especially
if you have used Eiffel before: many people have noted that
you can become a better C++ programmer, for example, if you have
been trained in Eiffel; see e.g the educators' testimonials at
http://www.eiffel.com/services/university/). But you will
get much more in Eiffel.


-- 
Bertrand Meyer, President, ISE Inc.
ISE Building, 2nd floor, 270 Storke Road, Goleta CA 93117
805-685-1006, fax 805-685-6869, <Bertrand.Meyer@eiffel.com>
http://www.eiffel.com, with download instructions

Re: Critique of Ariane 5 paper (finally!)

[W. Wesley Groleau x4923]

> > what some people disagree with is the use of two definitions of
> > DBC by the same advocates.
> >
> > Def'n 1 - Associating specifications with software elements" and
> > related practices.
> > Def'n 2 - The Eiffel implementation of a subset of these practices.
> >
> > Now before anyone denies having used definition 2,  please explain
> > what definition of DBC satisfies the claim "only Eiffel directly
> > supports DBC"
> 
> This has been explained many times. One can to some degree
> apply the principles of Design by Contract in any language,
> but among commercially commercially available languages Eiffel
> is the only one to offer direct support for "associating
> specifications with software elements" as part of the software
> elements themselves, i.e. using language constructs --
> preconditions, postconditions, class invariants, loop invariants,
> loop variants.

I say again: If DBC is defined as "Associating specifications 
with software elements", then Eiffel indeed directly supports 
a SUBSET of DBC.  Ada directly supports another subset of DBC.
C++ (shudder) directly supports yet another subset of DBC.  And 
with a little extra work, the user of any of these languages can 
handle a larger subset.  But none of them can completely support 
the whole thing.  And the intersection of the subsets is not empty.

I keep seeing most of the Eiffel advocates and a few of the Ada
advocates arguing, "My language supports DBC; your language does 
not look like my language; therefore your language does not 
support DBC."  Yet when they post specific examples, the syntax 
differences cannot hide (from me at least) the obvious similarities.

> In other languages ... does not go as far, because
> the presence of the assertion constructs provides Eiffel
> users with major benefits such as:

[ a long list of things that ARE being done with Ada and other
  languages ]

[ followed by the claim that only Eiffel users do these things. ]

-- 
----------------------------------------------------------------------
    Wes Groleau, Hughes Defense Communications, Fort Wayne, IN USA
Senior Software Engineer - AFATDS                  Tool-smith Wanna-be
                    wwgrol AT pseserv3.fw.hac.com

Don't send advertisements to this domain unless asked!  All disk space
on fw.hac.com hosts belongs to either Hughes Defense Communications or 
the United States government.  Using email to store YOUR advertising 
on them is trespassing!
----------------------------------------------------------------------

Re: Critique of Ariane 5 paper (finally!)

[Robert Dewar]

<<I think one of the primary differences between correctness and
reliability is
that the former can be defined crisply while the latter cannot.>>


No, I disagree, these are completely different qualities, they are related,
but as I said earlier, you can have reliable programs that are not correct, and correct
programs that are not realiable.

Re: Critique of Ariane 5 paper (finally!)

[Robert S. White]

In article <33FC66AD.9A0799D4@calfp.co.uk>, nickle@calfp.co.uk says...

>Let us say for the moment that in some circumstances DBC helps.
>For those that have been critising DBC, since DBC is optional, and is an
     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^


  Nobody that I know of on this thread has been "critising DBC"!
What all the furor is about, is the claims that DBC _must_ be
used to create reliable software.  

  Ken, myself, and a few others have been arguing that you can 
not always employ the executable code aspects of DBC (or Ada
run time checks) in hard real time systems with constrained
resources.  

  The other issue is that in the Ariane 5 case, the 
methodology that was in place (system requirements review and
software requirements specification), was not followed 
adequately.  To quote the inquiry report once more: 

  "the overriding means of preventing failures are the reviews 
   which are an integral part of the design and qualification 
   process, and which are carried out at all levels and involve 
   all major partners in the project (as well as external 
   experts)"

The Ariane 4 IRS software as-is reuse should not have made it
by such reviews.  Please read Ken's rebuttal paper:

  http://www.progsoc.uts.edu.au/~geldridg/eiffel/ariane/

  My reading of it does not indicate a general "critising DBC"
but rather it summerizes:

  "In the specific case of the Ariane IRS design fault, there
  is not clear and compelling evidence that DBC/Eiffel 
  assertions were likely to have uncovered the fault prior to 
  operational use, either through their documentation, test, 
  or execution value. Furthermore, alternative means were 
  available to the Ariane team to isolate the particular 
  fault, even without the use of DBC/Eiffel. Therefore, 
  although there may be a compelling claim to use DBC/Eiffel 
  in real-time safety-critical systems, the Ariane case 
  (and the Eiffel paper describing this case) does not 
  support such a claim."

  My complaint is against the claim in the Eiffel paper:

  "Does this mean that the [Ariane 5] crash would 
  automatically have been avoided had the mission used 
  a language and method supporting built-in assertions 
  and Design by Contract? Although it is always risky 
  to draw such after-the-fact conclusions, the answer 
  is probably yes..."
     ^^^^^^^^^^^^

    I say, IMO, probably no for the Ariane 5 case.  But I
think it is a "good thing" to use assertions and/or a DBC
methodology whenever practical.  Unfortunately, IME, it is
often not practical for resource constrained hard real time
systems.  
_____________________________________________________________________
Robert S. White         -- An embedded systems software engineer

Re: Critique of Ariane 5 paper (finally!)

[Samuel Mize]

Robert S. White wrote:
>   The other issue is that in the Ariane 5 case, the
> methodology that was in place (system requirements review and
> software requirements specification), was not followed
> adequately.  To quote the inquiry report once more:
...
>   My complaint is against the claim in the Eiffel paper:
> 
>   "Does this mean that the [Ariane 5] crash would
>   automatically have been avoided had the mission used
>   a language and method supporting built-in assertions
>   and Design by Contract? Although it is always risky
>   to draw such after-the-fact conclusions, the answer
>   is probably yes..."
>      ^^^^^^^^^^^^
> 
>     I say, IMO, probably no for the Ariane 5 case.

I'd even tolerate the "probably yes," if it weren't
explicitly stated that DBC is the ONLY method that would
probably have avoided the crash.

No discussion about whether DBC would have helped is relevant
to my point.  I concede that DBC is one method that would
have helped.

I'll say that again:

    I CONCEDE THAT DBC WOULD PROBABLY HAVE HELPED, IF ONLY
    BECAUSE IT ISN'T DBC WITHOUT REVIEWS AND TEST.

But the paper says that ONLY DBC would have helped, as I'll
show below.  If the authors had said "Yes, the claim was too
strong, sorry," a lot of us would have shut up and gone away.

Co-author Jean-Marc Jezequel has said that this does not
characterize what the paper was MEANT to say.  I have
little dispute with what he says they MEANT to say[1]:

  Let's finally sum up what I perceive as the most important
  claims in this paper:
  - reusing a component without checking its full specification
    is dangerous, which means that simple minded CORBA-like
    approaches at building components for mission-critical
    software are doomed.
  - using design by contract is an interesting way to specify the
    behavior of a component
  - at least in the case of Ariane 501, simple assertions (a la
    Eiffel and other languages) would have been expressive enough
    to specify the fatal hidden assumption.

But the paper DOES state explicitly that DBC is the ONLY method
that would have avoided the crash, and that existing methods were
applied but did not succeed.

Following are my reasons for stating that the paper says this.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
THE PAPER CLAIMS THAT ARIANE 5 APPLIED ALL RELEVANT EXISTING METHODS

At the very least it strongly implies this.

From the paper:
    The ESA's software people knew what they were doing and applied
    widely accepted industry practices.

No, the project made an explicit decision to NOT apply widely
accepted industry practices.

I have yet to see any support for this assertion: not in the Eiffel
paper, not in the ESA report, not in the net traffic.


From the paper:
    Is it a testing error?  Not really.  ...  But if one can test more
    one cannot test all.  Testing, we all know, can show the presence
    of errors, not their absence.

This implies that the error in question would not likely have been
found through normal testing.  Yet it is, in fact, one that would
have blown out a normal test scenario the first time they tried it.
The addition of DBC would have made no more difference than would
the addition of paper hats.  There may indeed be errors that cannot
be found through testing, and that DBC would find, but this is NOT
demonstrated by the Ariane 5 case.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
THE PAPER CLAIMS THAT ONLY DBC WOULD HAVE PREVENTED THIS ERROR

The concluding section, labelled "The lesson for every software
developer," is clearly meant to state what the Ariane 5 crash shows.
It states:

    To attempt to reuse software without Eiffel-like assertions is
    to invite failures of potentially disastrous consequences.
    ...
    For reuse to be effective, Design by Contract is a requirement.
    Without a precise specification attached to each reusable
    component -- precondition, postcondition, invariant -- no one
    can trust a supposedly reusable component.

This says that, if a reused component has not been analyzed with
DBC, you cannot trust it, and you are inviting failure.  No other
method is sufficient, by this statement.


From the paper:
    Reuse without a contract is sheer folly.

Does this state that Ariane 5 would have crashed, no matter what
other methods were used, unless DBC were also used?  In context,
yes.  It's from the conclusions section, "The lesson for every
software developer."  Surely this is meant to be the lesson of the
Ariane 5 crash.  Surely "contract" in this context is intended to
refer specifically to a Design By Contract artifact.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

[1] I'd dispute his first point if "full specification" is limited
    to pre/post conditions and invariants.  In some cases these are
    too little, in others they may not be needed.  However, it's
    certainly failure-prone to reuse components without reviewing
    their specs and designs.  I know too little about CORBA to have
    an opinion on its (in)sufficiency.

Sam Mize

Re: Critique of Ariane 5 paper (finally!)

[Samuel Mize]

Samuel Mize wrote:

> Co-author Jean-Marc Jezequel has said that this does not
> characterize what the paper was MEANT to say.  I have
> little dispute with what he says they MEANT to say[1]:
...
> [1] I'd dispute his first point if ...

This footnote looks like a reference.  I apologize for the
inclarity.

I should have referenced the text.  According to Deja News,
Mr. Jezequel posted this statement on 1997/03/18 to
comp.lang.eiffel, comp.object, comp.software-eng,
comp.programming.threads, comp.lang.ada, and comp.lang.java.tech.

Sam Mize

Re: Critique of Ariane 5 paper (finally!)

[Marin David Condic, 561.796.8997, M/S 731-96]

Robert Dewar <dewar@MERV.CS.NYU.EDU> writes:
>Note incidentally tha DBC seems to be a slippery character in this argument.
>At one end it is a very specific technique embodied in syntax in the tools
>being used, and at the other (see quoted paragraph above), it is referred
>to as though it is little more than the idea of saying what components of
>a program should do.
>
>Well of course if we take the second view, then it is certainly true that
>far more programs meet this criterion, which has by now been watered down
>to little more than "you should comment your programs properly".
>
>However, even with this watered down viewpoint, not all reliable programs
>meet the criterion. From time to time, there have been people holding the
>strong position that code should be self-documenting and that comments or
>documentation of any kind of the code is evil, becuse it could be wrong.
>
>I personally think this viewpoint is ludicrous and off the wall, BUT
>I would not for a moment claim that people following this viewpoint cannot
>produce reliable software (I know of counter examples -- yes, they surprise
>me, but the fact is that competent people can do almost anything with almost
>any tools, so general rules of good practice are almost never absolute).
>
    How about this: Build a random code generator which fills the
    first hundred or so words of memory with instructions. Execute
    those instructions. If the instructions output the string "Hello
    World!" once and then halt, the code has met the requirements and
    the image is to be saved for future executions.

    No DBC. No Object Oriented Design/Programming. No structured
    programming. No code walk-throughs. No methodology. No nothing. Yet
    I'd bet that a program could be constructed this way and be 100%,
    double-your-money-back-guaranteed reliable (Barring "Acts Of God"
    such as power outages! But then, that's a *hardware* reliability
    problem.:-) Software doesn't rot, rust or wear out, so the MTBF
    can be considered infinite.

    In other words: If you can think of only one way to solve a
    problem, then clearly you have not thought about it long enough. A
    jingoistic support of some method or technique as "the only"
    method or technique that can produce reliable, safe, "good"
    software is not particularly helpful. It also ends up insulting
    the folks who *do* produce reliable, safe, "good" software using
    other techniques as if they don't know their business.

    MDC

Marin David Condic, Senior Computer Engineer     ATT:        561.796.8997
Pratt & Whitney GESP, M/S 731-96, P.O.B. 109600  Fax:        561.796.4669
West Palm Beach, FL, 33410-9600                  Internet:   CONDICMA@PWFL.COM
===============================================================================
  "I saw a bank that said "24 Hour Banking", but I don't have that much time."
        --  Steven Wright
===============================================================================

Critique of Ariane 5 paper (finally)

[AdaWorks]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 3649 bytes --]


From: Richard Riehle
To:   DBC Discussion

Design by Contract (DBC) is a powerful idea.  It is more powerful than
the notion of assertions, alone. Although, I believe assertions can
be an important aspect of DBC, restricting our discussion to assertions
probably leads us to taking too small a view of DBC.    

I think DBC includes:

  1) Explicit separation of the contract from the implementation
  2) Ability to maximize contract violations at compile time
  3) Clear statement to client of the contract of what is promised
  4) Conditions under which the contract would be violated
  5) Conditions under which the contract would be broken
 
�There are probably others.

Ada supports all five of these in very important ways. In particular,
Ada provides the superb support for 1), separation of contract
from implementation through the specification/body model for packages.
Number 2) is also a strong point of Ada because the language is designed
with exactly that goal in mind.  

To expand a little on item number 2). No programming language is
more effective at supporting this than Ada. The package specification 
rigorously defines the profile of every public method (subprogram) thereby 
permitting strict compile-time checking on any call; the scope and
visibility rules, draconion in the view of some programmers, prevents
a client from falling into some ambiguity trap,  concientious use of
named association further ensures compile-time checking, the type
model guarantees that collisons between incompatible data elements
will not occur, and the accessiblity rules raise the probability that
pointer-related errors will be caught at compile time.

Number three implies both traceability and understandability. This is
well-supported in Ada through both the scope and visibility rules and
the separation of the contract into a visible part and a non-visible
part.  The addition of private children in Ada 95 allows us to improve
upon this even further.  Ada does not take a back seat to any other
language on this point, even though Eiffel also supports this 
very well.

The fourth and fifth points are closely-related yet subtly different.
For this discussion, we consider them together.  Ada has taken a more
conservative view of the contract than some languages.  The type-safe
model does include an invariant for the type.  It also includes a
simple pre- and post-condition in the form of range constraints. While
this is not as sophisticated as explicit assertions, it works quite
well when coupled with the other rules for subprogram invocation and
visibility control.  

All of that begin said, I do like the idea of adding assertions to
Ada as additional support for 5) & 6).  It is important to realize,
though, that an assertion may be incorrectly stated more easily than
one would like.  A wrongly-formed assertion might be more of a problem
than no assertion at all.  

We would probably profit from exploring the DBC notion in greater
depth.  But the benfits from that exploration will be greatest if
we define it a higher level of abstraction than simple assertions.
I think Bertrand would agree with this "assertion".  
I hope others will.

Richard Riehle
AdaWorks
Suite 30
2555 Park Boulevard
Palo Alto, CA 94306
(415) 328-1815

                    P.S. My News Server is acting funny right now so
                         it is difficult for me to respond directly to
                         postings until it gets fixed.
                                               RR

 
-- 

richard@adaworks.com
AdaWorks Software Engineering
Suite 30
2555 Park Boulevard
Palo Alto, CA 94306
(415) 328-1815
FAX  328-1112

Re: Critique of Ariane 5 paper (finally!)

[Ken Garlington]

Robert Dewar wrote:
> 
> DBC in the sense in which Bertrand means it is a possible tool. It is
> neither necessary nor sufficient, but it is one more useful tool (I cannot
> imagine anyone contesting this point). However, use of DBC does not ensure
> reliability, and failure to use it does not guarantee unreliability!

I think we can go further. With respect to the available techniques
(Musa
et. al.) to _quantify_ software reliability, none of the models to my
knowledge require DBC. You can argue that DBC would _improve_ the
quantified values (or argue that none of the models are useful), but
there is no evidence that DBC is _required_ to have a given level
of reliability, as measured by these models. Furthermore, of the
widely-used certification techniques that attempt to _qualitatively_
establish reliablilty, either of the product directly (882C, 178B,
etc.), the only one I can think of that even potentially could be
read as "requiring" DBC is 00-55/00-56, and I know of at least
one system qualified under that standard that did not use executable
assertions. Thus, I would assume that Mr. Meyer's relucance to
cite a specific case where DBC was required to satisfy a customer
as to the reliability of a system is... because he doesn't know of
a system that requires it!

Note that reliability (like safety) is not an absolute measure.
Systems are not "reliable" or "safe," they are merely more or
less reliable/safe than other alternatives. This is a big point
of "Safeware," which I would recommend highly to anyone participating
in a discussion of safety or reliability.

Re: Critique of Ariane 5 paper (finally!)

[Ken Garlington]

Robert S. White wrote:
> 
> The Ariane 4 IRS software as-is reuse should not have made it
> by such reviews.  Please read Ken's rebuttal paper:
> 
>   http://www.progsoc.uts.edu.au/~geldridg/eiffel/ariane/
> 
>   My reading of it does not indicate a general "critising DBC"
> but rather it summerizes:
> 
>   "In the specific case of the Ariane IRS design fault, there
>   is not clear and compelling evidence that DBC/Eiffel
>   assertions were likely to have uncovered the fault prior to
>   operational use, either through their documentation, test,
>   or execution value. Furthermore, alternative means were
>   available to the Ariane team to isolate the particular
>   fault, even without the use of DBC/Eiffel. Therefore,
>   although there may be a compelling claim to use DBC/Eiffel
>   in real-time safety-critical systems, the Ariane case
>   (and the Eiffel paper describing this case) does not
>   support such a claim."

In addition, it states in section 6:

"It would not be appropriate to use the criticisms here to say
in the general case that assertions have no value, anywhere
("casuistry"), but this criticism does not attempt to do this.
It focuses on the specific claim in the Eiffel paper in the
context of the Ariane IRS fault.... 

"Several Eiffel advocates will attest that they like the use of
Eiffel for their domain. Eiffel may be very useful in some domains,
however Ariane is a real-time embedded safety-critical system and
has unique properties (as described above). Again, this is a specific,
not a general, criticism of DBC/Eiffel."

Perhaps my paper is so boring that no one made it to this
section :)

Re: Critique of Ariane 5 paper (finally!)

[Jon S Anthony]

In article <3401811D.1700E7BE@stratasys.com> Jeff Kotula <jkotula@stratasys.com> writes:

> Robert Dewar wrote:
> 
> > No, I disagree, these are completely different qualities, they are
> > related,
> > but as I said earlier, you can have reliable programs that are not
> > correct, and correct
> > programs that are not realiable.
> 
> This is a nonsensical statement. In the first case, you will have a
> program
> that reliably does the *wrong* thing, and in the second case, you
> will have a program that doesn't do the right thing all the time.
> Neither of these cases is of any interest to the goal of creating
> high quality functioning software.

Wow.  This is amazingly naive.  You don't actually believe that in
anything but trivial cases you can prove correctness, do you?????

/Jon
-- 
Jon Anthony
OMI, Belmont, MA 02178, 617.484.3383 
"Nightmares - Ha!  The way my life's been going lately,
 Who'd notice?"  -- Londo Mollari

Re: Critique of Ariane 5 paper (finally!)

[Nick Leaton]

Thomas Beale wrote:

> Where I think there is more room for debate is in how DBC should be
> implemented. I have used Eiffel's DBC for some time, and it is miles
> ahead of no DBC. On the other hand, I am sure that there is empirical
> and theoretical work to be done which will show how the current idea
> in Eiffel can be improved, for example, how to determine the
> minimum set of orthogonal assertions for a precondition or invariant,
> and whether "synonymous" assertions are good or not. Quantitative
> statements have also been mentioned as a desirable thing. I believe
> gap between requirements specification and implementation is narrowing:
> better formal expression of at least some requirements may soon have
> a 1:1 counterpart in an implementation formalism, paving the way for
> very seamless software.

Explain a bit more about what you mean by 'synonymous'?

-- 

Nick

Re: Critique of Ariane 5 paper (finally!)

[Nick Leaton]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 8512 bytes --]


Ken Garlington wrote:
> 
> Nick Leaton wrote:
> >
> > Let us say for the moment that in some circumstances DBC helps.
> > For those that have been critising DBC, since DBC is optional, and is an
> > addition to the current methodology I think the only valid criticism one
> > make, is if you can find a situation where DBC causes a problem.
> >
> > That is, where does DBC screw up, and the current methodologies do not.
> 
> See my Ariane paper for some examples of such information:
> 
>   http://www.flash.net/~kennieg/ariane.html#s3.1.6

QUOTE -----------------------------------------------------------
3.1.6 Adverse effects of documenting assertions

     There is a line of reasoning that says, "Even if DBC/assertions
would have been of minimal use, why not include them anyway just in case
they do
     some good?" Such a statement assumes there are no adverse impacts
to including assertions in code for documentation purposes. However,
there
     are always adverse effects to including additional code: 

         As with any other project, there are management pressures to
meet cost and schedule drivers. Additional effort, therefore, is
discouraged
         unless justified.

         More importantly, all projects have finite time and other
resources available. Time spent on writing and analyzing assertions is
time not
         spent elsewhere. If the work that is not performed as a result
would have been more valuable (in terms of its effect on the safety and
         integrity of the system) than the time spent with assertions,
then the resulting system may be less safe. 

     There is a growing consensus in the safety-critical software field
that simpler software tends to be safer software [21]. With this
philosophy,
     additional code such as assertions should only be added where there
is a clear benefit to the overall safety of the particular system being
     developed. 
END QUOTE --------------------------------------------------------

1) Additional effort.
There is plenty of evidence that the cost of finding and fixing a bug
early in the development process is much cheaper that fixing it later.
Now *IF* DBC as a methodology produces earlier detection of faults, then
this 'additional effort' is justified, as it is effort early in the
development process. I believe this to be the case from practical
experience. Developing code last week I detected errors in my code very
early, even though the fault would have only shown itself in an
exception. I had an invariant that made sure I had an error handler in a
class. It broke as soon as I tried to construct the class, it didn't
wait until I tried to generate an error, all because I hadn't set the
handler up. That combined with permantant testing of assertions has the
effect of producing less effort.

2) Time spent else where.
Is this the case? Some of it may be, but I believe if you cannot be
rigourous about what your software is trying to do, by writing
assertions, then you are unlikely to produce a quality system. The
effect of writing assertions overlaps with the design process. It is not
wasted time, it just comes under a different heading. If your design
process listed the assertions in the specification, would implementing
them be a waste of effort?

3) Simple software. 
You bet. The simpler the better. Occams Razor rules. Now here there is a
split between DBC a la Eiffel and DBC, say in C++. In Eiffel it is
simple. In C++ it is hard, particularly with inheritance of assertions.
One common theme from Eiffel practitioners is their support for DBC.
Why? They are simple to write. Prior to using Eiffel I had read about
the language. I was extremely sceptical about assertions because in my
experience with C++ and C++ programmers, no one writes them, mainly
because it is hassle. Take away the hassle and people will write them
because of the benefits.



>   http://www.flash.net/~kennieg/ariane.html#s3.2.2

QUOTE --------------------------------------------------------
3.2.2 Adverse effects of testing with assertions

     Assume for a moment that the proper testing environment and data
had been available. Putting aside for the moment the question as to
whether
     assertions would have been necessary to detect the fault (see
section 4.2), are there any disadvantages to using assertions during
testing, then
     disabling them for the operational system? In the author's
experience, there are some concerns about using this approach for
safety-critical
     systems: 

         The addition of code at the object level obviously affects the
time it takes for an object to complete execution. Particularly for
real-time
         systems (such as the Ariane IRS), differences in timing between
the system being tested and the operational system may cause timing
         faults, such as race conditions or deadlock, to go undetected.
Such timing faults are serious failures in real-time systems, and a test
which
         is hindered from detected them loses some of its effectiveness.

         In addition, the differences in object code between the tested
and operational systems raise the issue of errors in the object code for
the
         operational system. Such errors are most likely to occur due to
an error in the compilation environment, although it is possible that
other
         factors, such as human error (e.g. specifying the wrong version
of a file when the code is recompiled) can be involved. For example, the
         author has documented cases where Ada compilers generate the
correct code when exceptions are not suppressed, but generate
         incorrect code (beyond the language's definition of
"erroneous") when they are suppressed. This is not entirely unexpected;
given the
         number of user-selectable options present with most compilation
environments, it is difficult if not impossible to perform adequate
toolset
         testing over all combinations of options. Nothing in the Eiffel
paper indicates that Eiffel compilers are any better (or worse) than
other
         compilers in this area. Although this is a fault of the
implementation, not the language or methodology, it is nonetheless a
practical limitation
         for safety-critical systems, where one object code error can
have devastating results. 

     One possible approach to resolving this issue is to completely test
the system twice; once with assertions on and another time with
assertions
     suppressed. However, the adverse factors described in section 3.1.6
then come into play: By adding to the test time in this manner, other
useful
     test techniques (which might have greater value) are not performed.
Generally, it is difficult to completely test such systems once, never
mind
     twice! This effect is worse for safety-critical systems that
perform object-code branch coverage testing, since this testing is
completely
     invalidated when the object code changes [25]. 

     Overall, there are significant limitations to using this technique
for safety-critical systems, and in particular real-time systems such as
the Ariane
     5 IRS. 

END QUOTE --------------------------------------------------------

Assertions affect timing in safety critical systems.

Firstly it depends on the implementation. It is easy to envisage a
system where the assertions are redundantly executed. But you would only
want to do that if you were running with faulty software ?!*^�%

I also find it worrying that systems are being used for safety critical
apps where there is the possibility of a race or deadlock to occur. 

Compilation problems.
These can occur in any system as you are aware. From discussions with
some of the people involved in writting Eiffel compilers, the enabling,
disabling of assertions has a very trivial implementation, which is very
unlikely to go wrong. It has also be extensively tested in the field by
end users.
Do you trust your compiler? If not, you shouldn't be writing safety
critical software with it. Period.

Next I find some of the logic of your arguments here very weak.
Paraphrasing. We have trouble testing safety critical systems, but we
will use them anyway. Hmmm.


>   http://www.flash.net/~kennieg/ariane.html#s3.3
> 
> There are approaches that can avoid such costs, particularly those of
> 3.2.2 and 3.3 (by not requiring object code modification). 3.1.6 can
> be mitigated through the use of techniques that minimize cost (e.g.
> automated structural testing analysis).
> 
> >
> > --
> >
> > Nick

-- 

Nick

Re: Critique of Ariane 5 paper (finally!)

[Samuel Mize]

Bertrand Meyer wrote:

> When the discussion turned to ... distortions of our views
> (e.g. that we had claimed that "DBC Eiffel is
> THE ONE method that could have found the Ariane 5 flaw"),

I'm glad to hear that this is a distortion of your views.

Unfortunately, your Ariane 5 column distorts your views the same
way.  The applicable quotes have been repeatedly posted.  If you
have responded to them before now, I've missed it.  This is the
first time I've seen you specifically disclaim the notion that
the Ariane 5 crash demonstrated the failure of current methods
(which were, in fact, unused on the Ariane 5 INS).

I'll be happy to re-post the quotes if you wish to explain how I
am mis-interpreting them.  I sincerely don't see how your paper
can support any other meaning.

Sam Mize

Re: Critique of Ariane 5 paper (finally!)

[Ken Garlington]

Thomas Beale wrote:
> 
> Ken Garlington wrote:
> >
> I've been caught out using a very informal argument and relying
> on "commonly agreed" norms of software engineering. But as you
> imply, (and I have done often enough, many of these can be
> wrong). So...
> 
> > > Jeff Kotula wrote:
> > > > 1. Any evaluation of a system's reliability must be based on its
> > > > requirements specification.
> >
> > False. The Musa model, for example, does not require examination
> > of a requirements specification.
> 
> I don't know what the Musa model is; does its definition of reliability
> of a system rely in no way on the requirements (or any transformation of
> any expression of it)? If not, does its idea of reliability not relate
> to any expression of the correctness of the system (of which the
> requirements spec is an instance)?

In a grossly oversimplified sense, it requires a set of tests performed
according to certain groundrules, the collection of faults detected over
time from those tests, and the application of a model to determine the
residual faults. I suppose you could claim that the tests would be
based on the requirements specification, although they could just as
easily be generated by the end user without reference to that
specification.
It certainly relates to an expression of the "correctness" of the
system,
but I never claimed otherwise.

The key of Musa (and several other models) is to execute the system over
time based on it's predicted operational profile (how it is expected to
be used). This is not necessarily the same as what's in the requirements
spec, which may be written more in terms how it _could_ be used.

There's a large amount of information available on quantitative
software reliability; I've been planning to get Lyu's "Handbook
of Quantitative Software Reliability," which I've heard is good.

> > > > 2. Any evaluation of a system's reliability will directly correlate with
> > > > the reliability of its component's reliability.
> >
> > For software, not true. See "Safeware." The sum of the component
> > reliability
> > does not necessarily equal the system's reliability. (This is a common
> > mistake made when attempting to apply hardware reliability models to
> > software.)
> 
> I would have to agree here. A better statement would have been:
> "A system's unreliability will strongly correlate to the unreliability
> of the components". I.e., you can build rubbish out of good bits, but
> its unlikely (if not impossible) to build a good system out of
> broken components.

There's _some_ correlation; it depends upon the system design, etc. as
to
the strength of the correlation for a given component. In the
extreme case, a completely broken component that's never executed
will not degrade system reliability.

More to the point, individual components that, individually, have high
reliability may exhibit low reliability when used together.

> > > > 3. Any evalution of a component's reliability must be based on its
> > > > requirements specification.
> >
> > Again, not true. See above.
> 
> I would need to be convinced by this "Musa" argument...
> 
> > > which closes the loop from requirements to implementation. I think
> > > this argument is a pretty fair statement of why DBC is not just another
> > > tool in the toolkit: (to paraphrase the above) DBC is a way of
> > > formalising, implementing, and enforcing the requirements specification
> > > at the design and implementation levels of abstraction.
> >
> > Unfortunately, there can be elements of the requirements specification
> > which cannot be enforced at the implementation level (e.g. certain
> > classes of assumptions about the external environment).
> 
> Naturally... probably the majority (measured by pages of document or
> whatever) is informal or semi-formal, and does not map directly to
> the various levels of design/implementation. But on the other hand,
> if contracts were used as a formalism in the reqts spec, then the
> ability to map through would be better.

Or (and IMHO, this is the more likely outcome) the informal requirements
are merely pushed up a level, into a system document for example. There
can certainly be exceptions; a spec for math routines can be formalized,
for example, but a general-purpose system involving a human in the
specification will likely, at some point, require a natural language
description of the expectations.

> > > Where I think there is more room for debate is in how DBC should be
> > > implemented. I have used Eiffel's DBC for some time, and it is miles
> > > ahead of no DBC.
> >
> > What did you use before DBC?
> 
> At university (Qld, Australia), we used formal assertions (a la
> Dijkstra) to specify the meaning of program components. There
> were no tools to support this, but we could easily code up test
> cases to test correctness by encoding these statements (in
> "pascal plus").
> 
> I spent about 6 years in distributed real-time SCADA systems
> coded in K&R C, and at the end, had come to the conclusion (along
> with other team members and developers) that modules built using
> PRE_CONDITION, POST_CONDITION, and CHECK (macro) statements
> in the code were better because:
> 
> a) they were more readable & thus maintainable;
> 
> b) they produced much more useful man pages (we built automated
>    back-extraction tools) - developers could avoid defensive
>    programming & complain to the authors if their program
>    died in a way forbidden by the assertion statements;
> 
> c) they were _much_ easier to debug (we used the Fred Fish dbug
>    macros as well, and integrated them with the assertions);
> 
> d) they acted as a repository for design-level decisions, which
>    began to replace voluminous design documentation which was
>    immediately out of date;
> 
> e) the incentive was there to write more generic, re-usable
>    modules, since for the first time, client developers had
>    a trustworthy way of knowing what a given module did,
>    without having to look at the code: the (automatically
>    extracted) man page with both function signatures (like
>    you get if you do "man strcat" for example, on a unix
>    system), and the pre- and post-condition statements....
>    this was treated as the "truth". A new experience -
>    trust in software not written by me - was discovered.
> 
> Although we never had the means or time to make comparative
> studies of code done this way versus the old way, I  believe
> we were able to write much better software, more quickly, and
> that it was more reliable because a) our ability to test it
> was much better, and b) it tended to get re-used more often
> (and hence further tested).
> 
> I should mention that we weren't doing this for amusement
> either: these systems control e.g. gas pipelines, electric
> power transmission and distribution systems (e.g. Sydney...),
> and passenger rail systems. Reliability was the most
> important thing in the world.
> 
> As well as the eariler Dijkstra/Hoare etc work, Meyer's
> original OOSC was a major influence on this work, since it
> encouraged us to think it would be possible to do this
> kind of thing in actual software (not hidden away in
> some document), and it showed us what reasonable syntax
> might look like for object-oriented software.
> 
> Since then I have felt that formalisms that don't support
> these ideas are way behind those that do. Eiffel is the
> only out-of-the-box formalism in this category that is
> available for building object-oriented software, but other
> formalisms (even the augmented C "formalism" I describe
> above) have great merit. It just depends on how much pain
> you want to go through to make reliable software.
> 
> I have since used Eiffel for some years because
> - I am allergic to pain,
> - it does the rest of the OO paradigm so well.
> 
> - thomas beale

Re: Critique of Ariane 5 paper (finally!)

[Jeff Kotula]

Ken Garlington wrote:

> Jeff Kotula wrote:
> >
> > Robert Dewar wrote:
> >
> > > <<I think one of the primary differences between correctness and
> > > reliability is
> > > that the former can be defined crisply while the latter cannot.>>
> > >
> > > No, I disagree, these are completely different qualities, they are
>
> > > related,
> > > but as I said earlier, you can have reliable programs that are not
>
> > > correct, and correct
> > > programs that are not realiable.
> [snip]
> More to the point, it is critically important not to confuse these
> attributes, since they (as Dr. Dewar has noted) are not synonyms.

I never said they were the same thing (see my orginal post included
in above. I only said that one could be quantified more easily
than the other.

Re: Critique of Ariane 5 paper (finally!)

[Ken Garlington]

Jeff Kotula wrote:
> 
> Robert Dewar wrote:
> 
> > <<I think one of the primary differences between correctness and
> > reliability is
> > that the former can be defined crisply while the latter cannot.>>
> >
> > No, I disagree, these are completely different qualities, they are
> > related,
> > but as I said earlier, you can have reliable programs that are not
> > correct, and correct
> > programs that are not realiable.
> 
> This is a nonsensical statement. In the first case, you will have a
> program
> that reliably does the *wrong* thing,

Exactly. Note that you used the word "reliable" to describe a reliable
program. Thus, it must be reliable!

> and in the second case, you
> will have a program that doesn't do the right thing all the time.

However, "correct" is not the same as "does the right thing".
Presumably,
"corrects" means "conforms to some testable definition of requirements."
A "correct" program may not be reliable, however, if used in some
context not anticipated by the requirements, for example.

> Neither of these cases is of any interest to the goal of creating
> high quality functioning software.

Actually, _both_ are of interest in terms of quality, as are "safe"
(which is something different from either reliability or correctness),
"portable," "maintainable," etc.

More to the point, it is critically important not to confuse these
attributes, since they (as Dr. Dewar has noted) are not synonyms.

Re: Critique of Ariane 5 paper (finally!)

[Ken Garlington]

Bertrand Meyer wrote:
> 
> Kenneth Almquist wrote (only last part of message quoted):
> 
> > This thread was started when Ken Garlington wrote a response to your
> > article about the Ariane V.  So far you have been unable or unwilling
> > to respond to the arguments contained in Ken Garlington's response.
> 
> That is incorrect. Jean-Marc Jezequel and I, plus many others
> familiar with the ideas of Design by Contract, have responsed
> repeatedly and in detail. Consult DejaNews for the record.

Could you provide a specific item? I reviewed DejaNews for my response.

I think it is fair to say that Jezequel and others have responsed in
detail
to the arguments in my response.

> When the discussion turned to ad hominem arguments (that I
> have a "vested interest", that our article was "advertising
> puffery" and "position[ed] deceptively as a technical item
> instead of an ad") and distortions of our views
> (e.g. that we had claimed that "DBC Eiffel is
> THE ONE method that could have found the Ariane 5 flaw"),
>
> I stopped, as there is no point in continuing in that mood.
> 
> The arguments and counter-arguments are widely available
> from the newsgroups and Web pages.
> 
> --
> Bertrand Meyer, President, ISE Inc.
> ISE Building, 2nd floor, 270 Storke Road, Goleta CA 93117
> 805-685-1006, fax 805-685-6869, <Bertrand.Meyer@eiffel.com>
> http://www.eiffel.com, with instructions for download

Not that Mr. Meyer has a vested interest in Eiffel, of
course :)

Re: Critique of Ariane 5 paper (finally!)

[Ken Garlington]

Jeff Kotula wrote:
> 
> Ken Garlington wrote:
> 
> > Jeff Kotula wrote:
> > >
> > > Robert Dewar wrote:
> > >
> > > > <<I think one of the primary differences between correctness and
> > > > reliability is
> > > > that the former can be defined crisply while the latter cannot.>>
> > > >
> > > > No, I disagree, these are completely different qualities, they are
> >
> > > > related,
> > > > but as I said earlier, you can have reliable programs that are not
> >
> > > > correct, and correct
> > > > programs that are not realiable.
> > [snip]
> > More to the point, it is critically important not to confuse these
> > attributes, since they (as Dr. Dewar has noted) are not synonyms.
> 
> I never said they were the same thing (see my orginal post included
> in above. I only said that one could be quantified more easily
> than the other.

Which is also not true. Quantitative software reliability models have
been around for many years.

In fact, "correctness" (depending upon your definition) can be much
harder to _quantify_ (assign a number) than "reliability"!

Re: Critique of Ariane 5 paper (finally!)

[Ken Garlington]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 17192 bytes --]



Nick Leaton wrote:
> 
> Ken Garlington wrote:
> >
> > Nick Leaton wrote:
> > >
> > > 1) Additional effort.
> > > There is plenty of evidence that the cost of finding and fixing a bug
> > > early in the development process is much cheaper that fixing it later.
> > > Now *IF* DBC as a methodology produces earlier detection of faults, then
> > > this 'additional effort' is justified, as it is effort early in the
> > > development process.
> >
> > No, re-read the quote: It can't just produce earlier detection of
> > faults;
> > it must be _better_ at detecting them early than _all_ other available
> > alternatives. Otherwise, you're not using your resources effectively.
> 
> Wrong. You have to cost the resource. 

Which is exactly what I am saying! You have to assign a cost to DBC, and
compare
it to the alternatives.


> I am asserting
> 
> 1. It catches many bugs early. Evidence is personal experience and
> others
>    have posted the same.

This statement provides no guidance as to assigning cost. It catches
bugs "early" - but not earlier than analysis of the UML (a la Jezequel)
for example. Thus, if this were the only criteria, DBC would not be
the best alternative.

> 
> 2. It is not expensive adding in assertions. There is a huge overlap in
>    writting an assertion and producing a correct specification so it is
> not
>    wasted work.

"Expensive" is a relative term, and this statement provides no guidance
as
to the _comparison_ (vs. what baseline?) you are making. If you are
saying, "it is
less expensive than writing a poor specification," then we agree.
However,
DBC is not the only alternative to writing a good specification, nor is
writing a good specification the only factor in producing quality
software.

> You now have to say what ALL the other resources are and what the costs
> are
> in using them to justify your statement.

My statement is "you now have to say what ALL the other resources are
and
what the costs are in using the to justify the statement: DBC is the
best
alternative." I'm glad we agree on this point.

Remember, fellow debators: The affirmative (DBC) has the burden of
proof, not
the negative! See: the principles of (a) status quo and (b) inherency.

> 
> > Furthermore, if it detects 20% of the total system faults early, but
> > as a result there isn't sufficient resources to use other techniques
> > to find the resulting 80% that are left, then although you fixed the
> > 20% more cheaply than some other alternative, you have still delivered
> > a system with 80% of the faults remaining. So, coverage of the faults
> > by quantity is an issue as well as speed.
> 
> But you assume that the other resources will find the faults within your
> resource budget. What if you haven't the resources, which is your
> argument against
> DBC? Can you fit a quart in a pint pot?

No, no, Nanette! I make no such assumption. YOU have the burden of proof
here.
I don't have to _assume_ anything!

You're repeating my argument exactly. You _can't fit a quart in a pint
pot.
Therefore, DBC must not only be effective; it must the the _most_
effective
of the alternatives.

> You have assumed that using DBC uses up resources at such a rate that it
> stops you producing product.  Implementing DBC does not take up huge
> resources at the coding stage. 

"Huge..." what a (relative) concept!

> At the design stage you should be
> specifying what you want to have happen at the level required to
> implement DBC, so there is no additional resources going to waste here.

Now you have made a statement you will have to defend vigourously. This
statement says that DBC takes _zero_ resources more than the
alternatives.

> 
> DBC is cost effective

Prove it!

> as a method of producing quality code, and does
> not take major resources

Post some numbers!

> to achieve. Having a compiler help you makes a
> big difference, which is where the Eiffel part comes into play.

"big"... compared to what!

> 
> > Finally, even if DBC discovers 95% of the faults, but the remaining 5%
> > are the most serious, you still have a problem. So, the ability to
> > detect the most serious faults, as well as the quantity and speed, must
> > be evaluated vs. _all_ alternatives.
> 
> All programmers reading this, if offered a method that found 95% of bugs
> the first time they executed a particular path in their code, would
> welcome it. It leaves them with more resources to work on the hard 5%

Not if the effort to get the 95% fails to get the 5%.

By the way, _safety-critical_ programmers do not agree with this
statement.
I will gladly release software with several minor problems in order to
make sure I have found and corrected the critical problems.


> 
> > That's the common fallacy of the Eiffel advocate argument: "It's better
> > than what I was doing, so go use it." However, was what you were doing
> > before the best approach (excluding Eiffel)?
> 
> I was not necessarily using the ideal approach before, but I haven't
> made this statement.

Actually, you have. You just don't realize it.

> I have found in practice, that DBC as a method
> works. You have to propose other ways of achieving what DBC achieves,
> with less effort.

Again, it's not my argument to make. You have the burden of proof.

2+2=4, therefore God exists! Dispute it!

> 
> > > 2) Time spent else where.
> > > Is this the case? Some of it may be, but I believe if you cannot be
> > > rigourous about what your software is trying to do, by writing
> > > assertions, then you are unlikely to produce a quality system.
> >
> > Bad argument. You haven't proven that "writing software rigorously"
> > is _best_ done by "writing assertions." For that matter, you haven't
> > shown that the "writing" (coding?) phase is the most critical with
> > respect to producing a quality system. Consider, for example, that
> > there are approaches that can generate software with no manual coding...
> 
> Eh? (the last sentence)

Welcome to the world of autocoding! See http://www.ise.com for one
of many examples.

> 
> > > The
> > > effect of writing assertions overlaps with the design process. It is not
> > > wasted time, it just comes under a different heading. If your design
> > > process listed the assertions in the specification, would implementing
> > > them be a waste of effort?
> >
> > Yes, quite possibly, implementing assertions in the code may be both
> > ineffective and counter-productive, compared to other approaches,
> > depending
> > upon circumstances.
> 
> What other approaches? What evidence do you have? (PS I have read your
> critique)

What other approaches are there to quality software? Let us list the
ways:
Formal reasoning, use cases, patterns, cleanroom, N-version programming,
PSP, power CASE, FMET, autocoding, reuse... (everbody sing along!)

Without quantitative measures, you're preaching a religion, not
evaluating
a technique.

> > > 3) Simple software.
> > > You bet. The simpler the better. Occams Razor rules. Now here there is a
> > > split between DBC a la Eiffel and DBC, say in C++. In Eiffel it is
> > > simple.
> >
> > "Simple software" is not synonymous with "easiler to write code." The
> > question
> > is: is software with assertions less complex than software without
> > assertions.
> > Based on measures of merit such as lines of code, paths, etc. the answer
> > is
> > "no".
> 
> Given a complex solution and a simple solution to the same problem, the
> simple solution is the one with merit. Period.
> 
> If you don't assert your software, then you have to write test code to
> test the same assertions (specification) in order to satisfy yourself it
> is correct.

Test oracles, debugger scripts, reliability models, structural testing,
etc.
There is a whole world of means to "satisfy yourself it is correct"
beyond
writing test code (who does this sort of trash anymore, anyway? We
haven't
done it in at least 5 years!).

> DBC on your measures of merit stated above, when you take the whole
> system into account will be better.

Only if you're comparing it to an obsolete alternative (as you did).

> 
> > > In C++ it is hard, particularly with inheritance of assertions.
> >
> > Again, this is the fallacy of the argument. Why are you comparing Eiffel
> > to
> > C++? (The second fallacy: Why are you comparing languages when
> > discussing
> > DBC as a methodology?)
> >
> > > One common theme from Eiffel practitioners is their support for DBC.
> > > Why? They are simple to write.
> >
> > As a motivation, consider this:
> >
> > Here is a simple assertion:
> >
> > "The sun is hot."
> >
> > Now, put this in your code as a comment. Is the code simpler as a
> > result?
> > Put it in 1,000 times (simple to do with a text editor, right)? Is
> > the code simpler as a result?
> >
> > > Assertions affect timing in safety critical systems.
> > >
> > > Firstly it depends on the implementation. It is easy to envisage a
> > > system where the assertions are redundantly executed. But you would only
> > > want to do that if you were running with faulty software ?!*^�%
> > >
> > > I also find it worrying that systems are being used for safety critical
> > > apps where there is the possibility of a race or deadlock to occur.
> >
> > Then you would agree that it is critical to test systems to ensure that
> > such possibilities are not present, correct? And so, doing things which
> > mask the presence of such errors during testing are bad, right? So,
> > you agree with my argument!
> >
> > If we are to assume the system is implemented correctly, why bother with
> > assertions as a means to validate system correctness!
> 
> Because you want to be sure it is implemented correctly!

But you already assumed the system is implemented correctly!
Why are you now unsure!

> 
> > I am always amazed when people don't grasp this. It's not that the
> > system is designed intentionally to permit race or deadlock; the concern
> > is making sure that you didn't _inadvertantly_ do this! It's like
> > saying:
> >
> > "I also find it worrying that systems are being used for safety critical
> > apps where there is the possibility of faults to occur." as an argument
> > for not testing at all!
> >
> > > Compilation problems.
> > > These can occur in any system as you are aware. From discussions with
> > > some of the people involved in writting Eiffel compilers, the enabling,
> > > disabling of assertions has a very trivial implementation, which is very
> > > unlikely to go wrong.
> >
> > Define "unlikely". As I note in the paper, no evidence has been provided
> > regarding this issue. My argument is that, the more complexity added to
> > the software, the more likely the introduction of a fault due to a
> > compiler
> > bug.
> 
> Do some thinking about how you would implement DBC if you were a
> compiler writer. Consider how to enable, disable the assertions. Is this
> a 'hard problem' to solve?

Yes, based on real experience with real compilers.
(I know that real experience doesn't compete with theory, but I find it
comforting nonetheless :)

> 
> > Would you be willing to use a safety-critical system if the engineer
> > said,
> > "Well, we didn't actually test the code we fielded, but we tested code
> > that was pretty much the same."?
> 
> No. I would do the testing. I'm not stupid.

So much for zero cost DBC! You now get to do the testing twice.

Compare to just _one_ alternative: coding the assertions in a debugger
script using
real-time non-intrusive monitoring. Since the assertions are outside the
code, you can (1) test just once (since the assertions do not affect the
code) and (2) have the assertions reference objects not in the code
(e.g.
values in the external environment not directly available to the code
under test). These assertions can be generated from a test oracle hooked
up to a UML (or other notation) model of the system, if you like. They
can be incorporated into a "short form" of the spec for review. So,
compared
to DBC, they do the same thing (assuming you were not going to have
the assertions execute during production) and cost less. We do this
today,
so we know it works. 

Since we agree on comparitive cost as the criteria, you now have to show
why this
alternative costs more. (The negative has introduced an alternative
plan, with
no inherency! Incredible!)


> > > It has also be extensively tested in the field by
> > > end users.
> >
> > Insufficient. How many years had the Ariane IV IRS been flown before it
> > was installed in the Ariane V?
> 
> Agreed. But give yourself the choice.
> 
> a) Tested by software engineers.
> 
> b) Tested by software engineers and lots of users.

Invalid. Show that the Eiffel compiler meets the first clause of (a),
and to what extent. The true choice is:

a) Tested by software engineers to a known standard.
b) Used in various domains (not your own), and tested to some unknown
standard.

I'll be taking "a", please!


> 
> > > Do you trust your compiler? If not, you shouldn't be writing safety
> > > critical software with it. Period.
> >
> > Irrelevant. Much like saying, "I trust my development team, so I won't
> > test their products." Blindly "trusting" any part of the development
> > process is extremely dangerous.
> 
> See you argument above, I'll quote it again
> 
> * If we are to assume the system is implemented correctly, why bother
> with
> * assertions as a means to validate system correctness!
> 
> This is why assertions are good. You don't blindly trust.

Interesting that you quote your own argument to reinforce your own
argument! You are the one who assumed the system was implemented
correctly, remember? I was the one who pointed out this was a bad
idea.

Glad we could agree that we should not assume the compiler is
implemented
correctly!


> 
> > > Next I find some of the logic of your arguments here very weak.
> > > Paraphrasing. We have trouble testing safety critical systems, but we
> > > will use them anyway.
> >
> > If you have trouble with the argument, "real-world safety critical
> > systems
> > are difficult to develop and qualify," then I recommend you stay away
> > from all real-world safety critical systems.
> >
> > More to the point, the "paraphrasing" misses the argument. Here is
> > a better summary. "Keep it simple, stupid. Don't add complexity to the
> > product or the process unless it provides the best payoff vs. all
> > alternatives. Furthermore, _all_ alternatives have risks as well as
> > benefits; ignore anyone who says otherwise as a religious fanatic."
> 
> I'll agree with this point. The disagreement between us is over DBC
> being an aid to improving the process. From practical experience, I
> believe it does.

Without quantification, all we can do is bow our heads and pray at this
point.

Leaving the telephone off the hook is also an aid to improving the
process (this is a real result of a real study). Compare the costs
and benefits of this vs. DBC.

> I don't think that you have backed your arguments up with enough facts
> to justify your position.

Fortunately, I'm not the one advancing the assertion that DBC is the
best alternative. If I were, I would be worried.

> Now that is not saying your wrong, but part of
> the reason for havin a vocal discusion on the point is DBC advocates
> have taken an theory on board that they believe to be correct.
                                          *******

I "believe" that you have summarized my problem with your argument
quite neatly.


> As with
> all discusion on theory, you have to come up with examples that
> contradict, or a better theory.

Actually, no. The burden of proof is on the group advancing the
claim. All I've said is that you haven't proved your claim.

But, just for fun, I did give you a cheaper alternative, so you
can look at that if you want.

> The contradictions either show up
> missing parts of the theory, in which case it is improved, or you come
> up with a new theory which is better.

Actually, no. I think you dropped a few steps in the scientific method
(somewhere between "pose a hypothesis" and "accept or reject
hypothesis").

> Since you have strongly held
> views, I want find out what they are, since they might improve mine and
> others knowledge.

What makes you think I have strongly held views? For that matter, what
relevance is the _strength_ of the views? Sounds suspiciously like a
religious argument, which I find boring.

> 
> > > Hmmm.
> > >
> > > >   http://www.flash.net/~kennieg/ariane.html#s3.3
> > > >
> > > > There are approaches that can avoid such costs, particularly those of
> > > > 3.2.2 and 3.3 (by not requiring object code modification). 3.1.6 can
> > > > be mitigated through the use of techniques that minimize cost (e.g.
> > > > automated structural testing analysis).
> >
> > I note you didn't respond to this part. Hmmm indeed!
> 
> I'll look it up and reply.
> 
> >
> > In any case, I'm going to have a wonderful Labor Day, hope you have the
> > same. Just don't build any safety-critical systems that I might end
> > up using, OK?
> 
> Your pension probably depends on it. ;-) As I do when I go flying in
> Norfolk, lots of your products wizzing around the sky there.
> 
> --
> 
> Nick