Nick Leaton wrote:
> 
> Ken Garlington wrote:
> >
> > Nick Leaton wrote:
> > >
> > > 1) Additional effort.
> > > There is plenty of evidence that the cost of finding and fixing a bug
> > > early in the development process is much cheaper that fixing it later.
> > > Now *IF* DBC as a methodology produces earlier detection of faults, then
> > > this 'additional effort' is justified, as it is effort early in the
> > > development process.
> >
> > No, re-read the quote: It can't just produce earlier detection of
> > faults;
> > it must be _better_ at detecting them early than _all_ other available
> > alternatives. Otherwise, you're not using your resources effectively.
> 
> Wrong. You have to cost the resource. 

Which is exactly what I am saying! You have to assign a cost to DBC, and
compare
it to the alternatives.


> I am asserting
> 
> 1. It catches many bugs early. Evidence is personal experience and
> others
>    have posted the same.

This statement provides no guidance as to assigning cost. It catches
bugs "early" - but not earlier than analysis of the UML (a la Jezequel)
for example. Thus, if this were the only criteria, DBC would not be
the best alternative.

> 
> 2. It is not expensive adding in assertions. There is a huge overlap in
>    writting an assertion and producing a correct specification so it is
> not
>    wasted work.

"Expensive" is a relative term, and this statement provides no guidance
as
to the _comparison_ (vs. what baseline?) you are making. If you are
saying, "it is
less expensive than writing a poor specification," then we agree.
However,
DBC is not the only alternative to writing a good specification, nor is
writing a good specification the only factor in producing quality
software.

> You now have to say what ALL the other resources are and what the costs
> are
> in using them to justify your statement.

My statement is "you now have to say what ALL the other resources are
and
what the costs are in using the to justify the statement: DBC is the
best
alternative." I'm glad we agree on this point.

Remember, fellow debators: The affirmative (DBC) has the burden of
proof, not
the negative! See: the principles of (a) status quo and (b) inherency.

> 
> > Furthermore, if it detects 20% of the total system faults early, but
> > as a result there isn't sufficient resources to use other techniques
> > to find the resulting 80% that are left, then although you fixed the
> > 20% more cheaply than some other alternative, you have still delivered
> > a system with 80% of the faults remaining. So, coverage of the faults
> > by quantity is an issue as well as speed.
> 
> But you assume that the other resources will find the faults within your
> resource budget. What if you haven't the resources, which is your
> argument against
> DBC? Can you fit a quart in a pint pot?

No, no, Nanette! I make no such assumption. YOU have the burden of proof
here.
I don't have to _assume_ anything!

You're repeating my argument exactly. You _can't fit a quart in a pint
pot.
Therefore, DBC must not only be effective; it must the the _most_
effective
of the alternatives.

> You have assumed that using DBC uses up resources at such a rate that it
> stops you producing product.  Implementing DBC does not take up huge
> resources at the coding stage. 

"Huge..." what a (relative) concept!

> At the design stage you should be
> specifying what you want to have happen at the level required to
> implement DBC, so there is no additional resources going to waste here.

Now you have made a statement you will have to defend vigourously. This
statement says that DBC takes _zero_ resources more than the
alternatives.

> 
> DBC is cost effective

Prove it!

> as a method of producing quality code, and does
> not take major resources

Post some numbers!

> to achieve. Having a compiler help you makes a
> big difference, which is where the Eiffel part comes into play.

"big"... compared to what!

> 
> > Finally, even if DBC discovers 95% of the faults, but the remaining 5%
> > are the most serious, you still have a problem. So, the ability to
> > detect the most serious faults, as well as the quantity and speed, must
> > be evaluated vs. _all_ alternatives.
> 
> All programmers reading this, if offered a method that found 95% of bugs
> the first time they executed a particular path in their code, would
> welcome it. It leaves them with more resources to work on the hard 5%

Not if the effort to get the 95% fails to get the 5%.

By the way, _safety-critical_ programmers do not agree with this
statement.
I will gladly release software with several minor problems in order to
make sure I have found and corrected the critical problems.


> 
> > That's the common fallacy of the Eiffel advocate argument: "It's better
> > than what I was doing, so go use it." However, was what you were doing
> > before the best approach (excluding Eiffel)?
> 
> I was not necessarily using the ideal approach before, but I haven't
> made this statement.

Actually, you have. You just don't realize it.

> I have found in practice, that DBC as a method
> works. You have to propose other ways of achieving what DBC achieves,
> with less effort.

Again, it's not my argument to make. You have the burden of proof.

2+2=4, therefore God exists! Dispute it!

> 
> > > 2) Time spent else where.
> > > Is this the case? Some of it may be, but I believe if you cannot be
> > > rigourous about what your software is trying to do, by writing
> > > assertions, then you are unlikely to produce a quality system.
> >
> > Bad argument. You haven't proven that "writing software rigorously"
> > is _best_ done by "writing assertions." For that matter, you haven't
> > shown that the "writing" (coding?) phase is the most critical with
> > respect to producing a quality system. Consider, for example, that
> > there are approaches that can generate software with no manual coding...
> 
> Eh? (the last sentence)

Welcome to the world of autocoding! See http://www.ise.com for one
of many examples.

> 
> > > The
> > > effect of writing assertions overlaps with the design process. It is not
> > > wasted time, it just comes under a different heading. If your design
> > > process listed the assertions in the specification, would implementing
> > > them be a waste of effort?
> >
> > Yes, quite possibly, implementing assertions in the code may be both
> > ineffective and counter-productive, compared to other approaches,
> > depending
> > upon circumstances.
> 
> What other approaches? What evidence do you have? (PS I have read your
> critique)

What other approaches are there to quality software? Let us list the
ways:
Formal reasoning, use cases, patterns, cleanroom, N-version programming,
PSP, power CASE, FMET, autocoding, reuse... (everbody sing along!)

Without quantitative measures, you're preaching a religion, not
evaluating
a technique.

> > > 3) Simple software.
> > > You bet. The simpler the better. Occams Razor rules. Now here there is a
> > > split between DBC a la Eiffel and DBC, say in C++. In Eiffel it is
> > > simple.
> >
> > "Simple software" is not synonymous with "easiler to write code." The
> > question
> > is: is software with assertions less complex than software without
> > assertions.
> > Based on measures of merit such as lines of code, paths, etc. the answer
> > is
> > "no".
> 
> Given a complex solution and a simple solution to the same problem, the
> simple solution is the one with merit. Period.
> 
> If you don't assert your software, then you have to write test code to
> test the same assertions (specification) in order to satisfy yourself it
> is correct.

Test oracles, debugger scripts, reliability models, structural testing,
etc.
There is a whole world of means to "satisfy yourself it is correct"
beyond
writing test code (who does this sort of trash anymore, anyway? We
haven't
done it in at least 5 years!).

> DBC on your measures of merit stated above, when you take the whole
> system into account will be better.

Only if you're comparing it to an obsolete alternative (as you did).

> 
> > > In C++ it is hard, particularly with inheritance of assertions.
> >
> > Again, this is the fallacy of the argument. Why are you comparing Eiffel
> > to
> > C++? (The second fallacy: Why are you comparing languages when
> > discussing
> > DBC as a methodology?)
> >
> > > One common theme from Eiffel practitioners is their support for DBC.
> > > Why? They are simple to write.
> >
> > As a motivation, consider this:
> >
> > Here is a simple assertion:
> >
> > "The sun is hot."
> >
> > Now, put this in your code as a comment. Is the code simpler as a
> > result?
> > Put it in 1,000 times (simple to do with a text editor, right)? Is
> > the code simpler as a result?
> >
> > > Assertions affect timing in safety critical systems.
> > >
> > > Firstly it depends on the implementation. It is easy to envisage a
> > > system where the assertions are redundantly executed. But you would only
> > > want to do that if you were running with faulty software ?!*^�%
> > >
> > > I also find it worrying that systems are being used for safety critical
> > > apps where there is the possibility of a race or deadlock to occur.
> >
> > Then you would agree that it is critical to test systems to ensure that
> > such possibilities are not present, correct? And so, doing things which
> > mask the presence of such errors during testing are bad, right? So,
> > you agree with my argument!
> >
> > If we are to assume the system is implemented correctly, why bother with
> > assertions as a means to validate system correctness!
> 
> Because you want to be sure it is implemented correctly!

But you already assumed the system is implemented correctly!
Why are you now unsure!

> 
> > I am always amazed when people don't grasp this. It's not that the
> > system is designed intentionally to permit race or deadlock; the concern
> > is making sure that you didn't _inadvertantly_ do this! It's like
> > saying:
> >
> > "I also find it worrying that systems are being used for safety critical
> > apps where there is the possibility of faults to occur." as an argument
> > for not testing at all!
> >
> > > Compilation problems.
> > > These can occur in any system as you are aware. From discussions with
> > > some of the people involved in writting Eiffel compilers, the enabling,
> > > disabling of assertions has a very trivial implementation, which is very
> > > unlikely to go wrong.
> >
> > Define "unlikely". As I note in the paper, no evidence has been provided
> > regarding this issue. My argument is that, the more complexity added to
> > the software, the more likely the introduction of a fault due to a
> > compiler
> > bug.
> 
> Do some thinking about how you would implement DBC if you were a
> compiler writer. Consider how to enable, disable the assertions. Is this
> a 'hard problem' to solve?

Yes, based on real experience with real compilers.
(I know that real experience doesn't compete with theory, but I find it
comforting nonetheless :)

> 
> > Would you be willing to use a safety-critical system if the engineer
> > said,
> > "Well, we didn't actually test the code we fielded, but we tested code
> > that was pretty much the same."?
> 
> No. I would do the testing. I'm not stupid.

So much for zero cost DBC! You now get to do the testing twice.

Compare to just _one_ alternative: coding the assertions in a debugger
script using
real-time non-intrusive monitoring. Since the assertions are outside the
code, you can (1) test just once (since the assertions do not affect the
code) and (2) have the assertions reference objects not in the code
(e.g.
values in the external environment not directly available to the code
under test). These assertions can be generated from a test oracle hooked
up to a UML (or other notation) model of the system, if you like. They
can be incorporated into a "short form" of the spec for review. So,
compared
to DBC, they do the same thing (assuming you were not going to have
the assertions execute during production) and cost less. We do this
today,
so we know it works. 

Since we agree on comparitive cost as the criteria, you now have to show
why this
alternative costs more. (The negative has introduced an alternative
plan, with
no inherency! Incredible!)


> > > It has also be extensively tested in the field by
> > > end users.
> >
> > Insufficient. How many years had the Ariane IV IRS been flown before it
> > was installed in the Ariane V?
> 
> Agreed. But give yourself the choice.
> 
> a) Tested by software engineers.
> 
> b) Tested by software engineers and lots of users.

Invalid. Show that the Eiffel compiler meets the first clause of (a),
and to what extent. The true choice is:

a) Tested by software engineers to a known standard.
b) Used in various domains (not your own), and tested to some unknown
standard.

I'll be taking "a", please!


> 
> > > Do you trust your compiler? If not, you shouldn't be writing safety
> > > critical software with it. Period.
> >
> > Irrelevant. Much like saying, "I trust my development team, so I won't
> > test their products." Blindly "trusting" any part of the development
> > process is extremely dangerous.
> 
> See you argument above, I'll quote it again
> 
> * If we are to assume the system is implemented correctly, why bother
> with
> * assertions as a means to validate system correctness!
> 
> This is why assertions are good. You don't blindly trust.

Interesting that you quote your own argument to reinforce your own
argument! You are the one who assumed the system was implemented
correctly, remember? I was the one who pointed out this was a bad
idea.

Glad we could agree that we should not assume the compiler is
implemented
correctly!


> 
> > > Next I find some of the logic of your arguments here very weak.
> > > Paraphrasing. We have trouble testing safety critical systems, but we
> > > will use them anyway.
> >
> > If you have trouble with the argument, "real-world safety critical
> > systems
> > are difficult to develop and qualify," then I recommend you stay away
> > from all real-world safety critical systems.
> >
> > More to the point, the "paraphrasing" misses the argument. Here is
> > a better summary. "Keep it simple, stupid. Don't add complexity to the
> > product or the process unless it provides the best payoff vs. all
> > alternatives. Furthermore, _all_ alternatives have risks as well as
> > benefits; ignore anyone who says otherwise as a religious fanatic."
> 
> I'll agree with this point. The disagreement between us is over DBC
> being an aid to improving the process. From practical experience, I
> believe it does.

Without quantification, all we can do is bow our heads and pray at this
point.

Leaving the telephone off the hook is also an aid to improving the
process (this is a real result of a real study). Compare the costs
and benefits of this vs. DBC.

> I don't think that you have backed your arguments up with enough facts
> to justify your position.

Fortunately, I'm not the one advancing the assertion that DBC is the
best alternative. If I were, I would be worried.

> Now that is not saying your wrong, but part of
> the reason for havin a vocal discusion on the point is DBC advocates
> have taken an theory on board that they believe to be correct.
                                          *******

I "believe" that you have summarized my problem with your argument
quite neatly.


> As with
> all discusion on theory, you have to come up with examples that
> contradict, or a better theory.

Actually, no. The burden of proof is on the group advancing the
claim. All I've said is that you haven't proved your claim.

But, just for fun, I did give you a cheaper alternative, so you
can look at that if you want.

> The contradictions either show up
> missing parts of the theory, in which case it is improved, or you come
> up with a new theory which is better.

Actually, no. I think you dropped a few steps in the scientific method
(somewhere between "pose a hypothesis" and "accept or reject
hypothesis").

> Since you have strongly held
> views, I want find out what they are, since they might improve mine and
> others knowledge.

What makes you think I have strongly held views? For that matter, what
relevance is the _strength_ of the views? Sounds suspiciously like a
religious argument, which I find boring.

> 
> > > Hmmm.
> > >
> > > >   http://www.flash.net/~kennieg/ariane.html#s3.3
> > > >
> > > > There are approaches that can avoid such costs, particularly those of
> > > > 3.2.2 and 3.3 (by not requiring object code modification). 3.1.6 can
> > > > be mitigated through the use of techniques that minimize cost (e.g.
> > > > automated structural testing analysis).
> >
> > I note you didn't respond to this part. Hmmm indeed!
> 
> I'll look it up and reply.
> 
> >
> > In any case, I'm going to have a wonderful Labor Day, hope you have the
> > same. Just don't build any safety-critical systems that I might end
> > up using, OK?
> 
> Your pension probably depends on it. ;-) As I do when I go flying in
> Norfolk, lots of your products wizzing around the sky there.
> 
> --
> 
> Nick