Re: Bugs or Mistakes?

[Richard D Riehle <laoXhai@ix.netcom.com>]

My apologies for boring some of you with this thread.  My thanks
to those of you have sent email agreeing with me.  As Robert Dewar
has noted, this discussion has the potential for becoming "silly."
Some have taken it seriously.   For the record, I realize that almost
everyone takes programming errors seriously whether they are called
bugs, mistakes, or cucumbers.  

In article <383086EC.372545D1@mitre.org>,
	"Robert I. Eachus" <eachus@mitre.org> wrote:

>     If you design a building, run it through a finite-element
>simulation before signing the drawings, and find that the bolts
>specified are inadequate, this is a bug in your design, and you fix it
>and repeat the simulation.  

Reading your full text, I find we agree on a number of points. 
However, I would still classify the above example as an engineering 
error.  It still needs to be corrected.  There is no identifiable 
entity analogous to a "bug" lurking in the design.   

>If the simulation does meet spec, but
>someone supplies or uses sub-standard materials or doesn't build to
>spec, then they are liable.  (The suspended walkway collapse in St.
>Louis and the DC-10 crash on takeoff at O'Hare fall into this category.)

In this case, I would say there is an error in the specification.  

>     But if the bolts do meet spec, and the actual conditions don't
>match the simulation, then you have the case of the John Handcock
>building in the Back Bay.  The wind sometimes blew UP the building, and
>sucked out windows.  The solution was to replace the windows with
>stiffer windows at significant cost.  

Ignorance of the laws of physics is no excuse.  :-)  Seriously, this
kind of thing is quite common.  In physical engineering, we are often
faced with incomplete understanding of the properties of common 
materials under new conditions.   Is it a mistake to go ahead and 
put people's lives at risk even when we are working off our best
guess?   

Who was at fault?  

Really a different question.  This is why insurance companies sometimes
use an "act of God" clause in their policies.  
HmmmMMMMmmmm. God is a bug?

>In this case no
>one.  The engineers had not been negligent, they had even tested models
>in a wind tunnel.  But the models had not been large enough--and complex
>enough--to catch the effects of such things as the fresh water in Back
>Bay vs. salt water in the harbor.  The result was that the Handcock
>tower was uniquely placed to trigger the uninversion of atmospheric
>inversions, and create local updrafts that exceeded the building's
>specifications.  The Galloping Gertie bridge failed due to a similar
>local trigger, but in that case there was negligence.  The winds were
>there before the bridge was built.

No engineer would ever claim that that the data sheets were always
adequate.  In some cases the engineer will make a mistake in using
the data.  Other times, it will be insufficient testing.  Still 
other situations are governed by the trade off of economics versus
the cost of a lawsuit.  For example,

A famous case involving the design of an automobile led the engineers
to realize that, under some circumstances, the gas tank could explode
and kill the occupants of the car.   The problem was handed off to the
actuarial folks who determined that it would cost more to reengineer
the car than pay off the lawsuits, if any, that would develop from
such accidents.  Who made that mistake?  Is that a bug or an mistake?
Is it simply a _feature_ as some software humorists have suggested?

>    Now we can discuss when a bug is not a mistake.  A bug that never
>gets to the field can be the result of a mistake, but often a bug
>detected early is just that, a bug.  

In the words of William Gates, "We respectfully disagree..."  It is
simply an error that goes through a succession of corrections.

>For example, in Ada it is much more
>productive to let the compiler worry about elaboration order, and only
>consider it as a problem if the compiler can't find a solution.  

The compilers I use give me error messages.  If the program fails to
elaborate correctly, there is still a mistake in my design.  

>If
>judicious use of elaboration pragmas can sort it out, was there a
>mistake?  Not really.  It is much more efficient to allow the compiler
>to select an order to the extent the order irrelevant as long as it is
>within the rules.  

The mistake, here, would be failure to test the results for correct
execution.  Correct execution will probably not occur in an incorrectly
compiled [read elaboration order is incorrect] program.

>If you leave off a necessary pragma Elaborate_Body,
>and as a result use an uninitialized variable, that is a mistake, even
>if you do catch it during unit test.

No argument with this.  I fully agree.

>    What is my point in all this?  A bug may have resulted from many
>mistakes or none.  Going around assigning blame for bugs is
>conterproductive until you have found what is wrong and why.  

I fully agree. It is always counterproductive to engage in 
the blame game during development. OTOH, it is not counterproductive
to simply acknowledge that, as someone else noted, there are defects
in the program.  The defects did not create themselves.  They were
put there by someone through ommission or commission.  The defects
may be a function of inexperience, sloppiness, designing in an 
entirely unknown domain, or ...    There is still no creature 
spontaneously springing to life, worrying its way through 
the code.  

Sometimes the mistake is all the way back in the compiler.  We once
had a C compiler that had a strange behavior with certain uses of
pointer arithmetic.  A bug in the compiler?  No. A serious mistake
made by the developers of the compiler.  

It can
>also be counterproductive to label as mistakes errors in draft
>documents, schedules, or software.  

I think this is the very place they should be so identified. The
earlier in the process we identify errors as errors, the greater
the potential for eliminating them before they become part of the
actual design.

The process of accountability should
>only begin when someone signs off on something--or alternatively, when
>it is accepted or fielded without competent techincal approval.

You and I have both sent a lot of documents through the approval
process.  In the early stages, the differences of opinion are 
niether mistakes or bugs.  Often, they are simply differences 
in how we perceive the goals and suggested solutions.  So in this
general principle I agree with you.  

It does seem clear that we both agree that, once a software 
system is deployed, it is too late to euphemize the defects
as bugs.  At this level, given our increasingly litigious
society, we'd better get used to the idea that some parasitic
bottom dweller is going to rip open our brainchild, probing 
for the rotting tissue that proves we made some mistake. In
this regard, your question, "Who was at fault," will not be
satisfied by the answer, "Oh, it was just another bug."

Richard Riehle
http://www.adaworks.com