Re: Writing better software was: Design by Contract (was Re: Interesting thread in comp.lang.eiffel)

["Robert I. Eachus" <rieachus@earthlink.net>]

Simon Brady wrote:
 
> Now that I think about it, it's wimpy to say "potentially" here.
> Software quality isn't just about lack of bugs, it's about _confidence_
> that there's a lack of bugs. So if they cut testing (and hence lowered
> their confidence in the software), I would argue that they're still
> delivering a lower-quality product even if it happened to be bug-free.
> Of course, some might say such a purist approach is commercially naive,
> but then that's why I'm not a commercial programmer :-)

   There is a very subtle, but insidious fallacy in the above.  Not to
do ANY testing is quite likely to result in catastrophic failure.  On
the other hand, testing should always be one part, and often not the
largest part, of a well planned quality assurance program.  In fact, on
a well managed project, anything found at preliminary design review time
to be potentially difficult or impossible to test should be flagged for
further design effort in an attempt to
reduce overall cost.  Sometimes you need to do those 'expensive' tests,
that would have been cheap at almost any price on Ariane 5.  But if you
can change the design to make testing easier without impacting other
requirements, do so.

   For another example, it is costly to create a thread level model of
control flow in a large system.  But if it is necessary to do so, often
for timing analysis reasons, the model can also be used to do other
analysis of project software.  (Of course, this requires that the model
be kept up to date during development.)  On one project where this was
done, it was determined that there was a race condition that could
transition the system to a subset of states that could not be left. 
(This was done using hierarchical colored Petri nets.)  Fixing this bug,
which could have remained undiscovered for the life of the system, let
alone througout testing, was trivial.  Of course, if it had been
discovered in a fielded system, it would probably have caused a plane to
crash during landing.  No amount of testing could have produced that
level of trust
in the software, even if it had resulted in the discovery of that bug.