Re: Win2000 has 63,000 'defects'

["Robert I. Eachus" <rieachus@earthlink.net>]

Larry Kilgallen wrote:
 
> If one considers the possibility it is an _estimate_, based on typical
> industry figures, the very thought of only 1.5 defects per KLOC is out
> of the question.  Some people I know use 5 defects per KLOC for code
> that has been revised and 10 defects per KLOC for "new" code.  Reports
> indicate that Windows 2000 has a lot of new code.

   When I was at Stratus, there was a very common understanding.  I
think I heard it "officially" expressed at more than one engineering
meeting, but never saw any underlying documentation.
Basicly it goes like this, if your (OS, real-time, compiler, transaction
processing, etc.) code has more than one bug per thousand lines, it
could never be made robust enough to ship.  If your code has less than
one bug per 10,000 lines, it will cost less to get it to 1 bug per
million lines than to get the bug 1 per 1000 lines code to one bug per
2000.

   Of course, it cost at least twice as much to write 1 bug per 10,000
line code than to write one bug per thousand lines code.  But writing
the buggier code at Stratus for any critical code (read any part of the
OS or the tools to build it) was worthless.  So we had to spend the time
up front to get it right.  The key measure of quality was the MBTF
during beta test for new OS versions, and we aimed for about one year
when we released.  MBTF was much higher on operational systems, since
the customers would run the beta test versions only on development
machines.  By the time the customers applications were fielded, they had
run into most of the OS bugs that they could on the development
systems.  In any case, we never got to one bug per million lines, but we
were well above one per 10000 lines of fielded code.  (Most of this code
was writen in PL/I. C was only used on I/O cards with less than 64K of
memory.)

   So there are really two styles of programming.  One, which may or may
not be fine for computer games, etc., relies on debugging quality in. 
The other relies on building it in in the first place.  Most Ada
programmers are used to the second style, which is why we seldom (if
ever!) use a debugger.