Re: Win2000 has 63,000 'defects'

Re: Win2000 has 63,000 'defects'

[Brian Rogoff]

On 15 Feb 2000, Hyman Rosen wrote:
> kilgallen@eisner.decus.org (Larry Kilgallen) writes:
> > Why must you be so argumentative ?
> > I know how to spell the words "I have proof", and would have done so
> > if that were the case.
> 
> You're probably right. I was just defending the honor of
> "my favorite language" :-)

I'm surprised that you even have a favorite. ;-)

> It would be interesting to see a defect analysis similar to that
> Lucent study (which was for old pre-ANSI C code) done for a large Ada
> project, to see what kinds of bugs show up there. 

That would indeed be interesting. It would also be interesting to find out 
roughly what fraction of the installed base of C code is "new ISO C" rather 
than bad old K&R. My experience is that a lot of brand new C code out there 
is still pre-ANSI, meaning roughly "prototype free". Similarly, I'd like
to find out what fraction of C++ code is truly ISO C++ compliant. When I
was using C++ a lot (mid 90's) it seemed that no two compilers would
handle the same subset of templates. It would have helped C++ to have an
Ada like test suite so that there would be some minimal level of agreement.

-- Brian

Dang! (was Re: Win2000 has 63,000 'defects')

[Mike Silva]

Arrgh!  I've been bitten by wrap-around!  The doc_key is 337, not 3.  I'm
sending it again with the window widened a bit, but if it still doesn't put
the entire URL on one line you'll have to type the 37 on the end.

Mike Silva wrote in message ...
>

http://www.rational.com/sitewide/support/whitepapers/dynamic.jtmpl?doc_key=3
37

Re: Win2000 has 63,000 'defects'

[Keith Thompson]

"Marin D. Condic" <mcondic-nospam@quadruscorp.com> writes:
[...]
> That's an interesting factoid. I'm wondering what sort of automatic
> scanner they might have used? Something like lint? I'd guess if this is
> the case that a) there are certainly more errors than would be found by
> the scanner and b) the errors that are found are probably over-reported.
> (Think of how missing one period in a Cobol program could cascade error
> messages from the compiler for days.)

The latter is doubtful.  Presumably the code being scanned at least
compiles; warnings don't typically cascade the way syntax errors do.

-- 
Keith Thompson (The_Other_Keith) kst@cts.com  <http://www.ghoti.net/~kst>
San Diego Supercomputer Center           <*>  <http://www.sdsc.edu/~kst>
Welcome to the last year of the 20th century.

Win2000 has 63,000 'defects'

[Gautier]

Urging developers to clean up their code, a Microsoft exec
says: 'How many of you would spend $500 on a piece of
software with over 63,000 potential known defects?' It
ships Thursday.

Not everyone will be having fun at Microsoft Corp. next week. While
the software giant and its partners celebrate the arrival of Windows
2000 on Thursday, Feb. 17, hundreds of members of the Windows
development team will be busy cleaning up the mess.

---> http://www.zdnet.com/zdnn/stories/news/0,4586,2436920,00.html?chkpt=zdnntop

Someone to sell or install them poor an Ada compiler ? ;o) G.

Re: Win2000 has 63,000 'defects'

[Larry Kilgallen]

In article <t7u2ja5lpx.fsf@calumny.jyacc.com>, Hyman Rosen <hymie@prolifics.com> writes:
> Gautier <gautier.demontmollin@maths.unine.ch> writes:
>> Urging developers to clean up their code, a Microsoft exec
>> says: 'How many of you would spend $500 on a piece of
>> software with over 63,000 potential known defects?' It
>> ships Thursday.
>> 
>> Not everyone will be having fun at Microsoft Corp. next week. While
>> the software giant and its partners celebrate the arrival of Windows
>> 2000 on Thursday, Feb. 17, hundreds of members of the Windows
>> development team will be busy cleaning up the mess.
>> 
>> Someone to sell or install them poor an Ada compiler ? ;o) G.
> 
> Why do you think the defects have anything to do with the language
> used to develop Windows 2000? Do you know that the defects are
> coding errors, as opposed to being design errors or unimplemented
> features?

I am convinced that a great many of their defects are coding issues.

I am not convinced that fixing all the coding issues (by any method)
would not still leave a great many other defects, probably enough to
make the fact that coding issues had been fixed invisible to the user.

I am not at all convinced that defects enumerated by Microsoft (or
any other vendor) cover a reasonable fraction of the total defects
in the software.

Re: Win2000 has 63,000 'defects'

[Brian Rogoff]

On 15 Feb 2000, Hyman Rosen wrote:

> kilgallen@eisner.decus.org (Larry Kilgallen) writes:
> > I am convinced that a great many of their defects are coding issues.
>>
> ... snip ...
> 
> Do you have evidence for any of these convictions?

For this one

http://www.lucent.com/minds/techjournal/apr-jun1998/abstract.html

The paper by Yu on "Software Fault Prevention...". 

This also agrees with my own (biased, personal, insert disclaimer here...) 
observations. 

Interestingly, the Erlang guys work in a very similar space and the
language they push as leading to robust systems is dynamically typed. 
Go figure. 

-- Brian
 

Re: Win2000 has 63,000 'defects'

[Larry Kilgallen]

In article <87k8k69qm9.fsf@deneb.cygnus.argh.org>, Florian Weimer <fw@deneb.cygnus.argh.org> writes:

> 63,000 `potential known defects' (whatever this means)

A news report said that number was achieved by running an automated
scanner over the software.  Certainly there are many defects that
would not be uncovered by such a scanner.

Re: Win2000 has 63,000 'defects'

[Larry Kilgallen]

In article <t7r9ee5ihu.fsf@calumny.jyacc.com>, Hyman Rosen <hymie@prolifics.com> writes:

> Do you have evidence for any of these convictions?

Why must you be so argumentative ?

I know how to spell the words "I have proof", and would have done so
if that were the case.

Re: Win2000 has 63,000 'defects'

[Ed Falis]

"Hyman Rosen" <hymie@prolifics.com> wrote in message > It would be
interesting to see a defect analysis similar to that
> Lucent study (which was for old pre-ANSI C code) done for a large Ada
> project, to see what kinds of bugs show up there. I wonder if ACT
> keeps statistics on that sort of thing, and if so, whether they would
> be willing to comment.

Well, there is that Steve Zeigler study on the Verdix compiler technology.
Probably someone here has the URL for it.

- Ed

Re: Win2000 has 63,000 'defects'

[Hyman Rosen]

Gautier <gautier.demontmollin@maths.unine.ch> writes:
> Urging developers to clean up their code, a Microsoft exec
> says: 'How many of you would spend $500 on a piece of
> software with over 63,000 potential known defects?' It
> ships Thursday.
> 
> Not everyone will be having fun at Microsoft Corp. next week. While
> the software giant and its partners celebrate the arrival of Windows
> 2000 on Thursday, Feb. 17, hundreds of members of the Windows
> development team will be busy cleaning up the mess.
> 
> Someone to sell or install them poor an Ada compiler ? ;o) G.

Why do you think the defects have anything to do with the language
used to develop Windows 2000? Do you know that the defects are
coding errors, as opposed to being design errors or unimplemented
features?

Re: Win2000 has 63,000 'defects'

[Florian Weimer]

Gautier <gautier.demontmollin@maths.unine.ch> writes:

> Urging developers to clean up their code, a Microsoft exec
> says: 'How many of you would spend $500 on a piece of
> software with over 63,000 potential known defects?' It
> ships Thursday.

63,000 `potential known defects' (whatever this means) isn't too bad for
a software product consisting of over 30 million lines of code, is it?

Re: Win2000 has 63,000 'defects'

[Hyman Rosen]

kilgallen@eisner.decus.org (Larry Kilgallen) writes:
> I am convinced that a great many of their defects are coding issues.
> 
> I am not convinced that fixing all the coding issues (by any method)
> would not still leave a great many other defects, probably enough to
> make the fact that coding issues had been fixed invisible to the user.
> 
> I am not at all convinced that defects enumerated by Microsoft (or
> any other vendor) cover a reasonable fraction of the total defects
> in the software.

Do you have evidence for any of these convictions?

Re: Win2000 has 63,000 'defects'

[Marin D. Condic]

Florian Weimer wrote:
> 
> 63,000 `potential known defects' (whatever this means) isn't too bad for
> a software product consisting of over 30 million lines of code, is it?

From my experience, this number is probably derived from some internal
change request tracking system. That means that there are likely 63k
submittals of one form or another wherein there may be a large number of
duplications.

Of course not all bugs are equal in the eyes of their creator. The
"defects" could range from "Correct the spelling of XYZ in a comment in
module ABC" all the way up to "If I click on this icon the disk drive is
erased and the whole system locks up."

Still in all, I think a reasonable presumption about a large system
written in a weakly typed language would be that some significant
portion of the known defects would never occur if it had been programmed
in a more rigorous language such as Ada. In the past, I have collected
enough data on similar project comparisons to be able to convince myself
and others that Ada resulted in a decrease of defects by a factor of
four over similar systems with weak checking.

MDC
-- 
=============================================================
Marin David Condic   - Quadrus Corporation -   1.800.555.3393
1015-116 Atlantic Boulevard, Atlantic Beach, FL 32233
http://www.quadruscorp.com/
m c o n d i c @ q u a d r u s c o r p . c o m

Visit my web site at:  http://www.mcondic.com/

"Capitalism without failure is like religion without sin." 
        --  Allan Meltzer, Economist 
=============================================================

Re: Win2000 has 63,000 'defects'

[Hyman Rosen]

kilgallen@eisner.decus.org (Larry Kilgallen) writes:
> Why must you be so argumentative ?
> I know how to spell the words "I have proof", and would have done so
> if that were the case.

You're probably right. I was just defending the honor of
"my favorite language" :-)

It would be interesting to see a defect analysis similar to that
Lucent study (which was for old pre-ANSI C code) done for a large Ada
project, to see what kinds of bugs show up there. I wonder if ACT
keeps statistics on that sort of thing, and if so, whether they would
be willing to comment.

Re: Win2000 has 63,000 'defects'

[Hyman Rosen]

"Ed Falis" <falis@gnat.com> writes:
> Well, there is that Steve Zeigler study on the Verdix compiler technology.
> Probably someone here has the URL for it.

<http://wuarchive.wustl.edu/languages/ada/ajpo/docs/reports/cada/cada_art.html>

Unfortunately, that paper has no information as to the nature of the errors
found in either the C or the Ada code.

Re: Win2000 has 63,000 'defects'

[Gautier]

Hyman Rosen:

> > Someone to sell or install them poor an Ada compiler ? ;o) G.

> Why do you think the defects have anything to do with the language
> used to develop Windows 2000? Do you know that the defects are
> coding errors, as opposed to being design errors or unimplemented
> features?

That's not so simple... Some coding errors may be catched at compile time
in one language and appear as an bad crash on an user's screen months
later when coded in another one. As complexity grows, uncatched (coding
or design) bugs tend to combine with other. In some way the "shape"
of a bug disappears rapidly with the complexity and the time elapsed
after the coding.

Some concrete examples. In the code I'm developing (a comparison of
finite element methods for multiple partial differential equations)
there are *lots* of coding issues that have been catched by the compiler
because I'm using Ada's rich typing: simple things like exchanging
the equation and node index, and so on.

You forget that design features help avoiding other *bigger lots* of coding bugs.
Back to the example (a tiny excerpt)...

...
procedure FT_elem(gelem: t_element) is

  nsd:      constant positive:= spatial_dimension(gelem);
  iel:      constant positive:= nodes_per_element(gelem);
  nb_faces: constant positive:= faces_per_element(gelem);

  subtype sp_vector  is vector(1..nsd);         -- vecteur spatial
  subtype sp_matrix  is matrix(1..nsd,1..nsd);  -- matrice spatiale (codecond)
  subtype el_array   is vector(1..iel);         -- tableau sur un element
  subtype el_iarray  is ivector(1..iel);        -- idem, composantes entieres
  subtype el_matrix  is matrix(1..iel,1..iel);  -- matrice elementaire
  subtype el_grad    is matrix(1..iel,1..nsd);  -- gradients sur l'element
...

Needless to comment!

An extreme but true example: a program (some environmental predictor) had
a strange crash recently. There was a line in the 15-year old Fortran code that
"almost" never occured, where a variable name had 2 letters exchanged!...

2 things I'm sure of:
 - every mean is good to track a bug, and I prefer to have a compile-time error
   than a day of Sherlock-Homesque bug tracking with a debugger, after a week of
   disfunctionment
 - the bugfest tends to grow exponentially or factorially or something like that
   (specialists know that better) with code length (and its ratio of bugs...).

BTW you can make a naive probabilistic calculation. Say you have 1000-line
modules. Each of them has a probability of 0.00001 of crashing per hour because of 
an uncatched bug. Now you have a 30,000 module program (say Windows 2000) that
uses all of them during an hour. Each module can crash or not during the hour
independently of each other (a generous hypothesis since bugs normally help
each other). Then, the probability of a crash for the whole program during an hour
is 1.0 - (1.0-0.00001) ** 30_000 = 0.259 (26%).  8-|

Gautier

_____\\________________\_______\_________
http://members.xoom.com/gdemont/gsoft.htm

Windows TP (Re: Win2000 has 63,000 'defects')

[Vladimir Olensky]

Gautier wrote in message <38A989B7.2D4D6B56@maths.unine.ch>...
>Urging developers to clean up their code, a Microsoft exec
>says: 'How many of you would spend $500 on a piece of
>software with over 63,000 potential known defects?' It
>ships Thursday.


And how about that :

MICROSOFT ANNOUNCES BETA RELEASE OF WINDOWS TP

REDMOND, WA  BUSINESS WIRE - Microsoft Corp. announced
yesterday  that a beta release of Windows TP, the telepathic operating
system, was released to 1,500 test sites worldwide.

Developed using the soon-to-be released Microsoft C for Neurons,
Windows TP bypasses awkward user interfaces by interacting directly
with the user's brain. Using the Microsoft MindMouse, users can visualize
images in their mind, and the application associated with that image (or
"thought icon") is executed. Users can visualize pictures to create
 Windows Bitmap images, or think text directly into Windows applications.
 Windows TP is fully compatible with all previous versions of Windows.

Data stored under Windows TP can be copied into the user's short- term
memory (the Windows TP Clipboard), or transferred directly into the user's
long-term memory using Windows' new 32-bit Direct Neuron Access
technology. Users can then plug into other Windows TP systems to transfer
the data.

Microsoft also announced the first application developed exclusively for
Windows TP. CyberMail is a mental mail system designed to transfer
messages by thought. Users visualize the person or company logo they
want to send a message to, followed by the message to send. Microsoft
has had a beta version of the application in use for several months.

CONTACT: Microsoft Corporation Liz Wagthor, A short, dumpy lady, with
shiny red hair, and a really gross mole growing on the right side of her
lip.
A blue tattoo on her right arm says, "Billy G's the Man for Me.")

Regards,
Vladimir Olensky
\
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
\
P.S. Please do not take this too seriously :-)
This is an old joke but who knows may be in some
near future this may come true.

Re: Win2000 has 63,000 'defects'

[Samuel T. Harris]

"Stanley R. Allen" wrote:
> 
> As usual with American news broadcasts, the story was full of
> brief, inexplicable snippets of video taken by the reporting
> crew when they went to the site to cover the story.  In this
> instance, they had gone to the place where the supercomputer
> was and had taken some shots of the screens streaming with
> X-Window outputs.  Naturally there was no explanation of what
> was being displayed, and the shot didn't last for more than 5
> seconds before they were back to showing trees and power lines
> felled by the storms.  But in that brief sliver of time, I was
> able to perceive one of the lines of output:
> 
>         segmentation violation: core dumped
> 

I love that!
Ada, the language of choice for Poor Richard's Almanac!

-- 
Samuel T. Harris, Principal Engineer
Raytheon, Aerospace Engineering Services
"If you can make it, We can fake it!"

Re: Dang! (was Re: Win2000 has 63,000 'defects')

[Larry Kilgallen]

In article <m3itznrh1v.fsf@kiuk0156.chembio.ntnu.no>, Preben Randhol <randhol@pvv.org> writes:
> "Mike Silva" <mjsilva@jps.net> writes:
> 
> | Arrgh!  I've been bitten by wrap-around!  The doc_key is 337, not 3.  I'm
> | sending it again with the window widened a bit, but if it still doesn't put
> | the entire URL on one line you'll have to type the 37 on the end.
> [...]
> | http://www.rational.com/sitewide/support/whitepapers/dynamic.jtmpl?doc_key=3
> | 37
> 
> Present the links as this:
> 
> <URL: http://www.rational.com/sitewide/support/whitepapers/dynamic.jtmpl?doc_key=337>
> 
> then at least my mail-program (GNUS) is able to get the right link.

Present the links without any adjacent non-whitespace (just like you did)
and then _my_ computer gets it right.

Re: Win2000 has 63,000 'defects'

[Gautier]

Florian:

> > 63,000 `potential known defects' (whatever this means) isn't too bad
> > for a software product consisting of over 30 million lines of code, is
> > it?

Maybe you are true. But I doubt that the power of a bug -whatever it is-
is diluted by the number of lines. A fatal bug won't crash the system softer
if you add 30 million other lines...

Ted:

> Personally I agrree with that. However, ZDNet has an interesting article
> disagreeing at
> http://www.zdnet.com/zdhubs/stories/special2000/0,9968,2439261,00.html .
> I'm not sure I agree with that author's implication that its possible to
> write large sotware that is bug-free. But he goes into the economic cost
> of all those bugs, and the fact that most of the bugs were in a very few
> of the modules. Its one of the best unintentional bits of Ada advocacy
> I've ever seen.

It is quite reassuring that one finds "57 percent of the software errors in
only 7 percent of the studied modules" (the article). But it doesn't tell how
the 43 other % are distributed... (see my little probabilistic simulation
in another sub-thread). According to the stability of past Windows releases
"the good one" will be out in 2001 or 2002...

-- 
Gautier

_____\\________________\_______\
http://members.xoom.com/gdemont/

Re: Win2000 has 63,000 'defects'

[Charles Hixson]

Well, they may be design errors, but they aren't all merely unimplemented
features  (partially implemented, perhaps) as some of them are designated
as potentially serious problems.

Hyman Rosen wrote:

> Gautier <gautier.demontmollin@maths.unine.ch> writes:
> > Urging developers to clean up their code, a Microsoft exec
> > says: 'How many of you would spend $500 on a piece of
> > software with over 63,000 potential known defects?' It
> > ships Thursday.
> >
> > Not everyone will be having fun at Microsoft Corp. next week. While
> > the software giant and its partners celebrate the arrival of Windows
> > 2000 on Thursday, Feb. 17, hundreds of members of the Windows
> > development team will be busy cleaning up the mess.
> >
> > Someone to sell or install them poor an Ada compiler ? ;o) G.
>
> Why do you think the defects have anything to do with the language
> used to develop Windows 2000? Do you know that the defects are
> coding errors, as opposed to being design errors or unimplemented
> features?

Re: Win2000 has 63,000 'defects'

[Ted Dennison]

In article <87k8k69qm9.fsf@deneb.cygnus.argh.org>,
  Florian Weimer <fw-usenet@deneb.cygnus.argh.org> wrote:
> Gautier <gautier.demontmollin@maths.unine.ch> writes:
>
> > Urging developers to clean up their code, a Microsoft exec
> > says: 'How many of you would spend $500 on a piece of
> > software with over 63,000 potential known defects?' It
> > ships Thursday.
>
> 63,000 `potential known defects' (whatever this means) isn't too bad
> for a software product consisting of over 30 million lines of code, is
> it?

Personally I agrree with that. However, ZDNet has an interesting article
disagreeing at
http://www.zdnet.com/zdhubs/stories/special2000/0,9968,2439261,00.html .
I'm not sure I agree with that author's implication that its possible to
write large sotware that is bug-free. But he goes into the economic cost
of all those bugs, and the fact that most of the bugs were in a very few
of the modules. Its one of the best unintentional bits of Ada advocacy
I've ever seen.


--
T.E.D.

http://www.telepath.com/~dennison/Ted/TED.html


Sent via Deja.com http://www.deja.com/
Before you buy.

Re: Win2000 has 63,000 'defects'

[Preben Randhol]

Hyman Rosen <hymie@prolifics.com> writes:

| kilgallen@eisner.decus.org (Larry Kilgallen) writes:
| > I am convinced that a great many of their defects are coding issues.
| > 
| > I am not convinced that fixing all the coding issues (by any method)
| > would not still leave a great many other defects, probably enough to
| > make the fact that coding issues had been fixed invisible to the user.
| > 
| > I am not at all convinced that defects enumerated by Microsoft (or
| > any other vendor) cover a reasonable fraction of the total defects
| > in the software.
| 
| Do you have evidence for any of these convictions?

Usually one do _not_ spot all errors in a program. So if Microsoft has
spotted 63k errors out of which 27k are "real errors" (as said by M$),
I would think that there are probably far more bugs that has not been
detected. Other than that one can look at all the other software
Microsoft has produced and the quality of it.

What I cannot understand is that it is possible to shipping a product
you know has so many bugs and charge money for. Especially in the US
of A. Why companies do not sue Microsoft is beyond me.   

-- 
Preben Randhol -- [randhol@pvv.org] -- [http://www.pvv.org/~randhol/]     
         "Det eneste trygge stedet i verden er inne i en fortelling." 
                                                      -- Athol Fugard

Re: Dang! (was Re: Win2000 has 63,000 'defects')

[Preben Randhol]

"Mike Silva" <mjsilva@jps.net> writes:

| Arrgh!  I've been bitten by wrap-around!  The doc_key is 337, not 3.  I'm
| sending it again with the window widened a bit, but if it still doesn't put
| the entire URL on one line you'll have to type the 37 on the end.
[...]
| http://www.rational.com/sitewide/support/whitepapers/dynamic.jtmpl?doc_key=3
| 37

Present the links as this:

<URL: http://www.rational.com/sitewide/support/whitepapers/dynamic.jtmpl?doc_key=337>

then at least my mail-program (GNUS) is able to get the right link.

-- 
Preben Randhol -- [randhol@pvv.org] -- [http://www.pvv.org/~randhol/]     
         "Det eneste trygge stedet i verden er inne i en fortelling." 
                                                      -- Athol Fugard

Re: Win2000 has 63,000 'defects'

[Dale Pontius]

In article <150220001931201946%emery@grebyn.com>,
        David Emery <emery@grebyn.com> writes:
...
> On a related note, around 1990 or so, MITRE did a study on life-cycle
> costs of Ada83 vs other languages.  At that point, we had many
> large systems well into development, with few that had been deployed
> and were in real maintenance.
>
> The result:  Ada maintenance costs were shown to be linear on KSLOC.
> This is in stark opposition to Boehm and similar studies that showed
> cost to maintain code was exponental on KSLOC.

Now let's take these facts, and tie them back to the original topic
of the thread. Supposedly, Win2k has about 35MLOC. I've heard numbers
between 30 and 40 MLOC, pick one and let's not haggle. I'm also under
the impression that about 2/3 of that is enhancement of the stable
NT4 code base, and about 1/3 of that is new.

Presumably there are bodies of knowledge that indicate defect rates
on mature and new code.

Apply that knowledge to what we know of Win2k, and what does the
estimate come out at? Is 63000 defects a civil number for an estimated
24MLOC of old code and 12MLOC of new code? At this point, I'm not
saying anything good or bad about Microsoft. I'm merely questioning
the historical perspective on a body of code of this size.

Dale Pontius
NOT speaking for IBM

Re: Win2000 has 63,000 'defects'

[Joe Wisniewski]

In article <88hbpp$j4i$1@news.btv.ibm.com>,
  pontius@twonky.btv.MBI.com.invalid (Dale Pontius) wrote:
> In article <150220001931201946%emery@grebyn.com>,
>         David Emery <emery@grebyn.com> writes:
> ...
> > On a related note, around 1990 or so, MITRE did a study on
life-cycle
> > costs of Ada83 vs other languages.  At that point, we had many
> > large systems well into development, with few that had been deployed
> > and were in real maintenance.
> >
> > The result:  Ada maintenance costs were shown to be linear on KSLOC.
> > This is in stark opposition to Boehm and similar studies that showed
> > cost to maintain code was exponental on KSLOC.
>
> Now let's take these facts, and tie them back to the original topic
> of the thread. Supposedly, Win2k has about 35MLOC. I've heard numbers
> between 30 and 40 MLOC, pick one and let's not haggle. I'm also under
> the impression that about 2/3 of that is enhancement of the stable
> NT4 code base, and about 1/3 of that is new.
>
> Presumably there are bodies of knowledge that indicate defect rates
> on mature and new code.
>
> Apply that knowledge to what we know of Win2k, and what does the
> estimate come out at? Is 63000 defects a civil number for an estimated
> 24MLOC of old code and 12MLOC of new code? At this point, I'm not
> saying anything good or bad about Microsoft. I'm merely questioning
> the historical perspective on a body of code of this size.
>
> Dale Pontius
> NOT speaking for IBM
>

Also consider ..."
   Categorization of the "defects". If the 63K defects are
   accurate and account for EVERY possible "fix" that may or may
   not be a "defect" or even necessary such as a separate defect for
   every typo in the help file, ..., 63K may not be that big a deal,
   especially if that number includes enhancements, which are often
   included in normal defect tracking, just categorized differently.

   In fact one _could_ make the argument that this is a real step up
   wrt turning Windows releases into a real software engineering
   effort. That is, identify every possible thing wrong that has
   shown up in beta, not matter how trivial it may seem right now.

Joe


Sent via Deja.com http://www.deja.com/
Before you buy.

Re: Win2000 has 63,000 'defects'

[Nick Roberts]

I would suggest that a bug figure for an overall operating system is not
very meaningful. How many of those bugs apply to critical code? How many to
totally unimportant code (e.g. games)?

The total number of lines of code may actually be a much more meaningful
statistic than the number of putative bugs, when it comes to evaluating the
reliability of Windows 2000 (or parts thereof). If the figures indicate
serious 'bloat', this indicates poorly programmed software at a fundamental
level.

In fact, the citation of Windows 2000 being written in 30-40 million lines
of code raises a further question in my mind: which programming languages
does this figure cover, and in what proportions?

--
Nick Roberts
http://www.adapower.com/lab/adaos

Re: Win2000 has 63,000 'defects'

[Larry Kilgallen]

In article <88ma3c$p6a$1@nnrp1.deja.com>, Joe Wisniewski <wisniew@acm.org> writes:

> Also consider ..."
>    Categorization of the "defects". If the 63K defects are
>    accurate and account for EVERY possible "fix" that may or may
>    not be a "defect" or even necessary such as a separate defect for
>    every typo in the help file, ..., 63K may not be that big a deal,
>    especially if that number includes enhancements, which are often
>    included in normal defect tracking, just categorized differently.
> 
>    In fact one _could_ make the argument that this is a real step up
>    wrt turning Windows releases into a real software engineering
>    effort. That is, identify every possible thing wrong that has
>    shown up in beta, not matter how trivial it may seem right now.

Ignoring the previous discussion in this topic of how the 63,000 came
about, I doubt the ability of anyone to totally enumerate the defects
in a body of code this large.  This cannot be a total exact count.

If one considers the possibility it is an _estimate_, based on typical
industry figures, the very thought of only 1.5 defects per KLOC is out
of the question.  Some people I know use 5 defects per KLOC for code
that has been revised and 10 defects per KLOC for "new" code.  Reports
indicate that Windows 2000 has a lot of new code.

Thus, based only on the defect count and the 40 million LOC count,
this can only be a partial number, which gives credibility to the
press report that the 63,000 count was the result of one particular
source scanner which may very well have included a goodly number of
false positives. But certainly such a tool is incapable of "finding
all defects".

If Microsoft had a scanner that was able to "find all defects", they
could drop the Windows and Office products (or let the court give it
away) and make more money by renting out their defect scanner.  Of
course before signing up to rent the scanner, some potential customers
might want to see proof in terms of one program that had been scanned
and revised repeatedly until no defects remained.  To date Microsoft
has not offered such a program for sale, so they presumably are taking
the same approach as those who hide their use of Ada as a "strategic
advantage" :-)

Re: Win2000 has 63,000 'defects'

[Robert I. Eachus]

Larry Kilgallen wrote:
 
> If one considers the possibility it is an _estimate_, based on typical
> industry figures, the very thought of only 1.5 defects per KLOC is out
> of the question.  Some people I know use 5 defects per KLOC for code
> that has been revised and 10 defects per KLOC for "new" code.  Reports
> indicate that Windows 2000 has a lot of new code.

   When I was at Stratus, there was a very common understanding.  I
think I heard it "officially" expressed at more than one engineering
meeting, but never saw any underlying documentation.
Basicly it goes like this, if your (OS, real-time, compiler, transaction
processing, etc.) code has more than one bug per thousand lines, it
could never be made robust enough to ship.  If your code has less than
one bug per 10,000 lines, it will cost less to get it to 1 bug per
million lines than to get the bug 1 per 1000 lines code to one bug per
2000.

   Of course, it cost at least twice as much to write 1 bug per 10,000
line code than to write one bug per thousand lines code.  But writing
the buggier code at Stratus for any critical code (read any part of the
OS or the tools to build it) was worthless.  So we had to spend the time
up front to get it right.  The key measure of quality was the MBTF
during beta test for new OS versions, and we aimed for about one year
when we released.  MBTF was much higher on operational systems, since
the customers would run the beta test versions only on development
machines.  By the time the customers applications were fielded, they had
run into most of the OS bugs that they could on the development
systems.  In any case, we never got to one bug per million lines, but we
were well above one per 10000 lines of fielded code.  (Most of this code
was writen in PL/I. C was only used on I/O cards with less than 64K of
memory.)

   So there are really two styles of programming.  One, which may or may
not be fine for computer games, etc., relies on debugging quality in. 
The other relies on building it in in the first place.  Most Ada
programmers are used to the second style, which is why we seldom (if
ever!) use a debugger.

Re: Win2000 has 63,000 'defects'

[Robert Dewar]

In article <38C3D03D.FD5D39D1@earthlink.net>,
  Charles Hixson <charleshixsn@earthlink.net> wrote:
> Somehow I tend to feel that both extremes are wrong here.
Yes, one should
> write the best code that one can.  Yes, one should include all
reasonable
> error checks. (A few kind words for Design By Contract go
here.) But one
> still needs debugging tools.

Not necessarily, the clean-room style has no room whatsoever
for debugging tools, since in this approach the development
team does not run or test the code they write, let alone
use debuggers on it! Very high quality software can and
has been written using this approach.

Actually a lot of Ada programmers find that all they really
need is a good symbolic back trace, and many people use a
debugger only to obtain this information.


Sent via Deja.com http://www.deja.com/
Before you buy.

Re: Win2000 has 63,000 'defects'

[Charles Hixson]

"Robert I. Eachus" wrote:

> -- snip

> So there are really two styles of programming.  One, which may or may
> not be fine for computer games, etc., relies on debugging quality in.
> The other relies on building it in in the first place.  Most Ada
> programmers are used to the second style, which is why we seldom (if
> ever!) use a debugger.

Somehow I tend to feel that both extremes are wrong here.  Yes, one should
write the best code that one can.  Yes, one should include all reasonable
error checks. (A few kind words for Design By Contract go here.) But one
still needs debugging tools.  The less you use them, the friendlier they
need to be.  Once upon a time I used to read core dumps, but I haven't
looked at one in the last 10 years.  I haven't written assembler in the last
15+.  So to help me much debugger needs to be symbolic, and tied in a useful
way to the source level of the code that it is being applied to.

Re: Win2000 has 63,000 'defects'

[Mike Dimmick]

"Charles Hixson" <charleshixsn@earthlink.net> wrote in message
news:38AC41FE.73461614@earthlink.net...
> Well, they may be design errors, but they aren't all merely
unimplemented
> features  (partially implemented, perhaps) as some of them are
designated
> as potentially serious problems.

My take on this is as follows:

There's a lot of legacy stuff in Windows 2000 that's still written in C
(as opposed to C++).  Microsoft programmers seem not to take account of
the much better features of C++ as regards correctness (if you've
studied MFC, they tend to use the older (int) style casts as opposed to
the new static_cast<int> () syntax).  They also have a habit of writing
what might be considered odd memory allocation routines and
strategies -- look at CPlex in MFC for an example.

However, for an operating system, at least for the kernel-mode levels,
C++ is probably the right choice.  It doesn't have large run-time
overheads, the disadvantage being if you want bounds-checking etc you
have to write it yourself.  With the advent of a lot of the standard
library, especially standard collections, a lot of the work is done for
the programmer; it's just a shame that they're so often ignored.

Windows has to remain compatible with C programming.  There's just way
too much legacy code out there.  This often means that design choices
are limited.

The newer Windows stuff is mostly based on COM.  COM is really a C++
API; while you can write COM objects in other languages, forcing them to
produce an in-memory layout for vtables can be very difficult.  COM
looks like it could be a lot better for getting software interaction
right first time -- it's surprisingly strongly typed.

I did an analysis of some of the competition -- the one the press are
all hyping, Linux -- when I first heard about this article, and read it.
I examined kernels only, versions 2.0.0 thru 2.0.38.  These releases
occurred over 38 months, an average of one a month.  It's interesting to
note that 2.0.36 - 2.0.38 appeared six months apart, so some of the
releases occurred far more frequently.

What I did was as follows:
Obtained all the patches;
Ran them through 'grep' looking for the '@@@' sequence to mark the
beginning of a patch hunk.  I then used 'wc' to count the number of
hunks in each patch.
I don't have the complete listing available, but I do remember the final
figure was of the order of 16,000 fixes.  Given that the total source
for kernel 2.0.38 was around 5Mb, I'm quite worried by that statistic.
It also appears the trend is worsening; kernel 2.2 has been out for six
months and is on something like patchlevel 20 right now.

Yes, I know it was a crude measure, but I suspect some of the Win2000
errors are very superficial.  For example, there's something I've
noticed on Internet Explorer 5: sometimes when changing windows, the
menus no longer work.  And the correct button on the toolbar isn't
always pressed (doesn't always correspond to the selected window).
Opening a new folder window usually leaves the bottom row of icons
partially obscured, when it's supposed to fit the window to the size of
the icons (this has been present since IE4.0).  And I'm sure everyones
who's used Windows has spotted it sometimes gets its icon cache in a
twist.

If a defect is a user-interface glitch like those, I'm relatively happy.
Annoying, but not work-threatening.  63,000 listed defects is not a huge
amount, but I'll be waiting for the first service pack, I think.  And
some of them will be like Novell's big shout about security flaws in
Active Directory, for which you should see
http://www.microsoft.com/windows2000/news/bulletins/novellresponse.asp,
where a 'defect' means a customer has misunderstood how to use
something.  This particular set of dialogs does appear quite confusing,
so I'm not sure which way to side on this one.

[I'd be interested to know how many of those defects are in
Microsoft-supplied device drivers, actually -- Microsoft-written because
the hardware supplier has decided they'll no longer support the
device -- my SoundBlaster AWE32 falls into this category.  If some of
the defects are in drivers for devices I don't own, I'm even happier].

Now, before you all shout about being off-topic *grin*:

Ada is a very good language for some things.  Tasking and protection of
objects for concurrent programming are very easy to do.  Object
persistence is easy to support with the 'read, 'write, 'input and
'output attributes and Ada.Streams.Stream_IO.  However the tools really
aren't there to write applications like Word -- and it would be slow in
many cases, because of the amount of run-time overhead Ada adds.
Security doesn't come cheap.

Hope this doesn't offend people,

--
Mike Dimmick
BSc Computing Science level II
Aston University, Birmingham, UK

Re: Win2000 has 63,000 'defects'

[Brian Rogoff]

On Tue, 7 Mar 2000, Mike Dimmick wrote:
> "Charles Hixson" <charleshixsn@earthlink.net> wrote in message
> news:38AC41FE.73461614@earthlink.net...
> > Well, they may be design errors, but they aren't all merely
> unimplemented
> > features  (partially implemented, perhaps) as some of them are
> designated
> > as potentially serious problems.
> 
> My take on this is as follows:
> 
> There's a lot of legacy stuff in Windows 2000 that's still written in C
> (as opposed to C++).  Microsoft programmers seem not to take account of
> the much better features of C++ as regards correctness (if you've
> studied MFC, they tend to use the older (int) style casts as opposed to
> the new static_cast<int> () syntax).  They also have a habit of writing
> what might be considered odd memory allocation routines and
> strategies -- look at CPlex in MFC for an example.

The GTK authors chose to write in straight C, and they had no legacy
issues to contend with. Fancy that!

> Now, before you all shout about being off-topic *grin*:
> 
> Ada is a very good language for some things.  Tasking and protection of
> objects for concurrent programming are very easy to do.  Object
> persistence is easy to support with the 'read, 'write, 'input and
> 'output attributes and Ada.Streams.Stream_IO.  However the tools really
> aren't there to write applications like Word -- and it would be slow in
> many cases, because of the amount of run-time overhead Ada adds.

Mike, you don't really know Ada that well at all, do you? If you did,
you'd know that you can write C level code in Ada, omit runtime checking, 
and have absolutely no runtime hit. You can get addresses of local
variables, use function pointers, all that stuff. About the only thing you 
can't do as easily I think is "varargs" stuff, but it isn't too hard. 

> Security doesn't come cheap.
> 
> Hope this doesn't offend people,

No offense taken, but your beliefs are wrong. If you're offended, why
don't you post some sample Ada which you believe incurs this runtime hit, 
along with the corresponding C code? 

-- Brian

Re: Win2000 has 63,000 'defects'

[Ted Dennison]

In article <8a1g93$ijs$1@nnrp1.deja.com>,
  Robert Dewar <robert_dewar@my-deja.com> wrote:
> In article <38C3D03D.FD5D39D1@earthlink.net>,
>   Charles Hixson <charleshixsn@earthlink.net> wrote:
> > But one still needs debugging tools.
>
> Actually a lot of Ada programmers find that all they really
> need is a good symbolic back trace, and many people use a
> debugger only to obtain this information.

If the only bug under consideration is one that causes an exception, I
could see that. But I fail to see how that helps at all when the bug is
a behavioral or timing issue. Its true that the source of behavioral
bugs can usually be intuited by someone clever who knows all the code
involved intimately and takes the time to think it through. But in a
large project often there is just too much code involved for one person
to grok it all.

As for timing bugs (eg: this routine has 8ms to run, but takes 22),
trying to fix the problem without first using tools to locate it is a
damn good way to waste a lot of time and make a big mess.

As for me, give me all the tools available, and then some. Stack traces,
debuggers, profilers, timers, cpu monitors, data monitors. Love 'em all.

--
T.E.D.

http://www.telepath.com/~dennison/Ted/TED.html


Sent via Deja.com http://www.deja.com/
Before you buy.

Re: Win2000 has 63,000 'defects'

[Marin D. Condic]

Charles Hixson wrote:
> Somehow I tend to feel that both extremes are wrong here.  Yes, one should
> write the best code that one can.  Yes, one should include all reasonable
> error checks. (A few kind words for Design By Contract go here.) But one
> still needs debugging tools.  The less you use them, the friendlier they
> need to be.  Once upon a time I used to read core dumps, but I haven't
> looked at one in the last 10 years.  I haven't written assembler in the last
> 15+.  So to help me much debugger needs to be symbolic, and tied in a useful
> way to the source level of the code that it is being applied to.

In some environments, debuggers (or what would pass for one) are a way
of life. In embedded systems programming you need a debugger or
something very similar just to be able to watch the code execute and see
if it is behaving as expected.

When I build things for a workstation or a PC, often I find I build code
with built-in Put_Line statements that are toggleable on/off for similar
reasons. I either embed them in "pragma Debug" statements or have some
runtime selectable debug switch controlling output. It gives me the
ability to trace what the code is doing and look at important data as I
am developing the code. This may be why I have not turned on a debugger
in a workstation environment in years. I never get to the point where I
can't see what the code is doing without a debugger.

Of course, when called on to work on other people's code, we have a
whole different situation... :-)

MDC
-- 
=============================================================
Marin David Condic   - Quadrus Corporation -   1.800.555.3393
1015-116 Atlantic Boulevard, Atlantic Beach, FL 32233
http://www.quadruscorp.com/
m c o n d i c @ q u a d r u s c o r p . c o m

***PLEASE REMOVE THE "-NOSPAM" PART OF MY RETURN ADDRESS***

Visit my web site at:  http://www.mcondic.com/

"Because that's where they keep the money."
    --  Willie Sutton when asked why he robbed banks. 
=============================================================

Re: Win2000 has 63,000 'defects'

[Jean-Pierre Rosen]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 1047 bytes --]


Marin D. Condic <mcondic-nospam@quadruscorp.com> a �crit dans le message :
38C529E4.BE1ED0E@quadruscorp.com...
> When I build things for a workstation or a PC, often I find I build code
> with built-in Put_Line statements that are toggleable on/off for similar
> reasons. I either embed them in "pragma Debug" statements or have some
> runtime selectable debug switch controlling output. It gives me the
> ability to trace what the code is doing and look at important data as I
> am developing the code. This may be why I have not turned on a debugger
> in a workstation environment in years. I never get to the point where I
> can't see what the code is doing without a debugger.
>
Maybe you would be interested in checking package Debug, available from
Adalog's component page http://pro.wanadoo.fr/adalog/compo2.htm
It's *much* more sophisticated that plain put_Line...

--
---------------------------------------------------------
           J-P. Rosen (Rosen.Adalog@wanadoo.fr)
Visit Adalog's web site at http://pro.wanadoo.fr/adalog

Re: Win2000 has 63,000 'defects'

[David Starner]

On 8 Mar 2000 14:05:17 GMT, Dale Pontius <pontius@twonky.btv.MBI.com.invalid> wrote:
>Isn't GNAT supposed to get included into the mainline gcc distribution
>at some point? If/when that happens, how feasible would it be to
>write a Linux kernel module/device driver in Ada?

It doesn't matter whether it's in the mainline gcc distribution. It'll still
work the same. It should be no problem writting a driver/module in Ada -
just follow all the normal rules for interfacing with C and remember you
don't have the standard C or Ada libraries available to you.

-- 
David Starner - dstarner98@aasaa.ofe.org
Only a nerd would worry about wrong parentheses with
square brackets. But that's what mathematicians are.
   -- Dr. Burchard, math professor at OSU

Re: Win2000 has 63,000 'defects'

[Ted Dennison]

In article <8a5ms7$aai1@news.cis.okstate.edu>,
  dstarner98@aasaa.ofe.org wrote:
> On 8 Mar 2000 14:05:17 GMT, Dale Pontius
<pontius@twonky.btv.MBI.com.invalid> wrote:
> >at some point? If/when that happens, how feasible would it be to
> >write a Linux kernel module/device driver in Ada?
>
> work the same. It should be no problem writting a driver/module in Ada
> - just follow all the normal rules for interfacing with C and remember
> you don't have the standard C or Ada libraries available to you.

...that is, interfacing C to Ada. For instance, you'll probably need to
take steps to make sure that no elaboration code is needed, or that the
elaboration entry point gets called on or before the first invocation.


If device drivers written in Ada started popping up, folks might start
to take notice of that odd "Ada" language...

-
T.E.D.

http://www.telepath.com/~dennison/Ted/TED.html


Sent via Deja.com http://www.deja.com/
Before you buy.

Re: Win2000 has 63,000 'defects'

[Marin D. Condic]

Jean-Pierre Rosen wrote:
> Maybe you would be interested in checking package Debug, available from
> Adalog's component page http://pro.wanadoo.fr/adalog/compo2.htm
> It's *much* more sophisticated that plain put_Line...
> 
Thanks. I've bookmarked the page and will look the stuff over. I've
often built relatively small debug packages that just let you embed
string output which can be toggled on & off, but if this package does
that much and more, I may make it a development standard for my
projects.

MDC
-- 
=============================================================
Marin David Condic   - Quadrus Corporation -   1.800.555.3393
1015-116 Atlantic Boulevard, Atlantic Beach, FL 32233
http://www.quadruscorp.com/
m c o n d i c @ q u a d r u s c o r p . c o m

***PLEASE REMOVE THE "-NOSPAM" PART OF MY RETURN ADDRESS***

Visit my web site at:  http://www.mcondic.com/

"Because that's where they keep the money."
    --  Willie Sutton when asked why he robbed banks. 
=============================================================

Re: Win2000 has 63,000 'defects'

[Dale Pontius]

In article <952459361.11185.0.nnrp-12.d4e5bde1@news.demon.co.uk>,
        "Mike Dimmick" <mike@dimmick.demon.co.uk> writes:
...
> I did an analysis of some of the competition -- the one the press are
> all hyping, Linux -- when I first heard about this article, and read it.
> I examined kernels only, versions 2.0.0 thru 2.0.38.  These releases
> occurred over 38 months, an average of one a month.  It's interesting to
> note that 2.0.36 - 2.0.38 appeared six months apart, so some of the
> releases occurred far more frequently.
>
> What I did was as follows:
> Obtained all the patches;
> Ran them through 'grep' looking for the '@@@' sequence to mark the
> beginning of a patch hunk.  I then used 'wc' to count the number of
> hunks in each patch.
> I don't have the complete listing available, but I do remember the final
> figure was of the order of 16,000 fixes.  Given that the total source
> for kernel 2.0.38 was around 5Mb, I'm quite worried by that statistic.
> It also appears the trend is worsening; kernel 2.2 has been out for six
> months and is on something like patchlevel 20 right now.
>
The other art of this exercise would be examining the changelogs
for that same timeframe. There are bugfixes to the base OS, to be
sure. But there are also platform and hardware fixes - that same
source tree targets multiple platforms and CPU architectures, as
well including the device drivers - implemented as kernel modules.
There is also some feature creep. The 2.0 series was notorious for
that, but even if it's only the support of a new device, it still
shows up as a kernel change. So IMHO treating your count as bugs
is somewhat inaccurate. It would be interesting if the changelogs
could be correlated to your count to get a more accurate number.

Not to mention that one of those patches roughly in the 2.0.35
timeframe was a fix for the Intel F00F bug. NT had a fix for that
one, too. But in both cases, I'd count the OS patch as a plus.

Considering the hardware support issue, where does a bug cross the
line between an OS bug and a driver bug? In Linux, the drivers are
kernel modules, and show up in your analysis. NT ships with some
number of drivers, in the box. Have driver bugs been counted in the
63000 number? How complete is driver coverage of the GA ship CD?
How common is it to need hardware manufacturer CDs to install, even
before you consider hardware developed after GA ship?

Side question, though more relevant...

Isn't GNAT supposed to get included into the mainline gcc distribution
at some point? If/when that happens, how feasible would it be to
write a Linux kernel module/device driver in Ada?

Dale Pontius
NOT speaking for IBM

Re: Win2000 has 63,000 'defects'

[Laurent Guerby]

Ted Dennison <dennison@telepath.com> writes:
> ...that is, interfacing C to Ada. For instance, you'll probably need to
> take steps to make sure that no elaboration code is needed, or that the
> elaboration entry point gets called on or before the first invocation.

GNAT provides configuration pragma to check this (no run time and no
elaboration).

> If device drivers written in Ada started popping up, folks might start
> to take notice of that odd "Ada" language...

The Linux kernel is written in C (and Asm). Prototyping a Linux driver
in Ada could be a very interesting project, but for the sake of the
Linux project, the real driver ought to be contributed to the kernel
effort in C and C only.

--LG

Re: Win2000 has 63,000 'defects'

[Tarjei T. Jensen]

Ted Dennison
>If device drivers written in Ada started popping up, folks might start
>to take notice of that odd "Ada" language...


I do not agree. Drivers are close to invisible unless it is something
spectacular like the tcp/ip system. The point is to create an application or
similar that is important to the end-user (much like Zope is the reason for the
rise of Python). That will give Ada due attention. It might be a bit late to
write an spreadsheet or wordprocessor since both the KDE and Gnome projects
already have functional applications for this use. However there might be other
applications which may be useful. E.g. a powerpoint clone or a project
management tool.

It will also help if those who have the means put even more effort into
creating Ada infrastructure. This will lower the resistance towards using ada.

The Ada com stuff at http://www.adapower.com/ for windows is potentially a very
visible project which may lead to a acceptance of Ada. Especially when students
get to use it for free. They will want to use this later when they get out in
industry.

I think it is useful to have available interfaces to filetypes like excel,
dbase and xml. It might be useful for those who want to create these to have a
look at what has been done with this in e.g. gnumeric (the Gnome spreadsheet
application). In the ideal world the interfaces would be dynamically loadable
libraries which would be loaded on demand. If such an interface was successful
it might be extended to general database access. Or perhaps odbc would cover
this?

Other worthwile projects is an interface to either (Gnome) bonobo or
equivalent. Universal com functionality would be nice.

Greetings,



Greetings,