Enforcing good software process

Enforcing good software process

[Stephen Leake]

"AG" <ang@xtra.co.nz> writes:

> "W D Tate" <billtate@usermail.com> wrote in message
> news:ccf933d0.0304241145.2e224252@posting.google.com...
> > "Alexandre E. Kopilovitch" <aek@vib.usr.pu.ru> wrote in message
> news:<mailman.6.1051151811.13478.comp.lang.ada@ada.eu.org>...
> >
> > Actually I would prefer something akin to the U.S. Surgeon General's
> > warning on cigarette packages that could be afixed to the side of
> > every piece of critical-care medical equipment and airline cockpit.  I
> > shudder to think....
> 
> Actually, that could be quite interesting. Let's suppose that
> each and every manufacturer of a safety-critical equipment
> (such as medical X-Rays, Flight Control software or
> even ordinary traffic lights on you nearest corner) had to
> declare by law what language is inside and affix a prominently
> visible label on it stating so.

I think the best way to achieve higher quality software is to allow
people to sue manufacturers for negligence when they don't follow
accepted software production processes. Just as a surgeon can be sued
when he screws up, but can't when he follows the rules (even if the
patient dies), we need good "rules" for writing software that can be
enforced by lawsuits.

The language choice is part of this, but only a small part.

The Capability Maturity Model is a start on a process for defining
such rules.

> Let's take a poll: How many C/C++ advocates would *really* like
> those stickers? Especially when it comes to some critical things?

I'd much prefer CMM level 3 or above, independent of language.

ISO 9000 would also be a comfort, but less so (I've seen really bad
code from ISO 9000 certified shops).

-- 
-- Stephe

Re: Enforcing good software process

[John R. Strohm]

"Stephen Leake" <Stephe.Leake@nasa.gov> wrote in message
news:uu1cmfw37.fsf_-_@nasa.gov...
> I'd much prefer CMM level 3 or above, independent of language.

The only difference between CMM level 2 and CMM level 3 is this:  At CMM
level 2, every project at a company is using *A* defined, published process.
At CMM level 3, they are all using the *SAME* defined, published process.

> ISO 9000 would also be a comfort, but less so (I've seen really bad
> code from ISO 9000 certified shops).

ISO 9000 says nothing about quality.  It just says that they have written
down their process, they are following their written process, and
independent auditors have spot-checked them on it.

Re: Enforcing good software process

[Stephen Leake]

"John R. Strohm" <strohm@airmail.net> writes:

> "Stephen Leake" <Stephe.Leake@nasa.gov> wrote in message
> news:uu1cmfw37.fsf_-_@nasa.gov...
> > I'd much prefer CMM level 3 or above, independent of language.
> 
> The only difference between CMM level 2 and CMM level 3 is this:  At CMM
> level 2, every project at a company is using *A* defined, published process.
> At CMM level 3, they are all using the *SAME* defined, published process.

Yep. And there's a strong motivator (the bottom line) to be sure it is a
_good_ process. Unfortuneately, here at NASA, we don't have that
particular motivator (highly public accidental deaths are a pretty
good motivator, but they don't happen very often).

> > ISO 9000 would also be a comfort, but less so (I've seen really
> > bad code from ISO 9000 certified shops).
> 
> ISO 9000 says nothing about quality.  It just says that they have written
> down their process, they are following their written process, and
> independent auditors have spot-checked them on it.

You can boil down CMM to the same statement. The real point is that
the company believes in treating process issues seriously, and works
to improve the process. That's at the heart of CMM, but is left out of
ISO. 

-- 
-- Stephe

Re: Enforcing good software process

[Kevin Cline]

Stephen Leake <Stephe.Leake@nasa.gov> wrote in message news:<uu1cmfw37.fsf_-_@nasa.gov>...
> I think the best way to achieve higher quality software is to allow
> people to sue manufacturers for negligence when they don't follow
> accepted software production processes. Just as a surgeon can be sued
> when he screws up, but can't when he follows the rules (even if the
> patient dies), we need good "rules" for writing software that can be
> enforced by lawsuits.

Manufacturers can be sued for negligence when a software-controlled
product with an explicit or implied guarantee of safety malfunctions. 
But you can't sue Microsoft because you connected some safety-critical
device to a controller installed on a PC running Windows, and Windows
subsequently crashed.  If you want someone to write and guarantee
software for safety-critical applications, they will do it, but they
will want a lot of money.  Personally, I'm happy to be able to be able
to license highly functional operating systems for under $100, or even
for free.

> The Capability Maturity Model is a start on a process for defining
> such rules.

No process can guarantee software correctness, except perhaps actually
proving that the software is correct.  Even then the proof may be
incorrect.
 
> I'd much prefer CMM level 3 or above, independent of language.
> 
> ISO 9000 would also be a comfort, but less so (I've seen really bad
> code from ISO 9000 certified shops).

And I predict you'll also see really bad code from CMM level 3 shops.
Certification has never been a guarantee of competence in any field.

Re: Enforcing good software process

[Stephen Leake]

kcline17@hotmail.com (Kevin Cline) writes:

> Stephen Leake <Stephe.Leake@nasa.gov> wrote in message news:<uu1cmfw37.fsf_-_@nasa.gov>...
> > I think the best way to achieve higher quality software is to allow
> > people to sue manufacturers for negligence when they don't follow
> > accepted software production processes. Just as a surgeon can be sued
> > when he screws up, but can't when he follows the rules (even if the
> > patient dies), we need good "rules" for writing software that can be
> > enforced by lawsuits.
> 
> Manufacturers can be sued for negligence when a software-controlled
> product with an explicit or implied guarantee of safety malfunctions. 
> But you can't sue Microsoft because you connected some safety-critical
> device to a controller installed on a PC running Windows, and Windows
> subsequently crashed.  If you want someone to write and guarantee
> software for safety-critical applications, they will do it, but they
> will want a lot of money.  Personally, I'm happy to be able to be able
> to license highly functional operating systems for under $100, or even
> for free.

Yes, but I'd like a choice somewhere in between. Something along the
lines of an ACT support contract :).

> > The Capability Maturity Model is a start on a process for defining
> > such rules.
> 
> No process can guarantee software correctness, except perhaps actually
> proving that the software is correct.  Even then the proof may be
> incorrect.

I never said anything about "guarranteed correct". I was talking about
reliability, and about liability. Ford and GM are liable when their
cars break; it would be nice if there were more software companies
that took the same attitude.

> > I'd much prefer CMM level 3 or above, independent of language.
> > 
> > ISO 9000 would also be a comfort, but less so (I've seen really bad
> > code from ISO 9000 certified shops).
> 
> And I predict you'll also see really bad code from CMM level 3 shops.

Possibly. But I haven't yet.

> Certification has never been a guarantee of competence in any field.

Not an absolute guarrantee, that's true. But it is often well worth
it. Doctors and social workers have good certification programs; I
certainly would never allow an uncertified surgeon to operate on me. I
assume civil engineers do as well; that's why we don't kill many
people with bridges. Yes, there are always some people that fall thru
the cracks, but on the whole, certification can improve quality.

-- 
-- Stephe

Re: Enforcing good software process

[Rod Chapman]

Stephen Leake <Stephe.Leake@nasa.gov> wrote in message news:
> I never said anything about "guarranteed correct". I was talking about
> reliability, and about liability. Ford and GM are liable when their
> cars break; it would be nice if there were more software companies
> that took the same attitude.

We have delivered software with warranties, and we will continue to
do so.  Examples are CDIS (part of the London air-traffic management
systems), SHOLIS, and the MULTOS CA.  In the case of the latter
project, four defects were reported in the first year of operation - these
were corrected under our warranty at no cost to our client.
 - Rod, Praxis Critical Systems

Re: Enforcing good software process

[Robert I. Eachus]

Rod Chapman wrote:

> We have delivered software with warranties, and we will continue to
> do so.  Examples are CDIS (part of the London air-traffic management
> systems), SHOLIS, and the MULTOS CA.  In the case of the latter
> project, four defects were reported in the first year of operation - these
> were corrected under our warranty at no cost to our client.

That reminded me of one of my favorite Ada stories from way back. 
Honeywell Small Systems Division--its name changed several times while I 
was there and it is now part of Bull--did a study to see which systems 
programming language should be chosen to replace assembler.  Three 
languages were chosen for further testing C, Pascal, and Ada.  Two of 
six implementation projects for the next OS version all about the same 
size and complexity were chosen for the study.  One of the Pascal 
projects was shifted to Ada when it was determined during the detailed 
design phase that concurrency support was needed in the programming 
language.

The three Ada projects came in on time and on budget as did one of the C 
projects.  The other two projects were over schedule and budget, but 
that is not the story.

Six months after the OS version containing all these projects shipped, 
there was a meeting to review the support costs.  The OS support manager 
was a big C fan and dead set against Ada.  He showed lots of charts that 
indicated that when repairing STARS (bug reports) on the C projects 
turnaround was twice as fast as with assembler, and the cost per report 
was a third less.  "What about the Ada projects?" he was asked by the 
division manager.

"Oh, we don't know what it costs to fix bugs in the Ada systems.  There 
haven't been any reported."  To this day, I don't think he understands 
why everyone present started laughing uncontrollably.

Similarly I was involved with one DoD project where "safety critical" 
was a total understatement.  (One joke among the requirements analysis 
team was that if there was an accidental nuclear war, this system would 
insure that we won it.)

Several years after deployment, I was copied on an analysis of remaining 
problems with the system.  There was just one.  The decision on whether 
to have the original contractor maintain the software or to do it 
organically (within the Air Force) had never been made.  Under reasons? 
  "No software maintenance has been required."  That I could live with.