ANNOUNCE: Ada success story in IEEE Software

ANNOUNCE: Ada success story in IEEE Software

[Rod Chapman]

Those of you who are interested in Ada success stories might
be interested to see this month's IEEE Software Magazine (Jan/Feb 2002)
at www.computer.org/software

We have an article in this issue "Correctness by Construction: Developing
a Commercial Secure System" that describes our development of the MULTOS
CA system - a highly secure, distributed, fault tolerant certification
authority.  A significant proportion of the system is constructed
using Ada95 and SPARK95.

It's the kind of system where most people
would not even consider Ada, but we found its use to be a significant
factor in the success of the project.  The final system achieved
0.04 defects per kloc (that's 4 defects in 100,000 lines of code) post-
delivery, which compares favourably with industry norms.

All the best,
 Rod Chapman
 SPARK Team
 Praxis Critical Systems

Re: ANNOUNCE: Ada success story in IEEE Software

[Dale Stanbrough]

Rod Chapman wrote:

> It's the kind of system where most people
> would not even consider Ada, but we found its use to be a significant
> factor in the success of the project.  The final system achieved
> 0.04 defects per kloc (that's 4 defects in 100,000 lines of code) post-
> delivery, which compares favourably with industry norms.

I'm not sure I follow this. If this result is the industry norm, 
and most of industry doesn't use Ada, then why do you think Ada
was "a significant factor in the success of the project"?

Dale

Re: ANNOUNCE: Ada success story in IEEE Software

[Martin Dowie]

"Dale Stanbrough" <dstanbro@bigpond.net.au> wrote in message
news:dstanbro-FFA1D2.23113708012002@mec2.bigpond.net.au...
> Rod Chapman wrote:
>
> > It's the kind of system where most people
> > would not even consider Ada, but we found its use to be a significant
> > factor in the success of the project.  The final system achieved
> > 0.04 defects per kloc (that's 4 defects in 100,000 lines of code) post-
> > delivery, which compares favourably with industry norms.
>
> I'm not sure I follow this. If this result is the industry norm,
> and most of industry doesn't use Ada, then why do you think Ada
> was "a significant factor in the success of the project"?

Need to re-read the original post :-)

He said it "compares favourably with industry norms" not "matches the
industry norms"

Re: ANNOUNCE: Ada success story in IEEE Software

[Ted Dennison]

In article <3c3aea7b$1@pull.gecm.com>, Martin Dowie says...
>
>"Dale Stanbrough" <dstanbro@bigpond.net.au> wrote in message
>news:dstanbro-FFA1D2.23113708012002@mec2.bigpond.net.au...
>> Rod Chapman wrote:
>>
>> > factor in the success of the project.  The final system achieved
>> > 0.04 defects per kloc (that's 4 defects in 100,000 lines of code) post-
>> > delivery, which compares favourably with industry norms.
>>
>> I'm not sure I follow this. If this result is the industry norm,
..
>He said it "compares favourably with industry norms" not "matches the
>industry norms"

I suspect that may have been a bit of that famous British understatement too. I
think our last (non safety-critical) Ada project would have had a defect count
orders of magnitude higher than that.

---
T.E.D.    homepage   - http://www.telepath.com/dennison/Ted/TED.html

No trees were killed in the sending of this message. 
However a large number of electrons were terribly inconvenienced.

Re: ANNOUNCE: Ada success story in IEEE Software

[Jeffrey Carter]

Ted Dennison wrote:
> 
> In article <3c3aea7b$1@pull.gecm.com>, Martin Dowie says...
> >
> >"Dale Stanbrough" <dstanbro@bigpond.net.au> wrote in message
> >news:dstanbro-FFA1D2.23113708012002@mec2.bigpond.net.au...
> >> Rod Chapman wrote:
> >>
> >> > factor in the success of the project.  The final system achieved
> >> > 0.04 defects per kloc (that's 4 defects in 100,000 lines of code) post-
> >> > delivery, which compares favourably with industry norms.
> >>
> >> I'm not sure I follow this. If this result is the industry norm,
> ..
> >He said it "compares favourably with industry norms" not "matches the
> >industry norms"
> 
> I suspect that may have been a bit of that famous British understatement too. I
> think our last (non safety-critical) Ada project would have had a defect count
> orders of magnitude higher than that.

I recall seeing the "industry norm" given as 1 defect per kloc.

-- 
Jeff Carter
"Death awaits you all, with nasty, big, pointy teeth!"
Monty Python & the Holy Grail

Re: ANNOUNCE: Ada success story in IEEE Software

[Larry Kilgallen]

In article <3C3CB31E.BC6FDBED@acm.org>, Jeffrey Carter <jrcarter@acm.org> writes:
> Ted Dennison wrote:
>> 
>> In article <3c3aea7b$1@pull.gecm.com>, Martin Dowie says...
>> >
>> >"Dale Stanbrough" <dstanbro@bigpond.net.au> wrote in message
>> >news:dstanbro-FFA1D2.23113708012002@mec2.bigpond.net.au...
>> >> Rod Chapman wrote:
>> >>
>> >> > factor in the success of the project.  The final system achieved
>> >> > 0.04 defects per kloc (that's 4 defects in 100,000 lines of code) post-
>> >> > delivery, which compares favourably with industry norms.
>> >>
>> >> I'm not sure I follow this. If this result is the industry norm,
>> ..
>> >He said it "compares favourably with industry norms" not "matches the
>> >industry norms"
>> 
>> I suspect that may have been a bit of that famous British understatement too. I
>> think our last (non safety-critical) Ada project would have had a defect count
>> orders of magnitude higher than that.
> 
> I recall seeing the "industry norm" given as 1 defect per kloc.

I suppose it depends on what industry.  I have seen numbers of 10 and 5
defects per kloc on brand new code and modification of code, not specific
to Ada.

Re: ANNOUNCE: Ada success story in IEEE Software

[Pat Rogers]

"Jeffrey Carter" <jrcarter@acm.org> wrote in message
news:3C3CB31E.BC6FDBED@acm.org...
> Ted Dennison wrote:
> >
> > In article <3c3aea7b$1@pull.gecm.com>, Martin Dowie says...
> > >
> > >"Dale Stanbrough" <dstanbro@bigpond.net.au> wrote in message
> > >news:dstanbro-FFA1D2.23113708012002@mec2.bigpond.net.au...
> > >> Rod Chapman wrote:
> > >>
> > >> > factor in the success of the project.  The final system achieved
> > >> > 0.04 defects per kloc (that's 4 defects in 100,000 lines of code) post-
> > >> > delivery, which compares favourably with industry norms.
> > >>
> > >> I'm not sure I follow this. If this result is the industry norm,
> > ..
> > >He said it "compares favourably with industry norms" not "matches the
> > >industry norms"
> >
> > I suspect that may have been a bit of that famous British understatement
too. I
> > think our last (non safety-critical) Ada project would have had a defect
count
> > orders of magnitude higher than that.
>
> I recall seeing the "industry norm" given as 1 defect per kloc.

In a study described in a paper published in 1986, Herbert and Myron Hecht found
that for each million lines of code, 20,000 bugs existed.  Normally 90% would be
found by testing.  Another 200 would be found during the first year of operation
by users, leaving 18,000 undetected bugs.  Regular maintenance would fix 200
bugs, but also introduce 200 new ones.

Things have probably progressed *a little* since then, but not much; certainly
not orders of magnitude.  Too, that's just one study (of several large systems),
but the magnitude is astounding.  (Both individuals are extremely well respected
in the software fault tolerance field.)

The paper is:
H. Hecht and M. Hecht, "Software Reliability In The Systems Context," IEEE
Transactions On Software Engineering, vol. 12, no. 1, pp. 51-58, 1986.

---
Patrick Rogers                       Consulting and Training in:
http://www.classwide.com          Real-Time/OO Languages
progers@classwide.com               Hard Deadline Schedulability Analysis
(281)648-3165                                 Software Fault Tolerance

Re: ANNOUNCE: Ada success story in IEEE Software

[Wes Groleau]

Pat Rogers wrote:
> In a study described in a paper published in 1986, Herbert and Myron Hecht found
> that for each million lines of code, 20,000 bugs existed.  Normally 90% would be
> found by testing.  Another 200 would be found during the first year of operation
> by users, leaving 18,000 undetected bugs.  Regular maintenance would fix 200
> bugs, but also introduce 200 new ones.

This confuses me.  Typos?

Are the 90% found in testing not fixed?  How are 18,000 of them
still undetected a year after release.

And how can any study detect 18,000 bugs that neither the testers
nor the users can detect?

-- 
Wes Groleau
http://freepages.rootsweb.com/~wgroleau

Re: ANNOUNCE: Ada success story in IEEE Software

[Pat Rogers]

"Wes Groleau" <wesgroleau@spamcop.net> wrote in message
news:3C3DAF46.F75BBD@spamcop.net...
>
>
> Pat Rogers wrote:
> > In a study described in a paper published in 1986, Herbert and Myron Hecht
found
> > that for each million lines of code, 20,000 bugs existed.  Normally 90%
would be
> > found by testing.  Another 200 would be found during the first year of
operation
> > by users, leaving 18,000 undetected bugs.  Regular maintenance would fix 200
> > bugs, but also introduce 200 new ones.
>
> This confuses me.  Typos?
>
> Are the 90% found in testing not fixed?  How are 18,000 of them
> still undetected a year after release.

Yes, mea culpa -- a typo (in my original notes, which it cut-and-pasted). That
should be 1800, not 18,000.

> And how can any study detect 18,000 bugs that neither the testers
> nor the users can detect?

The users eventually detected them.

Re: ANNOUNCE: Ada success story in IEEE Software

[Larry Kilgallen]

In article <ba18d5cb.0201080316.3954569b@posting.google.com>, rod@praxis-cs.co.uk (Rod Chapman) writes:
> Those of you who are interested in Ada success stories might
> be interested to see this month's IEEE Software Magazine (Jan/Feb 2002)
> at www.computer.org/software
> 
> We have an article in this issue "Correctness by Construction: Developing
> a Commercial Secure System" that describes our development of the MULTOS
> CA system - a highly secure, distributed, fault tolerant certification
> authority.  A significant proportion of the system is constructed
> using Ada95 and SPARK95.
> 
> It's the kind of system where most people
> would not even consider Ada,

Why is that ?

Re: ANNOUNCE: Ada success story in IEEE Software

[Rod Chapman]

Kilgallen@SpamCop.net (Larry Kilgallen) wrote in message news:> > It's the kind of system where most people
> > would not even consider Ada,
> 
> Why is that ?

Mainly because it's an industry sector (banking) where the benefits
of more rigorous software engineering approaches are not that widely
recognized.  There's also some belief going around that
you can't build large, distributed applications in anything other than
C++, Java, CORBA, ObjectWibble## etc. etc. these days...The CA is actually
written in a combination of C++, Ada95, SPARK95, C and SQL, and it all
fits together very well.  Please read the article for more details.

 - Rod

Re: ANNOUNCE: Ada success story in IEEE Software

[Larry Kilgallen]

In article <ba18d5cb.0201080945.3b0c50b8@posting.google.com>, rod@praxis-cs.co.uk (Rod Chapman) writes:
> Kilgallen@SpamCop.net (Larry Kilgallen) wrote in message news:> > It's the kind of system where most people
>> > would not even consider Ada,
>> 
>> Why is that ?
> 
> Mainly because it's an industry sector (banking) where the benefits
> of more rigorous software engineering approaches are not that widely
> recognized.  There's also some belief going around that
> you can't build large, distributed applications in anything other than
> C++, Java, CORBA, ObjectWibble## etc. etc. these days...The CA is actually
> written in a combination of C++, Ada95, SPARK95, C and SQL, and it all

Thanks.

> fits together very well.  Please read the article for more details.

I tried, but they require cookies.

I realize that is not your doing.

Re: ANNOUNCE: Ada success story in IEEE Software

[Ian]

Kilgallen@SpamCop.net (Larry Kilgallen) wrote in message news:<uA2lVe9FJfDD@eisner.encompasserve.org>...
> In article <ba18d5cb.0201080316.3954569b@posting.google.com>, rod@praxis-cs.co.uk (Rod Chapman) writes:
> > Those of you who are interested in Ada success stories might
> > be interested to see this month's IEEE Software Magazine (Jan/Feb 2002)
> > at www.computer.org/software
> > 
> > We have an article in this issue "Correctness by Construction: Developing
> > a Commercial Secure System" that describes our development of the MULTOS
> > CA system - a highly secure, distributed, fault tolerant certification
> > authority.  A significant proportion of the system is constructed
> > using Ada95 and SPARK95.
> > 
> > It's the kind of system where most people
> > would not even consider Ada,

Thanks for the promotion of the IEEE but:

"The full text of IEEE Software is available to Computer Society members"

so could you post a portion of the relevent quotes or even just the abstract?


Thanks,
Ian

Re: ANNOUNCE: Ada success story in IEEE Software

[Larry Kilgallen]

In article <11bf7180.0201110237.4b6b5d00@posting.google.com>, ian0kerr@my-deja.com (Ian) writes:
> Kilgallen@SpamCop.net (Larry Kilgallen) wrote in message news:<uA2lVe9FJfDD@eisner.encompasserve.org>...
>> In article <ba18d5cb.0201080316.3954569b@posting.google.com>, rod@praxis-cs.co.uk (Rod Chapman) writes:
>> > Those of you who are interested in Ada success stories might
>> > be interested to see this month's IEEE Software Magazine (Jan/Feb 2002)
>> > at www.computer.org/software
>> > 
>> > We have an article in this issue "Correctness by Construction: Developing
>> > a Commercial Secure System" that describes our development of the MULTOS
>> > CA system - a highly secure, distributed, fault tolerant certification
>> > authority.  A significant proportion of the system is constructed
>> > using Ada95 and SPARK95.
>> > 
>> > It's the kind of system where most people
>> > would not even consider Ada,
> 
> Thanks for the promotion of the IEEE but:
> 
> "The full text of IEEE Software is available to Computer Society members"
> 
> so could you post a portion of the relevent quotes or even just the abstract?

As for relevant quotes, I wrote none of the above.

Re: ANNOUNCE: Ada success story in IEEE Software

[John English]

Brian Orpin wrote:
> 
> On 8 Jan 2002 03:16:41 -0800, rod@praxis-cs.co.uk (Rod Chapman) wrote:
> 
> >Those of you who are interested in Ada success stories might
> >be interested to see this month's IEEE Software Magazine (Jan/Feb 2002)
> >at www.computer.org/software
> 
> I'd like to but I am not paying for the privileged, sorry.

Seconded. But maybe someone with access could summarise it for us
here?

-----------------------------------------------------------------
 John English              | mailto:je@brighton.ac.uk
 Senior Lecturer           | http://www.it.bton.ac.uk/staff/je
 Dept. of Computing        | ** NON-PROFIT CD FOR CS STUDENTS **
 University of Brighton    |    -- see http://burks.bton.ac.uk
-----------------------------------------------------------------

Re: ANNOUNCE: Ada success story in IEEE Software

[Rod Chapman]

John English <je@brighton.ac.uk> wrote in message news:>
>> I'd like to but I am not paying for the privileged, sorry.
> 
> Seconded. But maybe someone with access could summarise it for us
> here?

IEEE Software is indeed only available electronically to IEEE Members.
I'm sure the old-fangled paper copy will appear in your local library
soon (don't tell me your library _doesn't_ get IEEE Sofware?  Scandalous! :-) )

I must admit my 5 complimentary author's copies haven't shown up yet,
so I guess you might have to wait a couple of weeks.  We've also
ordered a bunch of re-prints, but they will take some time to arrive.
 - Rod Chapman

Re: ANNOUNCE: Ada success story in IEEE Software

[John English]

Rod Chapman wrote:
> 
> John English <je@brighton.ac.uk> wrote in message news:>
> >> I'd like to but I am not paying for the privileged, sorry.
> >
> > Seconded. But maybe someone with access could summarise it for us
> > here?
> 
> IEEE Software is indeed only available electronically to IEEE Members.
> I'm sure the old-fangled paper copy will appear in your local library
> soon (don't tell me your library _doesn't_ get IEEE Sofware?  Scandalous! :-) )

It does, but it'll involve me remembering to go look for it next
time I go to the library (and remembering to take the reference).
Much easier for someone who's already read it to tell us in a
sentence or two what this particular success story relates to,
surely?

> I must admit my 5 complimentary author's copies haven't shown up yet,
> so I guess you might have to wait a couple of weeks.  We've also
> ordered a bunch of re-prints, but they will take some time to arrive.

OK. All I wanted was a quick summary...

-----------------------------------------------------------------
 John English              | mailto:je@brighton.ac.uk
 Senior Lecturer           | http://www.it.bton.ac.uk/staff/je
 Dept. of Computing        | ** NON-PROFIT CD FOR CS STUDENTS **
 University of Brighton    |    -- see http://burks.bton.ac.uk
-----------------------------------------------------------------

Re: ANNOUNCE: Ada success story in IEEE Software

[Rod Chapman]

John English <je@brighton.ac.uk> wrote in message news:<3C45770A.19185127@brighton.ac.uk>...
> OK. All I wanted was a quick summary...

Please see http://www.sparkada.com/downloads/iccc2001.pdf
or http://www.sparkada.com/downloads/ChapmanSigAda2000.pdf
for some overview of the MULTOS CA system.
 - Rod

Re: ANNOUNCE: Ada success story in IEEE Software

[John English]

Rod Chapman wrote:
> 
> John English <je@brighton.ac.uk> wrote in message news:<3C45770A.19185127@brighton.ac.uk>...
> > OK. All I wanted was a quick summary...
> 
> Please see http://www.sparkada.com/downloads/iccc2001.pdf
> or http://www.sparkada.com/downloads/ChapmanSigAda2000.pdf
> for some overview of the MULTOS CA system.

Thank you -- that's exactly what I wanted.

-----------------------------------------------------------------
 John English              | mailto:je@brighton.ac.uk
 Senior Lecturer           | http://www.it.bton.ac.uk/staff/je
 Dept. of Computing        | ** NON-PROFIT CD FOR CS STUDENTS **
 University of Brighton    |    -- see http://burks.bton.ac.uk
-----------------------------------------------------------------

Re: ANNOUNCE: Ada success story in IEEE Software

[Rod Chapman]

rod@praxis-cs.co.uk (Rod Chapman) wrote in message news:<ba18d5cb.0201080316.3954569b@posting.google.com>...
> Those of you who are interested in Ada success stories might
> be interested to see this month's IEEE Software Magazine (Jan/Feb 2002)
> at www.computer.org/software

PDF of this article is now available free of charge at
www.sparkada.com under "Publications".

The IEEE have asked me to point out the following:

"This material is presented to ensure timely dissemination of scholarly
and technical work. Copyright and all rights therein are retained by
authors or by other copyright holders. All persons copying this information
are expected to adhere to the terms and constraints invoked by each author's
copyright. In most cases, these works may not be reposted without the explicit
permission of the copyright holder."

 - Rod Chapman, SPARK Team, Praxis Critical Systems