Bug rate and choice of programming language

Bug rate and choice of programming language

[Maciej Sobczak]

Hello,

This is not a troll, really. :-)

By following some links that were posted during the discussion on one
of the Polish programming newsgroups I have found the following page:

http://www.leshatton.org/SCSS_595.html

This page seems to refer to some paper with an assessment of
programming languages for safety critical systems with the conclusion
that the choice of programming language has little impact on the
resulting bug rate and what actually matters is the programmer's
fluency in using it.
Or this is at least what I understand from the paper's description.

Unfortunately the paper itself is not available.

Is any of you aware of the content of this paper?
Is it possible for you to objectively comment on it?

References to other research results in this domain would be highly
welcome. The above paper alone is very suggestive, but certainly not
exhaustive.

--
Maciej Sobczak * http://www.inspirel.com

YAMI4 - Messaging Solution for Distributed Systems
http://www.inspirel.com/yami4

Re: Bug rate and choice of programming language

[Georg Bauhaus]

On 10.08.10 15:22, Maciej Sobczak wrote:

> http://www.leshatton.org/SCSS_595.html

> Unfortunately the paper itself is not available.

FWIW, http://www.safety-club.org.uk/~jepson/scsc/publish.php?opt=list&pid=17#R17
Achievement and Assurance of Safety
Proceedings of the Third Safety-critical Systems Symposium, Brighton, UK

Re: Bug rate and choice of programming language

[Yannick Duchêne (Hibou57)]

Le Tue, 10 Aug 2010 15:22:01 +0200, Maciej Sobczak  
<see.my.homepage@gmail.com> a écrit:
> programming languages for safety critical systems with the conclusion
> that the choice of programming language has little impact on the
> resulting bug rate and what actually matters is the programmer's
> fluency in using it.
Seems it is viewed like a personal affair (it seems "The" centered as it  
is expressed in the message)

OK, if he/she works alone, on something which will never be revisited by  
others, will never be maintained by others, if nobody else will have to  
understand what he/she did and he/she did so.

By the way, there is no need for public assessment in personal areas  
(joking).

The question is also: what about collective works (a common case) or about  
lonely works (a very common case) where the lonely author will have to  
come back to what he/she did months or years later. I believe you may  
already have read such a reply and there is nothing new in these words...  
and there indeed may be a reason for that: practice and real life.

If you ask any one, with its favorite who-know-which in hand, even in  
QBasic results may be good... for some time. If time or numerous other  
people takes part, this is not the same story.

Disclaimer: replied on the basis of this sole post, as the document does  
not seems to be available on-line (so could not read it).

-- 
There is even better than a pragma Assert: a SPARK --# check.
--# check C and WhoKnowWhat and YouKnowWho;
--# assert Ada;
--  i.e. forget about previous premises which leads to conclusion
--  and start with new conclusion as premise.

Re: Bug rate and choice of programming language

[Phil Thornley]

On 10 Aug, 14:22, Maciej Sobczak <see.my.homep...@gmail.com> wrote:
[...]
> This page seems to refer to some paper with an assessment of
> programming languages for safety critical systems with the conclusion
> that the choice of programming language has little impact on the
> resulting bug rate and what actually matters is the programmer's
> fluency in using it.
[...]
> References to other research results in this domain would be highly
> welcome. The above paper alone is very suggestive, but certainly not
> exhaustive.

There's this paper in CrossTalk, from 2003 - which comes to very
different conclusions to Les Hatton's paper (see the table about 2/3
into the paper).
http://www.stsc.hill.af.mil/crosstalk/2003/11/0311german.html

"Table 1 shows that the poorest language for safety-critical
applications is C with consistently high anomaly rates. The best
language found is SPARK (Ada), which consistently achieves one anomaly
per 250 software lines of code. The average number of safety-critical
anomalies found is a small percentage of the overall anomalies found
with about 1 percent identified as having safety implications.
Automatically generated code was found to have considerably reduced
syntactic and data flow errors."

Cheers,

Phil

Re: Bug rate and choice of programming language

[Yannick Duchêne (Hibou57)]

Le Tue, 10 Aug 2010 22:53:25 +0200, Phil Thornley  
<phil.jpthornley@gmail.com> a écrit:
> Automatically generated code was found to have considerably reduced
> syntactic and data flow errors."
This one is good to know. I was wondering about it these last few days  
(was also wondering about a good choice to generate this, may be a script  
language). Sometime, automatically generated sources seems difficult to  
avoid, even when one do not really enjoy the idea of preprocessor (may be  
some resilient fears due to older bad experiences with C). This may  
happens when an application has multiple parts using different languages  
or representations, and in the mean times must have some of their parts  
synchronized in some way. Ex. an application with an Ada or Pascal part,  
and HTML pages, and some JavaScript parts ; all with references to some  
kind of protocols, identifiers, standard data which may need to be  
changed, etc. Different worlds, which may refer to common stuffs. This is  
probably better when these dependencies are handled via automatic source  
generation using a single data source.

I was wondering about reliability of automatically generated parts. Your  
comment is interesting in this area.

-- 
There is even better than a pragma Assert: a SPARK --# check.
--# check C and WhoKnowWhat and YouKnowWho;
--# assert Ada;
--  i.e. forget about previous premises which leads to conclusion
--  and start with new conclusion as premise.

Re: Bug rate and choice of programming language

[Yannick Duchêne (Hibou57)]

Le Tue, 10 Aug 2010 22:53:25 +0200, Phil Thornley  
<phil.jpthornley@gmail.com> a écrit:
> http://www.stsc.hill.af.mil/crosstalk/2003/11/0311german.html
Would like to quote this from the latter link:

> The application of static code analysis techniques in retrospect
> is not ideal; the process is best suited and cheapest when applied
> during software development.

Not in retrospect, but during the process (implies from near the start).

-- 
There is even better than a pragma Assert: a SPARK --# check.
--# check C and WhoKnowWhat and YouKnowWho;
--# assert Ada;
--  i.e. forget about previous premises which leads to conclusion
--  and start with new conclusion as premise.

Re: Bug rate and choice of programming language

[Yannick Duchêne (Hibou57)]

Le Tue, 10 Aug 2010 22:53:25 +0200, Phil Thornley  
<phil.jpthornley@gmail.com> a écrit:
> http://www.stsc.hill.af.mil/crosstalk/2003/11/0311german.html

Another quote from the source

> Control flow analysis can be conducted using tools or done manually at
> various levels of abstraction (module, node, etc.) and is done for the
> following reasons:* Ensure the code is executed in the right sequence.
> * Ensure the code is well structured.
> * Locate any syntactically unreachable code.
> * Highlight the parts of the code (e.g., loops) where termination
>   needs to be considered.
> This may result in diagrammatic and graphical representations of the
> code being produced.

I feel the last sentence (previous text was quoted for context), may be a  
good reason to mitigate some (bad) evaluation of UML or UML- like  
representations. It seems their experience shows some graphical  
representation at some level may help. This does not really advocate for  
UML, rather for some kind of multi-dimensional representations, like  
graphical representations are.

-- 
There is even better than a pragma Assert: a SPARK --# check.
--# check C and WhoKnowWhat and YouKnowWho;
--# assert Ada;
--  i.e. forget about previous premises which leads to conclusion
--  and start with new conclusion as premise.

Re: Bug rate and choice of programming language

[Shark8]

On Aug 10, 6:19 pm, Yannick Duchêne (Hibou57)
<yannick_duch...@yahoo.fr> wrote:
> I feel the last sentence (previous text was quoted for context), may be a  
> good reason to mitigate some (bad) evaluation of UML or UML- like  
> representations. It seems their experience shows some graphical  
> representation at some level may help. This does not really advocate for  
> UML, rather for some kind of multi-dimensional representations, like  
> graphical representations are.

UML is, in my opinion, over-rated and over-hyped in that it tries to
do two distinct and separate things at once while not realizing/
recognizing that they are separate things: that is representing the
DESIGN of a system AND representing the FLOW of the system.

The correct solution is, again in my opinion, something that
acknowledges the two as being distinct.
As a small example consider an [OOP-]object and its associated
functions/procedures/methods; there is the HOW DOES THIS OBJECT FIT
WITH OTHER OBJECTS [IN THE PROGRAM]? question, as well as the WHAT
HAPPENS WHEN [MESSAGE]? question. Both can be represented as graphs,
but one [the method] is in actuality a flowchart. In a GUI-system this
would look like an object having methods that when you examine/double-
click/access expand into a flowchart representation for that method.
[And the converse as well, when the method/procedure/function that you
are in has locally-declared objects you should be able to "zoom-in" on
those... taking you back to the OO-view but with that particular class-
of-object as the subject; this would also be how the theoretical/
imaginary/'main'/default-task would be viewed: as a function/procedure
which declares variables & objects.]

As an interesting and perhaps-or-perhaps-not practical extension of
the previous concept would be [in systems programming] the OS viewing
a program as an object, with its own methods [perhaps memory-
management], the defaulting one being that "main" thread.

Re: Bug rate and choice of programming language

[Yannick Duchêne (Hibou57)]

Le Wed, 11 Aug 2010 02:40:15 +0200, Shark8 <onewingedshark@gmail.com> a  
écrit:
> UML is, in my opinion, over-rated and over-hyped
This is the paradox which need to be solved: many talks about it, far few  
use I have seen (to be honest, none at all, even with publicly available  
Ada source which are supposed to be more demanding)


-- 
There is even better than a pragma Assert: a SPARK --# check.
--# check C and WhoKnowWhat and YouKnowWho;
--# assert Ada;
--  i.e. forget about previous premises which leads to conclusion
--  and start with new conclusion as premise.

Re: Bug rate and choice of programming language

[Simon Wright]

"Yannick Duchêne (Hibou57)" <yannick_duchene@yahoo.fr> writes:

> Le Tue, 10 Aug 2010 22:53:25 +0200, Phil Thornley
> <phil.jpthornley@gmail.com> a écrit:
>> http://www.stsc.hill.af.mil/crosstalk/2003/11/0311german.html
>
> Another quote from the source
>
>> Control flow analysis can be conducted using tools or done manually at
>> various levels of abstraction (module, node, etc.) and is done for the
>> following reasons:* Ensure the code is executed in the right sequence.
>> * Ensure the code is well structured.
>> * Locate any syntactically unreachable code.
>> * Highlight the parts of the code (e.g., loops) where termination
>>   needs to be considered.
>> This may result in diagrammatic and graphical representations of the
>> code being produced.
>
> I feel the last sentence (previous text was quoted for context), may
> be a good reason to mitigate some (bad) evaluation of UML or UML- like
> representations. It seems their experience shows some graphical
> representation at some level may help. This does not really advocate
> for UML, rather for some kind of multi-dimensional representations,
> like graphical representations are.

What you would get from this is a representation of the code.

UML is not a good match for full Ada (not sure it's that good a match
for Java, come to that; mainly because it's a hotchpotch of
techniques without a huge amount of semantic precision).

What can be done is to specify a 'profile', ie tie down a subset of UML
and add extensions to give UML constructs a more precise meaning; and
then you can specify a translation from thence to the language of your
choice. I should add that I'm coming from a Shlaer-Mellor OOA background
(http://en.wikipedia.org/wiki/Shlaer-Mellor).

What this approach won't do is to support round-trip development (change
the model, generate code, change the code, automatically import the
changes back into the model), because the generation process chooses
appropriate templates based on model structure and annotations, and
these are necessarily a particular subset of full Ada.

Re: Bug rate and choice of programming language

[Yannick Duchêne (Hibou57)]

Le Wed, 11 Aug 2010 07:36:14 +0200, Simon Wright <simon@pushface.org> a  
écrit:
> UML is not a good match for full Ada (not sure it's that good a match
> for Java, come to that; mainly because it's a hotchpotch of
> techniques without a huge amount of semantic precision).

Here is the focus word: semantic. You gave me the idea to come back to the  
semantic point, dropping everything around (tools, methods, and others).

I get some time yesterday to select some document which I will read in the  
coming days to see if a useful semantic is really there or not. My  
requirement is that if there is really a clear semantic, it should be  
abstract enough, as for precisely expressing things, there are already  
many things else (other languages). Otherwise, this would just lead to  
something which would add unexpected complexity (this would be translating  
a language into another, which is just a source of errors, complexity, and  
no an aid).

If ever some ones are interested, here are the documents :

The first two ones are the most accessible.
http://www.ibm.com/developerworks/rational/library/05/317_semantic/semantic_foundations_uml_2_0.pdf
http://www.imamu.edu.sa/DContent/IT_Topics/UML%20Semantics%20FAQ.pdf
The third is from UML authors and is much longer (not well suited for an  
overview)
http://cuiwww.unige.ch/~guyot/TPBD1/UML11/ad970804_UML11_Semantics2.pdf
The fourth looks interresting, because it express a readily understandable  
model (documentations of a Java set of class which express a view of UML  
semantic in a terse way)
http://elmuth.tigris.org/nonav/doc/javadocs/overview-summary.html
And the the fifth last, is interesting in the area of ergonomy and  
usability:
http://isgwww.cs.uni-magdeburg.de/cvse/Forschung/Publikationen/2008-SoftVis-SemanticZooming.pdf

Will be a base material for future personal works


-- 
There is even better than a pragma Assert: a SPARK --# check.
--# check C and WhoKnowWhat and YouKnowWho;
--# assert Ada;
--  i.e. forget about previous premises which leads to conclusion
--  and start with new conclusion as premise.

Re: Bug rate and choice of programming language

[Charles H. Sampson]

Maciej Sobczak <see.my.homepage@gmail.com> wrote:

> Hello,
>
> ...
> 
> By following some links that were posted during the discussion on one
> of the Polish programming newsgroups I have found the following page:
> 
> http://www.leshatton.org/SCSS_595.html
> 
> This page seems to refer to some paper with an assessment of
> programming languages for safety critical systems with the conclusion
> that the choice of programming language has little impact on the
> resulting bug rate and what actually matters is the programmer's
> fluency in using it.
> Or this is at least what I understand from the paper's description.
> 
> ...

     Not really an answer to your question, but it gives me an opening
to relate a war story.

     20 or so years ago I was working on an Ada project with a young
programmer who was very fluent in C.  After he had worked on the project
for a year or so, someone asked which language he would use, Ada or C,
if the choice were entirely up to him.  He replied that he would only
use C if he were not concerned about debugging.

                        Charlie
-- 
All the world's a stage, and most 
of us are desperately unrehearsed.  Sean O'Casey