Software Quality in Science

Software Quality in Science

[Jerry]

Here is an interesting article on software quality in science:

http://www.guardian.co.uk/technology/2010/feb/05/science-climate-emails-code-release

From the article:

"There is enough evidence for us to regard a lot of scientific
software with worry. For example Professor Les Hatton, an
international expert in software testing resident in the Universities
of Kent and Kingston, carried out an extensive analysis of several
million lines of scientific code. He showed that the software had an
unacceptably high level of detectable inconsistencies.

"For example, interface inconsistencies between software modules which
pass data from one part of a program to another occurred at the rate
of one in every seven interfaces on average in the programming
language Fortran, and one in every 37 interfaces in the language C.
This is hugely worrying when you realise that just one error — just
one — will usually invalidate a computer program. What he also
discovered, even more worryingly, is that the accuracy of results
declined from six significant figures to one significant figure during
the running of programs.

"Hatton and other researchers' work indicates that scientific software
is often of poor quality. What is staggering about the research that
has been done is that it examines commercial scientific software –
produced by software engineers who have to undergo a regime of
thorough testing, quality assurance and a change control discipline
known as configuration management.

"By contrast scientific software developed in our universities and
research institutes is often produced by scientists with no training
in software engineering and with no quality mechanisms in place and
so, no doubt, the occurrence of errors will be even higher."


For several years, I have used Ada for as much of my engineering
research as I can, having switched from Pascal and having dumped
FORTRAN many years ago (although it was my first language). Ada offers
the same advantages in this application as in other applications.
However, in my experience, Ada is hampered by having limited support
for standard and popular libraries such as the GNU Scientific Library.
(Feel free to add your own favorite library which is not supported by
Ada.)

Jerry

Re: Software Quality in Science

[Jerry]

Here is the link in the Guardian article to the original work:

http://www.leshatton.org/Documents/Texp_ICSE297.pdf

It appears that this work was done in the 1990s. The paper is actually
fairly entertaining to read, if for nothing else how scary it is. Here
is a choice comment, about a commercial program for use in the nuclear
engineering industry:

"This package climbed to an awe-inspiring 140 weighted static faults
per 1000 lines of code, and in spite of the aspirations of its
designers, amounted to no more than a very expensive random number
generator."

And this comment which addresses the use of Ada:

"In C, note that function prototypes were well used only around 60% of
the time and as a result, interface faults accounted for about 24% of
the total. In other words, if function prototypes were mandated in all
C functions, 24% of all serious faults would disappear.  The
computational scientist should not use this as an argument in favour
of C++ or Ada in which they are mandated. A large number of new
failure modes result from this action, which lack of space prohibits
further discussion here. The net result of changing languages appears
to be that the overall defect density appears to be about the same,
(Hatton 1997). In other words, when a language corrects one
deficiency, it appears to add one of its own."

Jerry

Re: Software Quality in Science

[Hibou57 (Yannick Duchêne)]

On 9 fév, 22:51, Jerry <lancebo...@qwest.net> wrote:
> And this comment which addresses the use of Ada:
>
> "In C, note that function prototypes were well used only around 60% of
> the time and as a result, interface faults accounted for about 24% of
> the total. In other words, if function prototypes were mandated in all
> C functions, 24% of all serious faults would disappear.  The
> computational scientist should not use this as an argument in favour
> of C++ or Ada in which they are mandated. A large number of new
> failure modes result from this action, which lack of space prohibits
> further discussion here. The net result of changing languages appears
> to be that the overall defect density appears to be about the same,
> (Hatton 1997). In other words, when a language corrects one
> deficiency, it appears to add one of its own."

Switching from C to C++ is nearly the same as switching from C to
another C (C++ is backward compatible with C, as fat as I know), so
this is not surprising there.

About Ada now, this could nice to better advocate the assertion that
Ada may (as suggested) correct some error but will add some others.
What's funding this assertion ?

A question : was the studies on migration from C to Ada made by C
developers who had just learn the basics of Ada, or by Ada developers
with minimum experiencing in Ada ?

Re: Software Quality in Science

[Robert A Duff]

Jerry <lanceboyle@qwest.net> writes:

> Here is the link in the Guardian article to the original work:
>
> http://www.leshatton.org/Documents/Texp_ICSE297.pdf

Thanks for the link.

> And this comment which addresses the use of Ada:

Hmm.  Looks like it fails to address Ada (or C++)
due to "lack of space".  I skept.

> "In C, note that function prototypes were well used only around 60% of
> the time and as a result, interface faults accounted for about 24% of
> the total. In other words, if function prototypes were mandated in all
> C functions, 24% of all serious faults would disappear.

Surely that's no longer a problem in modern C!?

>...The
> computational scientist should not use this as an argument in favour
> of C++ or Ada in which they are mandated. A large number of new
> failure modes result from this action, which lack of space prohibits
> further discussion here.

Hmm...

>...The net result of changing languages appears
> to be that the overall defect density appears to be about the same,
> (Hatton 1997). In other words, when a language corrects one
> deficiency, it appears to add one of its own."

That assertion requires evidence, and I don't see it here!

- Bob

Re: Software Quality in Science

[Georg Bauhaus]

Robert A Duff wrote:
> Jerry <lanceboyle@qwest.net> writes:
> 
>> Here is the link in the Guardian article to the original work:
>>
>> http://www.leshatton.org/Documents/Texp_ICSE297.pdf

>> ...The
>> computational scientist should not use this as an argument in favour
>> of C++ or Ada in which they are mandated. A large number of new
>> failure modes result from this action, which lack of space prohibits
>> further discussion here.
> 
> Hmm...
> 
>> ...The net result of changing languages appears
>> to be that the overall defect density appears to be about the same,
>> (Hatton 1997). In other words, when a language corrects one
>> deficiency, it appears to add one of its own."
> 
> That assertion requires evidence, and I don't see it here!

Indeed, looking at some of the things that Les Hatton suggests
to be doing for a living, there might be an incentive not to
perform a comparative study of the effects of using statically
checked C (with Safer C (TM)) versus statically "checked" Ada
(Spark, or SofCheck Inspector (TM)). IOW, language choice does not
matter as long as you use our tools and participate in our training
courses.

His arguments still seem based on studies from the mid 1990s.
A study is something at least. Is there anything in the Tokeneer
data that could serve as a basis for a comparison?
What failure modes might Spark add?

Re: Software Quality in Science

[Thomas Wicklund]

On 2/9/2010 3:00 PM, Hibou57 (Yannick Duch�ne) wrote:
>
> Switching from C to C++ is nearly the same as switching from C to
> another C (C++ is backward compatible with C, as fat as I know), so
> this is not surprising there.

C++ is not backward compatible.  For instance, C++ requires function 
prototypes while C does not.  C++ strengthens type checking in some 
other ways.

Still, C++ type checking is nowhere near as strong as Ada.

Re: Software Quality in Science

[Hibou57 (Yannick Duchêne)]

On 10 fév, 05:13, Thomas Wicklund <wickl...@eskimo.com> wrote:
> C++ is not backward compatible.  For instance, C++ requires function
> prototypes while C does not.
So C++ (I've tried it many years ago, so I forget it, but I feel I
remember a bit now after your words) is making a requirement of what
is just a good recommended practice in C.

I still use C sometime, and I always declare a function before I use
it, and never rely on the strange implicit declaration which supposes
default return type int and the like.

Re: Software Quality in Science

[Florian Weimer]

* Robert A. Duff:

>>...The net result of changing languages appears
>> to be that the overall defect density appears to be about the same,
>> (Hatton 1997). In other words, when a language corrects one
>> deficiency, it appears to add one of its own."
>
> That assertion requires evidence, and I don't see it here!

(Hatton 1997) is here: <http://www.leshatton.org/IEEE_Soft_97b.html>
For Ada, it cites Compton & Withrow, "Improving Productivity: Using
Metrics to Predict and Control Defects in Ada Software", in "Second
Annual Oregon Workshop on Software Metrics", Oregon 1990.  It then
concludes that language choice had no impact on fault density
patterns.  Of course, the evidence is only anecdotal.

But the actual cross-language studies I've seen show that most metrics
one can conceive (such as defect rate, performance, development time,
even lines of code) vary as much among programmers as among
programming languages.  This is a bit sad because it means that
language design does not really matter as far as actual results are
concerned.

Ada subsets may be helpful if your goal is to avoid the last
(relevant) bug.  But that requires matching development practices,
which are in place for only very few code bases.  Certainly, these
practices aren't magically introduced if you just substitute Ada for C
or Fortran.

Re: Software Quality in Science

[(see below)]

On 10/02/2010 13:13, in article 87fx59p725.fsf@mid.deneb.enyo.de, "Florian
Weimer" <fw@deneb.enyo.de> wrote:

> * Robert A. Duff:
> 
>>> ...The net result of changing languages appears
>>> to be that the overall defect density appears to be about the same,
>>> (Hatton 1997). In other words, when a language corrects one
>>> deficiency, it appears to add one of its own."
>> 
>> That assertion requires evidence, and I don't see it here!
> 
> (Hatton 1997) is here: <http://www.leshatton.org/IEEE_Soft_97b.html>
> For Ada, it cites Compton & Withrow, "Improving Productivity: Using
> Metrics to Predict and Control Defects in Ada Software", in "Second
> Annual Oregon Workshop on Software Metrics", Oregon 1990.  It then
> concludes that language choice had no impact on fault density
> patterns.  Of course, the evidence is only anecdotal.

But surely this is comprehensively falsified by Rational's carefully
controlled study?

-- 
Bill Findlay
<surname><forename> chez blueyonder.co.uk

Re: Software Quality in Science

[Martin]

On Feb 10, 1:13 pm, Florian Weimer <f...@deneb.enyo.de> wrote:
> * Robert A. Duff:
>
> >>...The net result of changing languages appears
> >> to be that the overall defect density appears to be about the same,
> >> (Hatton 1997). In other words, when a language corrects one
> >> deficiency, it appears to add one of its own."
>
> > That assertion requires evidence, and I don't see it here!
>
> (Hatton 1997) is here: <http://www.leshatton.org/IEEE_Soft_97b.html>
> For Ada, it cites Compton & Withrow, "Improving Productivity: Using
> Metrics to Predict and Control Defects in Ada Software", in "Second
> Annual Oregon Workshop on Software Metrics", Oregon 1990.  It then
> concludes that language choice had no impact on fault density
> patterns.  Of course, the evidence is only anecdotal.
>
> But the actual cross-language studies I've seen show that most metrics
> one can conceive (such as defect rate, performance, development time,
> even lines of code) vary as much among programmers as among
> programming languages.  This is a bit sad because it means that
> language design does not really matter as far as actual results are
> concerned.
>
> Ada subsets may be helpful if your goal is to avoid the last
> (relevant) bug.  But that requires matching development practices,
> which are in place for only very few code bases.  Certainly, these
> practices aren't magically introduced if you just substitute Ada for C
> or Fortran.

What about McCormack (http://www.stsc.hill.af.mil/crossTalk/2000/08/
mccormick.html)?

Or Zeiger (Rational) (http://www.adaic.com/whyada/ada-vs-c/
cada_art.html)?

Both quite old now (but so is Hatton's) and they at least have
something more than anecdotes...

Cheers
-- Martin

Re: Software Quality in Science

[Ludovic Brenta]

On Feb 9, 10:51 pm, Jerry <lancebo...@qwest.net> wrote:
> "The net result of changing languages appears
> to be that the overall defect density appears to be about the same,
> (Hatton 1997). In other words, when a language corrects one
> deficiency, it appears to add one of its own."

Ah but the study must have overlooked one thing. A strongly-typed
language with an emphasis on software quality will naturally attract
developers who understand the benefits of strong typing and value
software quality. Conversely, a loosely-typed, lax language will
attract sloppy programmers who do not understand why typing should be
strong or who believe that bugs are good for their job security.

So, even if the author's assertion were true (i.e. Ada has just as
many traps and pitfalls as C, which I disagree with), developing in
Ada with Ada-minded programmers will always be safer than developing
in C with C-minded programmers. In fact, developing in C with Ada-
minded programmers is safer, too.

The same reasoning applies to Spark vs. Ada.

--
Ludovic Brenta.

Re: Software Quality in Science

[Hibou57 (Yannick Duchêne)]

On 10 fév, 14:55, Ludovic Brenta <ludo...@ludovic-brenta.org> wrote:
> The same reasoning applies to Spark vs. Ada.
Such a comparison would not come without a difference : Ada is safer
than C, while it is capable of more than C. SPARK is safer than Ada,
but although attractive at first sight, it imposes a lot of
restrictions which make it less capable than Ada (capable : I'm
talking about the extend of constructs you can create with it).

While a demanding C developer may be wonderfully happy to learn about
Ada, an Ada designer may possibly have less reasons to switch to SPARK
(unless otherwise required). Proof : near to all Adaiste know about
SPARK, but just a few of them use it.

The path from Ada to SPARK has nothing to be compared to the one of
from C to Ada.



Side note for little words about C : the typical laxity of C is
sometime useful for quick modeling when interfacing with system API is
involved. Out of subject, I know, it was just to tell C developers we
do not really hate them so much and give them a Welcome Here :p (I
myself occasionally relying on C by the way).

Re: Software Quality in Science

[Stephen Leake]

Robert A Duff <bobduff@shell01.TheWorld.com> writes:

> Jerry <lanceboyle@qwest.net> writes:
>
>> Here is the link in the Guardian article to the original work:
>>
>> http://www.leshatton.org/Documents/Texp_ICSE297.pdf
>
> Thanks for the link.
>
>> And this comment which addresses the use of Ada:
>
> Hmm.  Looks like it fails to address Ada (or C++)
> due to "lack of space".  I skept.

Me too. Paraphrasing Fermat doesn't make it true; it just makes you
look like you are being deliberately mysterious.

>> "In C, note that function prototypes were well used only around 60% of
>> the time and as a result, interface faults accounted for about 24% of
>> the total. In other words, if function prototypes were mandated in all
>> C functions, 24% of all serious faults would disappear.
>
> Surely that's no longer a problem in modern C!?

The programmers still have to specify the right switches, and pay
attention to the results, and know how to fix them, which is a
political/management/skill issue, not a language design issue.

I know from personal experience that there is a satellite about to be
launched who's control code has many remaining C warnings. Sigh.

-- 
-- Stephe

Re: Software Quality in Science

[Stephen Leake]

Georg Bauhaus <rm.tsoh.plus-bug.bauhaus@maps.futureapps.de> writes:

> Robert A Duff wrote:
>> Jerry <lanceboyle@qwest.net> writes:
>> 
>>> ...The net result of changing languages appears
>>> to be that the overall defect density appears to be about the same,
>>> (Hatton 1997). In other words, when a language corrects one
>>> deficiency, it appears to add one of its own."
>> 
>> That assertion requires evidence, and I don't see it here!
>
> Indeed, looking at some of the things that Les Hatton suggests
> to be doing for a living, there might be an incentive not to
> perform a comparative study of the effects of using statically
> checked C (with Safer C (TM)) versus statically "checked" Ada
> (Spark, or SofCheck Inspector (TM)). IOW, language choice does not
> matter as long as you use our tools and participate in our training
> courses.

That has almost always been my reaction when I attend a software tools
sales pitch; "I don't need that tool, the Ada compiler already does
all of it".

Adacontrol and SPARK are tools I would consider using (they add real
value beyond the Ada compiler), but I've never been to a sales pitch
for those :).

> His arguments still seem based on studies from the mid 1990s.
> A study is something at least. Is there anything in the Tokeneer
> data that could serve as a basis for a comparison?
> What failure modes might Spark add?

Brain pain, for one. Understanding how to use Ada well is an order of
magnitude harder than understanding how to use C; I suspect SPARK is
another large increase (I've only looked into it lightly, never tried
to seriously use it).

I suspect this is one reason Ada has failed to take over the
programming space; you have to be a really good programmer to fully
benefit from it, and it's just not worth it for a mediocre programmer.

-- 
-- Stephe

Re: Software Quality in Science

[Hyman Rosen]

On 2/10/2010 9:29 AM, Stephen Leake wrote:
> The programmers still have to specify the right switches, and pay
> attention to the results, and know how to fix them, which is a
> political/management/skill issue, not a language design issue.

The same is true for Ada, at least as implemented in GNAT.

> I know from personal experience that there is a satellite about to be
> launched who's control code has many remaining C warnings. Sigh.

Well, isn't that part of "pay attention to the results"?
Not all warnings represent things that need to be fixed.

Re: Software Quality in Science

[Martin]

On Feb 10, 2:34 pm, Stephen Leake <stephen_le...@stephe-leake.org>
wrote:
> Georg Bauhaus <rm.tsoh.plus-bug.bauh...@maps.futureapps.de> writes:
> > Robert A Duff wrote:
> >> Jerry <lancebo...@qwest.net> writes:
>
> >>> ...The net result of changing languages appears
> >>> to be that the overall defect density appears to be about the same,
> >>> (Hatton 1997). In other words, when a language corrects one
> >>> deficiency, it appears to add one of its own."
>
> >> That assertion requires evidence, and I don't see it here!
>
> > Indeed, looking at some of the things that Les Hatton suggests
> > to be doing for a living, there might be an incentive not to
> > perform a comparative study of the effects of using statically
> > checked C (with Safer C (TM)) versus statically "checked" Ada
> > (Spark, or SofCheck Inspector (TM)). IOW, language choice does not
> > matter as long as you use our tools and participate in our training
> > courses.
>
> That has almost always been my reaction when I attend a software tools
> sales pitch; "I don't need that tool, the Ada compiler already does
> all of it".
>
> Adacontrol and SPARK are tools I would consider using (they add real
> value beyond the Ada compiler), but I've never been to a sales pitch
> for those :).
>
> > His arguments still seem based on studies from the mid 1990s.
> > A study is something at least. Is there anything in the Tokeneer
> > data that could serve as a basis for a comparison?
> > What failure modes might Spark add?
>
> Brain pain, for one. Understanding how to use Ada well is an order of
> magnitude harder than understanding how to use C; I suspect SPARK is
> another large increase (I've only looked into it lightly, never tried
> to seriously use it).
>
> I suspect this is one reason Ada has failed to take over the
> programming space; you have to be a really good programmer to fully
> benefit from it, and it's just not worth it for a mediocre programmer.
>
> --
> -- Stephe

No, no!! I've worked with guys I won't trust to programme my VCR...but
with an Ada compiler they were able to generate something that worked
(and met timing/space requirements). If they'd been let loose with a C
compiler (or worse a C++ one) they'd still be at now (10 years on)...

I'd add a smiley but it's sad really...

Cheers
-- Martin

Re: Software Quality in Science

[Robert A Duff]

Hyman Rosen <hyrosen@mail.com> writes:

> On 2/10/2010 9:29 AM, Stephen Leake wrote:
>> The programmers still have to specify the right switches, and pay
>> attention to the results, and know how to fix them, which is a
>> political/management/skill issue, not a language design issue.
>
> The same is true for Ada, at least as implemented in GNAT.

Yes.  Seat belts only work if you buckle them.

Don't put pennies in your fuse box.

>> I know from personal experience that there is a satellite about to be
>> launched who's control code has many remaining C warnings. Sigh.
>
> Well, isn't that part of "pay attention to the results"?
> Not all warnings represent things that need to be fixed.

Yes, but I think you need a way to systematically mark
the ones that don't need to be fixed.  That's what
pragma Warnings(Off) is for.  At AdaCore, we insist
on warning-free builds for out Ada code, which requires
using that pragma sometimes.

- Bob

Re: Software Quality in Science

[Robert A Duff]

Stephen Leake <stephen_leake@stephe-leake.org> writes:

> Georg Bauhaus <rm.tsoh.plus-bug.bauhaus@maps.futureapps.de> writes:
>> His arguments still seem based on studies from the mid 1990s.
>> A study is something at least. Is there anything in the Tokeneer
>> data that could serve as a basis for a comparison?
>> What failure modes might Spark add?

I don't know if you'd call this a "failure mode", but SPARK
has heavy restrictions.  For example, recursion is not allowed.
If I'm writing a compiler, I _will_ be using recursion, because
languages have recursive features (expressions contain
subexpressions, etc).  If the language doesn't support
recursion directly, I will have to implement it myself, using
an explicit stack.  That's error prone, compared to using
recursion in Ada.  So I wouldn't want to write a compiler
in SPARK.

On the other hand, if you don't need recursion and various
other features, then SPARK is an excellent choice.

> Brain pain, for one. Understanding how to use Ada well is an order of
> magnitude harder than understanding how to use C; I suspect SPARK is
> another large increase (I've only looked into it lightly, never tried
> to seriously use it).
>
> I suspect this is one reason Ada has failed to take over the
> programming space; you have to be a really good programmer to fully
> benefit from it, and it's just not worth it for a mediocre programmer.

I'm not sure that's true.  McCormick's paper about toy trains
says otherwise (students were able to produce working code
in Ada, but not in C, even with extra help in the C case).

It's true that Ada is a big language, but a lot of it
can be safely ignored.  There's no need for an Ada
programmer to understand the overload resolution rules,
for example.

- Bob

Re: Software Quality in Science

[Martin]

On Feb 10, 4:37 pm, Robert A Duff <bobd...@shell01.TheWorld.com>
wrote:
> It's true that Ada is a big language, but a lot of it
> can be safely ignored.  There's no need for an Ada
> programmer to understand the overload resolution rules,
> for example.

Unlike C++'s (broken) ADL rules! :roll:

-- Martin

Re: Software Quality in Science

[Bob Spooner]

"Ludovic Brenta" <ludovic@ludovic-brenta.org> wrote in message 
news:ddb8b59c-d746-4228-b7a9-7ace7058018b@u26g2000yqm.googlegroups.com...
On Feb 9, 10:51 pm, Jerry <lancebo...@qwest.net> wrote:
> "The net result of changing languages appears
> to be that the overall defect density appears to be about the same,
> (Hatton 1997). In other words, when a language corrects one
> deficiency, it appears to add one of its own."

Ah but the study must have overlooked one thing. A strongly-typed
language with an emphasis on software quality will naturally attract
developers who understand the benefits of strong typing and value
software quality. Conversely, a loosely-typed, lax language will
attract sloppy programmers who do not understand why typing should be
strong or who believe that bugs are good for their job security.

So, even if the author's assertion were true (i.e. Ada has just as
many traps and pitfalls as C, which I disagree with), developing in
Ada with Ada-minded programmers will always be safer than developing
in C with C-minded programmers. In fact, developing in C with Ada-
minded programmers is safer, too.

The same reasoning applies to Spark vs. Ada.

--
Ludovic Brenta.

In my experience, a lot of the problem with lack of improvement in software 
quality and reliability when using Ada results from developers writing Ada 
code the same way they write C. Then you don't get Ada's advantages and 
debugging is just as much of a problem as with C, with problem symptoms 
showing up far from the cause, etc. If Ada isn't used the way it was 
designed to be used, you get results similar to when you try to use a pair 
of pliers as a hammer.

Bob Spooner 

Re: Software Quality in Science

[Rick]

Bob Spooner wrote:
> In my experience, a lot of the problem with lack of improvement in software
> quality and reliability when using Ada results from developers writing Ada
> code the same way they write C. Then you don't get Ada's advantages and
> debugging is just as much of a problem as with C, with problem symptoms
> showing up far from the cause, etc. If Ada isn't used the way it was
> designed to be used, you get results similar to when you try to use a pair
> of pliers as a hammer.

Darrel Ince (in the Guardian article) gave us an insight into the
problem when he wrote of scientific software being:
"... produced by software engineers who have to undergo a regime of
thorough testing, quality assurance and a change control discipline
known as configuration management."

Two things are worth noting in that quotation:
1.  My reading in the field leads me to believe that "thorough
testing, quality assurance and a change control discipline" are pretty
thin on the ground in COTS software houses;
2.  There aren't a lot of Software Engineers around (if you emphasise
'Engineer').  Bear in mind that Engineering is an attitude, not a
qualification.

I believe that these are two of the foci Bob is looking for.  Good
luck, Bob.

Re: Software Quality in Science

[Hibou57 (Yannick Duchêne)]

On 9 fév, 22:51, Jerry <lancebo...@qwest.net> wrote:
> Here is the link in the Guardian article to the original work:
>
> http://www.leshatton.org/Documents/Texp_ICSE297.pdf
Waw, this one is really not a love-letter :

After it introduced common pitfalls which may await the unwaries, it
goes straight to this words :
> In contrast, predicting the existence of a new sub-atomic particle
> seems a relatively straightforward exercise.

I surprised (if confirmed) that FORTRAN (which I've never experienced)
is even less safe than C.

Re: Software Quality in Science

[Brian Drummond]

On Thu, 11 Feb 2010 14:39:08 -0800 (PST), Hibou57 (Yannick Duch�ne)
<yannick_duchene@yahoo.fr> wrote:

>On 9 f�v, 22:51, Jerry <lancebo...@qwest.net> wrote:
>> Here is the link in the Guardian article to the original work:
>>
>> http://www.leshatton.org/Documents/Texp_ICSE297.pdf
>Waw, this one is really not a love-letter :
>
>After it introduced common pitfalls which may await the unwaries, it
>goes straight to this words :
>> In contrast, predicting the existence of a new sub-atomic particle
>> seems a relatively straightforward exercise.
>
>I surprised (if confirmed) that FORTRAN (which I've never experienced)
>is even less safe than C.

It certainly can be. 

Mis-spell a variable name, and you have just declared a new one, not accessed
the one you expected... (There are usually ways to turn implicit declarations
off, or make them compile time errors or warnings.)

- Brian

Re: Software Quality in Science

[Hibou57 (Yannick Duchêne)]

On 12 fév, 00:10, Brian Drummond <brian_drumm...@btconnect.com> wrote:
> It certainly can be.
>
> Mis-spell a variable name, and you have just declared a new one, not accessed
> the one you expected... (There are usually ways to turn implicit declarations
> off, or make them compile time errors or warnings.)
>
> - Brian

If you are not careful, a similar thing may occurs with Ada also :
derive from a tagged type, "redefine" a miss-spelled ancestor method,
and you've created a new method on your back.

But Ada provides a nice way to avoid it : systematically make use of
"overriding" / "not overriding". The dark side is that this is
optional and not required by the language rules.

Re: Software Quality in Science

[Hibou57 (Yannick Duchêne)]

On 12 fév, 00:21, Hibou57 (Yannick Duchêne) <yannick_duch...@yahoo.fr>
wrote:
> "overriding" / "not overriding". The dark side is that this is
> optional and not required by the language rules.
I've forgot to say an important thing : AdaControl may help a lot in
this area ;)

Re: Software Quality in Science

[Robert A Duff]

"Hibou57 (Yannick Duch�ne)" <yannick_duchene@yahoo.fr> writes:

> But Ada provides a nice way to avoid it : systematically make use of
> "overriding" / "not overriding". The dark side is that this is
> optional and not required by the language rules.

There's an option in GNAT to warn about this.
I don't remember if it's turned on by default.
(I think not.)

- Bob

Re: Software Quality in Science

[Hibou57 (Yannick Duchêne)]

On 12 fév, 01:06, Robert A Duff <bobd...@shell01.TheWorld.com> wrote:
> There's an option in GNAT to warn about this.
> I don't remember if it's turned on by default.
> (I think not.)
>
> - Bob

It's not turned by default with GNAT GPL, but do not know about GNAT
Pro.

As I've later commented in the previous post, AdaControl may also help
here (make this be along with the GNAT compiler dedicated option).

Re: Software Quality in Science

[Pascal Obry]

Le 12/02/2010 00:21, Hibou57 (Yannick Duch�ne) a �crit :
> But Ada provides a nice way to avoid it : systematically make use of
> "overriding" / "not overriding". The dark side is that this is
> optional and not required by the language rules.

Optional, yes. But GNAT has options to enforce use of the overriding
keywords. I'm using it is AWS for example.

Pascal.

-- 

--|------------------------------------------------------
--| Pascal Obry                           Team-Ada Member
--| 45, rue Gabriel Peri - 78114 Magny Les Hameaux FRANCE
--|------------------------------------------------------
--|    http://www.obry.net  -  http://v2p.fr.eu.org
--| "The best way to travel is by means of imagination"
--|
--| gpg --keyserver keys.gnupg.net --recv-key F949BD3B

Re: Software Quality in Science

[Pascal Obry]

Le 12/02/2010 01:06, Robert A Duff a �crit :
> There's an option in GNAT to warn about this.
> I don't remember if it's turned on by default.
> (I think not.)

For sure it's not.

Pascal.

-- 

--|------------------------------------------------------
--| Pascal Obry                           Team-Ada Member
--| 45, rue Gabriel Peri - 78114 Magny Les Hameaux FRANCE
--|------------------------------------------------------
--|    http://www.obry.net  -  http://v2p.fr.eu.org
--| "The best way to travel is by means of imagination"
--|
--| gpg --keyserver keys.gnupg.net --recv-key F949BD3B

Re: Software Quality in Science

[Stephen Leake]

Robert A Duff <bobduff@shell01.TheWorld.com> writes:

> "Hibou57 (Yannick Duchêne)" <yannick_duchene@yahoo.fr> writes:
>
>> But Ada provides a nice way to avoid it : systematically make use of
>> "overriding" / "not overriding". The dark side is that this is
>> optional and not required by the language rules.
>
> There's an option in GNAT to warn about this.
> I don't remember if it's turned on by default.
> (I think not.)

It's -gnatyO, and it's off by default. And turning it on exposes
missing 'overriding' keywords in the standard library. So it's not
quite ready for real time yet. (yes, I've filed bug reports)

-- 
-- Stephe

Re: Software Quality in Science

[Robert A Duff]

Stephen Leake <stephen_leake@stephe-leake.org> writes:

> It's -gnatyO, and it's off by default. And turning it on exposes
> missing 'overriding' keywords in the standard library. So it's not
> quite ready for real time yet. (yes, I've filed bug reports)

Thanks for the bug reports.  We should really fix those!
In fact, we should probably be compiling the runtimes
with -gnatyO.

- Bob

Re: Software Quality in Science

[Alex R. Mosteo]

Robert A Duff wrote:

> "Hibou57 (Yannick Duchêne)" <yannick_duchene@yahoo.fr> writes:
> 
>> But Ada provides a nice way to avoid it : systematically make use of
>> "overriding" / "not overriding". The dark side is that this is
>> optional and not required by the language rules.
> 
> There's an option in GNAT to warn about this.
> I don't remember if it's turned on by default.
> (I think not.)

I requested this feature at AdaEurope'08 IIRC. Nice to know it's there :)

Re: Software Quality in Science

[Colin Paul Gloster]

On Tue, 9 Feb 2010, Jerry sent:

|----------------------------------------------------------------------|
|"Here is the link in the Guardian article to the original work:       |
|                                                                      |
|http://www.leshatton.org/Documents/Texp_ICSE297.pdf                   |
|                                                                      |
|It appears that this work was done in the 1990s. The paper is actually|
|fairly entertaining to read, if for nothing else how scary it is. Here|
|is a choice comment, about a commercial program for use in the nuclear|
|engineering industry:                                                 |
|                                                                      |
|"This package climbed to an awe-inspiring 140 weighted static faults  |
|per 1000 lines of code, and in spite of the aspirations of its        |
|designers, amounted to no more than a very expensive random number    |
|generator.""                                                          |
|----------------------------------------------------------------------|

I am a nuclear scientist, and I confirm that software used for
supposedly simulating nuclear reactions is dangerous. For example,
HTTP://HyperNews.SLAC.Stanford.edu/HyperNews/geant4/get/phys-list/545.html

Nuclear electricity should be illegal.

|----------------------------------------------------------------------|
|"And this comment which addresses the use of Ada:                     |
|                                                                      |
|"In C, note that function prototypes were well used only around 60% of|
|the time and as a result, interface faults accounted for about 24% of |
|the total. In other words, if function prototypes were mandated in all|
|C functions, 24% of all serious faults would disappear.  The          |
|computational scientist should not use this as an argument in favour  |
|of C++ or Ada in which they are mandated. A large number of new       |
|failure modes result from this action, which lack of space prohibits  |
|further discussion here. The net result of changing languages appears |
|to be that the overall defect density appears to be about the same,   |
|(Hatton 1997). In other words, when a language corrects one           |
|deficiency, it appears to add one of its own.""                       |
|----------------------------------------------------------------------|

Why did that entertain you?

Re: Software Quality in Science

[Colin Paul Gloster]

On Wed, 10 Feb 2010, Florian Weimer wrote:

|-------------------------------------------------------------------|
|"[..]                                                              |
|                                                                   |
|even lines of code) vary as much among programmers [..]"           |
|-------------------------------------------------------------------|

Of course. In C++ I produce
if(condition)
{
  statement;
}
but I have been profiling C++ code by someone else which contained
instead
if (condition) statement;
which with a profiler which profiles per line instead of per statement
does not enlighten me as to whether or not the condition has ever been
true.

|-------------------------------------------------------------------|
|"[..]  This is a bit sad because it means that                     |
|language design does not really matter as far as actual results are|
|concerned.                                                         |
|                                                                   |
|[..]"                                                              |
|-------------------------------------------------------------------|

You were mistaken.

Re: Software Quality in Science

[Colin Paul Gloster]

On Wed, 10 Feb 2010, Stephen Leake wrote:

|---------------------------------------------------------------------|
|"[..]                                                                |
|                                                                     |
|The programmers still have to specify the right switches, and pay    |
|attention to the results, and know how to fix them, which is a       |
|political/management/skill issue, not a language design issue.       |
|                                                                     |
|I know from personal experience that there is a satellite about to be|
|launched who's control code has many remaining C warnings. Sigh."    |
|---------------------------------------------------------------------|

Yes, people are idiots.

Colin Paul Gloster, who used to work for the European Space Agency and
who was forced to use C in a squanderous waste of taxpayers' monies

Re: Software Quality in Science

[Colin Paul Gloster]

On Thu, 11 Feb 2010, Brian Drummond wrote:

|-----------------------------------------------------------------------------|
|"[..]                                                                        |
|>                                                                            |
|>I surprised (if confirmed) that FORTRAN (which I've never experienced)      |
|>is even less safe than C.                                                   |
|                                                                             |
|It certainly can be.                                                         |
|                                                                             |
|Mis-spell a variable name, and you have just declared a new one, not accessed|
|the one you expected... (There are usually ways to turn implicit declarations|
|off, or make them compile time errors or warnings.)"                         |
|-----------------------------------------------------------------------------|

Some compilers for Fortran are better than Fortran. There is no way in
Fortran to enforce these sanity checks. Unlike the Ada standards, the
Fortran standards do not require illegal programs to be rejected by
the compilers: merely that legal programs be run correctly.

Re: Software Quality in Science

[Georg Bauhaus]

Colin Paul Gloster schrieb:
> On Wed, 10 Feb 2010, Stephen Leake wrote:
> 
> |---------------------------------------------------------------------|
> |"[..]                                                                |
> |                                                                     |
> |The programmers still have to specify the right switches, and pay    |
> |attention to the results, and know how to fix them, which is a       |
> |political/management/skill issue, not a language design issue.       |
> |                                                                     |
> |I know from personal experience that there is a satellite about to be|
> |launched who's control code has many remaining C warnings. Sigh."    |
> |---------------------------------------------------------------------|
> 
> Yes, people are idiots.

For making decisions (e.g. about ignoring warnings), programmers just have
a proven set of rules.  This set is defined by a certain rationality,
which is bounded, and not just technical:  it is also imposed by
tradition, knowledge, and management. It does sometimes make sense
---post hoc at least.  If this situation helps you live a peaceful life
with your family and friends, why bother with messy warnings at work?

At work, if the leading staff, adequately payed, thinks you should not
ignore warnings, then have them say when, and when not, to treat warnings
as errors. Or, at least have them delegate responsibility for defects
to you. OK, the latter is an unrelated issue.

Re: Software Quality in Science

[Hibou57 (Yannick Duchêne)]

Le Mon, 15 Feb 2010 12:08:11 +0100, Colin Paul Gloster  
<Colin_Paul_Gloster@acm.org> a écrit:
> I am a nuclear scientist, and I confirm that software used for
> supposedly simulating nuclear reactions is dangerous. For example,
> HTTP://HyperNews.SLAC.Stanford.edu/HyperNews/geant4/get/phys-list/545.html
Strange, he looks hesitating

-- 
No-no, this isn't an oops ...or I hope (TM) - Don't blame me... I'm just  
not lucky

Re: Software Quality in Science

[Maciej Sobczak]

On 15 Lut, 12:20, Colin Paul Gloster <Colin_Paul_Glos...@ACM.org>
wrote:

> if (condition) statement;

In Ada that would be:

if Condition then Statement; end if;

which is equally ugly.
Could you clarify your point?

--
Maciej Sobczak * www.msobczak.com * www.inspirel.com

Database Access Library for Ada: www.inspirel.com/soci-ada

Re: Software Quality in Science

[Colin Paul Gloster]

On Wed, 10 Feb 2010, Florian Weimer wrote:

|-------------------------------------------------------------------|
|"[..]                                                              |
|                                                                   |
|even lines of code) vary as much among programmers [..]"           |
|-------------------------------------------------------------------|

Colin Paul Gloster responded:

|----------------|
|"Of course. [..]|
|                |
|[..]"           |
|----------------|



On Mon, 15 Feb 2010, Maciej Sobczak wrote:

|------------------------------------------------------------------|
|"On 15 Lut, 12:20, Colin Paul Gloster <Colin_Paul_Glos...@ACM.org>|
|wrote:                                                            |
|                                                                  |
|> if (condition) statement;                                       |
|                                                                  |
|In Ada that would be:                                             |
|                                                                  |
|if Condition then Statement; end if;                              |
|                                                                  |
|which is equally ugly.                                            |
|Could you clarify your point?"                                    |
|------------------------------------------------------------------|


Of course <=> Yes. Yes even #lines of code vary among
programmers. Given one programmer and C++ then
if (condition) statement;
would result whereas changing the language but not the programmer
would result in
if Condition then Statement; end if;

A different programmer would produce
if (condition)
{
  statement;
}
or
if Condition
then
  Statement;
end if;
depending on language.

Yet a different programmer would produce
if (condition {
  statement;
}
or
if Condition then
  Statement;
end if;
depending on language.

Re: Software Quality in Science

[Colin Paul Gloster]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 2065 bytes --]

On Mon, 15 Feb 2010, Hibou57 (Yannick Duchêne) wrote:

|----------------------------------------------------------------------------|
|"Le Mon, 15 Feb 2010 12:08:11 +0100, Colin Paul Gloster                     |
|<Colin_Paul_Gloster@acm.org> a écrit:                                       |
|> I am a nuclear scientist, and I confirm that software used for            |
|> supposedly simulating nuclear reactions is dangerous. For example,        |
|> HTTP://HyperNews.SLAC.Stanford.edu/HyperNews/geant4/get/phys-list/545.html|
|Strange, he looks hesitating"                                               |
|----------------------------------------------------------------------------|

It is true that that person was unsure. I correctly asserted that that
code was buggy (as I found many bugs in that code myself) in
HTTP://HyperNews.SLAC.Stanford.edu/HyperNews/geant4/get/phys-list/545/1.html
and it was not contradicted, unless off the forum.

A developer of that code sent to me by email: "I'm thankful for you
criticism and your opinion, but before we can communicate you should
introduce yourself and explain, why you are looking inside the Geant4
implementation of the CHIPS physics package.

         The present status is: according to the contract with CERN
I've implemented in Geant4 a significant part of the CHIPS physics
package, which is enough for the independent CHIPS physics list in
Geant4. The only implementation, which I did not finish is the
implementation of the CHIPS elastic cross-sections. Now I've done it
only for nucleons and pions, but during the last three days of my
contract I can not manage the implementation of the elastic scattering
for kaons, hyperons, and anti-baryons. As I've already demonstrated it
even forpions and nucleons, the existing Geant4 cross-sections are
about 30% wrong. Everything (including the inelastic cross-sections,
which you mentioned) was made in a harry, as I had only one year for
the enormously big implementation. That is why you have so many
questions.

[..]"