C.A.R. Hoare on liability

C.A.R. Hoare on liability

[Wes Groleau]

Tying together two recent threads,
a quote from "The Emperor's Old Clothes":

    ... we asked our customers whether they wished us
    to provide an option to switch off these checks
    in the interests of efficiency on production runs. 
    Unanimously, they urged us not to--they already knew
    how frequently subscript errors occur on production
    runs where failure to detect them would be disastrous. 
    I note with fear and horror that even in 1980, language
    designers and users ahve not learned this lesson. 
    In any respectable branch of engineering, failure
    to observe such elementary precautions would have
    long been against the law.

-- 
Wes Groleau
http://freepages.rootsweb.com/~wgroleau

Re: C.A.R. Hoare on liability

[Mike Silva]

Wes Groleau <wesgroleau@despammed.com> wrote in message news:<3D0E09BA.A492AA3D@despammed.com>...
> Tying together two recent threads,
> a quote from "The Emperor's Old Clothes":
> 
>     ... we asked our customers whether they wished us
>     to provide an option to switch off these checks
>     in the interests of efficiency on production runs. 
>     Unanimously, they urged us not to--they already knew
>     how frequently subscript errors occur on production
>     runs where failure to detect them would be disastrous. 
>     I note with fear and horror that even in 1980, language
>     designers and users ahve not learned this lesson.

I sure don't understand this part.  It sounds as if "our customers"
were saying that, yes, we know how important it is to have checks
turned on even in production code, but, notwithstanding that
knowledge, we simply can't be trusted to leave them on ourselves, and
so we must be forced by the compiler to keep them turned on.  Very
odd!
 
>     In any respectable branch of engineering, failure
>     to observe such elementary precautions would have
>     long been against the law.

Yep!

Re: C.A.R. Hoare on liability

[Darren New]

Mike Silva wrote:
> I sure don't understand this part.  It sounds as if "our customers"
> were saying that, yes, we know how important it is to have checks
> turned on even in production code, but, notwithstanding that
> knowledge, we simply can't be trusted to leave them on ourselves

Or "It's so important to keep them on that it's so unlikely we'll want to
turn them off that it's not worth us paying you to develop the option to do
so."

-- 
Darren New 
San Diego, CA, USA (PST). Cryptokeys on demand.
** http://home.san.rr.com/dnew/DNResume.html **
** http://images.fbrtech.com/dnew/ **

     My brain needs a "back" button so I can
         remember where I left my coffee mug.

Re: C.A.R. Hoare on liability

[Larry Kilgallen]

In article <27085883.0206190814.67fc4825@posting.google.com>, mjsilva697@earthlink.net (Mike Silva) writes:
> Wes Groleau <wesgroleau@despammed.com> wrote in message news:<3D0E09BA.A492AA3D@despammed.com>...
>> Tying together two recent threads,
>> a quote from "The Emperor's Old Clothes":
>> 
>>     ... we asked our customers whether they wished us
>>     to provide an option to switch off these checks
>>     in the interests of efficiency on production runs. 
>>     Unanimously, they urged us not to--they already knew
>>     how frequently subscript errors occur on production
>>     runs where failure to detect them would be disastrous. 
>>     I note with fear and horror that even in 1980, language
>>     designers and users ahve not learned this lesson.
> 
> I sure don't understand this part.  It sounds as if "our customers"
> were saying that, yes, we know how important it is to have checks
> turned on even in production code, but, notwithstanding that
> knowledge, we simply can't be trusted to leave them on ourselves, and
> so we must be forced by the compiler to keep them turned on.  Very
> odd!

How about:

	We are never going to turn them off, so please don't make
	our user interface any more complicated than it already is.

Re: C.A.R. Hoare on liability

[Wes Groleau]

> > I sure don't understand this part.  It sounds as if "our customers"
> > were saying that, yes, we know how important it is to have checks
> > turned on even in production code, but, notwithstanding that
> > knowledge, we simply can't be trusted to leave them on ourselves, and
> > so we must be forced by the compiler to keep them turned on.  Very
> > odd!
> 
> How about:
> 
>         We are never going to turn them off, so please don't make
>         our user interface any more complicated than it already is.

How about:

   We know how stupid it would be to turn them off,
   and we don't want to risk having them turned off
   by the inevitable idiot that will slip past
   even the best employment screening.

or

   _We_ understand, but we don't want to take the
   chance that _management_ will order them turned
   off under pressure from _our_ customer to squeeze
   one more millisecond out of the cycle time.

-- 
Wes Groleau
http://freepages.rootsweb.com/~wgroleau

Re: C.A.R. Hoare on liability

[Marin David Condic]

I'm confused about where this is going. Are we talking about creating some
sort of "User Selectable Speed Option" in an application that allows the
user to turn off runtime checks? (Never seen that.) In which case, if the
"customer" doesn't want it, then you don't put one of those in. Or is the
notion that the "customer" wants the program image built with all runtime
checks enabled - implying that the compiler should never allow them to be
turned off?

If it is the latter, it seems intuitively obvious to even the most casual
observer that unless 100% of all customers everywhere are *insisting* on
full runtime checking (Never had one who went beyond "It needs to work all
the time...") that a developer needs to have the option of turning the
checks off for whatever percentage of the customers out there want to insist
that you do so.

So maybe I don't get where C.A.R. Hoare is going with this - you want to be
"customer driven"? Or is it more important that you build "good" software -
which *can* be done with runtime checks turned off when necessary? (Surely,
he can't be convinced that it is *never* necessary to turn off runtime
checks, can he?)

MDC
--
Marin David Condic
Senior Software Engineer
Pace Micro Technology Americas    www.pacemicro.com
Enabling the digital revolution
e-Mail:    marin.condic@pacemicro.com


"Wes Groleau" <wesgroleau@despammed.com> wrote in message
news:3D10C557.11C9DBC2@despammed.com...
>
> How about:
>
>    We know how stupid it would be to turn them off,
>    and we don't want to risk having them turned off
>    by the inevitable idiot that will slip past
>    even the best employment screening.
>
> or
>
>    _We_ understand, but we don't want to take the
>    chance that _management_ will order them turned
>    off under pressure from _our_ customer to squeeze
>    one more millisecond out of the cycle time.

Re: C.A.R. Hoare on liability

[Wes Groleau]

> So maybe I don't get where C.A.R. Hoare is going with this - you want to be
> "customer driven"? Or is it more important that you build "good" software -
> which *can* be done with runtime checks turned off when necessary? (Surely,
> he can't be convinced that it is *never* necessary to turn off runtime
> checks, can he?)

I have no idea what he thinks now.  :-)

But in the quote, he states that he (or his
company) suggested an option to turn off the checks
and the customers said "No!"

The other extreme is a language that does not
have the checks, and when offered a "helper app"
that might do _some_ of the checks, some users
say "No!"

Seems like Ada has a nice balance between those
extremes.

-- 
Wes Groleau
http://freepages.rootsweb.com/~wgroleau

Re: C.A.R. Hoare on liability

[Marin David Condic]

That would be my opinion - provide checks as the default and if I really,
really, really need that extra bit of speed on some segment of code, I'll
turn them off manually. For most programmers, I wouldn't even tell them how
to do it because they'd never need it, but it should be there as a back
door. (But that's the kind of thing you do as a policy for the app
development - stipulate what options are the norm for the project.)

MDC
--
Marin David Condic
Senior Software Engineer
Pace Micro Technology Americas    www.pacemicro.com
Enabling the digital revolution
e-Mail:    marin.condic@pacemicro.com


"Wes Groleau" <wesgroleau@despammed.com> wrote in message
news:3D1338A5.A34E6C7A@despammed.com...
>
> Seems like Ada has a nice balance between those
> extremes.
>

Re: C.A.R. Hoare on liability

[Robert Dewar]

Wes Groleau <wesgroleau@despammed.com> wrote in message news:<3D0E09BA.A492AA3D@despammed.com>...
> Tying together two recent threads,
> a quote from "The Emperor's Old Clothes":
> 
>     ... we asked our customers whether they wished us
>     to provide an option to switch off these checks
>     in the interests of efficiency on production runs. 
>     Unanimously, they urged us not to--they already knew
>     how frequently subscript errors occur on production
>     runs where failure to detect them would be disastrous. 
>     I note with fear and horror that even in 1980, language
>     designers and users ahve not learned this lesson. 
>     In any respectable branch of engineering, failure
>     to observe such elementary precautions would have
>     long been against the law.

Three comments

First, runtime checks can be deadly if you have not done proper
analysis
of how they can be handled, since they can turn trivial errors that
would
not intefere with overall correct function into disasters (Ariane5 is
an example of this in action).

Second, in safety critical code you often turn run time checks off,
because
you rely on other means to ensure that these checks can never fail.
See for
example the work that Praxis has done in proving programs to be
exception
free. With such a proof in hand, run time checks can be a menace for
certification since you have a whole bunch of useless deactivated
code.

Third, there are situations in which the extra overhead from runtime
checks,
small though it may be, is unacceptable. It is no use saying to
someone, sorry
we know that if the checks could be turned off, you could use Ada just
fine,
but we have decided in Ada 0X that it was a terrible idea to allow
people to
do this, so you will have to use C instead.

As to customers who can't trust themselves to follow their own
procedures, I
have no sympathy whatever. Procedures should be enforced by a
combination of
review and tools, and failure to put in the effort for such
enforcement is
sloppy workmanship in my view.

For example, I would think that project files that describe the
required
compilation options should be under very strict configuration control,
and
not something that can casually be modified by someone who does not
know
what they are doing.

Re: C.A.R. Hoare on liability

[Robert I. Eachus]

Robert Dewar wrote:

  
> Three comments
> 
> First, runtime checks can be deadly if you have not done proper
> analysis of how they can be handled, since they can turn trivial errors that
> would not intefere with overall correct function into disasters (Ariane5 is
> an example of this in action).


I agree with the point, but not the example.  For Ariane 4, the analysis 
was carried out, and whether or not you agree with the final decision 
for Ariane 4, the decision was well thought out.  The disaster was that 
the Araine 4 analysis was carried out absent the Ariane 5 requirements 
for political reasons, and the Ariane 5 requirements analysis was never 
done.

Was it the first TriAda where I first heard that the problem with reuse 
is that the Mark II tank is not the same as the Mark I tank?



> Second, in safety critical code you often turn run time checks off,
> because you rely on other means to ensure that these checks can never fail.
> See for example the work that Praxis has done in proving programs to be
> exception free. With such a proof in hand, run time checks can be a menace for
> certification since you have a whole bunch of useless deactivated code.


Agreed, but see above.  The analysis must be redone for reuse.

 
> Third, there are situations in which the extra overhead from runtime checks,
> small though it may be, is unacceptable. It is no use saying to someone, sorry
> we know that if the checks could be turned off, you could use Ada just fine,
> but we have decided in Ada 0X that it was a terrible idea to allow people to
> do this, so you will have to use C instead.


Agreed, and I really like the methods you use in GNAT.  If code depends 
on a particular exception, explicitly turn it on in the code.  If code 
depends on some exception not occcuring, turn the check off.  That way 
changes in global settings don't affect the correctness of the program.


> For example, I would think that project files that describe the required
> compilation options should be under very strict configuration control, and
> not something that can casually be modified by someone who does not know
> what they are doing.


Definitely right.  But the problem is that if you drop the configuration 
change request on some VP's desk for his signature, you are likely to 
get hauled on the carpet if not fired.  Or worse yet, the VP will sign 
off without doing the engineering analysis needed.  Look at what 
happened on Challenger.  The engineer on the spot refused to sign off, 
and a corporate VP two time zones away--and not a licensed professional 
engineer--signed off on the boosters and faxed the OK to NASA.

(NASA should not have gone along with this.  But this was the first 
shuttle flight where an engineer was not in the NASA chain of command. 
Beggs had been asked to step aside due to some accounting irregularities 
on government programs while he was on the Board of Directors at 
Lockheed.  The acting head of NASA was not an engineer, and by the time 
Beggs was cleared, it was too late.)

 

Re: C.A.R. Hoare on liability

[Robert Dewar]

"Robert I. Eachus" <rieachus@attbi.com> wrote in message news:<3D1390D0.7040709@attbi.com>...

> I agree with the point, but not the example.  For Ariane 4, the analysis 
> was carried out, and whether or not you agree with the final decision 
> for Ariane 4, the decision was well thought out.  The disaster was that 
> the Araine 4 analysis was carried out absent the Ariane 5 requirements 
> for political reasons, and the Ariane 5 requirements analysis was never 
> done.

I disagree. Here you have a case in the Ariane4 code where a check was being
made at runtime which had the quality that if the check failed, disaster
would occur. There are two possibilities

1. In the Ariane4 code, it was demonstrated that this check could never fail.
In that case, the check should not have been there.

2. In the Ariane4 code, it was NOT demonstrated that this check could
never fail. In that case, they were just lucky that no Ariane4 blew up.

I will repeat. You should NEVER have a runtime check in your code where it
is the case that failing the check is a more serious situation than not doing
it at all. Casually putting in checks is very likely to generate such cases.

My understanding of the Ariane case is that this check was casually put in,
in other words it was put in WITHOUT any analysis that said this check was
needed. Deployed code should not have such checks.

If you leave checks on throughout an Ada program, then a complex analysis is
required to ensure that if any check fails, it is properly handled in a manner
that does not do more damage than failing the check.

If you do not leave checks on, then a complex analysis is required to ensure
that nothing serious fails that would have been caught by a check.

In the absence of any careful analysis, it is not clear whether it is better
to leave on all checks or not. The important thing is to realize that in the
absence of a careful analysis, turning all checks on is not necessarily
safer than leaving them off.

Re: C.A.R. Hoare on liability

[Jack Flynn]

"Robert Dewar" <dewar@gnat.com> wrote in message
news:5ee5b646.0206220514.55f8cf9a@posting.google.com...
>
> My understanding of the Ariane case is that this check was casually put
in,
> in other words it was put in WITHOUT any analysis that said this check was
> needed. Deployed code should not have such checks.
>
In the Ariane IV case, analysis was done to convince themselves that
they did not have a "generic software fault". The checks were to trap
faulty hardware and get a failed channel offline to prevent actuator force
fights
or other interference with the unfailed channel.

Re: C.A.R. Hoare on liability

[Mark Biggar]

Robert Dewar wrote:
> 
> "Robert I. Eachus" <rieachus@attbi.com> wrote in message news:<3D1390D0.7040709@attbi.com>...
> 
> > I agree with the point, but not the example.  For Ariane 4, the analysis
> > was carried out, and whether or not you agree with the final decision
> > for Ariane 4, the decision was well thought out.  The disaster was that
> > the Araine 4 analysis was carried out absent the Ariane 5 requirements
> > for political reasons, and the Ariane 5 requirements analysis was never
> > done.
> 
> I disagree. Here you have a case in the Ariane4 code where a check was being
> made at runtime which had the quality that if the check failed, disaster
> would occur. There are two possibilities
> 
> 1. In the Ariane4 code, it was demonstrated that this check could never fail.
> In that case, the check should not have been there.
> 
> 2. In the Ariane4 code, it was NOT demonstrated that this check could
> never fail. In that case, they were just lucky that no Ariane4 blew up.
> 
> I will repeat. You should NEVER have a runtime check in your code where it
> is the case that failing the check is a more serious situation than not doing
> it at all. Casually putting in checks is very likely to generate such cases.
> 
> My understanding of the Ariane case is that this check was casually put in,
> in other words it was put in WITHOUT any analysis that said this check was
> needed. Deployed code should not have such checks.

No the situation was more complicated then that.  Ariane 4 
had, like most rocket systems, redundant hardware sets each 
with its own, independent sensor set.  Careful analysis 
was done that showed that given the possible flight profiles, 
the only way the code should see an overflow was if the 
sensor suite was packed up and sending bogus data.  The check 
for overflow was thus used to detect hardware problems and 
the exception raised was to notify the upper control code 
that that particular hardware set was offline and to ignore 
it.  Ariane 5 had a very different flight profile including 
much higher accelerations.  So when the software was reused, 
without rechecking the analysis against the new Ariane 5 
flight profile, it saw a real overflow that didn't have 
anything to do with hardware faults, so all the redundent
hardware suites overflowed and threw an exception all at the 
same time.  This put the rocket out of control and I believe 
it was destroyed by the range safety system.

So both the check and the exception were reasonable and 
correct for the Ariane 4 and would not have crashed it.  So it
was reusing the code without redoing the analysis or even 
testing it against the new flight profile that cause the 
problems, not the check in the code.  This problem would have 
arisen regardless of the language used.

--
Mark Biggar
mark.a.biggar@attbi.com

Re: C.A.R. Hoare on liability

[Robert I. Eachus]

Mark Biggar <mark.a.biggar@attbi.com> wrote in message news:<3D14AA34.E8FFBBBB@attbi.com>...
 
> So both the check and the exception were reasonable and 
> correct for the Ariane 4 and would not have crashed it.  So it
> was reusing the code without redoing the analysis or even 
> testing it against the new flight profile that cause the 
> problems, not the check in the code.  This problem would have 
> arisen regardless of the language used.

Right, in fact the actual situation was worse than that.  I won't go
into all the gory details, but part of the assumption of hardware
failure was to dump raw data from the intertial guidance system on the
buss to the engine control system.  This caused the engines to deflect
enough to cause the "stack" to break up, and at this point the range
safety officer had to destroy the rocket.

Where were the limits on the engine deflection that should have
prevented this?  They were set for the Ariane 4.  The Ariane 5 stack
was more fragile, and the engines more powerful.  So if the Arianne 4
software developers had gotten permission to put in a local exception
handler for the piece of software that failed on the Ariane 5, the
Ariane 5 would have made it past 38 seconds into the flight--and then
would have destroyed itself if it hit wind shear at a higher altitude.

As far as I am concerned, that was the real disaster.  The complex
path that started the actual failure involved an exception raised
because the Ariane 5 exceeded one of the physical limits for the
Ariane 4.  But there were dozens of Ariane 4 physical parameters built
into the software which could have been the primary cause of failure. 
The actual failure only ran into three of them.  (Horizontal movement
from point of launch in the first 40 seconds of flight, stack moment
of inertia, maximum engine deflection.)

Re: C.A.R. Hoare on liability

[SteveD]

"Robert Dewar" <dewar@gnat.com> wrote in message
news:5ee5b646.0206210355.3533be8f@posting.google.com...
[snip]
>
> First, runtime checks can be deadly if you have not done proper analysis
> of how they can be handled, since they can turn trivial errors that would
> not intefere with overall correct function into disasters (Ariane5 is
> an example of this in action).
>
Of course the errors detected by runtime checks might not be trivial errors
as well.  I can recall a case where an occasional overflow in calculations
in an old FORTRAN program caused some very large machinery to move to a very
awkward position.

As you say there is no substitute for doing a proper analysis.

SteveD