Re: Martian Rover Spirit's mulitasking software reset.

Re: Martian Rover Spirit's mulitasking software reset.

[jimmaureenrogers]

ME wrote:
> The mars rover Spirit suffered a software reset as a result of a multitasking problem.
>
> "The apparent cause was that the rover's central processing unit was overworked as several tasks were running in parallel at the time."
>
>
>
> The rover uses VX works and I believe C. Testing obviously did not catch this fault. Would the use of Ada  made any difference?

Not necessarily. While there could be many reasons for this problem,
one likely one was the dynamic creation of Pthreads. A similar problem
could have been designed into an Ada program using dynamic creation of
tasks from a task type.

The problem appears to be fundamentally caused by poor design. Ada does
not prevent poor design.

On the other hand, it may have been easier to spot the problem during
code reviews using Ada, since Ada tasking is done at a much higher
level of abstraction than Pthreads using C. The higher level of
abstraction often aids in human understanding of code.

Jim Rogers

Re: Martian Rover Spirit's mulitasking software reset.

[Pascal Obry]

jimmaureenrogers@worldnet.att.net a �crit :
> Not necessarily. While there could be many reasons for this problem,
> one likely one was the dynamic creation of Pthreads. A similar problem
> could have been designed into an Ada program using dynamic creation of
> tasks from a task type.
> 
> The problem appears to be fundamentally caused by poor design. Ada does
> not prevent poor design.

Agreed, but the time you spent on "fighting the language" would have
better used on thinking about a good design. This is at least what Ada
gives you, the time to think to useful problem on your project.

Pascal.

-- 

--|------------------------------------------------------
--| Pascal Obry                           Team-Ada Member
--| 45, rue Gabriel Peri - 78114 Magny Les Hameaux FRANCE
--|------------------------------------------------------
--|              http://www.obry.net
--| "The best way to travel is by means of imagination"
--|
--| gpg --keyserver wwwkeys.pgp.net --recv-key C1082595

Re: Martian Rover Spirit's mulitasking software reset.

[Martin Dowie]

ME wrote:
> The mars rover Spirit suffered a software reset as a result of a 
> multitasking problem.
> 
> "The apparent cause was that the rover's central processing unit was 
> overworked as several tasks were running in parallel at the time."
> 
>  
> 
> The rover uses VX works and I believe C. Testing obviously did not catch 
> this fault. Would the use of Ada  made any difference?
> 

My understanding is that there was a "priority inverstion", so had they 
used "pragma Locking_Policy (Ceiling_Locking);" (RM  D.3 Priority 
Ceiling Locking) then the problem could have been avioded.

Of course, they might not have used it even if they were using Ada. :-)
Cheers
-- Martin

Re: Martian Rover Spirit's mulitasking software reset.

[jimmaureenrogers]

Martin Dowie wrote:
> My understanding is that there was a "priority inverstion", so had they
> used "pragma Locking_Policy (Ceiling_Locking);" (RM  D.3 Priority
> Ceiling Locking) then the problem could have been avioded.
>
> Of course, they might not have used it even if they were using Ada. :-)
> Cheers

Problems such as priority inversions, deadlocks, livelocks, and race
conditions are far too common when writing Pthreads. While Pthreads
provides tools which, when used in the proper combination, allow the
programmer to avoid such problems, the programmer is expected to
know all the necessary combinations.

This follows the C programming philosophy. Assume that the programmer
is an expert and makes no mistakes.

Another possible contributor to the problem when using C is that
concurrency issues are not common considerations for C programmers.
C provides no syntax or for concurrency. C programmers are not as
familiar with the design of concurrent systems as are Ada programmers.

None of these reasons ensures that Ada programmers could not have
produced an equivalent mess. They do provide a higher likelihood of
producing a successful program.

Jim Rogers

Re: Martian Rover Spirit's mulitasking software reset.

[Bob Spooner]

<jimmaureenrogers@worldnet.att.net> wrote in message 
news:1157547128.205328.146920@m79g2000cwm.googlegroups.com...
> Martin Dowie wrote:
> ...
> This follows the C programming philosophy. Assume that the programmer
> is an expert and makes no mistakes.
> ...
> Jim Rogers
>

C-type languages assume that you mean what you say.
Ada insists that you say what you mean.

Bob Spooner 

Re: Martian Rover Spirit's mulitasking software reset.

[Mark Lorenzen]

Martin Dowie <martin.dowie@afraidof.co.uk> writes:

> ME wrote:
> > The mars rover Spirit suffered a software reset as a result of a
> > multitasking problem.
> > "The apparent cause was that the rover's central processing unit was
> > overworked as several tasks were running in parallel at the time."
> >  The rover uses VX works and I believe C. Testing obviously did not
> > catch this fault. Would the use of Ada  made any difference?
> >
> 
> My understanding is that there was a "priority inverstion", so had

That is also my understanding.

> they used "pragma Locking_Policy (Ceiling_Locking);" (RM  D.3 Priority
> Ceiling Locking) then the problem could have been avioded.
>
> Of course, they might not have used it even if they were using Ada. :-)

As far as I know, they were aware of the priority inversion problem
before launch, but did not consider it important enough to warrant a
fix. I guess that a watchdog device simply kicks in and reboots the
system, should it hang for too long.

- Mark

Re: Martian Rover Spirit's mulitasking software reset.

[ME]

"Mark Lorenzen" <mark.lorenzen@surfpost.dk> wrote in message 
news:m34pvkobtg.fsf@localhost.localdomain...
> Martin Dowie <martin.dowie@afraidof.co.uk> writes:
>
>> ME wrote:
>> > The mars rover Spirit suffered a software reset as a result of a
>> > multitasking problem.
>> > "The apparent cause was that the rover's central processing unit was
>> > overworked as several tasks were running in parallel at the time."
>> >  The rover uses VX works and I believe C. Testing obviously did not
>> > catch this fault. Would the use of Ada  made any difference?
>> >
>>
>> My understanding is that there was a "priority inverstion", so had
>
> That is also my understanding.
>
>> they used "pragma Locking_Policy (Ceiling_Locking);" (RM  D.3 Priority
>> Ceiling Locking) then the problem could have been avioded.
>>
>> Of course, they might not have used it even if they were using Ada. :-)
>
> As far as I know, they were aware of the priority inversion problem
> before launch, but did not consider it important enough to warrant a
> fix. I guess that a watchdog device simply kicks in and reboots the
> system, should it hang for too long.
>

But one of the original problems with Spirit was that it was that it had 
reset itself once every hour until it almost ran out of power. This was 
originally due to a flash memory management error which in turn was due to a 
complication due to new software uploaded to it because the system was 
launched with a serious flaw that was identified later.  I recall a JPL 
scientist admitting that the software was written last because they could 
upload it. The entire mission was on a very tight schedule and as is far too 
often the case the software comes last.
> - Mark 

Re: Martian Rover Spirit's mulitasking software reset.

[Paul Anderson]

All:

>>>>The mars rover Spirit suffered a software reset as a result of a
>>>>multitasking problem.
>>>>"The apparent cause was that the rover's central processing unit was
>>>>overworked as several tasks were running in parallel at the time."
>>>> The rover uses VX works and I believe C. Testing obviously did not
>>>>catch this fault. Would the use of Ada  made any difference?
>>>>
>>>
>>>My understanding is that there was a "priority inverstion", so had
>>
>>That is also my understanding.

This isn't right.  The problem with Spirit was with the flash
memory.  It is hard to find a good description of exactly what
went wrong, but the reboots happened because there were too
many files open.  Whether Ada might have helped is really
impossible to say, given the paucity of information available.

The Mars Pathfinder mission from 1997 was the one that had the
priority inversion problem.  I doubt that they were aware of
the problem before launch, as it turned out to be so easy
to fix.  In VxWorks, when you create a semaphore you do
something like:

   sem_id = semMCreate(options);

where options is a bitmask that specifies the set of options
you want for the semaphore.  VxWorks implements the priority
inheritance protocol, so if you want to use it, you would do:

   sem_id = semMCreate(SEM_Q_PRIORITY | SEM_INVERSION_SAFE);

On the Pathfinder, they simply did not specify the
SEM_INVERSION_SAFE option.

Fortunately, the options were held in a global variable, and
they were able to upload a patch that updated that variable to
specify that option.

I don't believe that VxWorks had an implementation of the
priority ceiling protocol at that time.

For more information, see:
http://en.wikipedia.org/wiki/Priority_inversion
http://research.microsoft.com/~mbj/Mars_Pathfinder/Mars_Pathfinder.html

There was a third Mars mission that had a software problem: the Mars
Polar Lander.  In this case, during entry to the atmosphere, a sensor
generated a spurious signal indicating that the craft had landed, and
the software set a bit accordingly.  This shut down the descent engines
prematurely.  The craft then plummetted to the surface and was
destroyed.  The software should have detected that the sensor input
was invalid at that time, and unset the problematic bit.  Apparently
the addition of a single line of code would have prevented the problem.

It is hard to see how Ada could have helped here either.

Paul

Re: Martian Rover Spirit's mulitasking software reset.

[ME]

Paul,

I think that you are thinking go an earlier problem which was the flash but 
this latest problem was as described above. There have been others as 
well...
"Paul Anderson" <notme@nowhere.com> wrote in message 
news:44fff0ed$1_4@newsfeed.slurp.net...
> All:
>
>>>>>The mars rover Spirit suffered a software reset as a result of a
>>>>>multitasking problem.
>>>>>"The apparent cause was that the rover's central processing unit was
>>>>>overworked as several tasks were running in parallel at the time."
>>>>> The rover uses VX works and I believe C. Testing obviously did not
>>>>>catch this fault. Would the use of Ada  made any difference?
>>>>>
>>>>
>>>>My understanding is that there was a "priority inverstion", so had
>>>
>>>That is also my understanding.
>
> This isn't right.  The problem with Spirit was with the flash
> memory.  It is hard to find a good description of exactly what
> went wrong, but the reboots happened because there were too
> many files open.  Whether Ada might have helped is really
> impossible to say, given the paucity of information available.
>
> The Mars Pathfinder mission from 1997 was the one that had the
> priority inversion problem.  I doubt that they were aware of
> the problem before launch, as it turned out to be so easy
> to fix.  In VxWorks, when you create a semaphore you do
> something like:
>
>   sem_id = semMCreate(options);
>
> where options is a bitmask that specifies the set of options
> you want for the semaphore.  VxWorks implements the priority
> inheritance protocol, so if you want to use it, you would do:
>
>   sem_id = semMCreate(SEM_Q_PRIORITY | SEM_INVERSION_SAFE);
>
> On the Pathfinder, they simply did not specify the
> SEM_INVERSION_SAFE option.
>
> Fortunately, the options were held in a global variable, and
> they were able to upload a patch that updated that variable to
> specify that option.
>
> I don't believe that VxWorks had an implementation of the
> priority ceiling protocol at that time.
>
> For more information, see:
> http://en.wikipedia.org/wiki/Priority_inversion
> http://research.microsoft.com/~mbj/Mars_Pathfinder/Mars_Pathfinder.html
>
> There was a third Mars mission that had a software problem: the Mars
> Polar Lander.  In this case, during entry to the atmosphere, a sensor
> generated a spurious signal indicating that the craft had landed, and
> the software set a bit accordingly.  This shut down the descent engines
> prematurely.  The craft then plummetted to the surface and was
> destroyed.  The software should have detected that the sensor input
> was invalid at that time, and unset the problematic bit.  Apparently
> the addition of a single line of code would have prevented the problem.
>
> It is hard to see how Ada could have helped here either.
>
> Paul 

Re: Martian Rover Spirit's mulitasking software reset.

[Paul Anderson]

ME:

> I think that you are thinking go an earlier problem which was the flash but 
> this latest problem was as described above. There have been others as 
> well...

My bad!  I hadn't noticed this latest glitch.

Paul

Re: Martian Rover Spirit's mulitasking software reset.

[Jeffrey R. Carter]

ME wrote:
> 
> The rover uses VX works and I believe C. Testing obviously did not catch 
> this fault. Would the use of Ada  made any difference?

In theory, no.

In practice, yes.

-- 
Jeff Carter
"The time has come to act, and act fast. I'm leaving."
Blazing Saddles
36