Re: Are un-validated compilers unsafe?

[Jim Chelini <jchelini@ma.aonix.com>]

Mark Elson wrote:
> 
> This question was prompted by the fact that a new space project may be
> using GNAT in conjunction with an un-validated RTOS on the grounds that
> the combination is in widespread use and that GNAT is a "very good"
> compiler (also due to the abundance of developers as well as users). I
> was somewhat surprised that they could get away with this (although
> their requirement is more reliability than safety). Does this mean there
> is not much motivation for vendors to get their compilers validated
> these days?

Don't confuse compiler validation with safety.  Compiler validation is a 
determination that the compiler conforms to the language definition.  It 
is not a measure of assurance or reliability.  Although it is a large
test suite.

> 
> In any case, does the fact that a compiler-OS-processor has not been
> validated mean that it is unsafe (or unreliable), i.e. that it is not
> suitable for use in safety-critical applications? I'm guessing, looking
> at a number of software safety requirements, that if you don't use a
> validated combination then the onus is on you is to verify down to
> object code level, i.e. validation may save you work.

For safety critical applications, there should be a governing safety
standard for the
project such as Do-178B (avionics), IEC-880 (Nuclear), NASA's Safety
Standard (don't remember the
title of the top of my head), etc.

These standards define the necessary process requirements and help to
determine (based on a system safety
analysis) the level of assurance the software must satisfy.  Any
software in the fielded system
must undergo testing and analysis applicable for the given safety
level.  This includes any runtime/os code.
For the most critical systems this typically  requires full disclosure
of the source and significant testing, review, analysis, and
documentation.

The use of a validated compiler does not reduce this burden. For a Level
A application under DO-178B, structural coverage is typically performed
at the machine code level. Under Do-178B, someone may choose to
"qualify" the compiler as a development tool.  This requires that the
compiler meet ALL of the objectives of DO-178B that apply to the level
of criticality for the application.  In other words, if you want to take
credit for using a "qualified" compiler for a Level A system to avoid
coverage testing at the machine code level, you would have to do the
coverage analysis on the compiler itself and provide a complete mapping
of source to object code.  To date, this has proven too great a cost to
be practical.

Instead, find a vendor who has worked to these standards and can provide
the materials for the runtime and help guide the testing for the
application.


> 
> I've had a look at the EDS site and the choice for embedded applications
> using Ada 95 seems restricted, especially wrt. to the RTOS choice. Are
> vendors not bothering to validate their compilers & OSs (or is it
> something that's done on demand and so additions are only likely to
> occur if a particular project can afford the validation). Is obtaining
> validation an expensive exercise anyway? Do vendors subsidise it if a
> projects chooses to go that route?
> 
> Are there other means by which compiler/OS/target combinations get
> certified or even proven by common use? Is there a list of such?


Common use is not generally accepted for safety critical system. 
Service history must be carefully documented and shown to be relevant to
the new application.  


Jim Chelini
Aonix
Mgr, Safety Critical Software
> 
> Many thanks for any replies. Sorry for all the questions - I'm new to
> Ada and safety-critical software.
> 
> --
> Mark Elson