ACVC, was Re: Ada Core Technologies and Ada95 Standards

ACVC, was Re: Ada Core Technologies and Ada95 Standards

[tmoran]

One view I've heard is that the ACVC doesn't test compilers so much
as it tests compiler writers.  Their understanding, as demonstrated
by an implementation, of the language.
  I wonder what fraction of production Ada programs are actually
compiled with the validated version of the compiler, with the
validated compiler option settings, etc.  Or even with 'home
validated' versions.

Re: ACVC, was Re: Ada Core Technologies and Ada95 Standards

[Robert Dewar]

tmoran says

"  I wonder what fraction of production Ada programs are actually
compiled with the validated version of the compiler, with the
validated compiler option settings, etc.  Or even with 'home
validated' versions."

By "the validated version", I suspect that Ted means the version that
was actually witness verified by the AVF. If so, the answer I would
guess is very few programs are actually compiled with this compiler.
However, this is misleading. If a vendor validates a compiler, and
subsequently updates the compiler, then providing the vendor runs
the full suite, and verifies compliant results as defined in the
validation procedures, the resulting upgraded compiler is in all
official respects considered to be fully validated (validation
would not mean much if this were not the case).

So, given the official definition of validated, the answer is that
many production Ada programs are compiled with validated compilers.
Actually the transition to Ada 95 has undoubtedly temporarily
decreased this percentage, since people are using Ada 95 compilers
before they are officially validated beause they want to use ada 95
ASAP. FOr example, the SOlaris version of GNAT is not yet validated,
but there are many production programs using this version (it is the
second most popular platform for supported use of GNAT -- the first
is SGI).

But, the issue of compiler option settings is a valid one. Given that
typical compilers have a HUGE number of possible settings of compiler
options (for GNAT it is in the tens of millions, 2**k gets quite big
as k increases :-), it is quite unlikely that there are many programs
that use identical settings. 

If passing the ACVC suite is critical enough, then it makes sense to do
a special run of the ACVC with the exact switches you wlil use, either
on your own, or asking the vendor to do it. However, in practice, as 
another thread of discussion has exhaustively examined, ACVC conformance
is not some kind of guarantee of conformance in general, so this kind of
special testing is reasonable only if you have reason to believe that
ACVC cnformance will be a weak link in the total reliability chain.
Otherwise you may find yourself spending a lot of resources showing that
one obscure ACVC test fails that does not concern you, resources that
would have better been spent in other compiler testing that may be more
effective in your particular setting.