Re: ACVC and Compiler Quality

[gmckee@cloudnine.com (Gary McKee)]

In article <316B9234.35AA@lfwc.lockheed.com>,
Ken Garlington <garlingtonke@lfwc.lockheed.com> wrote:
 > Robert Dewar wrote:
 > > 
 > >    The point is not that the ACVC suite does not measure product
quality,
 > >    it is that it does not guarantee product quality, because it only
 > >    addresses certain quality aspects.
 > 
 > OK, let's say that ACVC measures some aspects of product quality, and
that
 > I misunderstood the statement in the old AJPO guide about not using ACVC
to
 > determine "suitability for any particular purpose," or words to that
effect.
 > Is what it measures meaningful to the end user? In other words, does the
 > presence of the ACVC have a _measureable_ effect on the portability of
code
 > between compilers? How is this _measured_?
 > 
 > It's clear you can respond by saying, "Well, without the ACVC, a
reasonable
 > person would conclude that these attributes would be worse." However,
you've
 > implied that the ACVC suite _measures_ relevant aspects of product
quality.
 > From this, one could assume that changes in the ACVC test suite, say
from 1.x
 > to 2.x, might change this measure. How did it change, or how is it
expected
 > to change, such measures as portability, etc?
 > ...
 --------------------------------------------------------
There is a significant difference in purpose between validation (ACVC) and
evaluation (ACES). 

A different test suite exists PRECISELY for the purpose of
measuring/evaluating the "quality" of Ada compilation systems. It is quite
large, complex, and very useful.

I have used this system extensively and I do recommend it for the quality
assessment work that you are interested in.
--------------------------------------------------------
INTRODUCTION TO THE ADA COMPILER EVALUATION SYSTEM (ACES)

The Ada Compiler Evaluation System (ACES) provides performance tests, test
management software, and analysis software for assessing the performance
characteristics of Ada compilation and execution systems.
Functionality/usability assessor tools are also provided for examining the
implementation's diagnostic system, library management system, and symbolic
debugger, as well as for determining compile-time and run-time capacities
of the implementation.

http://sw-eng.falls-church.va.us/AdaIC/testing/aces/



--------------------------------------------------------------------
Gary McKee                           McKee Consulting
gmckee@cloudnine.com                 P. O. Box 3009
voice: (303) 795-7287                Littleton, CO 80161-3009
WWW home page =>                     <http://www.csn.net/~gmckee/>
--------------------------------------------------------------------