Ada Compiler Evaluation System Version 2.0

Ada Compiler Evaluation System Version 2.0

[Philip Brashear]

TO ALL WHO ARE INTERESTED IN PERFORMANCE MEASUREMENT OF ADA CODE:

Version 2.0 of the Ada Compiler Evaluation System (ACES) has just been
released for public review.  It is available at the AJPO host:

    sw-eng.falls-church.va.us: /public/AdaIC/testing/aces

The ACES provides performance tests, test management software, and
analysis software for assessing the performance characteristics of
Ada compilation and execution systems.  Functionality/usability assessor 
tools are also provided for examining the implementation's diagnostic
system, library management system, and symbolic debugger, as well as
for determining compile-time and run-time capacities of the implementation.

Version 2.0 enhancements include:

1. Setup
   The Setup facility, introduced in Version 1.1, replaces the old PRETEST 
   process.  Setup prompts the user for information about the operating
   system and compiler/linker commands, and uses this information to generate
   command script files for compiling global support modules, creating test
   programs for the support modules, creating test management tools, and
   creating analysis tools.  The following improvements are offered in this
   version:

   a. Default information may be used for several common operating systems
      and compilers, minimizing the amount of data supplied by the user.

   b. The user may interactively modify selected operating system or compiler 
      information and automatically re-generate the command scripts.

   c. The user must designate the tested system as an implementation of Ada 83
      or of Ada 95.  This information is used to take advantage of Ada 95's
      required mathematics packages and to suppress or allow the selection
      of Ada 95 performance tests.

2. Performance tests

   Version 2.0 includes a limited number of tests for the performance of 
   Ada 95 features.  Some new Ada 83 tests are provided for comparisons of
   Ada 95 and Ada 83 coding styles.  There are 19 Ada 95 tests and 9 new
   Ada 83 tests.

3. Compatibility

   An effort has been made to remove all incompatibilities with Ada 95.
   That is, all software should be compatible with Ada 95 requirements.
   Code that is not compatible with Ada 83 is isolated so that the Ada 83
   user should never encounter it.

4. Quick-Look

   A stand-alone Quick-Look facility has been provided that should enable
   a user to get an execution-speed profile of an implemenation in less than
   a day (from download to report generation).  This facility provides 60
   performance tests and all the necessary support software and data for
   compiling them, executing them, and generating reports from the results.
   All the new ACES performance tests are included, along with 28 previously
   existing ACES performance tests.  The latter 28 tests were carefully
   chosen to provide the same degree of language coverage as the most useful
   of the PIWG benchmark suite.

   By default, Quick-Look reports elapsed (wall-clock) time, using the Ada
   Calendar.Clock function.  CPU time measurements can be used, but the user
   must supply a CPU time function.

   A report generator extracts the execution time data (and other
   ancillary data) from the log file and produces three reports: a textual
   report showing the execution times; an ancillary report showing other useful
   data output by the tests; and an optional comma-delimited report of the
   execution times.  The comma-delimited report can be imported into many
   popular spreadsheet programs for further analysis.

   All the files needed for Quick-Look are contained in the single compressed
   file   ...aces/v2.0/qlook.zip.  Instructions for its use are provided in
   the ACES Primer, which is included in "qlook.zip" as files "zd_primr.txt"
   (ASCII) and "zd_primr.ps" (PostScript).

5. Single-System Analysis

   The Single-System Analysis subsystem has been enhanced to perform 
   comparisons of the performance of Ada 95 and Ada 83 methods of achieving
   the same goals.  For example, comparisons are done to determine the
   relative efficiency of monitor tasks and protected objects when used to
   provide coordinated access to protected data.



The ACES development team has established a group mail-alias (mailing list)
for discussing ACES-related issues.  Mail sent to 
      perform@sw-eng.falls-church.va.us
is automatically distributed to all addresses in the list.

If you wish to be added to the list, please express your wish in an e-mail
message to Phil Brashear: "brashear@sw-eng.falls-church.va.us".