Production Quality Ada Compiler Criteria

[macrakis@harvard.HARVARD.EDU (Stavros Macrakis)]

Compiler requirements beyond simple validation are very important in
procuring Ada compilers.  I agree in general with Hogan's list, but in
some places he requires too little, and in others too much.

Herewith, my comments.

2.1.1,2 ACVC tests are unusual and unrepresentative of real code: They
    are very short: startup time becomes disproportionate.  They have
    hardly any computation: runtime figures will be tiny and not
    terribly useful.  They bang on all parts of the language evenly,
    not in proportion to actual usage: real programs' efficiency
    depends on a very few constructs: inline code, array indexing,
    record selection, loops, subprograms.  You should choose better
    benchmarks.

2.2.1,2 How do you make the object code criteria meaningful?
    Problems: (1) what is an equivalent program; (2) what does this
    program do; (3) what sort of assembly coding are we talking about.

 (1) You must be sure that the assembler program is equally secure
    (Range checks, deref checks, etc.) and has the same runtime
    conventions.  For instance, the Intermetrics Ada compiler
    guarantees that every assignment statement will store to memory.
    Although it does excellent register allocation, it refuses to keep
    a variable's value only in a register between statements, to
    improve debuggability, error-tolerance, etc.  The compiler cannot
    understand algorithms well enough to suppress apparently necessary
    checks (e.g. sentinel termination).

 (2) Certain specialized applications may be amenable to assembler
    tricks beyond current compiler technology.

 (3) Quality of hand-written assembly code varies widely.

    Therefore, you should avoid comparing tuned assembler code with
    ordinary Ada code.  Of course, there are many applications where
    current compiler technology does better than typical hand code.


Capacity

-- Capacity is very important.  Your numbers for number of CU's and
    size of CU's seem reasonable; for very large projects, perhaps,
    the number of CU's (remember is-separate's) could be larger.

-- You should discuss program library size as well as number of CU's
    per program.  You should discuss program library functionality,
    capacity, and speed.  Support for sharing of program sublibraries
    is very important.

-- Source lines are probably not a good measure of capacity.  I would
    substitute something equally easy to count but more closely
    related to the content of the code, e.g. number of lexical tokens
    excluding comments, or number of semicolons.

-- No limit should be imposed on the frames an exception propagates
    through.  If an infinite recursion causes stack overflow, the
    resultant Storage_error should be trappable.  Good implementations
    allow as many levels of exception propagation as of subroutine
    calling (limited, I hope, only by stack size).

-- String is defined as array(Positive range <>).  You can just
    specify that Integer'last > 50,000.  Strings with smaller indexing
    types can always be defined by the user.

-- 256 enumeration literals is too few.  Many systems have > 256
    system calls, error types, syntactic productions, part types...

5.3.3 Hard packing is often undesirable on machines with expensive
    byte extraction.  This is what rep specs, Pack, and Optimize are for.

5.3.8 Contiguous bits is too tight: should a record composed of a
    boolean and a byte be stored with the byte non-aligned to be
    continguous to the bit?  Or do you allow the bit to be stored in a
    full byte for quicker access?

5.5.5 This is not reasonable.  A task creation, e.g., will take more
    resources than a subprogram call simply because there is more to
    initialize, there is synchronization overhead, and a new stack
    needs to be allocated.  I agree that task operations should not
    take <much> more than subprograms: `no more than' is too strong.

5.5.6 This criterion would be satisfied by having the compiler
    pre-sort the select alternatives alphabetically.  This is not what
    you have in mind, but it is unclear how to write a reasonable
    `fairness' spec.  Indeed, it is probably a bad idea.  A
    non-erroneous Ada program will work correctly and efficiently
    regardless of the implementation's select strategy.  (A debugging/
    validation tool that made one select alternative higher or lower
    priority than others could be useful.)  Relying on the fairness
    of the implementation is dangerous.

5.5.7 Not necessarily an ideal scheduling strategy.  You are specifying
    some sort of time-slicing: reasonable in many applications, not all.

    A greedy task merely needs to split into two to double its time
    slice, a common pathology in Unix.  To be complete, you need to
    mention priority in this section.

5.5.9 Add: or the finest time interval available in the hardware/OS,
    whichever is larger.

5.5.10 Is this needed?  Doesn't the ref man (9.6, 13.7.1) define it well
    enough?

5.6.3  This is too strong (halting problem).

5.7.1 This is too strong if taken literally, and too vague otherwise.
    The compiler should not maximize (sensu strictu) code sharing if
    runtime is an issue and the two instantiations are different enough.
    This is another reason for Pragma Optimize.

5.8.1 Add: The documentation should fully specify the machine language
    interface for subprograms with arguments and results of any Ada type.

5.8.2 Agreed, in general, but you should not expect that every language
    system on your machine will be able to be made to interface easily.
    I would tighten this requirement to something like `shall provide
    (in order of importance) the pragma Interface for (1) the standard
    systems programming language of the machine/OS (e.g. C for Unix,
    PL/I for Multics, ...)  (2) the standard applications programming
    language(s) of the machine/OS (e.g. Fortran, Cobol) and (3) Fortran
    (as the language most used for large portable libraries).'  Even
    then, there may be some difficulties because of different runtime
    structures and different initialization, etc., requirements.  You
    may have to pay many dollars for this capability.

5.10.1 What do you have in mind?  Fortran formats?  Why not say `an
    implementation shall provide appropriate libraries to allow
    convenient reading and writing of Fortran format text files'?



    -s

	Stavros Macrakis
	Harvard and Intermetrics

Note: I consult for Intermetrics' Ada group.