Re: Five Years After

[Robert Eachus <rieachus@comcast.net>]

On Thursday, January 25, 2018 at 4:09:28 PM UTC-5, Maciej Sobczak wrote:
> > Everyone agrees that the ACATS test suite offers a lot of value to both 
> > implementers and users (implementers because an independently sourced and 
> > trustworthy set of tests is very helpful to creating and maintaining an Ada 
> > compiler -- it is especially good IMHO in preventing errors of omission, 
> > which are often the hardest to find;
> 
> Sorry, but I cannot agree here. There is nothing that saves ACATS authors from errors of omission. The language is itself so complex that creating a complete (and I really mean formally complete) set of requirements is impractical and without such a set it is not possible to create a complete (again, I mean it) set of tests...

Almost every compiler developer is aware of the Halting Problem andGödel's incompleteness theorems.  However many programmers and software engineers, even if they are aware of them, do not 'get' the fact that you are up against them in a practical way whenever you use a compiler.  (Translating: There are legal Ada programs such that your compiler will either incorrectly reject or fail to halt when asked to compile one of those programs.  All compilers have bugs, at least one of which cannot be eliminated.)  In other words, no matter how much testing you do, you cannot eliminate all the bugs in a compiler.

Note that organizations which develop compilers tend to have an informal hierarchy of bugs.  The worst is usually assigned to programs that compile with no error messages but produce incorrect output or behavior.  Cases where the compiler crashes or rejects legal Ada programs tend to come next. Near the bottom of the list are incorrect or misleading error messages.  There is no requirement that some of those exist, but since there are a (countable) infinity of legal Ada programs, and also of illegal Ada programs, there will always be some cases where the compiler can't guess what the programmer had in mind.

> Unfortunately, no. Every single version of whatever compiler accepts a 
> different language than its previous version. This is how the bugs are fixed
> (or introduced), among others.

This is true in theory, but much less true in practice.  The Unix/Linux practice for distributing applications, including compilers is a case in point.  Often you will compile tens of thousands of lines of code with no error messages.  If you do get errors, they are much more likely to be from improper path arguments or directory locations for the tarball.

I used to say that there were two ways I could program (in Ada, Algol, Lisp, Pascal, PL/I and a few other languages):  In one I would produce tens of thousands of clean code with no compiler bugs found.  In the other I felt I was failing if I had less than one compiler bug discovered per 20 source lines. (I don't include C and C++ because even with lint, I have a long way to go from a clean compile to a working program.  In the languages listed, I almost never read the debugger documentation, and often never used the debugger.)

What is the difference?  In one case I would use only vanilla code.  I have more important things to do than debugging or sending in compiler bug reports.  In the other case I would use extreme corners of the language which often went untested by normal test suites.  An example:

with System; use Ascii; procedure Foo;

Many Ada programmers don't know that you can have more than one use clause following any with clause in a context clause.  Or that the use clauses don't have to relate to the with clause in any (semantic) way.  And Ascii is super special, it is a package that can appear in a use clause without being in any preceding with clause.

So should you use corner cases (with sharp corners)?  Only if you are trying to find compiler bugs.  Don't let this prevent you from using most, if not all of Ada.  But if you do need some tricky tasking code, try to keep any generics that interact with the tasking relatively simple.  Using multiple corner cases in one compilation unit is asking for trouble.