Re: FYI - ACATS test results for GCC 8.1.1 (mingw)

[Simon Wright <simon@pushface.org>]

alby.gamper@gmail.com writes:

> Thanks to the effort of Simon Wright's on his GitHub project, With
> very little tweaking, I was able to run the ACATS's test on my mingw64
> environment, with what I believe is a very successful result !

Great!

>                 === acats support ===
> Generating support files... done.
> Compiling support files... done.
>
>                 === acats Summary ===
> # of expected passes            2504
> # of unexpected failures        8
> # of expected failures          1458
> # of unresolved testcases       11
> # of unsupported tests          124
> *** FAILURES: c250002 c380004 c52103x c52104x c611a04 c760a02 c760a03
> cxag003

On macOS, with 8.1.0,

		=== acats Summary ===
# of expected passes		2501
# of unexpected failures	11
# of expected failures		1458
# of unresolved testcases	11
# of unsupported tests		124
*** FAILURES: c250002 c611a04 c760a02 c760a03 cxag003 cxd1003 cxd1004 cxd1005 cxd2006 cxd3001 cxd3002 

Comments:

c250002: GNAT fails to take account of case-preserving UTF8 filesystems

c380004: discriminants & protected, task types

c52103x: (large) arrays

c52104x: likewise

c611a04: class-wide preconditions

c760a02: function call intializing immutably limited type

c760a03: likewise

cxag003: Name_Case_Kind, _Equivalence not in Ada.Directories

cxd1003: rendezvous inherits active priority of caller

cxd1004: task inherits the active priority of its parent

cxd1005: likewise

cxd2006: priority inherited after it should be

cxd3001: ceiling locking

cxd3002: likewise

========================================================================

Note that GCC regards darwin and mingw32 as secondary platforms, and the
release criteria <https://gcc.gnu.org/gcc-8/criteria.html> are

  * The compiler bootstraps successfully, and the C++ runtime library
    builds.
    
  * The DejaGNU testsuite has been run, and a substantial majority of
    the tests pass.

so I need to run the suite on a *86* linux box.

Note that "unsupported" means (in this suite) that the test was either
deemed not applicable by the test itself (for example, C45322A requires
Machine_Overflows to be True), not supported by the compiler (for
example, CXAG002 will not compile because it requires support for
Hierarchical_File_Names), or not compatible with simple use of gnatchop
(because the test contains the same unit in several source files,
expecting them to be used in multiple passes).

Note that "expected failures" includes the B tests, which are
expected to fail, and the L tests, which may fail. This suite does not
check that the test fails in the expected way.

Note that I should maybe categorise some of the tests as XFAIL (expected
to fail), e.g. cxag003, so that when the bug is corrected it will show
up as an XPASS (wasn't expected to pass). I can't do this with most of
the others, since the pass/fail seems to be machine-dependent. I'm
reluctant to not even attempt them!

I'd like to try some of the tasking-related tests on a Ravenscar target,
but it's quite a lot of work (easier on one of the AdaCore -full- RTSs,
because they support exception propagation).