Apex vs GNAT on solaris

Apex vs GNAT on solaris

[reason67]

I am working on a project which employs the Apex compilation system to
compile Ada95. I was in the process of testing some performance issues
in Apex and decided t check and see what the GNAT 3.11 results were as
well.

To give you ideas of my test environment, I am running on a Sun Ultra-2
using solaris 2.6. I am using the standard Apex Ada 95 threaded model
(uses posix threads for tasks) and the out of the box gnat 3.11.

I thought these general tests would be of interest to some of you here:

An outline of the tests I ran so far, are as follows:
   1 - Context Switch timing
      1.0 15 iterations of 1000 delays of duration'small.
          report worst time for the delay and the average time.
      1.1 15 iterations of 1000 delays of 0.0
      1.2 15 iterations of 100 delays of duration'small in 100 tasks.
      1.3 15 iterations of 100 delays of 0.0 in 100 tasks.
   2 - TCP/IP timing (not run yet)
   3 - heap allocation/deallocation
      3.0 mem map of allocation of 1 .. 1 meg of memory keeping 1
previous allocation
      3.1 timing of allocation and deallocation of 1 .. 1 meg of memory
      3.2 same as 3.0 except keep 100 previous pointers (not run yet)
   4 - Rendezvous
      4.0 configurable number of task pairs one calling the others
rendezvous.
      4.1 same as 4.0 except data is passed.(not run yet)
   5 - timing of exception handling propagated over 2 levels (not run
yet)

I will respond to this message the code and the results of each test.
Any ideas of future testing or what you consider mistakes in the testing
would be appreciated.
---
Jeffrey S. Blatt


Sent via Deja.com http://www.deja.com/
Before you buy.

Re: Apex vs GNAT on solaris

[reason67]

>       3.0 mem map of allocation of 1 .. 1 meg of memory keeping 1
> previous allocation

-=-=-=-=-=-=-=-

with Ada.Text_Io;
with Ada.Unchecked_Deallocation;

with Common_Utilities;
with System;
with System.Address_To_Access_Conversions;
with System.Storage_Elements;
use type System.Storage_Elements.Integer_Address;


procedure Aaaa_3_0_Malloc is
    type Byte is mod 256;

    subtype Index_Type is Integer range 0 .. 1048476;
    type Byte_Array_Type is array (Index_Type range <>) of Byte;

    package Converter is
       new System.Address_To_Access_Conversions (Byte_Array_Type);

    procedure Free is new Ada.Unchecked_Deallocation
                             (Byte_Array_Type,
Converter.Object_Pointer);


    Current : Converter.Object_Pointer;
    Last : Converter.Object_Pointer;
--    Address : System.Address;
    Address : System.Storage_Elements.Integer_Address;
    File : Ada.Text_Io.File_Type;

    function Create (Length : in Index_Type) return
Converter.Object_Pointer is
        Result : Converter.Object_Pointer;
    begin
        return new Byte_Array_Type (1 .. Length);
    end Create;

begin

    Ada.Text_Io.Create (File => File, Name => "3.0_malloc_apex");

    Current := Create (1);

    Address := System.Storage_Elements.To_Integer
                  (Converter.To_Address (Current));

    Ada.Text_Io.Put_Line
       (File,
        "1 " & Common_Utilities.Strip_Spaces
                  (System.Storage_Elements.Integer_Address'Image
(Address)) &
           " " & Common_Utilities.Strip_Spaces
                    (System.Storage_Elements.Integer_Address'Image
(Address)));

    Last := Current;
    Current := Create (2);

    Address := System.Storage_Elements.To_Integer
                  (Converter.To_Address (Current));


    Ada.Text_Io.Put_Line
       (File,
        "2 " & Common_Utilities.Strip_Spaces
                  (System.Storage_Elements.Integer_Address'Image
(Address)) &
           " " & Common_Utilities.Strip_Spaces
                    (System.Storage_Elements.Integer_Address'Image
                        (Address + 1)));

    for Index in 3 .. Index_Type'Last loop

        Free (Last);
        Last := Current;
        Current := Create (Index);

        Address := System.Storage_Elements.To_Integer
                      (Converter.To_Address (Current));


        Ada.Text_Io.Put_Line
           (File,
            Common_Utilities.Strip_Spaces (Index_Type'Image (Index)) & "
" &
               Common_Utilities.Strip_Spaces
                  (System.Storage_Elements.Integer_Address'Image
(Address)) &
               " " &
               Common_Utilities.Strip_Spaces
                  (System.Storage_Elements.Integer_Address'Image
                      (Address +
                       System.Storage_Elements.Integer_Address (Index) -
1)));

    end loop;

    Ada.Text_Io.Close (File);

end Aaaa_3_0_Malloc;

-=-=-=-=-=-=-=-=-=-=-

This produces alot of output. I graphed it and found that both Apex and
GNAT did a good job of memory management on solaris


Sent via Deja.com http://www.deja.com/
Before you buy.

Re: Apex vs GNAT on solaris

[Robert Dewar]

In article <82hk18$7up$1@nnrp1.deja.com>,
  reason67@my-deja.com wrote:
> This produces alot of output. I graphed it and found that both
> Apex and GNAT did a good job of memory management on solaris

I would be surprised if both compilers did not simply use the
underlying storage allocation mechanism provided by Solaris.
Do you have any reason to think otherwise?


Sent via Deja.com http://www.deja.com/
Before you buy.

Re: Apex vs GNAT on solaris

[reason67]

In article <82ho1n$asm$1@nnrp1.deja.com>,
  Robert Dewar <robert_dewar@my-deja.com> wrote:

> I would be surprised if both compilers did not simply use the
> underlying storage allocation mechanism provided by Solaris.
> Do you have any reason to think otherwise?

Nope. In fact, I am sure that is correct now that I have run my test. I
was told that Apex had problems with heap memory and at times could
cause memory leaks. This is what lead to this test. My assumption now is
someone was blaming their coding errors on the compiler. :)
---
Jeffrey S. Blatt


Sent via Deja.com http://www.deja.com/
Before you buy.

Re: Apex vs GNAT on solaris

[reason67]

>       1.3 15 iterations of 100 delays of 0.0 in 100 tasks.

-=-=-=-=-=-=-=-=-

with Ada.Calendar;
with Ada.Text_Io;
with Common_Utilities;

procedure Aaaa_1_3_Timing_Test is

    package Long_Float_Io is new Ada.Text_Io.Float_Io (Long_Float);

    Compiler : constant String := "Apex";
    Instances : constant := 15;
    Iterations : constant := 100;
    Task_Number : constant := 100;

    type Time_Stamp_Type is array (0 .. Iterations) of
Ada.Calendar.Time;


    task type Time_Context_Switches is
        entry Start;
    end Time_Context_Switches;
    for Time_Context_Switches'Storage_Size use 100000;

    type Timer_Array_Type is array (1 .. Task_Number) of
Time_Context_Switches;
    Timer : Timer_Array_Type;

    task Controller is
        entry Finish (Biggest : in Long_Float; Average : in Long_Float);
    end Controller;

    task body Time_Context_Switches is
        Biggest : Long_Float := -1.0;
        Average : Long_Float := 0.0;
        Length : Long_Float;
        Time_Stamp : Time_Stamp_Type;

    begin
        Forever:
            loop
                Biggest := -1.0;
                Average := 0.0;

                select
                    accept Start;
                or
                    terminate;
                end select;

                Time_Stamp (0) := Ada.Calendar.Clock;
                for Index in 1 .. Iterations loop
                    delay 0.0;
                    Time_Stamp (Index) := Ada.Calendar.Clock;
                end loop;

                for Index in 1 .. Iterations loop
                    Length := Long_Float
                                 (Ada.Calendar."-" (Time_Stamp (Index),
                                                    Time_Stamp (Index -
1)));

                    if Biggest < Length then
                        Biggest := Length;
                    end if;

                    Average := Average + Length;

                end loop;

                Average := Average / Long_Float (Iterations);

                Controller.Finish (Biggest => Biggest, Average =>
Average);

            end loop Forever;
    end Time_Context_Switches;

    task body Controller is
        Local_Biggest : Long_Float := -1.0;
        Local_Average : Long_Float := 0.0;
        Instance_Length : Integer;
    begin
        Ada.Text_Io.Put_Line ("::: [Aaaa_1_3_Timing_Test]");

        Ada.Text_Io.Put_Line
           (Compiler & " tasking timing test running " &
            Common_Utilities.Strip_Spaces (Integer'Image (Instances)) &
            " instances of " &
            Common_Utilities.Strip_Spaces (Integer'Image (Iterations)) &
            " delays of 0, in " &
            Common_Utilities.Strip_Spaces (Integer'Image (Task_Number))
&
            " tasks");

        Instance_Length := Common_Utilities.Strip_Spaces
                              (Integer'Image (Instances))'Length;

        for Counter in 1 .. Instances loop
            for Index in 1 .. Task_Number loop
                Timer (Index).Start;
            end loop;

            Local_Biggest := -1.0;
            Local_Average := 0.0;

            for Index in 1 .. Task_Number loop
                accept Finish (Biggest : in Long_Float;
                               Average : in Long_Float) do
                    if Biggest > Local_Biggest then
                        Local_Biggest := Biggest;
                    end if;

                    Local_Average := Average + Local_Average;
                end Finish;

            end loop;

            Local_Average := Local_Average / Long_Float (Task_Number);

            Ada.Text_Io.Put ("Instance " & Common_Utilities.String_Of
                                              (Number => Counter,
                                               Length =>
Instance_Length,
                                               Show_Leading_Zeros =>
False) &
                             ": Maximum time => ");
            Long_Float_Io.Put
               (Item => Local_Biggest, Fore => 1, Aft => 4, Exp => 0);
            Ada.Text_Io.Put (Ascii.Ht & Ascii.Ht & "Average time => ");
            Long_Float_Io.Put
               (Item => Local_Average, Fore => 1, Aft => 4, Exp => 0);
            Ada.Text_Io.New_Line;
        end loop;

        Ada.Text_Io.Put_Line ("::: [Aaaa_1_3_Timing_Test has
finished]");
    end Controller;

begin
    null;
end Aaaa_1_3_Timing_Test;

-=-=-=-=-=-=-=-=-=

Apex tasking timing test running 15 instances of 100 delays of 0, in 100
tasks
Instance  1: Maximum time => 0.1668		Average time => 0.0000
Instance  2: Maximum time => 0.0076		Average time => 0.0000
Instance  3: Maximum time => 0.0023		Average time => 0.0000
Instance  4: Maximum time => 0.0103		Average time => 0.0000
Instance  5: Maximum time => 0.0026		Average time => 0.0000
Instance  6: Maximum time => 0.0027		Average time => 0.0000
Instance  7: Maximum time => 0.0022		Average time => 0.0000
Instance  8: Maximum time => 0.0139		Average time => 0.0000
Instance  9: Maximum time => 0.0006		Average time => 0.0000
Instance 10: Maximum time => 0.0103		Average time => 0.0000
Instance 11: Maximum time => 0.0089		Average time => 0.0000
Instance 12: Maximum time => 0.0026		Average time => 0.0000
Instance 13: Maximum time => 0.0110		Average time => 0.0000
Instance 14: Maximum time => 0.0050		Average time => 0.0000
Instance 15: Maximum time => 0.0117		Average time => 0.0000

GNAT tasking timing test running 15 instances of 100 delays of 0, in 100
tasks
Instance  1: Maximum time => 0.0152		Average time => 0.0011
Instance  2: Maximum time => 0.0130		Average time => 0.0012
Instance  3: Maximum time => 0.0143		Average time => 0.0013
Instance  4: Maximum time => 0.0142		Average time => 0.0013
Instance  5: Maximum time => 0.0139		Average time => 0.0013
Instance  6: Maximum time => 0.0681		Average time => 0.0012
Instance  7: Maximum time => 0.1722		Average time => 0.0001
Instance  8: Maximum time => 0.0036		Average time => 0.0001
Instance  9: Maximum time => 0.1307		Average time => 0.0001
Instance 10: Maximum time => 0.1089		Average time => 0.0001
Instance 11: Maximum time => 0.0988		Average time => 0.0001
Instance 12: Maximum time => 0.0907		Average time => 0.0001
Instance 13: Maximum time => 0.0141		Average time => 0.0001
Instance 14: Maximum time => 0.0333		Average time => 0.0001
Instance 15: Maximum time => 0.0318		Average time => 0.0001


Sent via Deja.com http://www.deja.com/
Before you buy.

Re: Apex vs GNAT on solaris

[reason67]

>       3.1 timing of allocation and deallocation of 1 .. 1 meg of
memory

-=-=-=-=-=-=-=-

with Ada.Calendar;
with Ada.Text_Io;
with Ada.Unchecked_Deallocation;

with System;
with System.Address_To_Access_Conversions;
with System.Storage_Elements;


procedure Aaaa_3_1_Malloc is

    package D_Io is new Ada.Text_Io.Fixed_Io (Duration);

    function "-" (Left : in Ada.Calendar.Time; Right : in
Ada.Calendar.Time)
                 return Duration renames Ada.Calendar."-";

    type Byte is mod 256;

    subtype Index_Type is Integer range 0 .. 1048476;
    type Byte_Array_Type is array (Index_Type range <>) of Byte;

    package Converter is
       new System.Address_To_Access_Conversions (Byte_Array_Type);

    procedure Free is new Ada.Unchecked_Deallocation
                             (Byte_Array_Type,
Converter.Object_Pointer);


    Current : Converter.Object_Pointer;
    Last : Converter.Object_Pointer;
    Address : System.Address;
    Start_Time : Ada.Calendar.Time;
    Stop_Time : Ada.Calendar.Time;
    Alloc_File : Ada.Text_Io.File_Type;
    Dealloc_File : Ada.Text_Io.File_Type;


    function Create (Length : in Index_Type) return
Converter.Object_Pointer is
        Result : Converter.Object_Pointer;
    begin
        Start_Time := Ada.Calendar.Clock;
        Result := new Byte_Array_Type (1 .. Length);
        Stop_Time := Ada.Calendar.Clock;
        return Result;
    end Create;

begin

    Ada.Text_Io.Create (File => Alloc_File, Name =>
"allocation_times.txt");
    Ada.Text_Io.Create (File => Dealloc_File, Name =>
"deallocation_times.txt");

    Current := Create (1);
    D_Io.Put (Alloc_File, (Stop_Time - Start_Time));
    Ada.Text_Io.New_Line (Alloc_File);

    Last := Current;
    Current := Create (2);
    D_Io.Put (Alloc_File, (Stop_Time - Start_Time));
    Ada.Text_Io.New_Line (Alloc_File);

    for Index in 3 .. Index_Type'Last loop

        Start_Time := Ada.Calendar.Clock;
        Free (Last);
        Stop_Time := Ada.Calendar.Clock;
        D_Io.Put (Dealloc_File, (Stop_Time - Start_Time));
        Ada.Text_Io.New_Line (Dealloc_File);

        Last := Current;
        Current := Create (Index);
        D_Io.Put (Alloc_File, (Stop_Time - Start_Time));
        Ada.Text_Io.New_Line (Alloc_File);

    end loop;

    Start_Time := Ada.Calendar.Clock;
    Free (Last);
    Stop_Time := Ada.Calendar.Clock;
    D_Io.Put (Dealloc_File, (Stop_Time - Start_Time));
    Ada.Text_Io.New_Line (Dealloc_File);

    Start_Time := Ada.Calendar.Clock;
    Free (Current);
    Stop_Time := Ada.Calendar.Clock;
    D_Io.Put (Dealloc_File, (Stop_Time - Start_Time));
    Ada.Text_Io.New_Line (Dealloc_File);

end Aaaa_3_1_Malloc;

-=-=-=-=-=-=-=-

On both Apex and GNAT every result was 0.0. Memory allocation seems to
fast to measure. Any ideas of how to get accurate timing would be
appreciated.


Sent via Deja.com http://www.deja.com/
Before you buy.

Re: Apex vs GNAT on solaris

[Robert Dewar]

In article <82hk5c$80p$1@nnrp1.deja.com>,
  reason67@my-deja.com wrote:
>     Start_Time := Ada.Calendar.Clock;
>     Free (Last);
>     Stop_Time := Ada.Calendar.Clock;

This is completely hopeless methodology, of course you will
not get useful results this way, the time for a Free is
orders of magnitude faster than the clock resolution.

I recommend looking at the PIWG tests or at the ACES tests
for examples of how to do timing in a meaningful way.

In fact unless you have a lot of experience in the construction
of meaningful benchmarks, I think you will find the ACES
quicklook to be a much more meaningful comparison that tests
that you prepare yourself. There are simply too many pitfalls
in benchmark construction :-)


Sent via Deja.com http://www.deja.com/
Before you buy.

Re: Apex vs GNAT on solaris

[Vladimir Olensky]

Robert Dewar wrote in message <82hob6$avp$1@nnrp1.deja.com>...
>In article <82hk5c$80p$1@nnrp1.deja.com>,
>  reason67@my-deja.com wrote:
>>     Start_Time := Ada.Calendar.Clock;
>>     Free (Last);
>>     Stop_Time := Ada.Calendar.Clock;
>
>This is completely hopeless methodology, of course you will
>not get useful results this way, the time for a Free is
>orders of magnitude faster than the clock resolution.


When I did some tests with GNAT 3.12p on Windows NT
I used Ada.Real_Time.Clock to measure elapsed time.

I do not know about  GNAT 3.11 for Solaris but
GNAT 3.12p Ada.Real_Time.Clock  gives one
nanosecond resolution on Windows NT (uses NT
high performance counter).
This information for particular platform could be
found in GNAT reference manual.
GNAT 3.11p for WinNT did not used NT high performance
counter.

In any case if OS has such counter it could be accessed
via assembler routine.

Here is a snippet from my code:

with Ada.Real_Time;
Procedure Test_Something;
   <...>
 use type Real_Time.Time;
   <...>
begin
  <...>
  Start_Time  := Ada.Real_Time.Clock;

  Do_Something (Number_Of_Iterations);
  -- contains loop which performs some action
  -- Number_Of_Iterations times

  Finish_Time  := Ada.Real_Time.Clock;
  Elapsed_Time := Ada.Real_Time.To_Duration (Finish_Time - Start_Time);
  <...>
end Test_Something;


Regards,
Vladimir_OLensky

Re: Apex vs GNAT on solaris

[Vladimir Olensky]

Vladimir Olensky wrote in message ...

>When I did some tests with GNAT 3.12p on Windows NT
>I used Ada.Real_Time.Clock to measure elapsed time.
>

Forgot to mention that I also used all possible
GNAT/GCC optimization options in different combinations
and    pragma Suppress (All_Checks);

Regards,
Vladimir Olensky

Re: Apex vs GNAT on solaris

[Geoff Bull]

Vladimir Olensky wrote:
> 
> GNAT 3.12p Ada.Real_Time.Clock  gives one
> nanosecond resolution on Windows NT

Now that's clever, on a system with a 3ns (or whatever) clock!
:-)

Re: Apex vs GNAT on solaris

[Vladimir Olensky]

Geoff Bull wrote in message <384F9FAE.8FEFEB30@acenet.com.au>...
>
>
>Vladimir Olensky wrote:
>>
>> GNAT 3.12p Ada.Real_Time.Clock  gives one
>> nanosecond resolution on Windows NT
>
>Now that's clever, on a system with a 3ns (or whatever) clock!
>:-)

Yes, you are right here about system clock.
From here a can make a guess that you are using 300 MHz machine.
On 300MHz machine clock cycle is about 3 ns
On 700MHz machine clock cycle is about 1.4 ns

Finest possible resolution for GNAT is 1ns.
(See GNAT ref.manual and implementation)

In reality GNAT resolution depends on hardware
high-resolution performance counter as
Ada.Real_Time.Clock calls QueryPerformanceCounter
from kernel32.

------------------------------------------
from System.OS_Primitives:
  -----------
   -- Clock --
   -----------

   --  this is a high resolution clock implemented using
   --  QueryPerformanceCounter. This call return a 64 bits values (based on
   --  the 8253 16 bits counter). This counter is updated every 1/1_193_182
   --  times per seconds. The call to QueryPerformanceCounter takes 6
   --  microsecs to complete.

   function Clock return Duration is
      Current_Ticks : aliased LARGE_INTEGER;
   begin
      if not QueryPerformanceCounter (Current_Ticks'Access) then
         return 0.0;
      end if;

      return Base_Clock +
        Duration (Long_Long_Float (Current_Ticks - Base_Ticks) /
                  Long_Long_Float (Tick_Frequency));
   end Clock;
-----------------
package Ada.Real_Time is

   type Time is private;
   Time_First : constant Time;
   Time_Last  : constant Time;
   Time_Unit  : constant := 10#1.0#E-9;
    ......
----------------------------------------------------


So all depends on hardware implementation.


Regards,
Vladimir Olensky

Re: Apex vs GNAT on solaris

[Vladimir Olensky]

Vladimir Olensky wrote in message ...
>Geoff Bull wrote in message <384F9FAE.8FEFEB30@acenet.com.au>...


>>Vladimir Olensky wrote:
>>>
>>> GNAT 3.12p Ada.Real_Time.Clock  gives one
>>> nanosecond resolution on Windows NT
>>
>>Now that's clever, on a system with a 3ns (or whatever) clock!
>>:-)
>
>Yes, you are right here about system clock.
>From here a can make a guess that you are using 300 MHz machine.
>On 300MHz machine clock cycle is about 3 ns
>On 700MHz machine clock cycle is about 1.4 ns
>
>Finest possible resolution for GNAT is 1ns.
>(See GNAT ref.manual and implementation)
>
>In reality GNAT resolution depends on hardware
>high-resolution performance counter as
>Ada.Real_Time.Clock calls QueryPerformanceCounter
>from kernel32.
<..>
>So all depends on hardware implementation.


e.g:

Pentium-III instruction RDTSC (Read Time-Stamp Counter):

This instruction loads the current value of the processor's
time-stamp counter into the EDX:EAX registers.
<..>
The processor increments the time-stamp counter
MSR every clock cycle and resets it to 0 whenever
the processor is reset.
------------------------------------
So on 700MHz  PIII machine resolution could be up to 1.4 ns.
Anyone can use this instruction directly to obtain the best
possible time resolution and not to wait when Performance
Counter will be updated in the next tick with the data from
this TSC.


Regards,
Vladimir Olensky

Re: Apex vs GNAT on solaris

[Vladimir Olensky]

Geoff Bull wrote in message <384F9FAE.8FEFEB30@acenet.com.au>...
>
>
>Vladimir Olensky wrote:
>>
>> GNAT 3.12p Ada.Real_Time.Clock  gives one
>> nanosecond resolution on Windows NT
>
>Now that's clever, on a system with a 3ns (or whatever) clock!
>:-)


Yes, you are right here about system clock.
From here a can make a guess that you are using 300 MHz machine.
On 300MHz machine clock cycle is about 3 ns
On 700MHz machine clock cycle is about 1.4 ns

Finest possible resolution for GNAT is 1ns.
(See GNAT ref.manual and implementation)

In reality GNAT resolution depends on hardware
high-resolution performance counter as
Ada.Real_Time.Clock calls QueryPerformanceCounter
from kernel32.

------------------------------------------
from System.OS_Primitives:
  -----------
   -- Clock --
   -----------

   --  this is a high resolution clock implemented using
   --  QueryPerformanceCounter. This call return a 64 bits values (based on
   --  the 8253 16 bits counter). This counter is updated every 1/1_193_182
   --  times per seconds. The call to QueryPerformanceCounter takes 6
   --  microsecs to complete.

   function Clock return Duration is
      Current_Ticks : aliased LARGE_INTEGER;
   begin
      if not QueryPerformanceCounter (Current_Ticks'Access) then
         return 0.0;
      end if;

      return Base_Clock +
        Duration (Long_Long_Float (Current_Ticks - Base_Ticks) /
                  Long_Long_Float (Tick_Frequency));
   end Clock;
-----------------
package Ada.Real_Time is

   type Time is private;
   Time_First : constant Time;
   Time_Last  : constant Time;
   Time_Unit  : constant := 10#1.0#E-9;
    ......
----------------------------------------------------


So all depends on hardware implementation.


Regards,
Vladimir Olensky

Re: Apex vs GNAT on solaris

[reason67]

In article <82hob6$avp$1@nnrp1.deja.com>,
  Robert Dewar <robert_dewar@my-deja.com> wrote:
> In article <82hk5c$80p$1@nnrp1.deja.com>,
>   reason67@my-deja.com wrote:
> >     Start_Time := Ada.Calendar.Clock;
> >     Free (Last);
> >     Stop_Time := Ada.Calendar.Clock;
>
> This is completely hopeless methodology, of course you will
> not get useful results this way, the time for a Free is
> orders of magnitude faster than the clock resolution.

Good to know. That would explain why it did not work :)

> I recommend looking at the PIWG tests or at the ACES tests
> for examples of how to do timing in a meaningful way.

I will look into it. Thanks for the info.

> In fact unless you have a lot of experience in the construction
> of meaningful benchmarks, I think you will find the ACES
> quicklook to be a much more meaningful comparison that tests
> that you prepare yourself. There are simply too many pitfalls
> in benchmark construction :-)

Nope. I have alot of experience writing Ada and working for companies
that say things are too slow when no one has ever checked. In this case,
I decided to check. I have found alot of discrepencies so far. :)
---
Jeffrey S. Blatt


Sent via Deja.com http://www.deja.com/
Before you buy.

Re: Apex vs GNAT on solaris

[Robert Dewar]

In article <82hpbb$bod$1@nnrp1.deja.com>,
  reason67@my-deja.com wrote:
=> Nope. I have alot of experience writing Ada and working for
> companies that say things are too slow when no one has ever
> checked. In this case, I decided to check. I have found alot
> of discrepencies so far. :)


My point was that "lot of experience writing Ada", does not
mean you know how to write meaningful benchmarks. This is a
special skill. Just doing the timing correctly is far from
trivial (naive attempts to just read the clock almost always
result in bogus data).

Once again, I definitely recommend that you look at the ACES
quicklook, which has been carefully designed to yield meaningful
results.


Sent via Deja.com http://www.deja.com/
Before you buy.

Re: Apex vs GNAT on solaris

[reason67]

In article <82kt9f$j20$1@nnrp1.deja.com>,
  Robert Dewar <robert_dewar@my-deja.com> wrote:

> My point was that "lot of experience writing Ada", does not
> mean you know how to write meaningful benchmarks. This is a
> special skill. Just doing the timing correctly is far from
> trivial (naive attempts to just read the clock almost always
> result in bogus data).

Again, you have to look at my original intent for writing these tests.
It was not to benchmark 2 compilers (although that did start an
interesting thread), but to see if some of the untested performance
assumptions. My goal was to show the number bogus. I have done so
already.

As for ACES, I am running the ACES tests now but am rather disappointed
in ACES. I have had to modify some of the code already (They had a
'small on a float) and I can not run the scripts in Apex as they put
spec's/bodies/many packages in the same file. I am able to get the
results I want, but it takes far longer than it should. At least the
ACES results will look better on a slide to management. :)
---
Jeffrey S. Blatt


Sent via Deja.com http://www.deja.com/
Before you buy.

Re: Apex vs GNAT on solaris

[reason67]

>       1.1 15 iterations of 1000 delays of 0.0

-=-=-=-=-=-=-

with Ada.Calendar;
with Ada.Text_Io;
with Common_Utilities;

procedure Aaaa_1_1_Timing_Test is

    package Long_Float_Io is new Ada.Text_Io.Float_Io (Long_Float);

    type Time_Stamp_Type is array (0 .. 1_000) of Ada.Calendar.Time;

    Compiler : constant String := "Apex";
    Instances : constant := 15;
    Iterations : constant := 1000;

    Biggest_Number : Integer;
    Smallest_Number : Integer;

    Biggest : Long_Float := -1.0;
    Smallest : Long_Float := 100.0;
    Average : Long_Float := 0.0;
    Length : Long_Float;
    Time_Stamp : Time_Stamp_Type;
    Instance_Length : Integer;

begin

    Ada.Text_Io.Put_Line ("::: [Aaaa_1_1_Timing_Test]");

    Ada.Text_Io.Put_Line
       (Compiler & " timing test running " &
        Common_Utilities.Strip_Spaces (Integer'Image (Instances)) &
        " instances of " &
        Common_Utilities.Strip_Spaces (Integer'Image (Iterations)) &
        " delays of 0");

    Instance_Length := Common_Utilities.Strip_Spaces
                          (Integer'Image (Instances))'Length;

    for Counter in 1 .. Instances loop

        Biggest := -1.0;
        Smallest := 100.0;
        Average := 0.0;

        Time_Stamp (0) := Ada.Calendar.Clock;

        for Index in 1 .. Iterations loop
            delay 0.0;
            Time_Stamp (Index) := Ada.Calendar.Clock;
        end loop;

        for Index in 1 .. Iterations loop
            Length := Long_Float (Ada.Calendar."-" (Time_Stamp (Index),
                                                    Time_Stamp (Index -
1)));

            if Biggest < Length then
                Biggest := Length;
                Biggest_Number := Index;
            end if;

            if Smallest > Length then
                Smallest := Length;
                Smallest_Number := Index;
            end if;

            Average := Average + Length;

        end loop;

        Average := Average / Long_Float (Iterations);

        Ada.Text_Io.Put ("Instance " & Common_Utilities.String_Of
                                          (Number => Counter,
                                           Length => Instance_Length,
                                           Show_Leading_Zeros => False)
&
                         ": Maximum time => ");
        Long_Float_Io.Put (Item => Biggest, Fore => 1, Aft => 4, Exp =>
0);
        Ada.Text_Io.Put (Ascii.Ht & Ascii.Ht & "Average time => ");
        Long_Float_Io.Put (Item => Average, Fore => 1, Aft => 4, Exp =>
0);
        Ada.Text_Io.New_Line;
    end loop;

    Ada.Text_Io.Put_Line ("::: [Aaaa_1_1_Timing_Test has finished]");

end Aaaa_1_1_Timing_Test;

-=-=-=-=-=-=-=-=-

Apex timing test running 15 instances of 1000 delays of 0
Instance  1: Maximum time => 0.0010		Average time => 0.0000
Instance  2: Maximum time => 0.0211		Average time => 0.0000
Instance  3: Maximum time => 0.0002		Average time => 0.0000
Instance  4: Maximum time => 0.0058		Average time => 0.0000
Instance  5: Maximum time => 0.0100		Average time => 0.0000
Instance  6: Maximum time => 0.0054		Average time => 0.0000
Instance  7: Maximum time => 0.0048		Average time => 0.0000
Instance  8: Maximum time => 0.0001		Average time => 0.0000
Instance  9: Maximum time => 0.0001		Average time => 0.0000
Instance 10: Maximum time => 0.0017		Average time => 0.0000
Instance 11: Maximum time => 0.0014		Average time => 0.0000
Instance 12: Maximum time => 0.0001		Average time => 0.0000
Instance 13: Maximum time => 0.0010		Average time => 0.0000
Instance 14: Maximum time => 0.0016		Average time => 0.0000
Instance 15: Maximum time => 0.0014		Average time => 0.0000


GNAT timing test running 15 instances of 1000 delays of 0
Instance  1: Maximum time => 0.0000		Average time => 0.0000
Instance  2: Maximum time => 0.0000		Average time => 0.0000
Instance  3: Maximum time => 0.0000		Average time => 0.0000
Instance  4: Maximum time => 0.0000		Average time => 0.0000
Instance  5: Maximum time => 0.0000		Average time => 0.0000
Instance  6: Maximum time => 0.0036		Average time => 0.0000
Instance  7: Maximum time => 0.0000		Average time => 0.0000
Instance  8: Maximum time => 0.0000		Average time => 0.0000
Instance  9: Maximum time => 0.0000		Average time => 0.0000
Instance 10: Maximum time => 0.0000		Average time => 0.0000
Instance 11: Maximum time => 0.0000		Average time => 0.0000
Instance 12: Maximum time => 0.0001		Average time => 0.0000
Instance 13: Maximum time => 0.0000		Average time => 0.0000
Instance 14: Maximum time => 0.0000		Average time => 0.0000
Instance 15: Maximum time => 0.0001		Average time => 0.0000


Sent via Deja.com http://www.deja.com/
Before you buy.

Re: Apex vs GNAT on solaris

[reason67]

>       1.2 15 iterations of 100 delays of duration'small in 100 tasks.

-=-=-=-=-=-=-

with Ada.Calendar;
with Ada.Text_Io;
with Common_Utilities;

procedure Aaaa_1_2_Timing_Test is

    package Long_Float_Io is new Ada.Text_Io.Float_Io (Long_Float);

    Compiler : constant String := "Apex";
    Instances : constant := 15;
    Iterations : constant := 100;
    Task_Number : constant := 100;

    type Time_Stamp_Type is array (0 .. Iterations) of
Ada.Calendar.Time;


    task type Time_Context_Switches is
        entry Start;
    end Time_Context_Switches;
    for Time_Context_Switches'Storage_Size use 100000;

    type Timer_Array_Type is array (1 .. Task_Number) of
Time_Context_Switches;
    Timer : Timer_Array_Type;

    task Controller is
        entry Finish (Biggest : in Long_Float; Average : in Long_Float);
    end Controller;

    task body Time_Context_Switches is
        Biggest : Long_Float := -1.0;
        Average : Long_Float := 0.0;
        Length : Long_Float;
        Time_Stamp : Time_Stamp_Type;

    begin
        Forever:
            loop
                Biggest := -1.0;
                Average := 0.0;

                select
                    accept Start;
                or
                    terminate;
                end select;

                Time_Stamp (0) := Ada.Calendar.Clock;
                for Index in 1 .. Iterations loop
                    delay Duration'Small;
                    Time_Stamp (Index) := Ada.Calendar.Clock;
                end loop;

                for Index in 1 .. Iterations loop
                    Length := Long_Float
                                 (Ada.Calendar."-" (Time_Stamp (Index),
                                                    Time_Stamp (Index -
1))) -
                              Long_Float (Duration'Small);

                    if Biggest < Length then
                        Biggest := Length;
                    end if;

                    Average := Average + Length;

                end loop;

                Average := Average / Long_Float (Iterations);

                Controller.Finish (Biggest => Biggest, Average =>
Average);

            end loop Forever;
    end Time_Context_Switches;

    task body Controller is
        Local_Biggest : Long_Float := -1.0;
        Local_Average : Long_Float := 0.0;
        Instance_Length : Integer;
    begin
        Ada.Text_Io.Put_Line ("::: [Aaaa_1_2_Timing_Test]");

        Ada.Text_Io.Put_Line
           (Compiler & " tasking timing test running " &
            Common_Utilities.Strip_Spaces (Integer'Image (Instances)) &
            " instances of " &
            Common_Utilities.Strip_Spaces (Integer'Image (Iterations)) &
            " delays of duration'small, in " &
            Common_Utilities.Strip_Spaces (Integer'Image (Task_Number))
&
            " tasks");

        Instance_Length := Common_Utilities.Strip_Spaces
                              (Integer'Image (Instances))'Length;

        for Counter in 1 .. Instances loop
            for Index in 1 .. Task_Number loop
                Timer (Index).Start;
            end loop;

            Local_Biggest := -1.0;
            Local_Average := 0.0;

            for Index in 1 .. Task_Number loop
                accept Finish (Biggest : in Long_Float;
                               Average : in Long_Float) do
                    if Biggest > Local_Biggest then
                        Local_Biggest := Biggest;
                    end if;

                    Local_Average := Average + Local_Average;
                end Finish;

            end loop;

            Local_Average := Local_Average / Long_Float (Task_Number);

            Ada.Text_Io.Put ("Instance " & Common_Utilities.String_Of
                                              (Number => Counter,
                                               Length =>
Instance_Length,
                                               Show_Leading_Zeros =>
False) &
                             ": Maximum time => ");
            Long_Float_Io.Put
               (Item => Local_Biggest, Fore => 1, Aft => 4, Exp => 0);
            Ada.Text_Io.Put (Ascii.Ht & Ascii.Ht & "Average time => ");
            Long_Float_Io.Put
               (Item => Local_Average, Fore => 1, Aft => 4, Exp => 0);
            Ada.Text_Io.New_Line;
        end loop;

        Ada.Text_Io.Put_Line ("::: [Aaaa_1_2_Timing_Test has
finished]");
    end Controller;

begin
    null;
end Aaaa_1_2_Timing_Test;

-=-=-=-=-=-=-=-=-

Apex tasking timing test running 15 instances of 100 delays of
duration'small, in 100 tasks
Instance  1: Maximum time => 0.6406		Average time => 0.0257
Instance  2: Maximum time => 0.4628		Average time => 0.0350
Instance  3: Maximum time => 1.0903		Average time => 0.0335
Instance  4: Maximum time => 0.9894		Average time => 0.0327
Instance  5: Maximum time => 0.6160		Average time => 0.0300
Instance  6: Maximum time => 0.6201		Average time => 0.0421
Instance  7: Maximum time => 0.1503		Average time => 0.0374
Instance  8: Maximum time => 0.3606		Average time => 0.0375
Instance  9: Maximum time => 1.0131		Average time => 0.0277
Instance 10: Maximum time => 0.4931		Average time => 0.0329
Instance 11: Maximum time => 0.0979		Average time => 0.0392
Instance 12: Maximum time => 0.5545		Average time => 0.0359
Instance 13: Maximum time => 0.3654		Average time => 0.0329
Instance 14: Maximum time => 0.3740		Average time => 0.0375
Instance 15: Maximum time => 0.6872		Average time => 0.0385


GNAT tasking timing test running 15 instances of 100 delays of
duration'small, in 100 tasks
Instance  1: Maximum time => 0.0051		Average time => 0.0001
Instance  2: Maximum time => 0.0196		Average time => 0.0002
Instance  3: Maximum time => 0.0061		Average time => 0.0001
Instance  4: Maximum time => 0.0001		Average time => 0.0000
Instance  5: Maximum time => 0.0008		Average time => 0.0000
Instance  6: Maximum time => 0.0002		Average time => 0.0000
Instance  7: Maximum time => 0.0055		Average time => 0.0000
Instance  8: Maximum time => 0.0001		Average time => 0.0000
Instance  9: Maximum time => 0.0006		Average time => 0.0000
Instance 10: Maximum time => 0.0004		Average time => 0.0000
Instance 11: Maximum time => 0.0007		Average time => 0.0000
Instance 12: Maximum time => 0.0008		Average time => 0.0000
Instance 13: Maximum time => 0.0235		Average time => 0.0000
Instance 14: Maximum time => 0.0007		Average time => 0.0000
Instance 15: Maximum time => 0.0190		Average time => 0.0006


Sent via Deja.com http://www.deja.com/
Before you buy.

Re: Apex vs GNAT on solaris

[Robert Dewar]

In article <82hiuj$74o$1@nnrp1.deja.com>,
  reason67@my-deja.com wrote:


You need to say what options you are using for both compilers.
We have sometimes found people making the *amazing* mistake
of compiling GNAT with -O0. Generally the appropriate options
for benchmarking are -O2 -gnatn.

The options are critical, because otherwise you may simply
be measuring differences in choices of default options. For
example if one compiler inlines by default, the other one
does not, then the comparison may be meaningless.

You also need to specify all other parameters. For instance,
if you are running tasking, make sure you are comparing
comparable underlying threads implementations, otherwise you
again have apples and oranges (e.g. which of the two threads
libraries did you use for GNAT).

We often work with customers running benchmarks, and we often
find that the comparisons are meaningless for various reasons,
sometimes to the disadvantage of GNAT, sometimes to its bogus
advantage.

It is very hard to do meaningful comparison testing.

Robert Dewar
Ada Core Technologies


Sent via Deja.com http://www.deja.com/
Before you buy.

Re: Apex vs GNAT on solaris

[reason67]

In article <82hnll$ahu$1@nnrp1.deja.com>,
  Robert Dewar <dewar@gnat.com> wrote:
> In article <82hiuj$74o$1@nnrp1.deja.com>,
>   reason67@my-deja.com wrote:
>
> You need to say what options you are using for both compilers.
> We have sometimes found people making the *amazing* mistake
> of compiling GNAT with -O0. Generally the appropriate options
> for benchmarking are -O2 -gnatn.

Did a gnatmake with no options. so I got gcc -c.
In apex I did the default link, which is what we were doing in
production. (Apex's results were my primary concern here, benchmarking,
if you can call it that, was an afterthought).

> The options are critical, because otherwise you may simply
> be measuring differences in choices of default options. For
> example if one compiler inlines by default, the other one
> does not, then the comparison may be meaningless.

This is true. I was actually personally concerned with the defaults that
both compilers used. I am sure I could have made GNAT run faster, and I
am sure I could have made Apex run faster. My curiousity was how they
ran "out of the box" against one another. I thought others might find it
interesting as well. This was by no stretch of the imagination anything
like a formal benchmark. I did not take network considerations into
account nor did I take system load into account. I averaged multiple
runs to general ideas of how they worked against one another.

> You also need to specify all other parameters. For instance,
> if you are running tasking, make sure you are comparing
> comparable underlying threads implementations, otherwise you
> again have apples and oranges (e.g. which of the two threads
> libraries did you use for GNAT).

Apex is running POSIX Threads. I did not install GNAT on this machine,
but my assumption, given the similarity of results is that GNAT is also
using POSIX threads. Had the timing been significantly off, I would have
checked.

Let me re-iterate one more time... These results should not ever be
consdered formal bench marks. They were simply a comparison tests I ran,
out of curiosity, with the standard Apex Posix Thread model, and the
standard GNAT 3.11 installation. Default compilation and linking options
were used.
---
Jeffrey S. Blatt


Sent via Deja.com http://www.deja.com/
Before you buy.

Re: Apex vs GNAT on solaris

[Robert Dewar]

In article <82hokf$b9l$1@nnrp1.deja.com>,
  reason67@my-deja.com wrote:
> This is true. I was actually personally concerned with the
> defaults that both compilers used.

It is completely meaningless to measure GNAT at -O0. The basic
code generation technique used is designed to generate extremely
inefficient code at this level, in a manner which is often hard
to duplicate with a hand written code generator :-)

From time to time we have discussed making the default -O2
-gnatn, *precisely* so that naive benchmarks run with default
options look better :-)

Seriously, you have to decide what you are doing here. If
you are measuring performance, then you need to make sure
you are selecting comparable levels of optimization.

If all you want to do is to find out the default options used
for optimization, then it is more efficient to simply read the
manuals to find this out, than to run meaningless comparisons
of times.

> > You also need to specify all other parameters. For instance,
> > if you are running tasking, make sure you are comparing
> > comparable underlying threads implementations, otherwise you
> > again have apples and oranges (e.g. which of the two threads
> > libraries did you use for GNAT).


Sent via Deja.com http://www.deja.com/
Before you buy.

Re: Apex vs GNAT on solaris

[reason67]

In article <82ktik$j4c$1@nnrp1.deja.com>,
  Robert Dewar <robert_dewar@my-deja.com> wrote:

> It is completely meaningless to measure GNAT at -O0. The basic
> code generation technique used is designed to generate extremely
> inefficient code at this level, in a manner which is often hard
> to duplicate with a hand written code generator :-)
>
> From time to time we have discussed making the default -O2
> -gnatn, *precisely* so that naive benchmarks run with default
> options look better :-)
>
> Seriously, you have to decide what you are doing here. If
> you are measuring performance, then you need to make sure
> you are selecting comparable levels of optimization.
>
> If all you want to do is to find out the default options used
> for optimization, then it is more efficient to simply read the
> manuals to find this out, than to run meaningless comparisons
> of times.

BTW: Both compilers were set at no optimization.
---
Jeffrey S. Blatt


Sent via Deja.com http://www.deja.com/
Before you buy.

Re: Apex vs GNAT on solaris

[Larry Kilgallen]

In article <82lcr0$tsg$1@nnrp1.deja.com>, reason67@my-deja.com writes:

> BTW: Both compilers were set at no optimization.

So two compilers were told to build inefficient programs.

Clearly the one that created the slower program is the better compiler :-)

Re: Apex vs GNAT on solaris

[Robert Dewar]

In article <82lcr0$tsg$1@nnrp1.deja.com>,
  reason67@my-deja.com wrote:
> BTW: Both compilers were set at no optimization.


That statement tends to imply that you think the above statement
is relevant to someting or other in your measurements.

It is not!

The code generation techniques of Rational and GNAT are quite
different, and one thing you will find is that some compilers do
a fair amount of optimization even with all optimization turned
off (from one point of view, you could say that running faster
with optimization off is an indication that the compiler does
not have the (sometimes important) capability of really
suppressing all optimization.

In other words, we are reading your thread whose title sure
implies some competitive comparison, and now perhaps we think
that for some tests (but not all) running more SLOWLY might
indicate the preferred behavior. This is sure not the way most
people run or interpret benchmarks, and in particular you
mention showing the results to management -- I am pretty sure
they will be likely to misinterpret them :-)

Note that for a meaningful comparison of ACES quicklook, you
should most certainly make sure that all relevant optimizations
are on (probably -O2 -gnatn, maybe -funroll-loops, for GNAT, I
do not know what the appropriate Apex options might be).


Sent via Deja.com http://www.deja.com/
Before you buy.

Re: Apex vs GNAT on solaris

[Robert A Duff]

reason67@my-deja.com writes:

> BTW: Both compilers were set at no optimization.

Which is why the run-time measurements are completely meaningless.  If
you care about run-time efficiency, then you want to know how well the
compiler can do, not how poorly it can do if you turn off optimization.
The only thing that makes sense is to compare compilers at their highest
optimization levels.

The only reasons to turn optimization off are to make compile time
faster, to make debugging easier, or because you think the optimizer is
buggy.  But none of those things were measured!

- Bob

Re: Apex vs GNAT on solaris

[Roger Racine]

On Tue, 07 Dec 1999 01:25:13 GMT, Robert Dewar <dewar@gnat.com> wrote:

>In article <82hiuj$74o$1@nnrp1.deja.com>,
>  reason67@my-deja.com wrote:
>
>
>You need to say what options you are using for both compilers.
>We have sometimes found people making the *amazing* mistake
>of compiling GNAT with -O0. Generally the appropriate options
>for benchmarking are -O2 -gnatn.
>
>The options are critical, because otherwise you may simply
>be measuring differences in choices of default options. For
>example if one compiler inlines by default, the other one
>does not, then the comparison may be meaningless.
>
>You also need to specify all other parameters. For instance,
>if you are running tasking, make sure you are comparing
>comparable underlying threads implementations, otherwise you
>again have apples and oranges (e.g. which of the two threads
>libraries did you use for GNAT).
>

Is it a mistake to use the default options for a compiler?  Many
people will assume that the default options are the most reliable
options.  Others will assume that the default options are the best
options.  The default options generally turn out to be the most
reliable because most people use them, and bugs will be discovered and
corrected sooner than those to be found using the non-default options.

The same can be said for tasking.

I have been hit many times by optimizer bugs, and generally assume
that the compiler vendors knows what they are doing when the defaults
are created.  They are giving what they think should be the most
common values for the options.

Roger Racine

Re: Apex vs GNAT on solaris

[David Starner]

On Tue, 07 Dec 1999 12:38:16 GMT, Roger Racine <rracine@myremarq.com> wrote:
>Is it a mistake to use the default options for a compiler?

Yes. GCC defaults to fast compilation. If you want good code,
turn optimization on. It's definetly not fair to benchmark
GNAT at -O0 (the default) unless you're timing the compiler
itself. 

-- 
David Starner - dstarner98@aasaa.ofe.org
This message property of the Secret Masters. Unauthorized
duplication or reading will result in execution. Aren't
you glad nothing happens without the authorization of
the Secret Masters?

Re: Apex vs GNAT on solaris

[Robert Dewar]

In article <82j35q$9281@news.cis.okstate.edu>,
  dstarner98@aasaa.ofe.org wrote:
> Yes. GCC defaults to fast compilation. If you want good code,
> turn optimization on. It's definetly not fair to benchmark
                                           ^^^^

The issue is not so much fairness, as meaningfulness of the
results. When people publish comparative timings, you sort
of assume that the subtext is:

"look and see which compiler is faster"

But if you don't take care to make the figures meaningful,
the true subtext is something like:

"here are some random non-comparable figures from which nothing
 can be usefully deduced"

> GNAT at -O0 (the default) unless you're timing the compiler
> itself.


Sent via Deja.com http://www.deja.com/
Before you buy.

Re: Apex vs GNAT on solaris

[Jean-Pierre Rosen]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 807 bytes --]


Roger Racine <rracine@myremarq.com> a �crit dans le message :
384cfdb3.691883075@newsnew.draper.com...
> I have been hit many times by optimizer bugs, and generally assume
> that the compiler vendors knows what they are doing when the defaults
> are created.  They are giving what they think should be the most
> common values for the options.
>
In general, you compile many times before you get to a production version,
where efficiency really matters.
In that case, it makes sense to have default options that favour compilation
speed over execution speed.
It does not mean that theses options are appropriate for the final build.

--
---------------------------------------------------------
           J-P. Rosen (Rosen.Adalog@wanadoo.fr)
Visit Adalog's web site at http://pro.wanadoo.fr/adalog

Re: Apex vs GNAT on solaris

[Larry Kilgallen]

In article <384cfdb3.691883075@newsnew.draper.com>, rracine@myremarq.com (Roger Racine) writes:
> On Tue, 07 Dec 1999 01:25:13 GMT, Robert Dewar <dewar@gnat.com> wrote:
> 
>>In article <82hiuj$74o$1@nnrp1.deja.com>,
>>  reason67@my-deja.com wrote:
>>
>>
>>You need to say what options you are using for both compilers.
>>We have sometimes found people making the *amazing* mistake
>>of compiling GNAT with -O0. Generally the appropriate options
>>for benchmarking are -O2 -gnatn.
>>
>>The options are critical, because otherwise you may simply
>>be measuring differences in choices of default options. For
>>example if one compiler inlines by default, the other one
>>does not, then the comparison may be meaningless.


> I have been hit many times by optimizer bugs, and generally assume
> that the compiler vendors knows what they are doing when the defaults
> are created.  They are giving what they think should be the most
> common values for the options.

For you to say the vendor's default is always your favorite is certainly
within your power.  But then to say that your favorite settings for
compiler #1 produce a faster (or smaller, or smoother, or more gender-
neutral) executable than your favorite settings for compiler #2 is quite
meaningless if your basis for choosing favorite settings is unrelated to
the comparison metric.

A compiler vendor may choose their defaults as being "best for those who
don't care about settings", which to me pretty much translates into those
for whom performance is not a big deal.

Larry Kilgallen

Re: Apex vs GNAT on solaris

[Samuel T. Harris]

Roger Racine wrote:
> 
> On Tue, 07 Dec 1999 01:25:13 GMT, Robert Dewar <dewar@gnat.com> wrote:
> 
> >In article <82hiuj$74o$1@nnrp1.deja.com>,
> >  reason67@my-deja.com wrote:
> >
> >
> >You need to say what options you are using for both compilers.
> >We have sometimes found people making the *amazing* mistake
> >of compiling GNAT with -O0. Generally the appropriate options
> >for benchmarking are -O2 -gnatn.
> >
> >The options are critical, because otherwise you may simply
> >be measuring differences in choices of default options. For
> >example if one compiler inlines by default, the other one
> >does not, then the comparison may be meaningless.
> >
> >You also need to specify all other parameters. For instance,
> >if you are running tasking, make sure you are comparing
> >comparable underlying threads implementations, otherwise you
> >again have apples and oranges (e.g. which of the two threads
> >libraries did you use for GNAT).
> >
> 
> Is it a mistake to use the default options for a compiler?  Many
> people will assume that the default options are the most reliable
> options.  Others will assume that the default options are the best
> options.  The default options generally turn out to be the most
> reliable because most people use them, and bugs will be discovered and
> corrected sooner than those to be found using the non-default options.

As a general rule, I try to avoid relying upon "default" values.
Given any tool you wish to discuss, by not explicitly configuring
the tool I am exposed to the maintenance question of "Am I relying
upon the defaults or did I just not know how to configure the tool."
So I try to configure whatever seems reasonable, even if my explicit
values are the default values.

As far as settings for compilers for benchmarks, I really am not
particularly interested in the faster speed possible. I am primarily
concerned with execution performance of the actual deliverable loads.
Therefore, when I run benchmarks I run them with the compiler
settings I'll be using for the final builds.

> 
> The same can be said for tasking.
> 
> I have been hit many times by optimizer bugs, and generally assume
> that the compiler vendors knows what they are doing when the defaults
> are created.  They are giving what they think should be the most
> common values for the options.
> 
> Roger Racine

-- 
Samuel T. Harris, Principal Engineer
Raytheon, Scientific and Technical Systems
"If you can make it, We can fake it!"

Re: Apex vs GNAT on solaris

[Ted Dennison]

In article <384cfdb3.691883075@newsnew.draper.com>,
  rracine@myremarq.com (Roger Racine) wrote:
> On Tue, 07 Dec 1999 01:25:13 GMT, Robert Dewar <dewar@gnat.com> wrote:
>
> I have been hit many times by optimizer bugs, and generally assume
> that the compiler vendors knows what they are doing when the defaults
> are created.  They are giving what they think should be the most
> common values for the options.

For optimization options, I want the defaults to be what will make
compiling fastest and running under the debugger easiest. The reason is
that I do far more compiles during code & testing phases than I do after
delivery. During those early phases, I don't particularly care that the
code isn't running as fast as it can. (That also means the fact that
"-g" isn't a default in Gnat is quite annoying.)

Once final testing begins, the options used need to be carefully
selected for the application, and are quite unlikely to match any
default the vendor would choose anyway. Oftentimes I'll even go with
debug symbols and minimal optimization in the final product, for ease of
maintanence.

--
T.E.D.


Sent via Deja.com http://www.deja.com/
Before you buy.

Re: Apex vs GNAT on solaris

[Robert Dewar]

In article <384cfdb3.691883075@newsnew.draper.com>,
  rracine@myremarq.com (Roger Racine) wrote:
> Is it a mistake to use the default options for a compiler?

Yes, it is a mistake if the default options do not correspond
to your needs, there are many options available.

> Many people will assume that the default options are the most
> reliable options.

They will assume wrong. For example, in the case of GNAT,
we let -O0 be the default because that is traditional in
all gcc compilers. We make inlining off the default, because
that seems more compatible with -O0 as the default.

> Others will assume that the default options are the best
> options.

They will assume wrong, there is no "best options", there is
only the appropriate options for your particular use.

> The default options generally turn out to be the most
> reliable because most people use them

Not true in the GCC/GNAT world, virtually all users plan on
setting at least -O1 for actual production work (-O0 is almost
never suitable for production work, it is usually only used
while debugging, since it is easier to debug unoptimized code).

> and bugs will be discovered and corrected sooner than those to
> be found using the non-default options.

Well you may guess this, but it is wrong for gcc. Far more code
is run with optimization on. It is not common to find bugs that
depend on the optimization level in the gcc technology (since
the backend is mature), but when they do arise, they are just
as likely to be bugs that show up ONLY with optimization off.

> The same can be said for tasking.

Again, this can be misleading. For example, on Solaris, the
system thread support is definitely buggy, and indeed we have
found it impossible to provide full Annex D support on Solaris,
even in realtime mode (it is out of the question in non-real
time, i.e. non-root mode). The FSU threads is FAR more reliable.
The choice of defaults is not based on reliability or accuracy
[which would dictate FSU], but usability, most people want to
use system threads [I assume this is what you mean by Posix
threads, your designation is confusing, since of course FSU
is also Posix compliant].

> I have been hit many times by optimizer bugs

Yes, that is true of many compilers, especially those with Ada
specific home spun code generators, but even there I think this
is mostly a historic factor, perhaps users of non-GNAT
technologies can comment ... Certainly there are some old
Ada 83 compilers that show this phenomenon. We have several
customers who report they cannot use their old Ada 83 compilers
with optimization on for this reason. But to repeat, gcc is
almost always used with optimization on, except when doing
initial debugging, so this kind of reliability problem is
not an issue. Once again, I am not saying that there is no
such thing as an optimization bug in the gcc technology, we
have occasionally run into such bugs and fixed them. The point
is that turning optimization ON is just as likely to remove bugs
as to cause them (and both are low probability events).

> and generally assum  that the compiler vendors knows what they
> are doing when the defaults are created.

We know EXACTLY what we are doing when we create the defaults,
it just does not happen to correspond to any of your guesses.

> They are giving what they think should be the most
> common values for the options.

Nope, that's plain wrong in the case of gcc! The important thing
in our mind is to be consistent with other uses of gcc, so that
the usage will be the same for GNAT, gnu C, g++ etc.

Robert Dewar
Ada Core Technologies


Sent via Deja.com http://www.deja.com/
Before you buy.

Re: Apex vs GNAT on solaris

[Robert A Duff]

Robert Dewar <dewar@gnat.com> writes:

> In article <384cfdb3.691883075@newsnew.draper.com>,
>   rracine@myremarq.com (Roger Racine) wrote:

> > Others will assume that the default options are the best
> > options.
> 
> They will assume wrong, there is no "best options", there is
> only the appropriate options for your particular use.

Right!  If there were a "best" value for any given option, then the
compiler writer would be foolish to have the option.  Instead, the
compiler should just always do what is best.  Options are costly -- they
can greatly increase the difficulty of testing, for example.

The reason options exist is that there are tradeoffs -- sometimes you
want fast compile times, sometimes you're willing to put up with slow
compile times in order to get better efficiency.  Neither one is
"better".

- Bob

Re: Apex vs GNAT on solaris

[Roger Racine]

On Wed, 08 Dec 1999 06:35:08 GMT, Robert Dewar <dewar@gnat.com> wrote:

>In article <384cfdb3.691883075@newsnew.draper.com>,
>  rracine@myremarq.com (Roger Racine) wrote:
>> Many people will assume that the default options are the most
>> reliable options.
>
>They will assume wrong. For example, in the case of GNAT,
>we let -O0 be the default because that is traditional in
>all gcc compilers. We make inlining off the default, because
>that seems more compatible with -O0 as the default.
>

That is a pretty questionable reason for creating a default.  Perhaps
compiler documentation should state reasons for using the various
options.  Of course, a given vendor might not want to say "turn
optimization on only as a last resort, since it is not well tested",
but certainly ACT (from what you say) should tell their users to turn
it on.

>> Others will assume that the default options are the best
>> options.
>
>They will assume wrong, there is no "best options", there is
>only the appropriate options for your particular use.
>

I do not know about your experience, but for me, with embedded system
compilers (which tend to have comparatively few users compared to
workstation compilers), for numerous languages over the years, it has
always been dangerous to change the default optimization options.  ACT
might want to reconsider their default.

>> The default options generally turn out to be the most
>> reliable because most people use them
>
>Not true in the GCC/GNAT world, virtually all users plan on
>setting at least -O1 for actual production work (-O0 is almost
>never suitable for production work, it is usually only used
>while debugging, since it is easier to debug unoptimized code).
>

How do you know this?  As an ACT user, I have never been asked about
my settings (-O2).  And the GCC world is pretty big.  You certainly
have not asked all those users.

>> and bugs will be discovered and corrected sooner than those to
>> be found using the non-default options.
>
>Well you may guess this, but it is wrong for gcc. Far more code
>is run with optimization on. It is not common to find bugs that
>depend on the optimization level in the gcc technology (since
>the backend is mature), but when they do arise, they are just
>as likely to be bugs that show up ONLY with optimization off.
>

This is not a guess, it is my experience with other compilers.  And,
as you have said many times, do not believe everything you read on
newsgroups.  :-)

As I said above, for GNAT I personally do use -O2, but only because I
did a bit of checking (such as the optimization level used in your
Make files and for the ACVC tests, or whatever they are called now),
but there are many others who have similar experiences and have not
done any checking.  They will use the default.  By the way, why don't
you use -O3?

>> The same can be said for tasking.
>
>Again, this can be misleading. For example, on Solaris, the
>system thread support is definitely buggy, and indeed we have
>found it impossible to provide full Annex D support on Solaris,
>even in realtime mode (it is out of the question in non-real
>time, i.e. non-root mode). The FSU threads is FAR more reliable.
>The choice of defaults is not based on reliability or accuracy
>[which would dictate FSU], but usability, most people want to
>use system threads [I assume this is what you mean by Posix
>threads, your designation is confusing, since of course FSU
>is also Posix compliant].
>

This is confused.  My message said nothing about Posix vs FSU.  But
you seem to be agreeing with me that it is better to use the default
for threads.  By the way, unless you have changed something, your
version of FSU Threads is -not- Posix compliant, as we have discussed
previously.  ACT is using an old version that was based on one of the
proposed Posix standards.

>> and generally assum  that the compiler vendors knows what they
>> are doing when the defaults are created.
>
>We know EXACTLY what we are doing when we create the defaults,
>it just does not happen to correspond to any of your guesses.
>

Nor does it seem to indicate much logic.  "We do it because everyone
else using the common back end does it.  We certainly do not want
anyone to use the default."  :-)

>> They are giving what they think should be the most
>> common values for the options.
>
>Nope, that's plain wrong in the case of gcc! The important thing
>in our mind is to be consistent with other uses of gcc, so that
>the usage will be the same for GNAT, gnu C, g++ etc.

How would the usage be different if you had a different (better)
default?  For gnu C, users would need to say "gcc -O1 XXX.c" to get
the first level of optimization.  For Ada, users could say the same
thing, or they could say "gcc XXX.adb" to get that same level
(assuming your default were -O1).  Why do you consider that to be an
overriding reason?

Roger Racine

Re: Apex vs GNAT on solaris

[Larry Kilgallen]

In article <384e52db.779218947@newsnew.draper.com>, rracine@myremarq.com (Roger Racine) writes:
> On Wed, 08 Dec 1999 06:35:08 GMT, Robert Dewar <dewar@gnat.com> wrote:
> 
>>In article <384cfdb3.691883075@newsnew.draper.com>,
>>  rracine@myremarq.com (Roger Racine) wrote:
>>> Many people will assume that the default options are the most
>>> reliable options.
>>
>>They will assume wrong. For example, in the case of GNAT,
>>we let -O0 be the default because that is traditional in
>>all gcc compilers. We make inlining off the default, because
>>that seems more compatible with -O0 as the default.
>>
> 
> That is a pretty questionable reason for creating a default.  Perhaps
> compiler documentation should state reasons for using the various
> options.  Of course, a given vendor might not want to say "turn
> optimization on only as a last resort, since it is not well tested",
> but certainly ACT (from what you say) should tell their users to turn
> it on.

Certainly compiler documentation should state the purpose of various
options.  I don't understand how, absent such a statement, one would
know that such an option exists.  Are there vendors whose documentation
says "we have an option called /FOOBAR, but we won't tell you what it
does" ?

>>They will assume wrong, there is no "best options", there is
>>only the appropriate options for your particular use.
>>
> 
> I do not know about your experience, but for me, with embedded system
> compilers (which tend to have comparatively few users compared to
> workstation compilers), for numerous languages over the years, it has
> always been dangerous to change the default optimization options.  ACT
> might want to reconsider their default.

I would believe Ada programmers to be a more discriminating bunch,
expecting features to have been tested, making incidents where an
adjustment to optimization options introduces "danger" to be less
frequent.  Furthermore, I would believe Ada programmers to expect
the vendor would fix anomolies encountered in this fashion.  Most
of all, I expect Ada programmers to test their resulting programs.
ACT might change their default, but misbehavior by other compilers
would be a silly reason to do so.

Larry Kilgallen

Re: Apex vs GNAT on solaris

[Roger Racine]

On Wed, 8 Dec 1999 15:39:22 GMT, kilgallen@eisner.decus.org (Larry
Kilgallen) wrote:

>In article <384e52db.779218947@newsnew.draper.com>, rracine@myremarq.com (Roger Racine) writes:
>> On Wed, 08 Dec 1999 06:35:08 GMT, Robert Dewar <dewar@gnat.com> wrote:
>> 
>>>In article <384cfdb3.691883075@newsnew.draper.com>,
>>>  rracine@myremarq.com (Roger Racine) wrote:
>>>> Many people will assume that the default options are the most
>>>> reliable options.
>>>
>>>They will assume wrong. For example, in the case of GNAT,
>>>we let -O0 be the default because that is traditional in
>>>all gcc compilers. We make inlining off the default, because
>>>that seems more compatible with -O0 as the default.
>>>
>> 
>> That is a pretty questionable reason for creating a default.  Perhaps
>> compiler documentation should state reasons for using the various
>> options.  Of course, a given vendor might not want to say "turn
>> optimization on only as a last resort, since it is not well tested",
>> but certainly ACT (from what you say) should tell their users to turn
>> it on.
>
>Certainly compiler documentation should state the purpose of various
>options.  I don't understand how, absent such a statement, one would
>know that such an option exists.  Are there vendors whose documentation
>says "we have an option called /FOOBAR, but we won't tell you what it
>does" ?
>

For example, the GNAT documentation simply states that you get no
optimization with -O0, "normal optimization" with -O1, "extensive
optimization" with -O2, and automatic inlining with -O3.  It does not
say anything about the speed of compilation difference, any debugger
deficiency with the various options, or any other reason one might
want to use one vs the other.  Is everyone supposed to have to check
the compile time for each option?

>>>They will assume wrong, there is no "best options", there is
>>>only the appropriate options for your particular use.
>>>
>> 
>> I do not know about your experience, but for me, with embedded system
>> compilers (which tend to have comparatively few users compared to
>> workstation compilers), for numerous languages over the years, it has
>> always been dangerous to change the default optimization options.  ACT
>> might want to reconsider their default.
>
>I would believe Ada programmers to be a more discriminating bunch,
>expecting features to have been tested, making incidents where an
>adjustment to optimization options introduces "danger" to be less
>frequent.  Furthermore, I would believe Ada programmers to expect
>the vendor would fix anomolies encountered in this fashion.  Most
>of all, I expect Ada programmers to test their resulting programs.
>ACT might change their default, but misbehavior by other compilers
>would be a silly reason to do so.
>

It is difficult to be discriminating when -all- compilers have bugs.
It is the natural result of compilers being big, complex software
systems.  No vendor will give you any sort of guarantee that their
code generator is perfect for all legal Ada programs.  And Ada is no
better or worse than any other language in this respect.

Of course the vendor is going to fix the bug (in the next release for
some vendors; in a patched version for others), but the problem comes
in the determination that there is a bug.  Remembering back to an Ada
83 program back in the early 1990s, some strange behavior took 2
people at least 2 full days to find the problem (at least 1 of those
days lasting until 2:00 in the morning).  This is not ancient history.
Now my first experience with a compiler bug might be considered by
some to be ancient (mid 1970s; in a programming course at college; the
computer consultants did not believe me at first, thinking I simply
wanted them to help me find a bug in my code), and my next (late
1970s), but it has been continuous throughout the 1980s and 1990s that
my colleagues and I have found bugs.  Most of these were found using
options that were other than the defaults.

Of course the resulting programs will be tested.  But I can only
assume you have never tracked down a compiler bug if you say that as a
reason not to worry about such problems.

ACT can do what they like, but they had better understand that there
are a lot of people with similar experiences to mine, at least in the
embedded world.  My general assumption (and I checked with a couple
colleagues today and got agreement) is that if the vendor has the
default optimization set to a given value, any lower value is OK, but
any higher value is dangerous and should be used with great care.  GCC
is a special case where the compiler build script can be examined.  If
the compiler itself is built with an optimization higher than the
default, I trust that value.

One more piece of history.  For the Space Shuttle, the primary
computer system and the backup system both use the HAL/S language.
However, the backup compiler is different from the primary compiler.
One of the differences was that the optimizer put in the primary
compiler was not allowed to be put in the backup version.  The
optimizer was considered an unnecessary risk factor.  Bugs were indeed
found in the primary compiler optimizer, but only, to my pre-1982
knowledge, in testing.

Roger Racine

Re: Apex vs GNAT on solaris

[Larry Kilgallen]

In article <384ea194.799371745@newsnew.draper.com>, rracine@myremarq.com (Roger Racine) writes:

> For example, the GNAT documentation simply states that you get no
> optimization with -O0, "normal optimization" with -O1, "extensive
> optimization" with -O2, and automatic inlining with -O3.  It does not
> say anything about the speed of compilation difference, any debugger
> deficiency with the various options, or any other reason one might
> want to use one vs the other.  Is everyone supposed to have to check
> the compile time for each option?

No, just those who care about the compile time.  Certainly ACT cannot
document the compile time, because differences in compile time will
depend on how heavily the program being compiled uses the features
that get more optimization.  A very good heuristic is that more
optimization takes longer during compilation.

Another heuristic is that more optimization makes debugging at the
machine code level harder, because the machine code layout bears
less resemblence to the source code layout.  ACT cannot document
how much harder, because that is in the eye of the beholder.
I have debugged a lot of Alpha machine code, but using GNAT
I would be less well equipped than someone else because I am
accustomed to looking at the output of a different code generator.

Both of these heuristics can be combined into one easy-to-remember saying:

	"There ain't no such thing as a free lunch."

Where the word "free" means anything I want it to mean :-).

Larry Kilgallen

Re: Apex vs GNAT on solaris

[tmoran]

>say anything about the speed of compilation difference, any debugger
>deficiency with the various options, or any other reason one might
>want to use one vs the other.
  Size of generated code, for instance.  It's not too hard to hand
optimize (in asm if necessary) 10% of the code if it's using 90% of
the CPU time, but you really are at the mercy of the compiler as far
as size of program is concerned.  Size still matters if you have
limited RAM or cache, or must send the code over a low speed link.

Re: Apex vs GNAT on solaris

[Robert Dewar]

In article <384e52db.779218947@newsnew.draper.com>,
  rracine@myremarq.com (Roger Racine) wrote:
> That is a pretty questionable reason for creating a default.

Not at all. You certainly want the various gcc front ends
to be compatible (lots of people create mixed language
programs). In any case, as people have pointed out here,
having the default be the fast compile option suitable
for development probably *is* the most convenient default
for most people.


>Perhaps
> compiler documentation should state reasons for using the
various
> options.  Of course, a given vendor might not want to say
"turn
> optimization on only as a last resort, since it is not well
tested",
> but certainly ACT (from what you say) should tell their users
to turn
> it on.

Perhaps you should read the documentation :-)

Here is the relevant excerpt

-----------
@item -O3
full optimization, and also attempt automatic inlining of small
subprograms within a unit (@pxref{Inlining of Subprograms}).
@end table

Higher optimization levels perform more global transformations
on the
program and apply more expensive analysis algorithms in order to
generate
faster and more compact code.  The price in compilation time,
and the
resulting improvement in execution time,
both depend on the particular application and the hardware
environment.
You should experiment to find the best level for your
application.

Note: Unlike some other compilation systems, @code{gcc} has
been tested extensively at all optimization levels. There are
some bugs
which appear only with optimization turned on, but there have
also been
bugs which show up only in @emph{unoptimized} code. Selecting a
lower
level of optimization does not improve the reliability of the
code
generator, which in practice is highly reliable at all
optimization
levels.
----------------------------

> I do not know about your experience, but for me, with embedded
system
> compilers (which tend to have comparatively few users compared
to
> workstation compilers), for numerous languages over the years,
it has
> always been dangerous to change the default optimization
options.

Not at all

> ACT
> might want to reconsider their default.

We don't know of any customers who have a problem with the
default now. If we did change the default, then for sure that
would create massive confusion.

> How do you know this?  As an ACT user, I have never been asked
> about my settings (-O2).  And the GCC world is pretty big.
> You certainly have not asked all those users.

Of course we have not asked "all those users", but we definitely
are aware enough of general usage to make me quite confident
in telling you that virtually all users of gcc go to at least
-O1 for production purposes. Sure there could be an exception,
but we work closely with many customers, and we never saw an
exception to this rule.

> This is not a guess, it is my experience with other compilers.
> And, as you have said many times, do not believe everything
> you read on newsgroups.  :-)

Well I am sorry other compilers have taught you to be afraid
to optimize. It is definitely not a good idea to assume that
all compilers are the same in this respect. If you insist on
believing your experience with other completely unrelated
compilers over our documentation that is of course your
decision, but it can lead to suboptimal decisions.

> As I said above, for GNAT I personally do use -O2, but only
because I
> did a bit of checking (such as the optimization level used in
your
> Make files and for the ACVC tests, or whatever they are called
now),
> but there are many others who have similar experiences and
have not
> done any checking.  They will use the default.  By the way,
why don't
> you use -O3?

We generally do not recommend -O3, though it depends on the
particular circumstance, -O3 will of course generally increase
the size of the executable, and icache effects can often cancel
out expected gains.


> This is confused.  My message said nothing about Posix vs FSU.

Actually your message talked about how you can expect the
default options to be more reliable, and then said "the same
can be said for tasking".

If, as seemed clear, you were saying that the default tasking
is more reliable, this is in fact incorrect.

> But you seem to be agreeing with me that it is better to use
> the default for threads.

Not at all. There is no "better to use", it depends on
particular customer needs. I think the majority of customers
will prefer the default, but there are many reasons to choose
the FSU package instead, one of which is higher performance,
the other is higher reliability and faithfulness to the
required Ada semantics (FSU was of course used for validation).

> By the way, unless you have changed something, your
> version of FSU Threads is -not- Posix compliant, as we have
> discussed previously.  ACT is using an old version that was
> based on one of the proposed Posix standards.

That's irrelevant, since the access to the tasking facilities
must be via the Ada tasking anyway. We strongly recommend
against any Ada program directly or indirectly making posix
threads calls other than via the Ada runtime, it almost always
gets people into trouble.

> Nor does it seem to indicate much logic.  "We do it because
> everyone else using the common back end does it."

We consider consistency of software interfaces to be valuable.

> We certainly do not want anyone to use the default."  :-)

Umm, that's a really strange claim. Most people use the default
most of the time, because during development you usually prefer
to use low optimization levels, giving better debugging behavior
and faster compilations. In fact the reason the gcc defaults
are set the way they are is PRECISELY this fact! And indeed
as you have seen in this thread, there are definitely users who
agree. If you use -O2 for all your development, there is nothing
wrong with that, but it is unusual.


> How would the usage be different if you had a different
> (better) default?

We don't see optimization on as a better default!

> For gnu C, users would need to say "gcc -O1 XXX.c" to get
> the first level of optimization.  For Ada, users could say the
> same thing, or they could say "gcc XXX.adb" to get that same
> level (assuming your default were -O1).  Why do you consider
> that to be an overriding reason?

Consistency is important here. Roger I know you are allergic
to C and want to do everything in 100% Ada, but actually the
more usual profile of our users is that they are doing at least
some mixed language programming.

If we really felt the default were wrong, we would consider
changing it, but in fact we agree with gcc's choice of defaults
anyway. There is certainly no disadvantage in practice (we don't
see any problem of users shying away from optimization merely
because it is not the default -- why, even you, who are SO
inclined to use the default, end up following normal practice
and using -O2 :-)

The one disadvantage of the no optimization default seems to
be the problem of naive benchmarking, which in fact is the
scenario that started this thread.

But I don't think it is right to change the default just because
of this concern. Indeed, anyone doing benchmarks who does not
understand the need for carefully studying what options should
be used is going to get meaningless results in any case.


Sent via Deja.com http://www.deja.com/
Before you buy.

Re: Apex vs GNAT on solaris

[Roger Racine]

On Thu, 09 Dec 1999 06:21:49 GMT, Robert Dewar
<robert_dewar@my-deja.com> wrote:

>In article <384e52db.779218947@newsnew.draper.com>,
>  rracine@myremarq.com (Roger Racine) wrote:
>If we really felt the default were wrong, we would consider
>changing it, but in fact we agree with gcc's choice of defaults
>anyway. There is certainly no disadvantage in practice (we don't
>see any problem of users shying away from optimization merely
>because it is not the default -- why, even you, who are SO
>inclined to use the default, end up following normal practice
>and using -O2 :-)
>

I am only trying to tell you that there are many people with my
experience.  I am trying to help ACT.  You do not need to convince
-me- to use a higher level of optimization.  I happen to have built
the GNU tools, and have enough sense to use the same options as are
used building those tools.  That, however, is not going to be the case
for many other people (they have not built the GNU tools).  I truly
believe you that all optimization levels are tested.  There are,
unfortunately, many people who are not subscribed to this newslist,
and there are many who would not believe the CEO of a compiler company
making this claim. :-)

>The one disadvantage of the no optimization default seems to
>be the problem of naive benchmarking, which in fact is the
>scenario that started this thread.
>

Exactly.  Which also could lead to selection of a compiler vendor for
the wrong reason (See.  I am really only concerned with ACT's
interests).  Or it could lead to programs losing time fixing
performance problems that would be unnecessary if they simply used a
higher optimization level (OK, so it is not -just- ACT's interests).

And the disadvantage of having -O1 or -O2 the default is what?  I have
not seen any difficulty debugging -O2 code yet.  Compilation speed is
probably at least one order of magnitude faster than what I was used
to 6 or 7 years ago.

>But I don't think it is right to change the default just because
>of this concern. Indeed, anyone doing benchmarks who does not
>understand the need for carefully studying what options should
>be used is going to get meaningless results in any case.

Again, I only am trying to make you understand that there are many
people who have experiences similar to mine.  I spoke to some
colleagues yesterday about this thread, and they were truly surprised
and shocked to learn that the default was picked -not- based on "best
optimization that is well-tested".

I think we can end this thread.  The decision is certainly the
compiler vendors' to make.  I think I have made my point that I
believe there are people (like my colleagues I talked to), large in
number, who will use the default, or less, with confidence, but will
not use higher without a good reason.

There are compiler vendors who pick their defaults based on the
criterion I mentioned.

Roger Racine

Re: Apex vs GNAT on solaris

[Mike Silva]

Roger Racine wrote in message <384fa0a4.864668076@newsnew.draper.com>...
>...I think I have made my point that I
>believe there are people (like my colleagues I talked to), large in
>number, who will use the default, or less, with confidence, but will
>not use higher without a good reason.


But surely, in the case where a mature compiler defaults to *no*
optimization, nobody should think that *any* optimization is suspect in that
compiler -- that's just not reasonable.

Mike

Re: Apex vs GNAT on solaris

[Robert Dewar]

In article <5OQ34.29$ia1.6535@news.wenet.net>,
  "Mike Silva" <mjsilva@jps.net> wrote:
> But surely, in the case where a mature compiler defaults to
> *no* optimization, nobody should think that *any* optimization
> is suspect in that compiler -- that's just not reasonable.
>
> Mike

Well nobody is a rather fierce word :-) But I can say that in
our experience this is not a problem.


Sent via Deja.com http://www.deja.com/
Before you buy.

Re: Apex vs GNAT on solaris

[Larry Kilgallen]

In article <384fa0a4.864668076@newsnew.draper.com>, rracine@myremarq.com (Roger Racine) writes:
> On Thu, 09 Dec 1999 06:21:49 GMT, Robert Dewar
> <robert_dewar@my-deja.com> wrote:

>>The one disadvantage of the no optimization default seems to
>>be the problem of naive benchmarking, which in fact is the
>>scenario that started this thread.
>>
> 
> Exactly.  Which also could lead to selection of a compiler vendor for
> the wrong reason (See.  I am really only concerned with ACT's
> interests).

But ACT does not really sell compilers (or compiler licenses).
They sell support.

It seems to me that someone who did not understand the effect that
options might have on performance would be a relatively expensive
customer to support.

Larry Kilgallen

Re: Apex vs GNAT on solaris

[Robert Dewar]

In article <1999Dec9.104359.1@eisner>,
  Kilgallen@eisner.decus.org.nospam wrote:
> But ACT does not really sell compilers (or compiler licenses).
> They sell support.

That's not true at all, we do indeed sell compiler licenses
together with support. True we place much more emphasis on
support, and provide a very permissive license, but I don't
think that there is that much basic difference in sales terms.
Sure, the permissive license maybe (and very often is) a strong
selling point, as is the commitment to high level support, but
from a typical users point of view, they are indeed "buying a
compiler".

> It seems to me that someone who did not understand the effect
> that options might have on performance would be a relatively
> expensive customer to support.

Well. First we certainly do not try to do some selection of
customers on the basis of how much support they will need, and
if we did, it wouldn't work, neither we nor the customer can
really guess how much support they will need in the future.

Second, I am not even sure that your claim is even approximately
true. Sometimes it is the really knowledgable uses who make the
most use of our support services. In any case we certainly do
not try to discourage customers from using our support services,
because we want them to find out how valuable they can be :-)

Robert Dewar
Ada Core Technologies


Sent via Deja.com http://www.deja.com/
Before you buy.

Re: Apex vs GNAT on solaris

[Stefan Skoglund]

Roger Racine wrote:
> unfortunately, many people who are not subscribed to this newslist,
> and there are many who would not believe the CEO of a compiler company
> making this claim. :-)
> 

Hm, i pretty much trust the optimizer technology in GCC.

Mandrake Linux have mad a business out of providing heavily
Pentium(Pro) optimized versions of RedHat Linux.

Which means an improvement of around 20-30% and things
haven't gotten less reliable.

Re: Apex vs GNAT on solaris

[reason67]

>       4.0 configurable number of task pairs one calling the others
> rendezvous.


-=-=-=-=-=-=-

with Ada.Calendar;
with Ada.Text_Io;
with Common_Utilities;

procedure Aaaa_4_0_Rendezvous is

    Compiler : constant String := "Apex";
    Maximum_Task_Pairs : constant := 10;
    Setup_Time : constant Duration := 5.0;
    Run_Duration : constant Duration := 60.0;

    function "<" (Left : in Ada.Calendar.Time; Right : in
Ada.Calendar.Time)
                 return Boolean renames Ada.Calendar."<";

    function "+" (Left : in Ada.Calendar.Time; Right : in Duration)
                 return Ada.Calendar.Time renames Ada.Calendar."+";

    task type Sender_Type is

        entry Set_Up (Number : in Integer;
                      Start_Time : in Ada.Calendar.Time;
                      End_Time : in Ada.Calendar.Time);

        entry Stop (Rendezvous_Count : out Integer);

    end Sender_Type;

    task type Receiver_Type is
        entry Timing_Entry;
    end Receiver_Type;

    type Sender_Array_Type is array (1 .. Maximum_Task_Pairs) of
Sender_Type;
    type Receiver_Array_Type is
       array (1 .. Maximum_Task_Pairs) of Receiver_Type;

    Sender_Array : Sender_Array_Type;
    Receiver_Array : Receiver_Array_Type;
    Begin_Time : Ada.Calendar.Time;
    End_Time : Ada.Calendar.Time;
    Max_Rendezvous : Integer := -1;
    Min_Rendezvous : Integer := Integer'Last;
    Total : Integer := 0;
    Average : Integer := 0;
    Current : Integer;

    task body Sender_Type is
        Receiver_Number : Integer;
        Start : Ada.Calendar.Time;
        Finish : Ada.Calendar.Time;
        Number_Of_Rendezvous : Integer := 0;
    begin
        accept Set_Up (Number : in Integer;
                       Start_Time : in Ada.Calendar.Time;
                       End_Time : in Ada.Calendar.Time) do

            Receiver_Number := Number;
            Start := Start_Time;
            Finish := End_Time;

        end Set_Up;

        delay until Start;

        --
        -- loop until the Finish time has occured
        --
        while Ada.Calendar.Clock < Finish loop

            Receiver_Array (Receiver_Number).Timing_Entry;
            Number_Of_Rendezvous := Number_Of_Rendezvous + 1;

            delay 0.0;

        end loop;

        --
        -- Gather statistics and terminate the task.
        --
        accept Stop (Rendezvous_Count : out Integer) do

            Rendezvous_Count := Number_Of_Rendezvous;

        end Stop;

    end Sender_Type;


    --
    -- Simple task that accepts rendezvous until the calling parent task
ends
    --
    task body Receiver_Type is
    begin
        loop
            select
                accept Timing_Entry;
            or
                terminate;
            end select;
        end loop;
    end Receiver_Type;


begin

    Ada.Text_Io.Put_Line ("::: [Aaaa_4_0_Rendezvous]");

    Begin_Time := Ada.Calendar.Clock + Setup_Time;
    End_Time := Begin_Time + Run_Duration;

    --
    -- Loop through all senders and tell it the common start time, end
time,
    -- and the number of the receiver task it should call.
    --
    for Index in 1 .. Maximum_Task_Pairs loop
        Sender_Array (Index).Set_Up
           (Number => Index, Start_Time => Begin_Time, End_Time =>
End_Time);
    end loop;

    --
    -- Gather the statistic information. It should hang on the Stop
entry
    -- until the completion time occurs.
    --
    for Index in 1 .. Maximum_Task_Pairs loop
        Sender_Array (Index).Stop (Rendezvous_Count => Current);

        Total := Total + Current;

        if Current > Max_Rendezvous then
            Max_Rendezvous := Current;
        end if;

        if Current < Min_Rendezvous then
            Min_Rendezvous := Current;
        end if;

    end loop;

    Average := Total / Maximum_Task_Pairs;


    --
    -- Output the results
    --
    Ada.Text_Io.Put_Line
       (Compiler & " reporting for " &
        Common_Utilities.Strip_Spaces (Duration'Image (Run_Duration)) &
        " sec. run with " &
        Common_Utilities.Strip_Spaces (Integer'Image
(Maximum_Task_Pairs)) &
        " pairs of tasks (no Data):");
    Ada.Text_Io.New_Line;
    Ada.Text_Io.Put_Line ("Total Rendezvous            : " &
                          Common_Utilities.Strip_Spaces
                             (Integer'Image (Total)));
    Ada.Text_Io.Put_Line ("Max Rendezvous in 1 pair    : " &
                          Common_Utilities.Strip_Spaces
                             (Integer'Image (Max_Rendezvous)));
    Ada.Text_Io.Put_Line ("Min Rendezvous in 1 pair    : " &
                          Common_Utilities.Strip_Spaces
                             (Integer'Image (Min_Rendezvous)));
    Ada.Text_Io.Put_Line ("Average Rendezvous per pair : " &
                          Common_Utilities.Strip_Spaces
                             (Integer'Image (Average)));

    Ada.Text_Io.Put_Line ("::: [Aaaa_4_0_Rendezvous has finished.]");

end Aaaa_4_0_Rendezvous;

-=-=-=-=-=-=-=-=-=

Apex Reporting for Run with 1 pair of tasks (no Data):

Total Rendesvous            : 263062

-=-=-=-=-

GNAT Reporting for Run with 1 pair of tasks (no Data):

Total Rendesvous            : 676564

-=-=-=-=-

Apex Reporting for Run with 10 pairs of tasks (No Data):

Total Rendesvous            : 703448
Max Rendesvous in 1 pair    : 73407
Min Rendesvous in 1 pair    : 67145
Average rendesvous per pair : 70344

-=-=-=-=-

GNAT Reporting for Run with 10 pairs of tasks (no Data):

Total Rendesvous            : 624886
Max Rendesvous in 1 pair    : 72561
Min Rendesvous in 1 pair    : 56665
Average rendesvous per pair : 62488

-=-=-=-=-

Apex Reporting for Run with 100 pairs of tasks (No Data):

Total Rendesvous            : 612376
Max Rendesvous in 1 pair    : 6229
Min Rendesvous in 1 pair    : 5971
Average rendesvous per pair : 6123

-=-=-=-=-

GNAT Reporting for Run with 100 pairs of tasks (no Data):

Total Rendesvous            : 578667
Max Rendesvous in 1 pair    : 6636
Min Rendesvous in 1 pair    : 5263
Average rendesvous per pair : 5786

-=-=-=-=-

Apex Reporting for Run with 1000 pairs of tasks (no Data):

Total Rendesvous            : 385579
Max Rendesvous in 1 pair    : 414
Min Rendesvous in 1 pair    : 360
Average rendesvous per pair : 385

-=-=-=-=-

GNAT Reporting for Run with 1000 pairs of tasks (no Data):

Total Rendesvous            : 113781
Max Rendesvous in 1 pair    : 221
Min Rendesvous in 1 pair    : 86
Average rendesvous per pair : 113


Sent via Deja.com http://www.deja.com/
Before you buy.

Re: Apex vs GNAT on solaris

[reason67]

>       1.0 15 iterations of 1000 delays of duration'small.
>           report worst time for the delay and the average time.


-=-=-=-=-=-=-=-=-

with Ada.Calendar;
with Ada.Text_Io;
with Common_Utilities;

procedure Aaaa_1_0_Timing_Test is

    package Long_Float_Io is new Ada.Text_Io.Float_Io (Long_Float);

    type Time_Stamp_Type is array (0 .. 1_000) of Ada.Calendar.Time;

    Compiler : constant String := "Apex";
    Instances : constant := 15;
    Iterations : constant := 1000;


    Biggest_Number : Integer;
    Smallest_Number : Integer;

    Biggest : Long_Float := -1.0;
    Smallest : Long_Float := 100.0;
    Average : Long_Float := 0.0;
    Length : Long_Float;
    Time_Stamp : Time_Stamp_Type;
    Instance_Length : Integer;

begin

    Ada.Text_Io.Put_Line ("::: [Aaaa_1_0_Timing_Test]");

    Ada.Text_Io.Put_Line
       (Compiler & " timing test running " &
        Common_Utilities.Strip_Spaces (Integer'Image (Instances)) &
        " instances of " &
        Common_Utilities.Strip_Spaces (Integer'Image (Iterations)) &
        " delays of duration'small");

    Instance_Length := Common_Utilities.Strip_Spaces
                          (Integer'Image (Instances))'Length;

    for Counter in 1 .. Instances loop

        Biggest := -1.0;
        Smallest := 100.0;
        Average := 0.0;

        Time_Stamp (0) := Ada.Calendar.Clock;

        for Index in 1 .. Iterations loop

            delay Duration'Small;
            Time_Stamp (Index) := Ada.Calendar.Clock;

        end loop;

        for Index in 1 .. Iterations loop

            Length := Long_Float (Ada.Calendar."-" (Time_Stamp (Index),
                                                    Time_Stamp (Index -
1))) -
                      Long_Float (Duration'Small);

            if Biggest < Length then
                Biggest := Length;
                Biggest_Number := Index;
            end if;

            if Smallest > Length then
                Smallest := Length;
                Smallest_Number := Index;
            end if;

            Average := Average + Length;

        end loop;

        Average := Average / Long_Float (Iterations);

        Ada.Text_Io.Put ("Instance " & Common_Utilities.String_Of
                                          (Number => Counter,
                                           Length => Instance_Length,
                                           Show_Leading_Zeros => False)
&
                         ": Maximum time => ");
        Long_Float_Io.Put (Item => Biggest, Fore => 1, Aft => 4, Exp =>
0);
        Ada.Text_Io.Put (Ascii.Ht & Ascii.Ht & "Average time => ");
        Long_Float_Io.Put (Item => Average, Fore => 1, Aft => 4, Exp =>
0);
        Ada.Text_Io.New_Line;
    end loop;

    Ada.Text_Io.Put_Line ("::: [Aaaa_1_0_Timing_Test has finished]");

end Aaaa_1_0_Timing_Test;


-=-=-=-=-=-=-=-=-=-=-=-

--
-- Note: Apex duration'Small is 0.0001
--
Apex timing test running 15 instances of 1000 delays of duration'small
Instance  1: Maximum time => 0.0670		Average time => 0.0003
Instance  2: Maximum time => 0.0006		Average time => 0.0001
Instance  3: Maximum time => 0.0108		Average time => 0.0001
Instance  4: Maximum time => 0.0006		Average time => 0.0001
Instance  5: Maximum time => 0.0006		Average time => 0.0001
Instance  6: Maximum time => 0.0002		Average time => 0.0001
Instance  7: Maximum time => 0.0002		Average time => 0.0001
Instance  8: Maximum time => 0.0007		Average time => 0.0001
Instance  9: Maximum time => 0.0005		Average time => 0.0001
Instance 10: Maximum time => 0.0006		Average time => 0.0001
Instance 11: Maximum time => 0.0108		Average time => 0.0001
Instance 12: Maximum time => 0.0380		Average time => 0.0001
Instance 13: Maximum time => 0.0010		Average time => 0.0001
Instance 14: Maximum time => 0.0062		Average time => 0.0001
Instance 15: Maximum time => 0.0108		Average time => 0.0001

--
-- Note: GNAT duration'Small is 0.000000001
--
GNAT timing test running 15 instances of 1000 delays of duration'small
Instance  1: Maximum time => 0.0494		Average time => 0.0101
Instance  2: Maximum time => 0.0270		Average time => 0.0100
Instance  3: Maximum time => 0.0268		Average time => 0.0100
Instance  4: Maximum time => 0.0251		Average time => 0.0100
Instance  5: Maximum time => 0.0271		Average time => 0.0100
Instance  6: Maximum time => 0.0420		Average time => 0.0100
Instance  7: Maximum time => 0.0249		Average time => 0.0100
Instance  8: Maximum time => 0.1332		Average time => 0.0109
Instance  9: Maximum time => 0.1102		Average time => 0.0108
Instance 10: Maximum time => 0.1700		Average time => 0.0106
Instance 11: Maximum time => 0.1914		Average time => 0.0105
Instance 12: Maximum time => 0.0420		Average time => 0.0103
Instance 13: Maximum time => 0.0483		Average time => 0.0103
Instance 14: Maximum time => 0.1302		Average time => 0.0104
Instance 15: Maximum time => 0.0281		Average time => 0.0101


Sent via Deja.com http://www.deja.com/
Before you buy.