* GNAT stuck, any idea on how to diagnose it?
@ 2014-09-14 9:21 Natasha Kerensikova
2014-09-14 11:59 ` Georg Bauhaus
` (2 more replies)
0 siblings, 3 replies; 13+ messages in thread
From: Natasha Kerensikova @ 2014-09-14 9:21 UTC (permalink / raw)
Hello,
I have a library project, called Natools, where I accumulate
miscellaneous small stuff. I also have a test suite, and a binary called
`test_all` that runs all the test suites.
Since I'm accumulating stuff, and test_all is accessing everything, it's
getting larger and larger, and takes more and more time to build. So far
so good.
However a few days ago it seems I've reached a threshold where gnatmake
takes a much longer time to finish. I've tried letting it run, and so
far it has taken 41 CPU hours without any sign of progress, while
without my latest package test it doesn't take more than a few wallclock
minutes. Moreover, the memory usage reported by the operating system
hasn't changed between a few minutes after start and now.
Here is the command line, word-wrapped to make it easier to read:
/usr/local/gcc-aux/libexec/gcc/x86_64-aux-freebsd9.2/4.9.0/gnat1 -I
--quiet -dumpbase natools-s_expressions-templates-integers.ads -auxbase
natools-s_expressions-templates-integers -O3 -fstack-check=specific
-gnatez -gnatea -gnatec=/home/nat/code/natools/obj/GNAT-TEMP-000004.TMP
-gnatafnovy -gnateE -gnatwae -gnatA
-gnatem=/home/nat/code/natools/obj/GNAT-TEMP-000005.TMP -gnatez
-mtune=generic -march=x86-64
/home/nat/code/natools/src/natools-s_expressions-templates-integers.ads
-o /tmp//ccd4TDPm.s
Would you know what is this `gnat1`?
What part of the build process is it?
What is the command above supposed to perform?
Is it known to occasionally take a lot of time?
Can I consider it eternally stuck, or am I just too impatient?
What kind of tool can I used to help answer those questions?
Thanks in advance for your help,
Natasha
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: GNAT stuck, any idea on how to diagnose it?
2014-09-14 9:21 GNAT stuck, any idea on how to diagnose it? Natasha Kerensikova
@ 2014-09-14 11:59 ` Georg Bauhaus
2014-09-14 14:49 ` Dmitry A. Kazakov
2014-09-14 17:45 ` Robert A Duff
2 siblings, 0 replies; 13+ messages in thread
From: Georg Bauhaus @ 2014-09-14 11:59 UTC (permalink / raw)
On 14.09.14 11:21, Natasha Kerensikova wrote:
> /usr/local/gcc-aux/libexec/gcc/x86_64-aux-freebsd9.2/4.9.0/gnat1 -I
> --quiet -dumpbase natools-s_expressions-templates-integers.ads -auxbase
> natools-s_expressions-templates-integers -O3 -fstack-check=specific
> -gnatez -gnatea -gnatec=/home/nat/code/natools/obj/GNAT-TEMP-000004.TMP
> -gnatafnovy -gnateE -gnatwae -gnatA
> -gnatem=/home/nat/code/natools/obj/GNAT-TEMP-000005.TMP -gnatez
> -mtune=generic -march=x86-64
> /home/nat/code/natools/src/natools-s_expressions-templates-integers.ads
> -o /tmp//ccd4TDPm.s
>
> Would you know what is this `gnat1`?
The GNAT User's Guide says this in "3.1 Compiling Programs":
`gcc' is actually a driver program that looks at the extensions of
the file arguments and loads the appropriate compiler. For example, the
GNU C compiler is `cc1', and the Ada compiler is `gnat1'.
The User's Guide is included with the compiler, in any formats the user
may likely prefer.
> What kind of tool can I used to help answer those questions?
My first attempt would be to obtain access to one of the platforms that
have receive more extensive support for the Ada part of GCC, for example
one of the platforms supported by GPL GNAT, or Debian/GNU. This should
help with excluding issues that may need to be attributed to GNAT on FreeBSD.
Then, also, some OS tracing facility may hint at loops.
Another option is to run the translation process from a debugger.
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: GNAT stuck, any idea on how to diagnose it?
2014-09-14 9:21 GNAT stuck, any idea on how to diagnose it? Natasha Kerensikova
2014-09-14 11:59 ` Georg Bauhaus
@ 2014-09-14 14:49 ` Dmitry A. Kazakov
2014-09-14 15:21 ` Brad Moore
` (2 more replies)
2014-09-14 17:45 ` Robert A Duff
2 siblings, 3 replies; 13+ messages in thread
From: Dmitry A. Kazakov @ 2014-09-14 14:49 UTC (permalink / raw)
On Sun, 14 Sep 2014 09:21:18 +0000 (UTC), Natasha Kerensikova wrote:
> Would you know what is this `gnat1`?
> What part of the build process is it?
> What is the command above supposed to perform?
> Is it known to occasionally take a lot of time?
Yes. It may take relatively much time on its own, but from my experience, I
can tell that GNAT consumes extremely and unreasonably much memory.
At some point it stops compiling under 32-bit systems because there is not
enough virtual memory.
You could check if your problem is caused by exhaustion of the physical
memory and turning to the swap. If swap gets involved it will never end.
--
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: GNAT stuck, any idea on how to diagnose it?
2014-09-14 14:49 ` Dmitry A. Kazakov
@ 2014-09-14 15:21 ` Brad Moore
2014-09-14 21:47 ` Natasha Kerensikova
2014-09-14 21:45 ` anon
2014-09-15 7:16 ` Natasha Kerensikova
2 siblings, 1 reply; 13+ messages in thread
From: Brad Moore @ 2014-09-14 15:21 UTC (permalink / raw)
On 2014-09-14 8:49 AM, Dmitry A. Kazakov wrote:
> On Sun, 14 Sep 2014 09:21:18 +0000 (UTC), Natasha Kerensikova wrote:
>
>> Would you know what is this `gnat1`?
>> What part of the build process is it?
>> What is the command above supposed to perform?
>> Is it known to occasionally take a lot of time?
>
> Yes. It may take relatively much time on its own, but from my experience, I
> can tell that GNAT consumes extremely and unreasonably much memory.
>
> At some point it stops compiling under 32-bit systems because there is not
> enough virtual memory.
>
> You could check if your problem is caused by exhaustion of the physical
> memory and turning to the swap. If swap gets involved it will never end.
>
I have on very rare occasion run into a case where the compiler appears
to be stuck in an endless loop when encountering some illegal code that
it couldn't handle. (More typically, this would involve a compiler crash
where at least you get some diagnostic message offering a clue to the
problem). In my case, I think it had something to do with a name
resolution problem I think. I found the source of the problem by
commenting out sections of new code until the
problem went away, then adding back smaller sections until the program
reappears, until I zeroed in on the problematic code. Once I realized my
mistake, I changed the code to use legal constructs, and the code
compiled right away.
Brad
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: GNAT stuck, any idea on how to diagnose it?
2014-09-14 15:21 ` Brad Moore
@ 2014-09-14 21:47 ` Natasha Kerensikova
2014-09-14 22:56 ` Anh Vo
0 siblings, 1 reply; 13+ messages in thread
From: Natasha Kerensikova @ 2014-09-14 21:47 UTC (permalink / raw)
Hello,
On 2014-09-14, Brad Moore <brad.moore@shaw.ca> wrote:
> On 2014-09-14 8:49 AM, Dmitry A. Kazakov wrote:
>> On Sun, 14 Sep 2014 09:21:18 +0000 (UTC), Natasha Kerensikova wrote:
> I have on very rare occasion run into a case where the compiler appears
> to be stuck in an endless loop when encountering some illegal code that
> it couldn't handle. (More typically, this would involve a compiler crash
> where at least you get some diagnostic message offering a clue to the
> problem). In my case, I think it had something to do with a name
> resolution problem I think. I found the source of the problem by
> commenting out sections of new code until the
> problem went away, then adding back smaller sections until the program
> reappears, until I zeroed in on the problematic code. Once I realized my
> mistake, I changed the code to use legal constructs, and the code
> compiled right away.
It turns out your message was the largest contributor to my diagnostic,
except I don't see any mistake in my code (though thinking about it now,
I seem to remember people advising against expression functions
returning unconstrained values).
The presumably-endless loop was triggered by the following expression:
(if Ref.Is_Empty then To_Atom (Fallback) else Ref.Query.Data.all);
It was originally in an expression function, but turning it into a
normal function whose body is only a return statement with the
expression above leads to the same symptoms.
However, turning it into an if statement in the normal function below
sloves the problem:
if Ref.Is_Empty then
return To_Atom (Fallback);
else
return Ref.Query.Data.all;
end if;
In case anyone is interested, the full code is published there
https://github.com/faelys/natools/blob/trunk/src/natools-s_expressions-templates-generic_integers.adb#L291
Now a secondary question would be, how long is it reasonable to let GNAT
run before assuming it won't terminate ever?
My 41h here was probably extremely conservative...
Thank you all for your help,
Natasha
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: GNAT stuck, any idea on how to diagnose it?
2014-09-14 21:47 ` Natasha Kerensikova
@ 2014-09-14 22:56 ` Anh Vo
2014-09-15 7:15 ` Dmitry A. Kazakov
0 siblings, 1 reply; 13+ messages in thread
From: Anh Vo @ 2014-09-14 22:56 UTC (permalink / raw)
On Sunday, September 14, 2014 2:47:47 PM UTC-7, Natasha Kerensikova wrote:
> Hello,
>
> On 2014-09-14, Brad Moore <brad.moore@shaw.ca> wrote:
> > On 2014-09-14 8:49 AM, Dmitry A. Kazakov wrote:
> >> On Sun, 14 Sep 2014 09:21:18 +0000 (UTC), Natasha Kerensikova wrote:
>
> Now a secondary question would be, how long is it reasonable to let GNAT
> run before assuming it won't terminate ever?
It depends on size of the code in question. For normal case if gnat1 occupies close to 99% CPU in 30 seconds or more, that means the compiler runs away. I would submit a bug report.
A. Vo
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: GNAT stuck, any idea on how to diagnose it?
2014-09-14 22:56 ` Anh Vo
@ 2014-09-15 7:15 ` Dmitry A. Kazakov
0 siblings, 0 replies; 13+ messages in thread
From: Dmitry A. Kazakov @ 2014-09-15 7:15 UTC (permalink / raw)
On Sun, 14 Sep 2014 15:56:35 -0700 (PDT), Anh Vo wrote:
> On Sunday, September 14, 2014 2:47:47 PM UTC-7, Natasha Kerensikova wrote:
>> Hello,
>>
>> On 2014-09-14, Brad Moore <brad.moore@shaw.ca> wrote:
>>> On 2014-09-14 8:49 AM, Dmitry A. Kazakov wrote:
>>>> On Sun, 14 Sep 2014 09:21:18 +0000 (UTC), Natasha Kerensikova wrote:
>>
>> Now a secondary question would be, how long is it reasonable to let GNAT
>> run before assuming it won't terminate ever?
>
> It depends on size of the code in question. For normal case if gnat1
> occupies close to 99% CPU in 30 seconds or more, that means the compiler
> runs away. I would submit a bug report.
I would say 1 hour for the compiler and 1/2 for the linker (which could be
very slow too).
--
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: GNAT stuck, any idea on how to diagnose it?
2014-09-14 14:49 ` Dmitry A. Kazakov
2014-09-14 15:21 ` Brad Moore
@ 2014-09-14 21:45 ` anon
2014-09-15 7:25 ` Dmitry A. Kazakov
2014-09-15 7:16 ` Natasha Kerensikova
2 siblings, 1 reply; 13+ messages in thread
From: anon @ 2014-09-14 21:45 UTC (permalink / raw)
GNAT is a command shell for Adacore's Ada compiler. And once it
determines what action to be done, GNAT runs a number of programs.
At first GNAT may call the GNAT1 (Ada compiler). This program converts
Ada to assembly language, if the source code is correct, else reports
errors.
"gnat compile <compiler options> <source filename>.ad[b|s]"
or
"gnat make <compiler options> <source filename>.ad[b|s]"
preforms
GNAT --> "gcc <compiler options> <source filename>.ad[b|s]"
gcc --> "gnat1 <compiler options> <source filename>.ad[b|s]"
Second, if GNAT1 returns a no errors condition then GNAT calls the gcc
assembler "as" (gcc assembler)
GNAT --> "as <assembly options> <source filename>"
You can bypass GNAT and the "gcc" steps, by using
gnat1 <compiler options> <source filename>.ad[b|s]
as <assembly options> <source filename>.s
Also, GNAT BIND works almost the same way as GNAT COMPILE except
BIND creates either a Ada package (default) or a C binding package
that can be linked to the runtime code. Note: The binder creates
a package which starts with "b~" added to the source filename.
GNAT --> "gnatbind <binder options> <source filename>.ali"
GNAT --> "gcc <binder options> <b~source filename>.adb"
gcc --> "gnat1 <binder options> <b~source filename>.adb"
GNAT --> "as <assembly options> <b~source filename>.s"
bypassing GNAT would be:
gnatbind <bind options> <source filename>.ali
gnat1 <bind options> options> <b~source filename>.adb
as <assembly options> <b~source filename>.s
But as for time GNAT does not take that much time. The part that takes
the longest is the gcc backend which GNAT uses to convert it internal
gcc IR language to assembly. Mostly it is the optimization phase
in the gcc backend that requires more time ( options -O2 (default),
-O3, -O4 ).
now the process that GNAT1 uses is
GNAT1 :== Ada source --> AST --> GCC IR
at this movement the GCC backend takes over and processes the GCC IR
into assembly
gcc backend :== GCC IR --> ??? --> assembly code
For speed, GNAT1 loads the entire code into memory at one time. Then
basically processes the source as a large data array, while others
compilers may read a line of code at a time (reduced speed). With all
of the checks that the Ada requires in compiling, GNAT1 is very
efficiency during the syntactic and semantic phases. Now the slow
down starts to occur in the backend converter "gigi" which is written
in C. The "gigi" uses the GCC backend which is efficiency for C/C++ but
is not as efficiency for other languages like Ada, Fortran, etc.
One explanation is in translating Ada into the "GCC IR" code the
"gigi" may use or alter the code that makes it difficult for the
GCC optimization to process.
Which means that Adacore's GNAT by now (20 years) should have its own
backend written in Ada, specifically for Ada. That way the world could
see the true speed of the GNAT compiler.
In <vlxfpcmsetjg$.1em03ijtae3i6$.dlg@40tude.net>, "Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> writes:
>On Sun, 14 Sep 2014 09:21:18 +0000 (UTC), Natasha Kerensikova wrote:
>
> Would you know what is this `gnat1`?
> What part of the build process is it?
> What is the command above supposed to perform?
> Is it known to occasionally take a lot of time?
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: GNAT stuck, any idea on how to diagnose it?
2014-09-14 21:45 ` anon
@ 2014-09-15 7:25 ` Dmitry A. Kazakov
2014-09-15 9:56 ` anon
0 siblings, 1 reply; 13+ messages in thread
From: Dmitry A. Kazakov @ 2014-09-15 7:25 UTC (permalink / raw)
On Sun, 14 Sep 2014 21:45:16 +0000 (UTC), anon@att.net wrote:
> One explanation is in translating Ada into the "GCC IR" code the
> "gigi" may use or alter the code that makes it difficult for the
> GCC optimization to process.
Interesting.
My impression based on how I dealt with the problem was that it was not the
compiler or the code generator which grabbed memory. Maybe it was the
symbols cross reference and other stuff that helped generating error
messages and prompts. For example, making a separate body or splitting off
a child package does not help. I seems that it must be a distance of two or
more packages apart from the offending body. So my wild guess was that GNAT
grabbed all symbols from all packages at the distance 1, even from the
invisible ones and their bodies or ones referenced from the bodies to make
suggestions on an error, but did not do that for the packages at the
distance 2.
--
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: GNAT stuck, any idea on how to diagnose it?
2014-09-15 7:25 ` Dmitry A. Kazakov
@ 2014-09-15 9:56 ` anon
0 siblings, 0 replies; 13+ messages in thread
From: anon @ 2014-09-15 9:56 UTC (permalink / raw)
To stop the call to gigi and the GCC backend. One can use the debug
option "-gnatdH" the "dH" must be lower case 'd' and uppercase 'H'.
Even though this option can be used on "gnat1" or "gnat compile" or
even "gnat make". It should be limited to "gnat1" or "gnat compile".
The compiler will generate a <source file>.ali and <source file>.s.
The <source file>.s will only contain a 3 line header for a gcc assembly
file.
Now all of GNAT debug options can be found in compiler source package
"debug" and a very simple explanation of each option in the file
"debug.adb".
Also, for those options that generate text output. Some of those
can generate 100s of lines, even for a simple hello world program.
An example is -gnatdt" outputs the a detailed AST listing.
Note: Some of the text debug option will also conflict with other
text debug options. So, you need to start slowly (one option
at a time) until you learn which options you need and which
option conflict with others.
In <mmnjpohtp615.1d24bw7p0fns8.dlg@40tude.net>, "Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> writes:
>On Sun, 14 Sep 2014 21:45:16 +0000 (UTC), anon@att.net wrote:
>
>> One explanation is in translating Ada into the "GCC IR" code the
>> "gigi" may use or alter the code that makes it difficult for the
>> GCC optimization to process.
>
>Interesting.
>
>My impression based on how I dealt with the problem was that it was not the
>compiler or the code generator which grabbed memory. Maybe it was the
>symbols cross reference and other stuff that helped generating error
>messages and prompts. For example, making a separate body or splitting off
>a child package does not help. I seems that it must be a distance of two or
>more packages apart from the offending body. So my wild guess was that GNAT
>grabbed all symbols from all packages at the distance 1, even from the
>invisible ones and their bodies or ones referenced from the bodies to make
>suggestions on an error, but did not do that for the packages at the
>distance 2.
>
>--
>Regards,
>Dmitry A. Kazakov
>http://www.dmitry-kazakov.de
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: GNAT stuck, any idea on how to diagnose it?
2014-09-14 14:49 ` Dmitry A. Kazakov
2014-09-14 15:21 ` Brad Moore
2014-09-14 21:45 ` anon
@ 2014-09-15 7:16 ` Natasha Kerensikova
2014-09-15 7:26 ` Jeffrey Carter
2 siblings, 1 reply; 13+ messages in thread
From: Natasha Kerensikova @ 2014-09-15 7:16 UTC (permalink / raw)
Hello,
On 2014-09-14, Dmitry A. Kazakov <mailbox@dmitry-kazakov.de> wrote:
> On Sun, 14 Sep 2014 09:21:18 +0000 (UTC), Natasha Kerensikova wrote:
>
>> Would you know what is this `gnat1`?
>> What part of the build process is it?
>> What is the command above supposed to perform?
>> Is it known to occasionally take a lot of time?
>
> Yes. It may take relatively much time on its own, but from my experience, I
> can tell that GNAT consumes extremely and unreasonably much memory.
>
> At some point it stops compiling under 32-bit systems because there is not
> enough virtual memory.
>
> You could check if your problem is caused by exhaustion of the physical
> memory and turning to the swap. If swap gets involved it will never end.
As I wrote in the OP, the memory used by gnat1 process, as reported by
the operating system, was constant. Actual numbers depend on what is
measured, as is often the case in modern OS, but all of them where
constant, from 72 MB of physical memory to about 140 MB of addressable
space.
From there, can we deduce that 100% CPU usage with constant memory usage
means the compiler is stuck?
Or are there parts of the compiling process that can legitimately a
significant amount of time without using extra memory?
Thanks for you comments,
Natasha
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: GNAT stuck, any idea on how to diagnose it?
2014-09-14 9:21 GNAT stuck, any idea on how to diagnose it? Natasha Kerensikova
2014-09-14 11:59 ` Georg Bauhaus
2014-09-14 14:49 ` Dmitry A. Kazakov
@ 2014-09-14 17:45 ` Robert A Duff
2 siblings, 0 replies; 13+ messages in thread
From: Robert A Duff @ 2014-09-14 17:45 UTC (permalink / raw)
Natasha Kerensikova <lithiumcat@instinctive.eu> writes:
> Since I'm accumulating stuff, and test_all is accessing everything, it's
> getting larger and larger, and takes more and more time to build. So far
> so good.
Try using the -gnatdO switch. That's a capital Oh at the end, not zero.
It's documented in debug.adb.
- Bob
^ permalink raw reply [flat|nested] 13+ messages in thread
end of thread, other threads:[~2014-09-15 9:56 UTC | newest]
Thread overview: 13+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2014-09-14 9:21 GNAT stuck, any idea on how to diagnose it? Natasha Kerensikova
2014-09-14 11:59 ` Georg Bauhaus
2014-09-14 14:49 ` Dmitry A. Kazakov
2014-09-14 15:21 ` Brad Moore
2014-09-14 21:47 ` Natasha Kerensikova
2014-09-14 22:56 ` Anh Vo
2014-09-15 7:15 ` Dmitry A. Kazakov
2014-09-14 21:45 ` anon
2014-09-15 7:25 ` Dmitry A. Kazakov
2014-09-15 9:56 ` anon
2014-09-15 7:16 ` Natasha Kerensikova
2014-09-15 7:26 ` Jeffrey Carter
2014-09-14 17:45 ` Robert A Duff
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox