GNAT GPL 2017 incremental compilation

GNAT GPL 2017 incremental compilation

[gautier_niouzes]

Hello,
On some projects, GNAT GPL 2017 (for Win32 at least) seems to forget about incremental compilation and recompiles everything every time, even if no source has changed. This happens with gnatmake or gprbuild indifferently.
Did anyone also notice it?
Is there some configuration parameter to fix this issue?
Otherwise I'll try to make a small reproducer.
_________________________ 
Gautier's Ada programming 
http://sf.net/users/gdemont/

Re: GNAT GPL 2017 incremental compilation

[Dmitry A. Kazakov]

On 2017-11-22 22:13, gautier_niouzes@hotmail.com wrote:

> On some projects, GNAT GPL 2017 (for Win32 at least) seems to forget 
> about incremental compilation and recompiles everything every time, 
> even if no source has changed. This happens with gnatmake or gprbuild
> indifferently. Did anyone also notice it?
Never in my case, unless network gets involved. I presume it is time 
stamping problem. Windows files have three of them. For me when files 
are checked out of the source control they get recompiled.

> Is there some configuration parameter to fix this issue?

I know that if all files are touched they do not recompile.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: GNAT GPL 2017 incremental compilation

[Simon Wright]

gautier_niouzes@hotmail.com writes:

> On some projects, GNAT GPL 2017 (for Win32 at least) seems to forget
> about incremental compilation and recompiles everything every time,
> even if no source has changed. This happens with gnatmake or gprbuild
> indifferently.

With GNAT 3.16a1 there was a problem with timestamp discrepancies
between the RTS ALI files and newly generated code; the effect was that
if you installed the compiler in winter, it would always recompile
everything.

The workround was, if installing in winter, you had to set the
computer's clock to a month that was in DST, reinstall the compiler, set
the clock back.

Anyway, gprbuild -vm (or -vh, not sure what the difference is supposed
to be) will show the reason gprbuild has decided to recompile a file:
e.g.

   Checking /Users/simon/Analytical_Engine/src/aes.adb ... 
   -> different time stamp for analytical_engine-store.ads

It is quite verbose!

Re: GNAT GPL 2017 incremental compilation

[gautier_niouzes]

Could be a time stamp issue indeed. However I'd expect some randomness to when GNAT wrongly recompiles an up-to-date file. Here it is as if the tool decided a "-f" (full recompile). Also editing or touch-ing some sources doesn't help: all subsequent builds will recompile all files.
So far the smallest reproducer is the following project:
  https://sourceforge.net/projects/apdf/
Works with both svn checkout and by unpacking the apdf_002.zip Zip archive.

Re: GNAT GPL 2017 incremental compilation

[Simon Wright]

gautier_niouzes@hotmail.com writes:

> Could be a time stamp issue indeed. However I'd expect some randomness
> to when GNAT wrongly recompiles an up-to-date file. Here it is as if
> the tool decided a "-f" (full recompile). Also editing or touch-ing
> some sources doesn't help: all subsequent builds will recompile all
> files.
> So far the smallest reproducer is the following project:
>   https://sourceforge.net/projects/apdf/
> Works with both svn checkout and by unpacking the apdf_002.zip Zip archive.

-vm says

   Checking /Users/simon/tmp/apdf/demo/pdf_out_demo.adb ... 
  -> "debug.pra" missing

because in Builder with Build_Mode=Default

   "-gnatecdebug.pra"

  

Re: GNAT GPL 2017 incremental compilation

[Dmitry A. Kazakov]

On 22/11/2017 23:45, Simon Wright wrote:
> gautier_niouzes@hotmail.com writes:
> 
>> On some projects, GNAT GPL 2017 (for Win32 at least) seems to forget
>> about incremental compilation and recompiles everything every time,
>> even if no source has changed. This happens with gnatmake or gprbuild
>> indifferently.
> 
> With GNAT 3.16a1 there was a problem with timestamp discrepancies
> between the RTS ALI files and newly generated code; the effect was that
> if you installed the compiler in winter, it would always recompile
> everything.
> 
> The workround was, if installing in winter, you had to set the
> computer's clock to a month that was in DST, reinstall the compiler, set
> the clock back.
> 
> Anyway, gprbuild -vm (or -vh, not sure what the difference is supposed
> to be) will show the reason gprbuild has decided to recompile a file:
> e.g.
> 
>     Checking /Users/simon/Analytical_Engine/src/aes.adb ...
>     -> different time stamp for analytical_engine-store.ads
> 
> It is quite verbose!

Another issue was/is with object files. I had a source control system 
settings which automatically reverted unchanged files. When compiling 
the same source the ALI file turns always different while the object 
file stays the same. Now, the object file gets reverted and ALI gets 
checked in. Here you are, they will have different time stamps ever!

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: GNAT GPL 2017 incremental compilation

[Simon Wright]

"Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> writes:

> Another issue was/is with object files. I had a source control system
> settings which automatically reverted unchanged files. When compiling
> the same source the ALI file turns always different while the object
> file stays the same. Now, the object file gets reverted and ALI gets
> checked in. Here you are, they will have different time stamps ever!

If it was a _source_ control system, why were object and ALI files being
checked in?

Re: GNAT GPL 2017 incremental compilation

[Dmitry A. Kazakov]

On 23/11/2017 10:11, Simon Wright wrote:
> "Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> writes:
> 
>> Another issue was/is with object files. I had a source control system
>> settings which automatically reverted unchanged files. When compiling
>> the same source the ALI file turns always different while the object
>> file stays the same. Now, the object file gets reverted and ALI gets
>> checked in. Here you are, they will have different time stamps ever!
> 
> If it was a _source_ control system, why were object and ALI files being
> checked in?

Where would you keep them otherwise?

(If you don't keep them you compile always everything)

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: GNAT GPL 2017 incremental compilation

[Jeffrey R. Carter]

On 11/23/2017 12:09 AM, gautier_niouzes@hotmail.com wrote:
> Could be a time stamp issue indeed. However I'd expect some randomness to when GNAT wrongly recompiles an up-to-date file. Here it is as if the tool decided a "-f" (full recompile). Also editing or touch-ing some sources doesn't help: all subsequent builds will recompile all files.
> So far the smallest reproducer is the following project:
>    https://sourceforge.net/projects/apdf/
> Works with both svn checkout and by unpacking the apdf_002.zip Zip archive.

Do you use the -m option?

-- 
Jeff Carter
"IMHO, Interfaces are worthless."
Randy Brukardt
117

Re: GNAT GPL 2017 incremental compilation

[gautier_niouzes]

> Where would you keep them otherwise?
> (If you don't keep them you compile always everything)

You just keep them local, invisible to the source control system.
In the local copy, the full compilation will occur only on first checkout - on a system with functioning incremental compilation.

Products of compilation (.ali, .o, .exe) should not appear in the source control system, otherwise it causes a mess at latest when there are more than one local copy (and one or more developers). If the development is done with different GNAT versions or on different operating systems, the mess becomes a big mess!...

Re: GNAT GPL 2017 incremental compilation

[Dmitry A. Kazakov]

On 23/11/2017 12:05, gautier_niouzes@hotmail.com wrote:
>> Where would you keep them otherwise?
>> (If you don't keep them you compile always everything)
> 
> You just keep them local, invisible to the source control system.

If I wanted to recompile always everything that would be the way. The 
point is to avoid recompilation of projects that were built once, no 
matter when or where.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: GNAT GPL 2017 incremental compilation

[Jacob Sparre Andersen]

Dmitry A. Kazakov wrote:
> On 23/11/2017 12:05, gautier_niouzes@hotmail.com wrote:

>>> Where would you keep them otherwise?
>>> (If you don't keep them you compile always everything)
>>
>> You just keep them local, invisible to the source control system.
>
> If I wanted to recompile always everything that would be the way. The
> point is to avoid recompilation of projects that were built once, no
> matter when or where.

Do you have ALI, Object and Exec directories for each compiler version
and target?

Or are you lucky enough only to have to worry about a single compiler
version and a single target?

Greetings,

Jacob
-- 
»What fun is it being "cool" if you can't wear a sombrero?«

Re: GNAT GPL 2017 incremental compilation

[Dmitry A. Kazakov]

On 23/11/2017 13:16, Jacob Sparre Andersen wrote:
> Dmitry A. Kazakov wrote:
>> On 23/11/2017 12:05, gautier_niouzes@hotmail.com wrote:
> 
>>>> Where would you keep them otherwise?
>>>> (If you don't keep them you compile always everything)
>>>
>>> You just keep them local, invisible to the source control system.
>>
>> If I wanted to recompile always everything that would be the way. The
>> point is to avoid recompilation of projects that were built once, no
>> matter when or where.
> 
> Do you have ALI, Object and Exec directories for each compiler version
> and target?

Yes, of course.

BTW, ALI files are duplicated. They are needed in one place in order to 
build a library and in another to use that library.

> Or are you lucky enough only to have to worry about a single compiler
> version and a single target?

I have all together 10 targets x86|armhf|armeabi x Windows|VxWorks|Linux 
x Debug|Release and 5 different GNAT compilers, two of them cross.

This is why recompilation is not an option to me. It takes 1 week time 
to rebuild everything (running on 3 separate hosts in parallel with -j0 on).

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: GNAT GPL 2017 incremental compilation

[Simon Wright]

"Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> writes:

> BTW, ALI files are duplicated. They are needed in one place in order
> to build a library and in another to use that library.

Only for a particular RTS/compiler pair?

Re: GNAT GPL 2017 incremental compilation

[Dmitry A. Kazakov]

On 23/11/2017 14:24, Simon Wright wrote:
> "Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> writes:
> 
>> BTW, ALI files are duplicated. They are needed in one place in order
>> to build a library and in another to use that library.
> 
> Only for a particular RTS/compiler pair?

For each target: platform x debug|release.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: GNAT GPL 2017 incremental compilation

[gautier_niouzes]

Le jeudi 23 novembre 2017 09:35:41 UTC+1, Simon Wright a écrit :

> -vm says
> 
>    Checking /Users/simon/tmp/apdf/demo/pdf_out_demo.adb ... 
>   -> "debug.pra" missing
> 
> because in Builder with Build_Mode=Default
> 
>    "-gnatecdebug.pra"

Thanks for spotting that.
The (more or less) funny thing is that the compiler finds the file, but not gprbuild (in the GPL 2017 version)...

Re: GNAT GPL 2017 incremental compilation

[Simon Wright]

gautier_niouzes@hotmail.com writes:

> Le jeudi 23 novembre 2017 09:35:41 UTC+1, Simon Wright a écrit :
>
>> -vm says
>> 
>>    Checking /Users/simon/tmp/apdf/demo/pdf_out_demo.adb ... 
>>   -> "debug.pra" missing
>> 
>> because in Builder with Build_Mode=Default
>> 
>>    "-gnatecdebug.pra"
>
> Thanks for spotting that.
> The (more or less) funny thing is that the compiler finds the file,
> but not gprbuild (in the GPL 2017 version)...

I think this is because gprbuild looks for files relative to itself, but
the compiler and linker look for files relative to the object directory.

This works (for me):

   "-gnatec=" & project'Project_Dir & project'Object_Dir & "/debug.pra"

though I'd move the file up a level, being in the habit of just
deleting the entire object directory, and say

   "-gnatec=" & project'Project_Dir & "debug.pra"

Re: GNAT GPL 2017 incremental compilation

[gautier_niouzes]

Le jeudi 23 novembre 2017 23:44:11 UTC+1, Simon Wright a écrit :

> This works (for me):
> 
>    "-gnatec=" & project'Project_Dir & project'Object_Dir & "/debug.pra"
> 
> though I'd move the file up a level, being in the habit of just
> deleting the entire object directory, and say
> 
>    "-gnatec=" & project'Project_Dir & "debug.pra"

Fantastic, it works - and yes, the variant is better. The only reason the configuration pragma file was put into the object directory was that it was the place where the compiler was looking for it, and I didn't know of this "project'Project_Dir"...
Thanks Simon!