Re: GNAT Modification_Time limitation

["Randy Brukardt" <randy@rrsoftware.com>]

<briot.emmanuel@gmail.com> wrote in message 
news:62ffa1fb-6733-4f97-ba87-ae3103bfc877@googlegroups.com...
>> I wouldn't claim that the situation is that dire; it seems to be related 
>> to
>> the particular implementation of a particular GNAT feature (project 
>> scenario
>> variables). If you're not implementing something where the source code
>> location can be changed for a particular build, then timestamps will work
>> (but you have to remember that they are quite granular).
>
> The trick of course is to define what a "build" is in your sentence.
> If it is one execution of the builder (gprbuild, make,...) then I think it 
> is indeed
> a reasonable assertion.
>
> If however a build is defined to something that amount to "in debug mode, 
> in production mode,..."
> then of course it might happen that the sources are changed and the 
> timestamp have a timestamp
> delta of less than 1s (when we generate code for instance).

I'd argue that these are something else on top of individual builds. And 
that it is a mistake trying to combine basic building with those 
higher-level configuration management things.

I've struggled with those higher level issues almost since the beginning of 
RR Software (we've almost always supported multiple targets for Janus/Ada). 
Neither conventional build tools nor configuration management tools are any 
help whatsoever for managing those situations. I've seen various attempts to 
do so, but none of them address the underlying issues very well.

> Furthermore, the actual scenario was the following: in the automatic 
> tests, I need to simulate the
> connection to the database, so that means I need to have support for 
> alternate bodies (but I still
> compile in debug mode, or production mode,...). Is that still the same 
> "build" ?

No, at least four separate builds.

> I would guess it is, but in the end we would end up with literally dozens 
> of "build" types, each with
> its own set of object files, and each taking 20 or 30 minutes to build 
> from scratch. Not realistic
> for continuous testing.

You need a faster compiler. :-) :-)

Seriously, at least debug vs. production has to be built over from scratch 
(at least the way I typically use those). The debug version uses different 
compiler options and the production version, as debug symbols need to be 
generated, some optimizations need to be turned off, and then the production 
version turns off various Ada checking. So to switch from one to the other 
requires a full rebuild anyway.

In recent years, I've avoided that problem by keeping multiple projects for 
debug and production and various targets, and thus (re)building each 
individually as needed. Disk space is plentiful on modern machines -- it's 
my time that's limited.

> I spent some time looking around at general builder tools around. Most of 
> them seem to
> advertise nowadays that they look at file contents, not timestamps. I 
> started from the list
> at https://en.wikipedia.org/wiki/List_of_build_automation_software, and 
> looked at a few of them.

For source code, I tend to agree. The Janus/Ada COrder tool always had an 
option to read the source files instead of depending on timestamps. And 
Janus/Ada puts the timestamps into the compilation results, so that they 
can't be clobbered by file operations. In any case, source code is only part 
of the picture.

>> It also seems to be related in part of source-based compilation (which
>> necessarily keeps less information between builds). In a Janus/Ada 
>> project
>> (which is very different than a GNAT project -- it's a binary DB-like 
>> file
>> of compilation information), changing the location of a source file would
>> invalidate the entire entry and essentially delete any existing
>> compilations. More likely, however, is that a scenario would be set up 
>> using
>> separate project files (most likely using Windows batch files/Unix
>> shell-scripts to automate), so each would have their own set of 
>> compilation
>> states.
>
> That's more or less what gprbuild does in practice. It uses a "distributed 
> database"
> via the .ALI files, which are found in the object directories, so for best 
> use each
> "build" should have a different object directories. And we are again 
> hitting the notion
> of "build".

Precisely. Higher-level things than raw builds are best kept separate at the 
compilation artifact level.

>> And it's completely impossible to bind multiple versions of a unit
>> into a single executable; only one or the other could be selected -
>
> That's indeed one of the ways gprbuild could detect the error. To me it is 
> a bug in
> gprbuild that it allows linking different files for the same unit into the 
> same executable.

I believe that is a result of the way GNAT compiles files -- the package 
specifications are never materialized, so it would be hard for it to have 
any compilation result which could tell which one is used. I've seen this 
sort of effect working on ACATS tests, and I've never had any reason to use 
GPRBuild for that.

>> somehow some files were compiled against the wrong one, some or all of 
>> the
>> compilation timestamps wouldn't match (which would cause binding 
>> failure).
>
> timestamps are not reliable enough, especially on modern fast machines. I 
> am pretty
> sure you will hit a similar issue I had, one day.

I'm sorry, I confused you here. I was talking about the timestamps that 
Janus/Ada records for compilation units when they are compiled. These are 
internal to the SYM files (which are a representation of the Ada symboltable 
for a library unit), and used to determine which version is "with"ed in 
other files. They're only compared for equality (other than for the purposes 
of error messages). Even on a FAT system, these have 2 second granularity. 
You could only have a problem if the same specification is recompiled twice 
in under 2 seconds.

It's hard to imagine a build taking less than two seconds; certainly not if 
a human is involved, and very unlikely even if automated. (The Janus/Ada 
binder is fairly slow as it removes unreachable subprograms recursively --  
that's required for Windows programs because the presence of binding for a 
variety of Windows versions -- and as such it takes multiple seconds for all 
but the most trivial programs.) On more modern systems, we're talking 
hundredths of seconds granularity; it's essentially impossible for multiple 
builds to happen that fast.

COrder (the Janus/Ada compilation order tool that's at the heart of any 
builds) has an old /T option that uses file timestamps, but it has not been 
recommended for a while. It's faster than the /I option that inspects the 
internal timestamps and the source code ('cause it doesn't have to open 
hundreds of files and read part of them), but it messes up so often it is 
not recommended anymore. (One nice side-effect of /I is that one can simply 
delete all of the SYM files to force a rebuild of everything; /T doesn't 
always rebuild everything in that case.)

In any case, timestamps have their place, but they have to be used 
carefully.

                         Randy.