* How to cache output of the compiler aka ccache @ 2005-03-17 18:54 i-google-iasuhdkajsh 2005-03-17 19:40 ` Robert A Duff ` (4 more replies) 0 siblings, 5 replies; 22+ messages in thread From: i-google-iasuhdkajsh @ 2005-03-17 18:54 UTC (permalink / raw) Hi I have a 1mio LOC project and it takes multiple hours to build and rebuild. The C/C++ world has a nice tools (ccache is well known here) which cache the output of the compiler. This especially speeds up the time for a "make clean;make". Is there such a thing for Ada in general and gcc's gnat specially? Any other way to speed up the compilation? Thomas ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: How to cache output of the compiler aka ccache 2005-03-17 18:54 How to cache output of the compiler aka ccache i-google-iasuhdkajsh @ 2005-03-17 19:40 ` Robert A Duff 2005-03-17 20:04 ` Ludovic Brenta 2005-03-17 19:47 ` Ludovic Brenta ` (3 subsequent siblings) 4 siblings, 1 reply; 22+ messages in thread From: Robert A Duff @ 2005-03-17 19:40 UTC (permalink / raw) i-google-iasuhdkajsh@rf.risimo.net writes: > Hi > > I have a 1mio LOC project and it takes multiple hours to > build and rebuild. The C/C++ world has a nice tools (ccache > is well known here) which cache the output of the compiler. > This especially speeds up the time for a "make clean;make". > > Is there such a thing for Ada in general and gcc's gnat > specially? Any other way to speed up the compilation? Well, I had never heard of ccache before, so I just went and read about it. I am rather mystified by the *point* of it. I mean, if I don't want to recompile stuff that hasn't changed, I just don't type "make clean". If for some reason I want to rebuild everything from scratch, I type "make clean" (or rm -rf build_area or whatever) -- but then ccache *defeats* that, and does *not* rebuild everything from scratch. I don't see the point of that. It seems like a contradiction -- I want to rebuild everything from scratch while avoiding rebuilding everything! The only thing I can think of is that if your makefile is buggy (missing dependences) it won't rebuild some things that depend on some .h files, and you'll get mysterious bugs. So rebuilding from scratch is more reliable. And ccache actually decides whether to run the compiler *reliably* (whereas 'make' does not). Is *that* why people use ccache? Don't people automatically generate the make-file dependences for C code these days? It seems crazy to try to keep the make file in synch with the #includes by hand! Anyway, the buggy make file problem does not happen with Ada. All Ada compilers come with a build tool that reliably decides what needs to be recompiled, and does it. In AdaMagic, it's called adabuild. In GNAT it's called gnatmake. Also, if you try to link an Ada program where the parts are inconsistently compiled you will get an error at link time. You never need to invoke the compiler directly -- just use the build tool. If you're writing a make file for an Ada program, you should *not* put in dependencies for the Ada code! Instead, use a .PHONY rule to unconditionally invoke the build tool. Don't type "make clean" (except perhaps in your nightly test script, where you don't have to wait for it), but trust the builder to decide what needs recompiling and to keep things consistent. You don't even need to use 'make' with Ada at all, if all you want to do is rebuild. But it's convenient to have make-file targets like "rebuild if necessary and run the regression tests". - Bob ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: How to cache output of the compiler aka ccache 2005-03-17 19:40 ` Robert A Duff @ 2005-03-17 20:04 ` Ludovic Brenta 2005-03-17 21:27 ` Marius Amado Alves 0 siblings, 1 reply; 22+ messages in thread From: Ludovic Brenta @ 2005-03-17 20:04 UTC (permalink / raw) Yes, Ada defines separate compilation cleanly. C does not, and Makefiles were created to make up for it. Makefiles can become very complex in the presence of preprocessors or rules that build a program that generates code which is compiled later on (this happens e.g. in the GCC bootstrap process). It takes a lot of care to get these Makefiles right, and even then they tend to be very brittle. ccache was created to make up for *that*. The Ada model is indeed clean, simple and reliable. Sometimes, it is necessary to supplement it to provide preprocessing facilities, or to select the units to be compiled from a CM system. Simple scripts or minimalistic makefiles can do that. -- Ludovic Brenta. ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: How to cache output of the compiler aka ccache 2005-03-17 20:04 ` Ludovic Brenta @ 2005-03-17 21:27 ` Marius Amado Alves 2005-03-18 1:24 ` Ludovic Brenta 0 siblings, 1 reply; 22+ messages in thread From: Marius Amado Alves @ 2005-03-17 21:27 UTC (permalink / raw) To: comp.lang.ada > The Ada model is indeed clean, simple and reliable. Sometimes, it is > necessary to supplement it to provide preprocessing facilities, or to > select the units to be compiled from a CM system. Simple scripts or > minimalistic makefiles can do that. Yeah. And nowadays makefiles is mostly used as an output language from Configure. Write-only makefiles. Completely unreadable and unmantainable. And in my experience, more often than not these makefiles don't work. For example, building Graphviz, or ZLib on a Mac OS X. I end up studying the macrostructure of the code (the C units, which ones are libraries, which ones are programs) and then compiling by hand, or writing a small, readable makefile by hand, and eventually using Libtool (and sometimes Ranlib, another idiotic tool). For me Configure was a waste of time. Its 72-page manual is in the bin. ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: How to cache output of the compiler aka ccache 2005-03-17 21:27 ` Marius Amado Alves @ 2005-03-18 1:24 ` Ludovic Brenta 2005-03-18 2:08 ` Robert A Duff 0 siblings, 1 reply; 22+ messages in thread From: Ludovic Brenta @ 2005-03-18 1:24 UTC (permalink / raw) Marius Amado Alves writes: >> The Ada model is indeed clean, simple and reliable. Sometimes, it >> is necessary to supplement it to provide preprocessing facilities, >> or to select the units to be compiled from a CM system. Simple >> scripts or minimalistic makefiles can do that. > > Yeah. And nowadays makefiles is mostly used as an output language > from Configure. Write-only makefiles. Completely unreadable and > unmantainable. And in my experience, more often than not these > makefiles don't work. For example, building Graphviz, or ZLib on a > Mac OS X. I end up studying the macrostructure of the code (the C > units, which ones are libraries, which ones are programs) and then > compiling by hand, or writing a small, readable makefile by hand, > and eventually using Libtool (and sometimes Ranlib, another idiotic > tool). For me Configure was a waste of time. Its 72-page manual is > in the bin. Hehe... C had separate compilation, but no dependency management. To overcome this limitation, Makefiles were created, but they were not portable enough. To overcome this limitation, configure scripts were created. At first simple, they had to take into account the idiosyncrasies of more and more host and target platforms and became unwieldy and fragile. To overcome this limitation, autoconf was created, but it still cannot possibly know everything about every host and target platform. To overcome this limitation, automake was created. Now, autoconf and automake work hand in hand, each using a mix of several cryptic languages. They have become so fiendlishly difficult to maintain that few people if any at all understand how they work. To overcome this limitation, ... Verily this is not the Tao of Programming :) -- Ludovic Brenta. ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: How to cache output of the compiler aka ccache 2005-03-18 1:24 ` Ludovic Brenta @ 2005-03-18 2:08 ` Robert A Duff 0 siblings, 0 replies; 22+ messages in thread From: Robert A Duff @ 2005-03-18 2:08 UTC (permalink / raw) Ludovic Brenta <ludovic.brenta@insalien.org> writes: > Hehe... C had separate compilation, but no dependency management. To > overcome this limitation, Makefiles were created, but they were not > portable enough. To overcome this limitation, configure scripts were > created. At first simple, they had to take into account the > idiosyncrasies of more and more host and target platforms and became > unwieldy and fragile. To overcome this limitation, autoconf was > created, but it still cannot possibly know everything about every host > and target platform. To overcome this limitation, automake was > created. > > Now, autoconf and automake work hand in hand, each using a mix of > several cryptic languages. They have become so fiendlishly difficult > to maintain that few people if any at all understand how they work. > To overcome this limitation, ... > > Verily this is not the Tao of Programming :) The strange thing is, it wouldn't be all that hard for a C compiler vendor to provide a tool for building C programs that would be just as reliable about maintaining consistency as an ada builder -- without having to type a bazillion (usually wrong) dependences into a make file. Seems a lot easier to deal with than automatically-generated make files. - Bob ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: How to cache output of the compiler aka ccache 2005-03-17 18:54 How to cache output of the compiler aka ccache i-google-iasuhdkajsh 2005-03-17 19:40 ` Robert A Duff @ 2005-03-17 19:47 ` Ludovic Brenta 2005-03-18 19:22 ` Simon Wright 2005-03-18 6:58 ` i-google-iasuhdkajsh ` (2 subsequent siblings) 4 siblings, 1 reply; 22+ messages in thread From: Ludovic Brenta @ 2005-03-17 19:47 UTC (permalink / raw) writes: > Hi > > I have a 1mio LOC project and it takes multiple hours to > build and rebuild. The C/C++ world has a nice tools (ccache > is well known here) which cache the output of the compiler. > This especially speeds up the time for a "make clean;make". > > Is there such a thing for Ada in general and gcc's gnat > specially? Any other way to speed up the compilation? ccache does not work with GNAT (I tried it). However, "make clean" is seldom necessary where I work, and "gnatmake -m" does minimal recompilation. This really speeds things up. -- Ludovic Brenta. ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: How to cache output of the compiler aka ccache 2005-03-17 19:47 ` Ludovic Brenta @ 2005-03-18 19:22 ` Simon Wright 2005-03-18 20:04 ` Randy Brukardt 0 siblings, 1 reply; 22+ messages in thread From: Simon Wright @ 2005-03-18 19:22 UTC (permalink / raw) Ludovic Brenta <ludovic.brenta@insalien.org> writes: > and "gnatmake -m" does minimal > recompilation. This really speeds things up. We absolutely rely on this; the handwritten code is in separate proper bodies, 99% of specs and package bodies are generated, and the only way to change them is to change the model and re-generate. One thing to watch out for is that when deciding whether a unit needs to be recompiled GNAT first checks the timestamp of each dependency; it the timestamp is different it checks the contents of the dependency and only recompiles if there's a semantic difference. So recreating lots of identical source will mean that the dependencies have to be at least parsed. This can take quite a time (I think mainly the overhead of opening/closing the files). We had an interesting bug with 3.16a1 on Windows where compilations ran slower if the compiler had been installed in the winter months (outside daylight saving time) -- a feature involving the Windows installer, I think; the timestamps in the library were all one hour out. -- Simon Wright 100% Ada, no bugs. ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: How to cache output of the compiler aka ccache 2005-03-18 19:22 ` Simon Wright @ 2005-03-18 20:04 ` Randy Brukardt 2005-03-18 22:10 ` Florian Weimer 2005-03-19 3:25 ` Larry Kilgallen 0 siblings, 2 replies; 22+ messages in thread From: Randy Brukardt @ 2005-03-18 20:04 UTC (permalink / raw) "Simon Wright" <simon@pushface.org> wrote in message news:x7vekeciwtj.fsf@smaug.pushface.org... ... > We had an interesting bug with 3.16a1 on Windows where compilations > ran slower if the compiler had been installed in the winter months > (outside daylight saving time) -- a feature involving the Windows > installer, I think; the timestamps in the library were all one hour > out. This is a bizarre Windows "feature". When daylight saving time changes, all of the time stamps on your existing files will change one hour. (If you don't believe me, check it out when the time changes.) If you have code the previously stored the time stamp somewhere (say in a dependency file), it will suddenly be off an hour, and thus will mismatch. Janus/Ada doesn't have the problem with the core runtime (since the source isn't supplied), but sometimes a daylight saving time change will force a rebuild of an entire project. I've never figured out a good solution, since the behavior is so odd. Randy. ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: How to cache output of the compiler aka ccache 2005-03-18 20:04 ` Randy Brukardt @ 2005-03-18 22:10 ` Florian Weimer 2005-03-19 3:25 ` Larry Kilgallen 1 sibling, 0 replies; 22+ messages in thread From: Florian Weimer @ 2005-03-18 22:10 UTC (permalink / raw) * Randy Brukardt: > Janus/Ada doesn't have the problem with the core runtime (since the source > isn't supplied), but sometimes a daylight saving time change will force a > rebuild of an entire project. I've never figured out a good solution, since > the behavior is so odd. If I read the specification of GetFileTime correctly, this API does indeed return values relative to UTC. Is this not the case? Ah, found it: | File Times and Daylight Saving Time | | You must take care using file times if the user has set the system to | automatically adjust for daylight saving time. | | To convert a file time to local time, use the FileTimeToLocalFileTime | function. However, FileTimeToLocalFileTime uses the current settings | for the time zone and daylight saving time. Therefore, if it is | daylight saving time, it will take daylight saving time into account, | even if the file time you are converting is in standard time. | | FAT records times on disk in local time. GetFileTime retrieves cached | UTC times from FAT. When it becomes daylight saving time, the time | retrieved by GetFileTime will be off an hour, because the cache has | not been updated. When you restart the machine, the cached time | retrieved by GetFileTime will be correct. FindFirstFile retrieves the | local time from FAT and converts it to UTC using the current settings | for the time zone and daylight saving time. Therefore, if it is | daylight saving time, FindFirstFile will take daylight saving time | into account, even if the file time you are converting is in standard | time. | | NTFS records times on disk in UTC. [...] <http://msdn.microsoft.com/library/en-us/sysinfo/base/file_times.asp> If there isn't any interface to read the file time as stored on FAT, you lose. Apparently, there isn't one. Great engineering. ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: How to cache output of the compiler aka ccache 2005-03-18 20:04 ` Randy Brukardt 2005-03-18 22:10 ` Florian Weimer @ 2005-03-19 3:25 ` Larry Kilgallen 1 sibling, 0 replies; 22+ messages in thread From: Larry Kilgallen @ 2005-03-19 3:25 UTC (permalink / raw) In article <pJqdnQxLIP40rqbfRVn-uw@megapath.net>, "Randy Brukardt" <randy@rrsoftware.com> writes: > "Simon Wright" <simon@pushface.org> wrote in message > news:x7vekeciwtj.fsf@smaug.pushface.org... > ... >> We had an interesting bug with 3.16a1 on Windows where compilations >> ran slower if the compiler had been installed in the winter months >> (outside daylight saving time) -- a feature involving the Windows >> installer, I think; the timestamps in the library were all one hour >> out. > > This is a bizarre Windows "feature". When daylight saving time changes, all > of the time stamps on your existing files will change one hour. (If you > don't believe me, check it out when the time changes.) > > If you have code the previously stored the time stamp somewhere (say in a > dependency file), it will suddenly be off an hour, and thus will mismatch. > > Janus/Ada doesn't have the problem with the core runtime (since the source > isn't supplied), but sometimes a daylight saving time change will force a > rebuild of an entire project. I've never figured out a good solution, since > the behavior is so odd. It seems to me that one should instruct Windows that one is in a place where Daylight Savings Time is not honored. To ask the computer to go switching times away from reality is asking for trouble. ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: How to cache output of the compiler aka ccache 2005-03-17 18:54 How to cache output of the compiler aka ccache i-google-iasuhdkajsh 2005-03-17 19:40 ` Robert A Duff 2005-03-17 19:47 ` Ludovic Brenta @ 2005-03-18 6:58 ` i-google-iasuhdkajsh 2005-03-18 13:33 ` Robert A Duff 2005-03-18 8:07 ` Martin Krischik 2005-03-18 18:50 ` i-google-iasuhdkajsh 4 siblings, 1 reply; 22+ messages in thread From: i-google-iasuhdkajsh @ 2005-03-18 6:58 UTC (permalink / raw) i-google-iasuhdkajsh@rf.risimo.net wrote in message news:<1111085641.211767.56950@f14g2000cwb.googlegroups.com>... > Hi > > I have a 1mio LOC project and it takes multiple hours to > build and rebuild. The C/C++ world has a nice tools (ccache > is well known here) which cache the output of the compiler. > This especially speeds up the time for a "make clean;make". > > Is there such a thing for Ada in general and gcc's gnat > specially? Any other way to speed up the compilation? Thanks for your input. I'm more a C/C++ guy than an Ada one. So please correct anything below. There are two main gains scenarios where a cache helps. 1) make clean;make You think that there are no reason for this. However I disagree. There are cases you need to rerun configure and afterwards the make clean;make idoim. Reasons for this could be: - you have changed the version/flags of tools you use - you have changed the configuration somehow (adding sound support for example) - poor/bad make files for non-Ada code In my project are multiple compilper generators and C/C++ and Java source code. So there is a configure and make clean;make is a good way to make sure you have a good build. 2) Sharing between trees I also want to give here two examples. The first is when a group of local developers share such a cache. If now an external CVS change comes in only one developer has to wait. In the best case you could do a background cache filling based on new CVS code with cron and co. The second case for sharing is multiple tress. Let me give you an example: $ cd tree.clean $ make $ cd ..;cp -a tree.clean tree.patched; cd tree.patched $ patch -p1 <.... $ make # nothing gained so far. Two days later however there # is an external CVS change. $ cd ../tree.clean $ cvs up;make # testing ... ok. Let's test the patched version $ cd ../tree.patched $ cvs up $ make # The last make should really be speed up. So you see there are more or less valid reasons for a compile cache. Thanks for the -m switch hint. I will try it. Thomas ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: How to cache output of the compiler aka ccache 2005-03-18 6:58 ` i-google-iasuhdkajsh @ 2005-03-18 13:33 ` Robert A Duff 2005-03-18 14:01 ` Vinzent 'Gadget' Hoefler 0 siblings, 1 reply; 22+ messages in thread From: Robert A Duff @ 2005-03-18 13:33 UTC (permalink / raw) i-google-iasuhdkajsh@rf.risimo.net writes: > There are two main gains scenarios where a cache helps. > > 1) make clean;make > You think that there are no reason for this. I don't think that. I just think it's rare. In my normal work, I don't use "make clean"; I can arrange for any necessary "make clean"s to happen at night, while I'm sleeping, and everything is rebuilt by the time I wake up. > However I disagree. There are cases you need to rerun > configure and afterwards the make clean;make idoim. Reasons > for this could be: > - you have changed the version/flags of tools you use ...in which case ccache will flush its cache and recompile everything, which is what you want! I guess the advantage here is that when you switch back to the previous version/flags, the stuff is still there in the cache, so the rebuild is quick. In Ada, I use separate build areas for separate versions/flags, which accomplishes the same thing "by hand". > - you have changed the configuration somehow (adding > sound support for example) OK. Seems similar to the version/flags case. > - poor/bad make files for non-Ada code OK; that's the case I speculated upon. > In my project are multiple compilper generators and > C/C++ and Java source code. So there is a configure and > make clean;make is a good way to make sure you have a > good build. Makes sense for C and C++ compilers, and various other tools. But I think Java compilers can be trusted to rebuild properly, just like Ada compilers. > 2) Sharing between trees That's a good point; I hadn't thought of that. I believe the Rational (IBM) Ada compiler supports sharing between trees. I suspect making a ccache-like tool for Ada would require some compiler-specific knowledge, because unlike C, the Ada compiler is required to store version information and prevent linking of version-skewed programs. > So you see there are more or less valid reasons for a compile cache. Yes, I agree. - Bob ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: How to cache output of the compiler aka ccache 2005-03-18 13:33 ` Robert A Duff @ 2005-03-18 14:01 ` Vinzent 'Gadget' Hoefler 2005-03-18 14:41 ` Florian Weimer 0 siblings, 1 reply; 22+ messages in thread From: Vinzent 'Gadget' Hoefler @ 2005-03-18 14:01 UTC (permalink / raw) Robert A Duff wrote: > But I think Java compilers can be trusted to rebuild > properly, just like Ada compilers. What makes you think that? |hovi@jellix:~/src/java/test> rm screencard/exceptions/CaffeineUnderflow.class |hovi@jellix:~/src/java/test> javac screencard/Test.java |hovi@jellix:~/src/java/test> java screencard/Test |Exception in thread "main" java.lang.NoClassDefFoundError: screencard/exceptions/CaffeineUnderflow | at screencard.Test.main(Test.java:42) |hovi@jellix:~/src/java/test> javac screencard/exceptions/CaffeineUnderflow.java |hovi@jellix:~/src/java/test> java screencard/Test |Target: 3015/192/1000 |[...] It obviously fails to recompile missing dependencies. From my tests it also seems to ignore file dates. So I'd say, the "Java Build System" is non-existent. Well, considering that Java is designed to be able can load classes at runtime (meaning: they are not required to have been compiled before they need to be loaded) this isn't a very big surprise. What's more surprising in this example: It fails to load an exception class which isn't even raised. Vinzent. -- worst case: The wrong assumption there actually is one. ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: How to cache output of the compiler aka ccache 2005-03-18 14:01 ` Vinzent 'Gadget' Hoefler @ 2005-03-18 14:41 ` Florian Weimer 2005-03-18 15:08 ` Vinzent 'Gadget' Hoefler 0 siblings, 1 reply; 22+ messages in thread From: Florian Weimer @ 2005-03-18 14:41 UTC (permalink / raw) * Vinzent Hoefler: > It obviously fails to recompile missing dependencies. From my tests > it also seems to ignore file dates. So I'd say, the "Java Build System" > is non-existent. It's not specified by the standard. Sun's implementation optionally recompiles out-of-date dependencies, though. ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: How to cache output of the compiler aka ccache 2005-03-18 14:41 ` Florian Weimer @ 2005-03-18 15:08 ` Vinzent 'Gadget' Hoefler 2005-03-18 16:24 ` Florian Weimer 0 siblings, 1 reply; 22+ messages in thread From: Vinzent 'Gadget' Hoefler @ 2005-03-18 15:08 UTC (permalink / raw) Florian Weimer wrote: > * Vinzent Hoefler: > >> It obviously fails to recompile missing dependencies. From my tests >> it also seems to ignore file dates. So I'd say, the "Java Build >> System" is non-existent. > > It's not specified by the standard. That's as good as "non-existent". :) > Sun's implementation optionally > recompiles out-of-date dependencies, though. Well, yes. I've tested some more, and from those tests it seems, Sun's "javac" recompiles dependencies if they are at most one level deep (or - to put it another way - direct dependencies of the file being compiled). It even does that recursively, so if I delete /all/ .class files it recompiles everything, of course. But if just *some* .class file somewhere in the middle of the hierarchy gets changed, it simply failes to detect that change. That's very different from what an Ada compiler is - by definition - required to do. Vinzent. -- worst case: The wrong assumption there actually is one. ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: How to cache output of the compiler aka ccache 2005-03-18 15:08 ` Vinzent 'Gadget' Hoefler @ 2005-03-18 16:24 ` Florian Weimer 2005-03-21 8:16 ` Vinzent 'Gadget' Hoefler 0 siblings, 1 reply; 22+ messages in thread From: Florian Weimer @ 2005-03-18 16:24 UTC (permalink / raw) * Vinzent Hoefler: > Florian Weimer wrote: > >> * Vinzent Hoefler: >> >>> It obviously fails to recompile missing dependencies. From my tests >>> it also seems to ignore file dates. So I'd say, the "Java Build >>> System" is non-existent. >> >> It's not specified by the standard. > > That's as good as "non-existent". :) Uh-oh, even with Ada, this is mostly a quality-of-implementation issue. Ada implementations only need to enforce consistency. There's no need to support build tools which recognize dependencies and automatically recompile them. It's also perfectly acceptable to always recompile all dependencies. >> Sun's implementation optionally >> recompiles out-of-date dependencies, though. > > Well, yes. I've tested some more, and from those tests it seems, Sun's > "javac" recompiles dependencies if they are at most one level deep (or > - to put it another way - direct dependencies of the file being > compiled). It even does that recursively, so if I delete /all/ .class > files it recompiles everything, of course. Oh, come on, have a closer look at the documentation. ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: How to cache output of the compiler aka ccache 2005-03-18 16:24 ` Florian Weimer @ 2005-03-21 8:16 ` Vinzent 'Gadget' Hoefler 2005-03-21 8:54 ` Florian Weimer 0 siblings, 1 reply; 22+ messages in thread From: Vinzent 'Gadget' Hoefler @ 2005-03-21 8:16 UTC (permalink / raw) Florian Weimer wrote: > * Vinzent Hoefler: > >> Florian Weimer wrote: >> >>> * Vinzent Hoefler: >>> >>>> It obviously fails to recompile missing dependencies. From my tests >>>> it also seems to ignore file dates. So I'd say, the "Java Build >>>> System" is non-existent. >>> >>> It's not specified by the standard. >> >> That's as good as "non-existent". :) > > Uh-oh, even with Ada, this is mostly a quality-of-implementation > issue. Ada implementations only need to enforce consistency. Yep. And talking about the compilation system: In which way is this different from taking all dependencies into account and check if they need to be recompiled? > There's > no need to support build tools which recognize dependencies and > automatically recompile them. It's also perfectly acceptable to > always recompile all dependencies. Ok, yes, that's the painful way. But at least it *would* recompile them then. >>> Sun's implementation optionally >>> recompiles out-of-date dependencies, though. >> >> Well, yes. I've tested some more, and from those tests it seems, >> Sun's "javac" recompiles dependencies if they are at most one level >> deep (or - to put it another way - direct dependencies of the file >> being compiled). It even does that recursively, so if I delete /all/ >> .class files it recompiles everything, of course. > > Oh, come on, have a closer look at the documentation. What documentation? hovi@jellix:~> man javac No manual entry for javac hovi@jellix:~> javac -help Usage: javac <options> <source files> where possible options include: -g Generate all debugging info -g:none Generate no debugging info -g:{lines,vars,source} Generate only some debugging info -nowarn Generate no warnings -verbose Output messages about what the compiler is doing -deprecation Output source locations where deprecated APIs are used -classpath <path> Specify where to find user class files -sourcepath <path> Specify where to find input source files -bootclasspath <path> Override location of bootstrap class files -extdirs <dirs> Override location of installed extensions -d <directory> Specify where to place generated class files -encoding <encoding> Specify character encoding used by source files -source <release> Provide source compatibility with specified release -target <release> Generate class files for specific VM version -help Print a synopsis of standard options Oh, obviously you mean <URI:http://java.sun.com/j2se/1.4.2/docs/tooldocs/solaris/javac.html>. Well, yes. The part about "searching for types" seems to imply it recompiles needed things. But in fact it doesn't (yes, I *did* use the -verbose option to check for that). The fact is: if it compiles class A which depends on class B which depends on class C and class C is out of date, but class B is not yet, it does *not* recompile it. It stops after checking class B. And from what I understand, this behaviour is indeed conforming to the specification (if you'd like to call the information on some company's web site a "specification"). Vinzent. -- worst case: The wrong assumption there actually is one. ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: How to cache output of the compiler aka ccache 2005-03-21 8:16 ` Vinzent 'Gadget' Hoefler @ 2005-03-21 8:54 ` Florian Weimer 2005-03-21 11:26 ` Vinzent 'Gadget' Hoefler 0 siblings, 1 reply; 22+ messages in thread From: Florian Weimer @ 2005-03-21 8:54 UTC (permalink / raw) * Vinzent Hoefler: > Oh, obviously you mean > <URI:http://java.sun.com/j2se/1.4.2/docs/tooldocs/solaris/javac.html>. Actually, I meant: <http://java.sun.com/j2se/1.3/docs/tooldocs/solaris/javac.html> Has Sun really removed that option? I can't believe it. ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: How to cache output of the compiler aka ccache 2005-03-21 8:54 ` Florian Weimer @ 2005-03-21 11:26 ` Vinzent 'Gadget' Hoefler 0 siblings, 0 replies; 22+ messages in thread From: Vinzent 'Gadget' Hoefler @ 2005-03-21 11:26 UTC (permalink / raw) Florian Weimer wrote: > <http://java.sun.com/j2se/1.3/docs/tooldocs/solaris/javac.html> > > Has Sun really removed that option? I can't believe it. I suppose, you mean the -Xdepend option? I don't see in in Java 1.5 (sorry, Java(tm) 2, of course) either, so I'd say "Yes, they did.". <sarcastic> Which part of "These options are non-standard and subject to change without notice." didn't you understand? </sarcastic> Vinzent. ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: How to cache output of the compiler aka ccache 2005-03-17 18:54 How to cache output of the compiler aka ccache i-google-iasuhdkajsh ` (2 preceding siblings ...) 2005-03-18 6:58 ` i-google-iasuhdkajsh @ 2005-03-18 8:07 ` Martin Krischik 2005-03-18 18:50 ` i-google-iasuhdkajsh 4 siblings, 0 replies; 22+ messages in thread From: Martin Krischik @ 2005-03-18 8:07 UTC (permalink / raw) Hi, i-google-iasuhdkajsh@rf.risimo.net wrote: > I have a 1mio LOC project and it takes multiple hours to > build and rebuild. The C/C++ world has a nice tools (ccache > is well known here) which cache the output of the compiler. > This especially speeds up the time for a "make clean;make". > Is there such a thing for Ada in general and gcc's gnat > specially? Any other way to speed up the compilation? Tools like ccache or precompiled header files usually optimise "#include" which is indeed the main performance killer in C/C++ compilations - however Ada does not have "#include" - the with command works differently. That much for a general comment - for more specific help we would need to know which compiler you are using. Martin -- mailto://krischik@users.sourceforge.net Ada programming at: http://ada.krischik.com ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: How to cache output of the compiler aka ccache 2005-03-17 18:54 How to cache output of the compiler aka ccache i-google-iasuhdkajsh ` (3 preceding siblings ...) 2005-03-18 8:07 ` Martin Krischik @ 2005-03-18 18:50 ` i-google-iasuhdkajsh 4 siblings, 0 replies; 22+ messages in thread From: i-google-iasuhdkajsh @ 2005-03-18 18:50 UTC (permalink / raw) > Hi > > I have a 1mio LOC project and it takes multiple hours to > build and rebuild. The C/C++ world has a nice tools (ccache > is well known here) which cache the output of the compiler. > This especially speeds up the time for a "make clean;make". > > Is there such a thing for Ada in general and gcc's gnat > specially? Any other way to speed up the compilation? Thanks for your input. I'm more a C/C++ guy than an Ada one. So please correct anything below. There are two main gains scenarios where a cache helps. 1) make clean;make You think that there are no reason for this. However I disagree. There are cases you need to rerun configure and afterwards the make clean;make idoim. Reasons for this could be: - you have changed the version/flags of tools you use - you have changed the configuration somehow (adding sound support for example) - poor/bad make files for non-Ada code In my project are multiple compilper generators and C/C++ and Java source code. So there is a configure and make clean;make is a good way to make sure you have a good build. 2) Sharing between trees I also want to give here two examples. The first is when a group of local developers share such a cache. If now an external CVS change comes in only one developer has to wait. In the best case you could do a background cache filling based on new CVS code with cron and co. The second case for sharing is multiple tress. Let me give you an example: $ cd tree.clean $ make $ cd ..;cp -a tree.clean tree.patched; cd tree.patched $ patch -p1 <.... $ make # nothing gained so far. Two days later however there # is an external CVS change. $ cd ../tree.clean $ cvs up;make # testing ... ok. Let's test the patched version $ cd ../tree.patched $ cvs up $ make # The last make should really be speed up. So you see there are more or less valid reasons for a compile cache. Thanks for the -m switch hint. I will try it. Thomas ^ permalink raw reply [flat|nested] 22+ messages in thread
end of thread, other threads:[~2005-03-21 11:26 UTC | newest] Thread overview: 22+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2005-03-17 18:54 How to cache output of the compiler aka ccache i-google-iasuhdkajsh 2005-03-17 19:40 ` Robert A Duff 2005-03-17 20:04 ` Ludovic Brenta 2005-03-17 21:27 ` Marius Amado Alves 2005-03-18 1:24 ` Ludovic Brenta 2005-03-18 2:08 ` Robert A Duff 2005-03-17 19:47 ` Ludovic Brenta 2005-03-18 19:22 ` Simon Wright 2005-03-18 20:04 ` Randy Brukardt 2005-03-18 22:10 ` Florian Weimer 2005-03-19 3:25 ` Larry Kilgallen 2005-03-18 6:58 ` i-google-iasuhdkajsh 2005-03-18 13:33 ` Robert A Duff 2005-03-18 14:01 ` Vinzent 'Gadget' Hoefler 2005-03-18 14:41 ` Florian Weimer 2005-03-18 15:08 ` Vinzent 'Gadget' Hoefler 2005-03-18 16:24 ` Florian Weimer 2005-03-21 8:16 ` Vinzent 'Gadget' Hoefler 2005-03-21 8:54 ` Florian Weimer 2005-03-21 11:26 ` Vinzent 'Gadget' Hoefler 2005-03-18 8:07 ` Martin Krischik 2005-03-18 18:50 ` i-google-iasuhdkajsh
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox