* many exceptions cause memory leak? @ 2005-10-21 10:07 Christopher Broeg 2005-10-21 15:38 ` Frank J. Lhota ` (2 more replies) 0 siblings, 3 replies; 16+ messages in thread From: Christopher Broeg @ 2005-10-21 10:07 UTC (permalink / raw) Hi, I am running some numeric code in ADA95 using GNAT Academic edition. When negative values occur or the integration routine fails exceptions are raised and propagated. This works fine. Surprisingly, the memory usage increases from an initial ~40MB to 100MB and more during one day of execution. the memory usage shouldn't increase in my eyes. Is it possible that raising many exceptions coninues to use up some memory? thanks for your ideas, Chris ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: many exceptions cause memory leak? 2005-10-21 10:07 many exceptions cause memory leak? Christopher Broeg @ 2005-10-21 15:38 ` Frank J. Lhota 2005-10-21 17:59 ` Robert Klungle 2005-10-24 11:29 ` Christopher Broeg 2005-10-26 12:45 ` Christopher Broeg 2 siblings, 1 reply; 16+ messages in thread From: Frank J. Lhota @ 2005-10-21 15:38 UTC (permalink / raw) Christopher Broeg wrote: > Hi, > > I am running some numeric code in ADA95 using GNAT Academic edition. > When negative values occur or the integration routine fails exceptions > are raised and propagated. This works fine. > > Surprisingly, the memory usage increases from an initial ~40MB to 100MB > and more during one day of execution. the memory usage shouldn't > increase in my eyes. Is it possible that raising many exceptions > coninues to use up some memory? > > thanks for your ideas, > > Chris Which platform is this running on? That could be helpful in determining the cause of your problem. On Windows XP, I wrote a GNAT GPL application that raised and handled a million exception occurrences. This application also does a call to the Win32 API function GlobalMemoryStatus both before and after handling a million exception occurrences. The memory status before and after handling these exceptions was virtually the same. This seems to indicate that on XP at least, GNAT exceptions do *not* leak memory. -- "All things extant in this world, Gods of Heaven, gods of Earth, Let everything be as it should be; Thus shall it be!" - Magical chant from "Magical Shopping Arcade Abenobashi" "Drizzle, Drazzle, Drozzle, Drome, Time for the this one to come home!" - Mr. Lizard from "Tutor Turtle" ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: many exceptions cause memory leak? 2005-10-21 15:38 ` Frank J. Lhota @ 2005-10-21 17:59 ` Robert Klungle 2005-10-21 18:13 ` Ed Falis 0 siblings, 1 reply; 16+ messages in thread From: Robert Klungle @ 2005-10-21 17:59 UTC (permalink / raw) For what it is worth, a few years ago my team developed a large system which ran on SGIs, IBM AIXs, and Intel systems. We found that over time, the memory usage by the system grew even though all memory was being returned. We found out that the Ada Run Time requested more memory from the OS even though it had memory available in its allocation pool. Eventually, this pool allocation stopped growing and occasionally dropped a little (we expect due to the action of other programs running). The Ada allocation pool grew to 4.6 GB before stabilizing. The system ran for weeks without crashing (due to memory exhaustion). We were told by the compiler vendor this was done to improve the dynamic memory allocation performance. Take it for what it is worth. Just an observation. cheers...bob "Frank J. Lhota" <NOSPAM.lhota@adarose.com> wrote in message news:yX76f.7$dW6.4@trndny09... > Christopher Broeg wrote: > > Hi, > > > > I am running some numeric code in ADA95 using GNAT Academic edition. > > When negative values occur or the integration routine fails exceptions > > are raised and propagated. This works fine. > > > > Surprisingly, the memory usage increases from an initial ~40MB to 100MB > > and more during one day of execution. the memory usage shouldn't > > increase in my eyes. Is it possible that raising many exceptions > > coninues to use up some memory? > > > > thanks for your ideas, > > > > Chris > > Which platform is this running on? That could be helpful in determining > the cause of your problem. > > On Windows XP, I wrote a GNAT GPL application that raised and handled a > million exception occurrences. This application also does a call to the > Win32 API function GlobalMemoryStatus both before and after handling a > million exception occurrences. The memory status before and after > handling these exceptions was virtually the same. This seems to indicate > that on XP at least, GNAT exceptions do *not* leak memory. > > -- > "All things extant in this world, > Gods of Heaven, gods of Earth, > Let everything be as it should be; > Thus shall it be!" > - Magical chant from "Magical Shopping Arcade Abenobashi" > > "Drizzle, Drazzle, Drozzle, Drome, > Time for the this one to come home!" > - Mr. Lizard from "Tutor Turtle" ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: many exceptions cause memory leak? 2005-10-21 17:59 ` Robert Klungle @ 2005-10-21 18:13 ` Ed Falis 0 siblings, 0 replies; 16+ messages in thread From: Ed Falis @ 2005-10-21 18:13 UTC (permalink / raw) On Fri, 21 Oct 2005 13:59:36 -0400, Robert Klungle <bklungle@adelphia.net> wrote: > We were told by the compiler vendor this was done to > improve the dynamic memory allocation performance. Take it for what it is > worth. Sounds like an old Alsys compiler that grabbed large chunks of memory, then managed the allocated chunks as the application's heap. This as opposed to using malloc directly. - Ed ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: many exceptions cause memory leak? 2005-10-21 10:07 many exceptions cause memory leak? Christopher Broeg 2005-10-21 15:38 ` Frank J. Lhota @ 2005-10-24 11:29 ` Christopher Broeg 2005-10-24 20:00 ` Simon Wright 2005-10-24 21:27 ` Andreas Schulz 2005-10-26 12:45 ` Christopher Broeg 2 siblings, 2 replies; 16+ messages in thread From: Christopher Broeg @ 2005-10-24 11:29 UTC (permalink / raw) Hi, sorry that I didn't reply immediately. I was away over the weekend. From your replys I assume that ada should not waste memory by exception calls. Thanks for that. Well, my platform is suse Linux and I compile using GNATMAKE GAP 1.1.0 (20041209-323) I am by no means an expert in programming but I figured that all memory assigned by pointers is returned. I do not use my own storage pool but allocate just by is new... so it is all on the heap, I guess. The application is not large on your terms. It is just an a little more complicated numerical algorithm finding a few thousand solutions to a set of ordinary differential equations. Using spline interpolation for equations of state, and other physical quantities, it gets a little more complicated, though. Furthermore my data i/o is via a c binding to HDF5. I am not sure if may be the HDF5 library wasting the memory. It is not real problem, because the program usually finishes around a few 100 MB, but when I run the calculations on other peoples machines I don't want it to use so much (it is nice of them to allow the cpu load, already). Upon starting it only uses 46 MB. But increases steadily. Any suggestions how to find the memory leak? Thanks, Chris ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: many exceptions cause memory leak? 2005-10-24 11:29 ` Christopher Broeg @ 2005-10-24 20:00 ` Simon Wright 2005-10-25 7:32 ` Christopher Broeg 2005-10-24 21:27 ` Andreas Schulz 1 sibling, 1 reply; 16+ messages in thread From: Simon Wright @ 2005-10-24 20:00 UTC (permalink / raw) Christopher Broeg <broeg@astro.uni-jena.de> writes: > I am by no means an expert in programming but I figured that all > memory assigned by pointers is returned. I do not use my own storage > pool but allocate just by > is new... > so it is all on the heap, I guess. GNAT doesn't automatically free this memory until the end of the program. You need to free it yourself. ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: many exceptions cause memory leak? 2005-10-24 20:00 ` Simon Wright @ 2005-10-25 7:32 ` Christopher Broeg 2005-10-25 7:42 ` Christopher Broeg 2005-10-25 13:09 ` Frank J. Lhota 0 siblings, 2 replies; 16+ messages in thread From: Christopher Broeg @ 2005-10-25 7:32 UTC (permalink / raw) Simon Wright wrote: > Christopher Broeg <broeg@astro.uni-jena.de> writes: > >>I am by no means an expert in programming but I figured that all >>memory assigned by pointers is returned. I do not use my own storage >>pool but allocate just by >>is new... >>so it is all on the heap, I guess. > > GNAT doesn't automatically free this memory until the end of the > program. You need to free it yourself. I do use unchecked deallocation on all pointers. I now have used gnatmem and found one memory leak that was my fault. There are still a few thousand "leaks" due to ada exception handling, though. Gnatmem output follows: Global information ------------------ Total number of allocations :109993 Total number of deallocations :97243 Final Water Mark (non freed mem) : 29.85 Megabytes High Water Mark : 29.90 Megabytes Allocation Root # 1 ------------------- Number of non freed allocations :12702 Final Water Mark (non freed mem) : 80.87 Kilobytes High Water Mark : 80.87 Kilobytes Backtrace : a-except.adb:1396 <ada__exceptions___elabb> I do think that the program now stays constnat in size, or at least asymtotically approaches a relatively small constant size in memory. running the code over night, the high water mark of the entire program was small, so I hope the problem is solved. But since the program had already finished by this morning, I couldn't check independently using top, also I could not check whether this means reserved memory only and what happens with the virtual memory. It used to be twice as large as the reserved. I'll know in another 10 hours if everything is fine now. Still, I am wondering if the above begaviour is normal. It is only 60kB, so no big deal, but 12702 non freed allocations? Is this normal? They always stem from line 1396 in a-except.adb. It is the place where a dummy procecdure called ZZZ is called. It appears to me that this should be a one-time per program lifetime event. Does this mean that there are 12702 somehow independent exception occurences in my program? thanks for all your help, Chris ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: many exceptions cause memory leak? 2005-10-25 7:32 ` Christopher Broeg @ 2005-10-25 7:42 ` Christopher Broeg 2005-10-25 12:34 ` Alex R. Mosteo 2005-10-25 18:50 ` Wiljan Derks 2005-10-25 13:09 ` Frank J. Lhota 1 sibling, 2 replies; 16+ messages in thread From: Christopher Broeg @ 2005-10-25 7:42 UTC (permalink / raw) Christopher Broeg wrote: > Simon Wright wrote: >> Christopher Broeg <broeg@astro.uni-jena.de> writes: >> >>> I am by no means an expert in programming but I figured that all >>> memory assigned by pointers is returned. I do not use my own storage >>> pool but allocate just by >>> is new... >>> so it is all on the heap, I guess. >> >> GNAT doesn't automatically free this memory until the end of the >> program. You need to free it yourself. > > I do use unchecked deallocation on all pointers. I now have used gnatmem > and found one memory leak that was my fault. There are still a few > thousand "leaks" due to ada exception handling, though. Gnatmem output > follows: > > Global information > ------------------ > Total number of allocations :109993 > Total number of deallocations :97243 > Final Water Mark (non freed mem) : 29.85 Megabytes > High Water Mark : 29.90 Megabytes > > Allocation Root # 1 > ------------------- > Number of non freed allocations :12702 > Final Water Mark (non freed mem) : 80.87 Kilobytes > High Water Mark : 80.87 Kilobytes > Backtrace : > a-except.adb:1396 <ada__exceptions___elabb> > > I do think that the program now stays constnat in size, or at least > asymtotically approaches a relatively small constant size in memory. > running the code over night, the high water mark of the entire program > was small, so I hope the problem is solved. But since the program had > already finished by this morning, I couldn't check independently using > top, also I could not check whether this means reserved memory only and > what happens with the virtual memory. It used to be twice as large as > the reserved. I'll know in another 10 hours if everything is fine now. > > Still, I am wondering if the above begaviour is normal. It is only 60kB, > so no big deal, but 12702 non freed allocations? Is this normal? They > always stem from line 1396 in a-except.adb. It is the place where a > dummy procecdure called ZZZ is called. It appears to me that this should > be a one-time per program lifetime event. Does this mean that there are > 12702 somehow independent exception occurences in my program? > > thanks for all your help, > > Chris Addition: Could compiling with debug information be the reason for the above behaviour? Chris ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: many exceptions cause memory leak? 2005-10-25 7:42 ` Christopher Broeg @ 2005-10-25 12:34 ` Alex R. Mosteo 2005-10-25 12:55 ` Alex R. Mosteo 2005-10-25 18:50 ` Wiljan Derks 1 sibling, 1 reply; 16+ messages in thread From: Alex R. Mosteo @ 2005-10-25 12:34 UTC (permalink / raw) Christopher Broeg wrote: > Addition: > Could compiling with debug information be the reason for the above > behaviour? Not in my experience. > Chris ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: many exceptions cause memory leak? 2005-10-25 12:34 ` Alex R. Mosteo @ 2005-10-25 12:55 ` Alex R. Mosteo 0 siblings, 0 replies; 16+ messages in thread From: Alex R. Mosteo @ 2005-10-25 12:55 UTC (permalink / raw) Alex R. Mosteo wrote: > Christopher Broeg wrote: > >> Addition: >> Could compiling with debug information be the reason for the above >> behaviour? > > > Not in my experience. Unless you're using the Gnat.Debug_Pools package. These pools store a list with accessed addresses or something like that, which grows with time. > >> Chris ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: many exceptions cause memory leak? 2005-10-25 7:42 ` Christopher Broeg 2005-10-25 12:34 ` Alex R. Mosteo @ 2005-10-25 18:50 ` Wiljan Derks 2005-10-26 10:56 ` Christopher Broeg 1 sibling, 1 reply; 16+ messages in thread From: Wiljan Derks @ 2005-10-25 18:50 UTC (permalink / raw) "Christopher Broeg" <broeg@astro.uni-jena.de> wrote in message news:djknki$cl6$1@lc03.rz.uni-jena.de... >> Global information >> ------------------ >> Total number of allocations :109993 >> Total number of deallocations :97243 >> Final Water Mark (non freed mem) : 29.85 Megabytes >> High Water Mark : 29.90 Megabytes Still looks like you are loosing memory. Maybe a good tip: When and exception occurs one can easily asume that an inout variable of a procedure alsready has the value assigned to its end location. Thus something like this: procedure zzz(x: in out access y) is begin x:=new ... exception here end; When an exception occurs, one could assume that the new pointer is already assigned to the structure that was passed to zzz. That is however not the case when using gnat. Typically, the inout parameter is a local variable for the procedure that is stored on its final location when doing a normal procedure exit. This is a typical reason for loosing memory when doing exception processing. Changing the mode from inout to and access parameter fixes that. ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: many exceptions cause memory leak? 2005-10-25 18:50 ` Wiljan Derks @ 2005-10-26 10:56 ` Christopher Broeg 0 siblings, 0 replies; 16+ messages in thread From: Christopher Broeg @ 2005-10-26 10:56 UTC (permalink / raw) Wiljan Derks wrote: > "Christopher Broeg" <broeg@astro.uni-jena.de> wrote in message > news:djknki$cl6$1@lc03.rz.uni-jena.de... >>>Global information >>>------------------ >>> Total number of allocations :109993 >>> Total number of deallocations :97243 >>> Final Water Mark (non freed mem) : 29.85 Megabytes >>> High Water Mark : 29.90 Megabytes > Still looks like you are loosing memory. > Maybe a good tip: > When and exception occurs one can easily asume that an inout variable of a > procedure > alsready has the value assigned to its end location. Thus something like > this: > procedure zzz(x: in out access y) is > begin > x:=new ... > exception here > end; > When an exception occurs, one could assume that the new pointer is already > assigned > to the structure that was passed to zzz. > That is however not the case when using gnat. > Typically, the inout parameter is a local variable for the procedure that is > stored on its final location when doing > a normal procedure exit. > This is a typical reason for loosing memory when doing exception processing. > Changing the mode from inout to and access parameter fixes that. > > Hi, the ZZZ procedure is a procedure in the GNAT exception library. There are no pointers there, actually. I think I might have found the reason for the memory loss, though: I am using unchecked_acces to the data for the external HDF5 calls. The memory leak seems to be originating there. If I have aliased variables do I have to manually free the memory? i.e. I have: declare Data : H5_double.Data_1d(0..2) ; begin ... Status := H5_Double.H5aread (Attr_Id => Attribute, Mem_Type_Id => H5t_Native_double, Buf => Data(0)'Unchecked_Access); data_1d is an array of aliased float. I don't do any manual deallocaton. cheers, Chris ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: many exceptions cause memory leak? 2005-10-25 7:32 ` Christopher Broeg 2005-10-25 7:42 ` Christopher Broeg @ 2005-10-25 13:09 ` Frank J. Lhota 2005-10-25 14:09 ` Christopher Broeg 1 sibling, 1 reply; 16+ messages in thread From: Frank J. Lhota @ 2005-10-25 13:09 UTC (permalink / raw) Do you ever call the following function from the Ada.Exceptions package? function Save_Occurrence (Source : Exception_Occurrence) return Exception_Occurrence_Access; This function allocates an Exception Occurrence from the heap and copies the data from Source into this new occurrence. It is up to the programmer to deallocate the result of this function. An instance of Unchecked_Conversion can be used for this purpose. Failure to deallocate the results of the Save_Occurrence function would certainly result in a memory leak. Also, is there any other Ada.Exceptions subprogram that you call? More info on how you are using exceptions might help us diagnose the problem. -- "All things extant in this world, Gods of Heaven, gods of Earth, Let everything be as it should be; Thus shall it be!" - Magical chant from "Magical Shopping Arcade Abenobashi" "Drizzle, Drazzle, Drozzle, Drome, Time for the this one to come home!" - Mr. Lizard from "Tutor Turtle" ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: many exceptions cause memory leak? 2005-10-25 13:09 ` Frank J. Lhota @ 2005-10-25 14:09 ` Christopher Broeg 0 siblings, 0 replies; 16+ messages in thread From: Christopher Broeg @ 2005-10-25 14:09 UTC (permalink / raw) Frank J. Lhota wrote: > Do you ever call the following function from the Ada.Exceptions package? > > function Save_Occurrence > (Source : Exception_Occurrence) > return Exception_Occurrence_Access; > > This function allocates an Exception Occurrence from the heap and copies > the data from Source into this new occurrence. It is up to the > programmer to deallocate the result of this function. An instance of > Unchecked_Conversion can be used for this purpose. > > Failure to deallocate the results of the Save_Occurrence function would > certainly result in a memory leak. > > Also, is there any other Ada.Exceptions subprogram that you call? More > info on how you are using exceptions might help us diagnose the problem. > Hi, no I do not use save_occurence. I do use Raise_Exception and exception_information / message, though. I am still running the code and it seems that the major memory leak problem is solved: Global information ------------------ Total number of allocations :4216666 Total number of deallocations :3823682 Final Water Mark (non freed mem) : 32.21 Megabytes High Water Mark : 32.26 Megabytes However, it appears strange to me that gnatmem detects so many non freed allocations originating in the exception package: Allocation Root # 1 ------------------- Number of non freed allocations :392937 Final Water Mark (non freed mem) : 2.44 Megabytes High Water Mark : 2.44 Megabytes Backtrace : a-except.adb:1396 <ada__exceptions___elabb> Of course, the program is still running, but in the past those occurences have not been freed upon termination. The memory use of 2.44 MB fits well with the fact that the code started out with a high water mark of roughly 29 MB and during the day raised the value to 32.21 MB. To be clear, this poses no problem at all to me, now. I am just curious and would like to understand what is going on. I thank you all for the helpful comments and suggestions. Chris ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: many exceptions cause memory leak? 2005-10-24 11:29 ` Christopher Broeg 2005-10-24 20:00 ` Simon Wright @ 2005-10-24 21:27 ` Andreas Schulz 1 sibling, 0 replies; 16+ messages in thread From: Andreas Schulz @ 2005-10-24 21:27 UTC (permalink / raw) Christopher Broeg wrote: > Any suggestions how to find the memory leak? gnatmem should be able to tell you - <http://gcc.gnu.org/onlinedocs/gcc-4.0.2/gnat_ugn_unw/The-gnatmem-Tool.html> -- For whatever-the NOSPAM address is valid and mine... ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: many exceptions cause memory leak? 2005-10-21 10:07 many exceptions cause memory leak? Christopher Broeg 2005-10-21 15:38 ` Frank J. Lhota 2005-10-24 11:29 ` Christopher Broeg @ 2005-10-26 12:45 ` Christopher Broeg 2 siblings, 0 replies; 16+ messages in thread From: Christopher Broeg @ 2005-10-26 12:45 UTC (permalink / raw) Hi everybody, I finally found the (second) reason for the memory leak: I was calling HDF5 library (c version) functions. When a string was needed I passed it using name => cs.new_string("whatever") this created a new c-pointer and i didn't deallocate it. Stupid mistake, really. It is the first time I call foreign libraries, so I didn't comprehend fully what the new_string does. Thank you all very much for your help! chris ^ permalink raw reply [flat|nested] 16+ messages in thread
end of thread, other threads:[~2005-10-26 12:45 UTC | newest] Thread overview: 16+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2005-10-21 10:07 many exceptions cause memory leak? Christopher Broeg 2005-10-21 15:38 ` Frank J. Lhota 2005-10-21 17:59 ` Robert Klungle 2005-10-21 18:13 ` Ed Falis 2005-10-24 11:29 ` Christopher Broeg 2005-10-24 20:00 ` Simon Wright 2005-10-25 7:32 ` Christopher Broeg 2005-10-25 7:42 ` Christopher Broeg 2005-10-25 12:34 ` Alex R. Mosteo 2005-10-25 12:55 ` Alex R. Mosteo 2005-10-25 18:50 ` Wiljan Derks 2005-10-26 10:56 ` Christopher Broeg 2005-10-25 13:09 ` Frank J. Lhota 2005-10-25 14:09 ` Christopher Broeg 2005-10-24 21:27 ` Andreas Schulz 2005-10-26 12:45 ` Christopher Broeg
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox