Re: Memory Useage

[mhamel_98@yahoo.com]

On Jun 8, 5:43 pm, Adam Beneschan <a...@irvine.com> wrote:
> On Jun 8, 1:38 pm, mhamel...@yahoo.com wrote:
>
>
>
> > Hello c.l.a.  Another question, I have a program that stores data on
> > the disk using sequential_io.  When I later read that data into an
> > array, the memory growth after ingesting a file is much much larger
> > than the disk footprint.  A file that takes 26.8MB on disk (over 134k
> > records) causes the program to swell by over 600MB!  Holy bloatware.
> > A short overview of what I'm trying to do - each sequential_io data
> > file has an associated header file with stuff like number of records,
> > etc.  The header is read, and an array is then created based on how
> > many records are said to be in the data file.  The data file is then
> > read, sticking a node into the array.  Some abbreviated code below,
> > the spec:
>
> > generic
> >   type Node_Type is private;
> > package Node_Manager is
>
> >   package Seq is new Sequential_Io (Node_Type);
>
> >   type Node_Array is array (positive range <>) of Node_Type;
> >   type Node_Ptr is access Node_Array;
>
> >   type Data_Rec is
> >     record
> >       Hdr : Node_Hdr;
> >       Data : Node_Ptr;
> >     end record;
>
> > Body stuff:
>
> >   procedure Free is new Unchecked_Deallocation (Node_Array, Node_Ptr);
> >   procedure Open (File : in out Data_Rec;
> >                            Name : in String) is
> >     Curr : Positive := 1;
> >     Node : Node_Type;
> >   begin
> >     Read_Hdr (Name, File.Hdr);
> >     File.Data := new Node_Array (1 .. File.Hdr.Size);
>
> >     Seq.Open (Dat_File, Seq.In_File, Name & ".dat");
> >     while not Seq.End_of_File (Dat_File) loop
> >       Seq.Read (Dat_File, Node);
> >       File.data.all (Curr) := Node;
> >       Curr := Curr + 1;
> >     end loop;
> >     Seq.Close (Dat_File);
> >     ...
>
> > The program works as I've wanted, though up until recently I've only
> > dealt with very small data sets, which is why I've never noticed undue
> > memory growth.  Now that I'm working with some "large" data sets, the
> > bloat is unbearable.  Any suggestions? (Besides look for another line
> > of work ;) )
> > Platform is ObjectAda 7.2 on WinNT.
>
> You sure File.Hdr.Size is correct?  (I.e. is it the same as the number
> of records in the file?)
>
>                    -- Adam


Yep, pretty sure the size is correct.  There is an internal confidence
test I developed sometime ago, I'll run it on this data set on Monday,
but things down the road in the program break if the size is
incorrectly over- or under-stated in the header.