Re: Problems with large records (GNAT) [continued]

[Dr Adrian Wrigley <amtw@linuxchip.demon.co.uk>]

Robert A Duff wrote:
> I'm curious: Why do you need such enormous records?
> And if your data structures are almost as big as your 32-bit address
> space, why don't you get a machine with a 64-bit address space?

I tried to explain that below... I want to map data from a filing system
directly into the program's memory space.

I would be using a 64-bit machine, but they're a bit more expensive,
and GNAT isn't released for Alpha/Linux :(
Since a PC can be loaded with 2048MB RAM nowadays, it is frustrating
to struggle with code hitting the 256M limit.

It is very easy to write code which breaks unexpectedly in "normal" use,
due to this capacity limit.  For example, in image processing, you might
read in a 6000x4000 pixel color image, and compute the spectrum (eg FFT)
as floats.  An apparently reasonable implementation in Ada/GNAT/Intel breaks
(whereas the equivalent program in C works fine).  Images of this
resolution are standard in the 70mm film world.

> > This time, I have come across another manifestation of the problem,
> > which appears rather strange.  The locations for different
> > elements in a record are the same.  In the example code below,
> > the elements First and Last are stored in the same place.
> > (you can verify this using 'Access on the two elements).
> 
> They're not aliased in the code below, so 'Access isn't allowed.
> Perhaps you meant 'Address?

I took the "aliased" out, since it was redundant in the actual code.

> Anyway, I can't explain that problem, but...
> 
> >    X := Malloc (Interfaces.C.Size_T (Size * Float'Size + 2*Float'Size)); -- Hmmmm
> 
> Float'Size is in bits, whereas malloc expects a size in bytes
> (or in units of sizeof(char), or whatever).

Damn!  I had originally used "4" in place of Float'Size. The value passed
the Malloc should be 16#4000004#, or thereabouts.  As written in my
post, it only gets 32, due to truncation.  I didn't notice
this because the code doesn't access the high end of the memory.

> You need to divide by System.Storage_Unit, and you need to write your
> code carefully to avoid overflow, since you're dealing with numbers
> close to Integer'Last.

This, of course is the crux of the problem.  We're dealing with
numbers exceeding Integer'Last, and wrapping makes it unworkable.

> Why are you using malloc anyway?  Why not "X := new Item_t;"?

I find that "new" gives me problems with large types.
I think maybe GNAT sometimes optimises "new" to use the stack, and
then runs out of space.  Malloc is happy to give all the memory the
system has (and on Linux, considerably more :().

> P.S. If you think you've found a bug in GNAT, report it, and maybe
> they'll fix it.

Probably a good idea.  But I'm impatient!
--
Dr Adrian Wrigley