gnat and heap size

gnat and heap size

[Claude Marinier]

Hi,

We want to use large arrays (well, large for us: 10000 x 10000 complex
numbers). We are using gnat 3.13p on Solaris 7. We run out of heap (heap
exhausted) and have not yet found a way to increase it.

Thanks.

P.S. Please reply by e-mail.

Re: gnat and heap size

[Ted Dennison]

In article <1001442590.557811@news.drenet.dnd.ca>, Claude Marinier says...
>We want to use large arrays (well, large for us: 10000 x 10000 complex
>numbers). We are using gnat 3.13p on Solaris 7. We run out of heap (heap
>exhausted) and have not yet found a way to increase it.

By my math, that's probably going to be using at least 800MB. Do you actually
have that much RAM in your system? It would have to be one kick'n system to have
nearly a gig of contiguous free RAM available on the heap...

---
T.E.D.    homepage   - http://www.telepath.com/dennison/Ted/TED.html
          home email - mailto:dennison@telepath.com

Re: gnat and heap size

[Marin David Condic]

It would help to know what hardware/OS this is trying to run on since this
sounds like it is probably running up against some system limitation. Things
like VMS had virtual memory restricted by various settable parameters, so it
*might* run if the quotas were increased appropriately. But you'd need to
know the system to make the recommendations.

It wouldn't necessarily have to be a kick-a** machine to run something
needing 800mb of memory. Assuming it had virtual memory and sufficient disk
space, you might be able to get it to work. Albeit, the page swapping would
probably kill you, but it might actually run in a finite amount of time.
Virtual memory can be a cool thing if you're not in a big rush.

If virtual memory didn't exist, we'd have to invent it.

MDC
--
Marin David Condic
Senior Software Engineer
Pace Micro Technology Americas    www.pacemicro.com
Enabling the digital revolution
e-Mail:    marin.condic@pacemicro.com
Web:      http://www.mcondic.com/


"Ted Dennison" <dennison@telepath.com> wrote in message
news:%26s7.4950$ev2.8194@www.newsranger.com...
> In article <1001442590.557811@news.drenet.dnd.ca>, Claude Marinier says...
> >We want to use large arrays (well, large for us: 10000 x 10000 complex
> >numbers). We are using gnat 3.13p on Solaris 7. We run out of heap (heap
> >exhausted) and have not yet found a way to increase it.
>
> By my math, that's probably going to be using at least 800MB. Do you
actually
> have that much RAM in your system? It would have to be one kick'n system
to have
> nearly a gig of contiguous free RAM available on the heap...
>
> ---
> T.E.D.    homepage   - http://www.telepath.com/dennison/Ted/TED.html
>           home email - mailto:dennison@telepath.com

Re: gnat and heap size

[Ted Dennison]

In article <9oqs5k$jjq$1@nh.pace.co.uk>, Marin David Condic says...
>
>It would help to know what hardware/OS this is trying to run on since this
..
>It wouldn't necessarily have to be a kick-a** machine to run something
>needing 800mb of memory. Assuming it had virtual memory and sufficient disk
>space, you might be able to get it to work. Albeit, the page swapping would
>probably kill you, but it might actually run in a finite amount of time.

Well, he did say Solaris 7. That narrows it down a bit. If its one of those new
Starcat servers, it might even have way more than that just in RAM alone....

---
T.E.D.    homepage   - http://www.telepath.com/dennison/Ted/TED.html
          home email - mailto:dennison@telepath.com

Re: gnat and heap size

[David Starner]

On Tue, 25 Sep 2001 20:46:51 GMT, Ted Dennison <dennison@telepath.com> wrote:
> By my math, that's probably going to be using at least 800MB. Do you actually
> have that much RAM in your system? It would have to be one kick'n system to have
> nearly a gig of contiguous free RAM available on the heap...

Why would it have to be contiguous? If I understand virtual memory right,
on almost any modern system, it'd just have to be a contiguous set of
virtual pages, which could then be placed individually throughout real
memory and swap space whereever.

-- 
David Starner - dstarner98@aasaa.ofe.org
Pointless website: http://dvdeug.dhis.org
When the aliens come, when the deathrays hum, when the bombers bomb,
we'll still be freakin' friends. - "Freakin' Friends"

Re: gnat and heap size

[Dr Adrian Wrigley]

Claude Marinier wrote:
> We want to use large arrays (well, large for us: 10000 x 10000 complex
> numbers). We are using gnat 3.13p on Solaris 7. We run out of heap (heap
> exhausted) and have not yet found a way to increase it.

I have had a similar problem, and raised it on this NG a few months ago...

I was trying to store the past five years of the US stock markets in a big array.
I wanted to reload the data from disk "instantly", so I could do statistics easily.

The solution I came up with was to use an "mmap" call to map the data.  That way,
I could get an access value to data as big as the OS could give.  This was big
enough for my data (about 0.5GB).  Mapping the data back in was extremely quick.
Unfortunately, it makes the code a bit less portable, but that wasn't too bad.

But there were problems with mapping big records, and taking 'size attributes.
Since Ada works in bits, the sizes can overflow (signed) 32-bit representation.
This makes handling data in excess of 256MB tricky, since locations of
record elements are wrong, as are size attributes (eg a problem in generics).
I solved this by calculating the sizes of each element, multiplying by the
number of elements, adding in each record component's size.  Nasty.
But it was 10-100 times the speed of more obvious solutions.

I suspect you have a 64-bit architecture, and *may* not hit the 'size problems
I had with GNAT.  If you stick to simple arrays, you might be OK. I was using
GNAT 3.12p on a '686 processor.

I don't know what these other guys seem so surprised at.  Doesn't everybody
else have 1280MB of RAM nowadays, too?  ;-)  It's only a few hundred bucks!
--
Adrian Wrigley

(by the way... what do you use such large matrices for?)

Re: gnat and heap size

[Robert Dewar]

Ted Dennison<dennison@telepath.com> wrote in message news:<%26s7.4950$ev2.8194@www.newsranger.com>...
> By my math, that's probably going to be using at least 800MB. Do you actually
> have that much RAM in your system? It would have to be one kick'n system to have
> nearly a gig of contiguous free RAM available on the heap...


What an extraordinarily peculiar comment :-)

We are talking about contiguous *virtual* memory, not
physical memory. This is 2001, not 1965.

Re: gnat and heap size

[Robert Dewar]

Ted Dennison<dennison@telepath.com> wrote in message news:<%26s7.4950$ev2.8194@www.newsranger.com>...
> By my math, that's probably going to be using at least 800MB. Do you actually
> have that much RAM in your system? It would have to be one kick'n system to have
> nearly a gig of contiguous free RAM available on the heap...


And by the way (this being 2001), machines with 1-2 gig
of real memory are common (my little portable notebook
has 512 meg ...)

Re: gnat and heap size

[Lutz Donnerhacke]

* Dr Adrian Wrigley wrote:
>I don't know what these other guys seem so surprised at.  Doesn't everybody
>else have 1280MB of RAM nowadays, too?  ;-)  It's only a few hundred bucks!

No. Availability of huge amounts of computing and storage ressources are the
main reason for decreasing programmer and program designer skills resulting
in ever decreasing software quality. :-(

Still using NeXT stations, i386/32MB, ...

Re: gnat and heap size

[Marin David Condic]

Must have missed the Solaris 7 part. Sorry. Not too familiar with how that
restricts memory use, but I seem to recall a similar situation quite a while
back in which there was one parameter that could be changed & a system
reboot performed that altered how much memory was available to a process.

My recollection was that Solaris did not have many memory management
parameters - VMS certainly had considerably more control over process quotas
on all sorts of resources.

MDC
--
Marin David Condic
Senior Software Engineer
Pace Micro Technology Americas    www.pacemicro.com
Enabling the digital revolution
e-Mail:    marin.condic@pacemicro.com
Web:      http://www.mcondic.com/


"Ted Dennison" <dennison@telepath.com> wrote in message
news:WZ6s7.5025$ev2.8658@www.newsranger.com...
> Well, he did say Solaris 7. That narrows it down a bit. If its one of
those new
> Starcat servers, it might even have way more than that just in RAM
alone....
>

Re: gnat and heap size

[Ted Dennison]

In article <5ee5b646.0109251813.3001cff7@posting.google.com>, Robert Dewar
says...
>
>Ted Dennison<dennison@telepath.com> wrote in message news:<%26s7.4950$ev2.8194@www.newsranger.com>...
>> By my math, that's probably going to be using at least 800MB. Do you actually
>> have that much RAM in your system? It would have to be one kick'n system to have
>> nearly a gig of contiguous free RAM available on the heap...
>
>
>And by the way (this being 2001), machines with 1-2 gig
>of real memory are common (my little portable notebook
>has 512 meg ...)

Well, I'm perhaps not as impressed with that as you intended. After 7 or so
years of reading your posts Robert, one of the (perhaps few) things I've learned
is that you keep your laptop systems pretty close to state of the art. You of
course will dispute that, but compared to the P75 laptop I have here at my desk,
you are practicly in a separate historical age. I'd wadger that your laptop is
probably better than any of the 9 systems I have access to. :-)

---
T.E.D.    homepage   - http://www.telepath.com/dennison/Ted/TED.html
          home email - mailto:dennison@telepath.com

Re: gnat and heap size

[Ted Dennison]

In article <5ee5b646.0109251812.47533415@posting.google.com>, Robert Dewar
says...
>
>Ted Dennison<dennison@telepath.com> wrote in message news:<%26s7.4950$ev2.8194@www.newsranger.com>...
>> By my math, that's probably going to be using at least 800MB. Do you actually
>> have that much RAM in your system? It would have to be one kick'n system to have
>> nearly a gig of contiguous free RAM available on the heap...
>
>
>What an extraordinarily peculiar comment :-)
>
>We are talking about contiguous *virtual* memory, not
>physical memory. This is 2001, not 1965.

Ahh yes, I had fogotten that. Realise that I do mostly RTOS work these days,
where we typically don't have virtual memory even here in 2001. But since he
said "Solaris 7", there definitely would be virtual memory involved.

---
T.E.D.    homepage   - http://www.telepath.com/dennison/Ted/TED.html
          home email - mailto:dennison@telepath.com

Re: gnat and heap size

[Ted Dennison]

In article <9osjpt$a5l$1@nh.pace.co.uk>, Marin David Condic says...
>
>Must have missed the Solaris 7 part. Sorry. Not too familiar with how that
>restricts memory use, but I seem to recall a similar situation quite a while
>back in which there was one parameter that could be changed & a system
>reboot performed that altered how much memory was available to a process.
>
>My recollection was that Solaris did not have many memory management
>parameters - VMS certainly had considerably more control over process quotas
>on all sorts of resources.

VMS also had a limit of about 1GB if I remember right. Its memory space was
something like 4GB, divided into 4 sections. At least that's what I remember
from 12 years ago or so when I was using it.

---
T.E.D.    homepage   - http://www.telepath.com/dennison/Ted/TED.html
          home email - mailto:dennison@telepath.com

Re: gnat and heap size

[Claude Marinier]

The application is an electromagnetic simulation. Yes, we could use sparse
matrix techniques but part of the program requires a full matrix. Perhaps
we are hitting a low level limit (is that what Adrian is saying?).

Here is a test program.

	procedure matrix is
	type matrix is array ( integer range <>, integer range <> ) of float;
	type matrix_a is access matrix;
	A: matrix_a := New matrix( 1..9000, 1..9000 );
	i : integer;
	begin
	i := 1;
	while i <= 9000 loop
	  A(i,i) := 1.0;
	  i := i + 1;
	end loop;
	end matrix;

On Tue, 25 Sep 2001, Dr Adrian Wrigley wrote:
> Claude Marinier wrote:
> > We want to use large arrays (well, large for us: 10000 x 10000 complex
> > numbers). We are using gnat 3.13p on Solaris 7. We run out of heap (heap
> > exhausted) and have not yet found a way to increase it.

> Since Ada works in bits, the sizes can overflow (signed) 32-bit
> representation. This makes handling data in excess of 256MB tricky,
> since locations of record elements are wrong, as are size attributes
> (eg a problem in generics). I solved this by calculating the sizes of
> each element, multiplying by the number of elements, adding in each
> record component's size.  Nasty. But it was 10-100 times the speed of
> more obvious solutions.
>
> I suspect you have a 64-bit architecture, and *may* not hit the 'size
> problems I had with GNAT.  If you stick to simple arrays, you might be
> OK. I was using GNAT 3.12p on a '686 processor.

> (by the way... what do you use such large matrices for?)

-- 
Claude Marinier, Information Technology Group    claude.marinier@dreo.dnd.ca
Defence Research Establishment Ottawa (DREO)    (613) 998-4901  FAX 998-2675
3701 Carling Avenue, Ottawa, Ontario  K1A 0Z4         http://www.dreo.dnd.ca

The decline of programming civilization (was: gnat and heap size)

[Ted Dennison]

In article <slrn9r36pi.jv.lutz@taranis.iks-jena.de>, Lutz Donnerhacke says...
>No. Availability of huge amounts of computing and storage ressources are the
>main reason for decreasing programmer and program designer skills resulting
>in ever decreasing software quality. :-(

I'm not entirely convinced that there are less smart programmers now than there
used to be. If anything I see the opposite trend. There may be a lot more
doufuses running around too, but that's just because there are so many more
programmers in general than there used to be. I don't think the percentage of
good ones has changed much.

If software quality is decreasing (something of which I'm not convinced either),
I'd blame it on the complexity of applications outstripping our advances in
being able to deal with complexity. The community's tendancy to cling to obsolte
tools certianly doesn't help matters either. One could also blame the rise of
the consumer market for software, and that market's inherent disincentive to
quality (price and schedule are vital there, and you only get to pick 2 of the
three).

Anyway, I'd personally blame all our troubles on "Visual" programming
environments. :-)

---
T.E.D.    homepage   - http://www.telepath.com/dennison/Ted/TED.html
          home email - mailto:dennison@telepath.com

Re: gnat and heap size

[Larry Kilgallen]

In article <9osjpt$a5l$1@nh.pace.co.uk>, "Marin David Condic" <dont.bother.mcondic.auntie.spam@[acm.org> writes:
> Must have missed the Solaris 7 part. Sorry. Not too familiar with how that
> restricts memory use, but I seem to recall a similar situation quite a while
> back in which there was one parameter that could be changed & a system
> reboot performed that altered how much memory was available to a process.
> 
> My recollection was that Solaris did not have many memory management
> parameters - VMS certainly had considerably more control over process quotas
> on all sorts of resources.

The latest version of VMS on Alpha has removed the former maximum
of 4 pagefiles serving any single process.

Re: gnat and heap size

[Larry Kilgallen]

In article <yUks7.5799$ev2.9466@www.newsranger.com>, Ted Dennison<dennison@telepath.com> writes:
> In article <9osjpt$a5l$1@nh.pace.co.uk>, Marin David Condic says...
>>
>>Must have missed the Solaris 7 part. Sorry. Not too familiar with how that
>>restricts memory use, but I seem to recall a similar situation quite a while
>>back in which there was one parameter that could be changed & a system
>>reboot performed that altered how much memory was available to a process.
>>
>>My recollection was that Solaris did not have many memory management
>>parameters - VMS certainly had considerably more control over process quotas
>>on all sorts of resources.
> 
> VMS also had a limit of about 1GB if I remember right. Its memory space was
> something like 4GB, divided into 4 sections. At least that's what I remember
> from 12 years ago or so when I was using it.

The 4GB memory space comes from the 2**32 address space on VAX.

Since then they have ported to Alpha, and the current page table
arrangement does not particularly limit virtual memory to be any
smaller than the amount of paging space you can fit on a disk.
But I do not believe Compaq offers any disks with 2**64 bytes
of space.  I don't believe they even offer any machines that
could hold that much disk space in total, even if you could
afford it.

Re: gnat and heap size

[Larry Kilgallen]

> In article <yUks7.5799$ev2.9466@www.newsranger.com>, Ted Dennison<dennison@telepath.com> writes:

>> VMS also had a limit of about 1GB if I remember right. Its memory space was
>> something like 4GB, divided into 4 sections. At least that's what I remember
>> from 12 years ago or so when I was using it.

GNAT for VMS is Alpha-only, but there is still a possibility it is
operating within the older VAX virtual memory scheme, in which
case even on the largest Alpha heap space would be limited to 1 GB.

If you encounter that as a problem, contact Sales@GNAT.com :-)

Re: gnat and heap size

[Marin David Condic]

I suspect that 2**64 is probably large enough to count up all of the atoms
in the entire universe or something similarly absurdly large. Or at least
all the iron oxide molecules on the planet Earth. :-) Something my
calculator computes as 1.844674407 * 10**19. (18 quintillion, if I'm
counting zeros correctly?)

 I doubt that it would be practically possible to construct a disk big
enough to hold 2**64 bytes of data as a result. :-)

MDC
--
Marin David Condic
Senior Software Engineer
Pace Micro Technology Americas    www.pacemicro.com
Enabling the digital revolution
e-Mail:    marin.condic@pacemicro.com
Web:      http://www.mcondic.com/


"Larry Kilgallen" <Kilgallen@SpamCop.net> wrote in message
news:RUhYu4tuVLZU@eisner.encompasserve.org...
>
> Since then they have ported to Alpha, and the current page table
> arrangement does not particularly limit virtual memory to be any
> smaller than the amount of paging space you can fit on a disk.
> But I do not believe Compaq offers any disks with 2**64 bytes
> of space.  I don't believe they even offer any machines that
> could hold that much disk space in total, even if you could
> afford it.

Re: gnat and heap size

[Ted Dennison]

In article <Pine.NEB.4.31.0109260932510.24582-100000@behemoth.dreo.dnd.ca>,
Claude Marinier says...
>
>The application is an electromagnetic simulation. Yes, we could use sparse
>matrix techniques but part of the program requires a full matrix. Perhaps
>we are hitting a low level limit (is that what Adrian is saying?).

If the small program you showed has the problem, then try the same thing with a
small C program. If *it* has the problem too, then you have an OS issue that you
should really take up with Solaris experts. If it doesn't have the problem, then
at the least you can call "malloc" from Ada to get your storage. :-)

---
T.E.D.    homepage   - http://www.telepath.com/dennison/Ted/TED.html
          home email - mailto:dennison@telepath.com

Re: gnat and heap size

[Larry Kilgallen]

In article <9osq5a$crn$1@nh.pace.co.uk>, "Marin David Condic" <dont.bother.mcondic.auntie.spam@[acm.org> writes:
> I suspect that 2**64 is probably large enough to count up all of the atoms
> in the entire universe or something similarly absurdly large. Or at least
> all the iron oxide molecules on the planet Earth. :-) Something my
> calculator computes as 1.844674407 * 10**19. (18 quintillion, if I'm
> counting zeros correctly?)
> 
>  I doubt that it would be practically possible to construct a disk big
> enough to hold 2**64 bytes of data as a result. :-)

Certainly not such a disk that would fit into Robert Dewar's famous laptop.

Re: gnat and heap size

[Marin David Condic]

"Larry Kilgallen" <Kilgallen@SpamCop.net> wrote in message
news:VtbOFeY4lZR2@eisner.encompasserve.org...
>
> Certainly not such a disk that would fit into Robert Dewar's famous
laptop.

Is this leading up to some sort of thological question? "Can IBM make a disk
so big that it won't end up in Robert Dewar's laptop?" :-)

--
Marin David Condic
Senior Software Engineer
Pace Micro Technology Americas    www.pacemicro.com
Enabling the digital revolution
e-Mail:    marin.condic@pacemicro.com
Web:      http://www.mcondic.com/

Re: gnat and heap size

[Marin David Condic]

Just because I got curious, I found this reference:

http://pages.prodigy.net/jhonig/bignum/qauniver.html

Seems we've got to get to around 10**78..10**81 to approximate the number of
atoms in the universe. But I don't think that this indicates we'd want to
try to build a disk drive with 2**64 bytes on it any time soon... :-)

MDC
--
Marin David Condic
Senior Software Engineer
Pace Micro Technology Americas    www.pacemicro.com
Enabling the digital revolution
e-Mail:    marin.condic@pacemicro.com
Web:      http://www.mcondic.com/


"Marin David Condic" <dont.bother.mcondic.auntie.spam@[acm.org> wrote in
message news:9osq5a$crn$1@nh.pace.co.uk...
> I suspect that 2**64 is probably large enough to count up all of the atoms
> in the entire universe or something similarly absurdly large. Or at least
> all the iron oxide molecules on the planet Earth. :-) Something my
> calculator computes as 1.844674407 * 10**19. (18 quintillion, if I'm
> counting zeros correctly?)
>

Re: gnat and heap size

[Erik Johannessen]

Ted Dennison<dennison@telepath.com> writes:

> In article <Pine.NEB.4.31.0109260932510.24582-100000@behemoth.dreo.dnd.ca>,
> Claude Marinier says...
> >
> >The application is an electromagnetic simulation. Yes, we could use sparse
> >matrix techniques but part of the program requires a full matrix. Perhaps
> >we are hitting a low level limit (is that what Adrian is saying?).
> 
> If the small program you showed has the problem, then try the same
> thing with a small C program. If *it* has the problem too, then you
> have an OS issue that you should really take up with Solaris
> experts. If it doesn't have the problem, then at the least you can

There's no problem with malloc()'ing a 9000*9000*4 byte array on solaris,
provided you have that much vmem available.

In Claude's example if the array is declared as a constrained type
gnat will warn that a storage error will be raised at runtime.  As far
as I can tell this happens when the size of the array in bits is
larger than the largest positive value for a signed 32-bit integer.

When using new to allocate the array gnat will apparently not do any
checks for this. For the 9000x9000 array of float example it will
request a negative size from gnat_malloc/malloc.  An interesting
consequence of this is that if you continue to increase the size of
the array it will eventually get positive again, but the size will of
course not match the real size of the array.

The fun part here is that you can stay within the bounds you've
declared for the array, but access memory outside of the area
allocated for the array.  If you've got other data on the heap after
the array you'll get some interesting bugs. :)

The following code demonstrates the problem

with ada.text_io;

procedure matrix is
   type matrix is array ( integer range <> ) of integer;
   type matrix_a is access matrix;
   a: matrix_a;
   b: matrix_a;
begin
   a := New matrix( 1..152000000 );
   b := New matrix( 1..100 );

   for i in b'range loop
      b(i) := 0;
   end loop;

   a(17782318) := 42;

   for i in b'range loop
      if b(i) = 42 then
         ada.text_io.put_line("Found 42 at index " &
                              integer'image(i) & " in b");
      end if;
      b(i) := 0;
   end loop;

end matrix;

Compiled with gnat-3.14a1 on sparc-solaris2.6 I get:
> ./matrix
Found 42 at index  42 in b


A quick glance at the gnat rm/ug did not turn up any information on
this.  Also, I haven't checked if there is a compiler switch which
will turn on checks for this.

-Erik
-- 
/d{def}def/m{mul}d/a{add}d/q{repeat}d/y 1 d 300{/x 1 d 600{/c x 600 div 4 m 2
sub d/z y 300 div 2 m 1 sub d/r 0 d/i 0 d/t 0 d{t 1 le{/n r r m i i m sub c a
d/b 2 r m i m z a d/r n d/i b d r r m i i m a 16 gt{exit}if} {exit}ifelse/t t 
.01 a d}loop t setgray x y moveto 1 1 rlineto stroke/x x 1 a d}q /y y 1 a d}q

Re: gnat and heap size

[Pascal Obry]

"Marin David Condic" <dont.bother.mcondic.auntie.spam@[acm.org> writes:

>  I doubt that it would be practically possible to construct a disk big
> enough to hold 2**64 bytes of data as a result. :-)

Well well... Somebody said that only 3 or 4 ENIAC would be needed in the
world because it was so powerful... 

What about holographic memory :)

<<
Holographic memory offers the possibility of storing one terabyte (TB) of data
in a sugar-cube-sized crystal. A terabyte of data equals 1,000 gigabytes, 1
million megabytes or 1 trillion bytes. Data from more than 1,000 CDs could fit
on a holographic memory system. Most computer hard drives only hold 10 to 40
GB of data, a small fraction of what a holographic memory system might
hold. There are several reasons for developing a three-dimensional data
storage. It will be able to store more information in a smaller space and
offer faster data transfer times. In this edition of How Stuff Will Work, you
will learn how a holographic storage system might be built in the next three
or four years, and what it will take to make a desktop version of such a
high-density storage system.
>>

Ok, that's only 1 terabyte on a sugar-sized disk... But what about next
revolution, the one that will dwarfed out the holographic memory :)

Pascal.

-- 

--|------------------------------------------------------
--| Pascal Obry                           Team-Ada Member
--| 45, rue Gabriel Peri - 78114 Magny Les Hameaux FRANCE
--|------------------------------------------------------
--|         http://perso.wanadoo.fr/pascal.obry
--|
--| "The best way to travel is by means of imagination"

Re: gnat and heap size

[Dr Adrian Wrigley]

Erik Johannessen wrote:
...
> The fun part here is that you can stay within the bounds you've
> declared for the array, but access memory outside of the area
> allocated for the array.  If you've got other data on the heap after
> the array you'll get some interesting bugs. :)
...

This seems to be the same problem I had, except I was finding
unexpected "aliases" between different fields in a record.
Potentially nasty, because it can happen in otherwise well tested
code used with unusually large data.  Be aware of and work around
the various limitations.  Doesn't GNAT 3.14 fix some of these?

Can (any of) you confirm whether the problem occurs with GNAT on
64-bit processors (Alpha etc.)?
--
Adrian Wrigley

Re: gnat and heap size

[Martin Dowie]

Marin David Condic <dont.bother.mcondic.auntie.spam@[acm.org> wrote in
message news:9ot75l$i64$1@nh.pace.co.uk...
> Just because I got curious, I found this reference:
>
> http://pages.prodigy.net/jhonig/bignum/qauniver.html
>
> Seems we've got to get to around 10**78..10**81 to approximate the number
of
> atoms in the universe. But I don't think that this indicates we'd want to
> try to build a disk drive with 2**64 bytes on it any time soon... :-)

Why would you stop at wanting to count things as big as an atom?... ;-)

Re: gnat and heap size

[Erik Johannessen]

Erik Johannessen <erik.johannessen@kongsberg.com> writes:

> The fun part here is that you can stay within the bounds you've
> declared for the array, but access memory outside of the area
> allocated for the array.  If you've got other data on the heap after
> the array you'll get some interesting bugs. :)
> 
> The following code demonstrates the problem

[...]

> Compiled with gnat-3.14a1 on sparc-solaris2.6 I get:
> > ./matrix
> Found 42 at index  42 in b

And in this example the array bounds for b can be overwritten as well
(since they are stored right before the first array element).

-Erik
-- 
/d{def}def/m{mul}d/a{add}d/q{repeat}d/y 1 d 300{/x 1 d 600{/c x 600 div 4 m 2
sub d/z y 300 div 2 m 1 sub d/r 0 d/i 0 d/t 0 d{t 1 le{/n r r m i i m sub c a
d/b 2 r m i m z a d/r n d/i b d r r m i i m a 16 gt{exit}if} {exit}ifelse/t t 
.01 a d}loop t setgray x y moveto 1 1 rlineto stroke/x x 1 a d}q /y y 1 a d}q

Re: gnat and heap size

[Ole-Hjalmar Kristensen]

Ted Dennison<dennison@telepath.com> writes:

> In article <9oqs5k$jjq$1@nh.pace.co.uk>, Marin David Condic says...
> >
> >It would help to know what hardware/OS this is trying to run on since this
> ..
> >It wouldn't necessarily have to be a kick-a** machine to run something
> >needing 800mb of memory. Assuming it had virtual memory and sufficient disk
> >space, you might be able to get it to work. Albeit, the page swapping would
> >probably kill you, but it might actually run in a finite amount of time.
> 
> Well, he did say Solaris 7. That narrows it down a bit. If its one of those new
> Starcat servers, it might even have way more than that just in RAM alone....
> 
> ---
> T.E.D.    homepage   - http://www.telepath.com/dennison/Ted/TED.html
>           home email - mailto:dennison@telepath.com

Why would he need one of the new Starcat servers?

SunOS maestro 5.7 Generic_106541-17 sun4u sparc SUNW,Ultra-4
load averages:  1.09,  1.07,  1.16                                     12:38:58
429 processes: 424 sleeping, 4 zombie, 1 on cpu
CPU states:     % idle,     % user,     % kernel,     % iowait,     % swap
Memory: 4096M real, 279M free, 831M swap in use, 7398M swap free

-- 
Kabelsalat ist gesund.

Ole-Hj. Kristensen

Re: gnat and heap size

[Marin David Condic]

Sure. But assuming that if you've got an ability to address 2**64 of
anything (bytes, sectors, whatever) then there must be something there
physically you want to retrieve. If its bytes, then you'd need something
storing the binary states of the bits. Presuming that you had a means of
detecting some sort of state change in a single atom, then you'd need 2**64
* 8 (or whatever your byte size is, for those about to observe that a byte
need not be 8 bits) atoms. That might start approaching something a little
too big to get into a laptop. :-)

I wouldn't mind having a 64 bit address - it would allow you to dedicate
whole banks of the address space to various purposes - but actually having
2*64 bytes available to address may be just a wee bit beyond current
technology.

I like holographic memory and remember reading about research on such
devices several years ago. I'm wondering why it never made it out of the
lab? Too expensive? Too hard to produce? Too unreliable? Something must have
got in its way to the market because its been a long time since I heard tell
of the devices working in labs and I don't see any on the shelves at
CompUSA... Hmmmmm.....

MDC
--
Marin David Condic
Senior Software Engineer
Pace Micro Technology Americas    www.pacemicro.com
Enabling the digital revolution
e-Mail:    marin.condic@pacemicro.com
Web:      http://www.mcondic.com/


"Pascal Obry" <p.obry@wanadoo.fr> wrote in message
news:ulmj1fyac.fsf@wanadoo.fr...
>
> "Marin David Condic" <dont.bother.mcondic.auntie.spam@[acm.org> writes:
>
> >  I doubt that it would be practically possible to construct a disk big
> > enough to hold 2**64 bytes of data as a result. :-)
>
> Well well... Somebody said that only 3 or 4 ENIAC would be needed in the
> world because it was so powerful...
>

Re: gnat and heap size

[Gerald Kasner]

> 
> > Compiled with gnat-3.14a1 on sparc-solaris2.6 I get:
> > > ./matrix
> > Found 42 at index  42 in b
> 
> And in this example the array bounds for b can be overwritten as well
> (since they are stored right before the first array element).
> 

RedHat Linux 6.2 on a sparc sun 4 ultra, gnat 3.12p :

No output. 

:-)

-Gerald

Re: gnat and heap size

[Peter F. Gath]

Erik Johannessen wrote:
> 
> 
> The following code demonstrates the problem
> 
<...>

With gnat 3.13p on WindowsNT I do not get any output.

-- 
____________________________________________________________

Peter F. Gath    University of Stuttgart
Dipl.-Ing.       Institute of Flight Mechanics and Control
                 Pfaffenwaldring 7a
                 D-70550 Stuttgart
                 Germany
                 Tel.: +49 711 - 6 85 - 66 67
                 Fax : +49 711 - 6 85 - 66 70

Email: peter.gath@ifr.uni-stuttgart.de
http://www.ifr.uni-stuttgart.de/
____________________________________________________________

Holographic memory (Was: gnat and heap size)

[Jacob Sparre Andersen]

Marin:

> I like holographic memory and remember reading about research on such
> devices several years ago. I'm wondering why it never made it out of the
> lab? Too expensive? Too hard to produce? Too unreliable? Something must have
> got in its way to the market because its been a long time since I heard tell
> of the devices working in labs and I don't see any on the shelves at
> CompUSA... Hmmmmm.....

I think the main problem is that the holographic memory
readers/writers still are slightly too large for the shelves
in CompUSA. The system at Ris� National Laboratory was close
to three cubic meter (most of it air, though) last time I
saw it.

Jacob
-- 
"There is nothing worse than having only one drunk head."

Re: gnat and heap size

[Dale Pennington]

You might note, that is one sugar cube was one terrabyte (i.e. 10**12
bytes), then it would take a MegaSugarCube with approximately 262 sugar
cubes to a side to hold 2**64 bytes of data. I am not sure of the dimensions
of a sugar cube, but if one were to assume a 0.25" sugar cube, that would be
be about 5 1/2" per side cube. Note too bad for a fixed site, but a but
large for a laptop.

"Marin David Condic" <dont.bother.mcondic.auntie.spam@[acm.org> wrote in
message news:9ov8a4$b71$1@nh.pace.co.uk...
> Sure. But assuming that if you've got an ability to address 2**64 of
> anything (bytes, sectors, whatever) then there must be something there
> physically you want to retrieve. If its bytes, then you'd need something
> storing the binary states of the bits. Presuming that you had a means of
> detecting some sort of state change in a single atom, then you'd need
2**64
> * 8 (or whatever your byte size is, for those about to observe that a byte
> need not be 8 bits) atoms. That might start approaching something a little
> too big to get into a laptop. :-)
>
> I wouldn't mind having a 64 bit address - it would allow you to dedicate
> whole banks of the address space to various purposes - but actually having
> 2*64 bytes available to address may be just a wee bit beyond current
> technology.
>
> I like holographic memory and remember reading about research on such
> devices several years ago. I'm wondering why it never made it out of the
> lab? Too expensive? Too hard to produce? Too unreliable? Something must
have
> got in its way to the market because its been a long time since I heard
tell
> of the devices working in labs and I don't see any on the shelves at
> CompUSA... Hmmmmm.....
>
> MDC
> --
> Marin David Condic
> Senior Software Engineer
> Pace Micro Technology Americas    www.pacemicro.com
> Enabling the digital revolution
> e-Mail:    marin.condic@pacemicro.com
> Web:      http://www.mcondic.com/
>
>
> "Pascal Obry" <p.obry@wanadoo.fr> wrote in message
> news:ulmj1fyac.fsf@wanadoo.fr...
> >
> > "Marin David Condic" <dont.bother.mcondic.auntie.spam@[acm.org> writes:
> >
> > >  I doubt that it would be practically possible to construct a disk big
> > > enough to hold 2**64 bytes of data as a result. :-)
> >
> > Well well... Somebody said that only 3 or 4 ENIAC would be needed in the
> > world because it was so powerful...
> >
>
>
>

Re: gnat and heap size

[Darren New]

Dale Pennington wrote:
> You might note, that is one sugar cube was one terrabyte (i.e. 10**12
> bytes), then it would take a MegaSugarCube with approximately 262 sugar
> cubes to a side to hold 2**64 bytes of data. I am not sure of the dimensions
> of a sugar cube, but if one were to assume a 0.25" sugar cube, that would be
> be about 5 1/2" per side cube. Note too bad for a fixed site, but a but
> large for a laptop.

Not really on topic, but...

When I read about these, they worked by focussing lasers thru the cube
to intersect at a point, causing the die in the cube to change color. I
don't imagine it would work as well through 5.5" of cube as it would
through 0.25" of cube. It was also write-once.

-- 
Darren New 
San Diego, CA, USA (PST). Cryptokeys on demand.
                   Who is this Dr. Ibid anyway, 
                  and how does he know so much?

Re: gnat and heap size

[Marin David Condic]

Are we getting dangerously close to discussing how many angels can dance on
the head of a pin? :-)

BTW: I think a sugar cube is a little closer to a half inch - maybe 3/4
inch - also depending on the particular brand. (Some are square while others
are more "Domino" shaped... Pun intended.)

I'll bet that we will continue to see some really major advances in storage
technology in the years to come. A 64-bit address is probably a little large
for anything that is likely to hit the market in the next couple of years,
so why design in that limitation? But is it reasonably forseeable that we
might want 128 bit addresses for anything? At some point, that starts
becoming the construction of the proverbial brick outhouse....

MDC
--
Marin David Condic
Senior Software Engineer
Pace Micro Technology Americas    www.pacemicro.com
Enabling the digital revolution
e-Mail:    marin.condic@pacemicro.com
Web:      http://www.mcondic.com/


"Dale Pennington" <Dale.K.Pennington@boeing.com> wrote in message
news:GKBxLF.ICJ@news.boeing.com...
> You might note, that is one sugar cube was one terrabyte (i.e. 10**12
> bytes), then it would take a MegaSugarCube with approximately 262 sugar
> cubes to a side to hold 2**64 bytes of data. I am not sure of the
dimensions
> of a sugar cube, but if one were to assume a 0.25" sugar cube, that would
be
> be about 5 1/2" per side cube. Note too bad for a fixed site, but a but
> large for a laptop.
>

Re: gnat and heap size

[erij]

Gerald Kasner <Gerald.Kasner@Physik.Uni-Magdeburg.DE> writes:

> > 
> > > Compiled with gnat-3.14a1 on sparc-solaris2.6 I get:
> > > > ./matrix
> > > Found 42 at index  42 in b
> > 
> > And in this example the array bounds for b can be overwritten as well
> > (since they are stored right before the first array element).
> > 
> 
> RedHat Linux 6.2 on a sparc sun 4 ultra, gnat 3.12p :
> 
> No output. 

The output of this program will depend on the heap layout of the
particular os you are running on.  By increasing the size of b (to a
few thousand) and adding a few thousand to the index in a I write to I
was able to reproduce the output in linux-x86 with gnat 3.13p as well.

-Erik
-- 
/d{def}def/m{mul}d/a{add}d/q{repeat}d/y 1 d 300{/x 1 d 600{/c x 600 div 4 m 2
sub d/z y 300 div 2 m 1 sub d/r 0 d/i 0 d/t 0 d{t 1 le{/n r r m i i m sub c a
d/b 2 r m i m z a d/r n d/i b d r r m i i m a 16 gt{exit}if} {exit}ifelse/t t 
.01 a d}loop t setgray x y moveto 1 1 rlineto stroke/x x 1 a d}q /y y 1 a d}q

Re: gnat and heap size

[erij]

Dr Adrian Wrigley <amtw@linuxchip.demon.co.uk> writes:

> Erik Johannessen wrote:
> ...
> > The fun part here is that you can stay within the bounds you've
> > declared for the array, but access memory outside of the area
> > allocated for the array.  If you've got other data on the heap after
> > the array you'll get some interesting bugs. :)
> ...
> 
> This seems to be the same problem I had, except I was finding
> unexpected "aliases" between different fields in a record.
> Potentially nasty, because it can happen in otherwise well tested
> code used with unusually large data.  Be aware of and work around
> the various limitations.  Doesn't GNAT 3.14 fix some of these?

As I wrote in my first post, this was tested with gnat-3.14.  I'm not
sure if ACT is aware of this (they should be now).  Our main gnat
person at work wasn't there today so I couldn't write a bug report.

-Erik
-- 
/d{def}def/m{mul}d/a{add}d/q{repeat}d/y 1 d 300{/x 1 d 600{/c x 600 div 4 m 2
sub d/z y 300 div 2 m 1 sub d/r 0 d/i 0 d/t 0 d{t 1 le{/n r r m i i m sub c a
d/b 2 r m i m z a d/r n d/i b d r r m i i m a 16 gt{exit}if} {exit}ifelse/t t 
.01 a d}loop t setgray x y moveto 1 1 rlineto stroke/x x 1 a d}q /y y 1 a d}q

Re: gnat and heap size

[tmoran]

>be about 5 1/2" per side cube. Note too bad for a fixed site, but a but
>large for a laptop.
  Disk storage capacities are going about 60%/yr, so 2**64 bytes should be
available in about 50 years.  Price is supposedly dropping even faster,
so it should be pretty cheap by then.