GNAT Allocation of a very large record

GNAT Allocation of a very large record

[hyunghwan.chung]

Hi, 

The program at the bottom of this message, when compiled with GNAT 4.6 on Ubuntu12/x86_64, seems to corrupt memory, ending up with a malloc error message.

$ ./x1
1. Kind: POINTER_OBJECT Size:  10
x1: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed.
^C

On some other platforms (i.e. debian/armv5tel, gcc/gnat 4.4.5), just segmentation fault.

I expect an output like this, meaning that the second call to 'new' should raise an exception like storage_error.

$ ./x1
1. Kind: POINTER_OBJECT Size:  10
2. Allocation Failed

You'd get the result like above if you changed the upper bound of the range of Pointer_Object_Size from Storage_Count'Last to  (OBJECT_DATA_BYTES / (Object_Pointer'Max_Size_In_Storage_Elements * System.Storage_Unit)).

The new upper bound has been defined so that Pointer_Object_Record'Size don't exceed 2 ** 63 - 1 (System.Max_Int). On an 32-bit platform, the upper bound seems to be able to reach OBJECT_DATA_BYTES / Object_Pointer'Max_Size_In_Storage_Elements. With this upper bound on the 32-bit platform, Pointer_Object_Record'Size may go beyond 2 ^ 31 - 1 but Pointer_Object_Record'Max_Size_In_Storage_Elements falls below 2 ^ 31 - 1. 

Anyway, when the upper bound is set to Storage_Count'Last, Pointer_Object_Record'Max_Size_In_Storage_Elements and Pointer_Object_Record'Size seem to wrap around to an undesired value

The problem is that neither Constraint_Error nor Storage_Error is raised when the size of a record is very large. I'd like to have a graceful way to catch a ridiculously large size given to the function and handle the exception properly without ending up with segmentation fault or something similar.

Is it a bug of the GNU Ada compiler or am i doing anything wrong?

Thanks,
Hyung-Hwan

------------------------------------------------------------------------------- 
with Ada.Text_IO; 
with System;
with Ada.Unchecked_Conversion;

procedure X1 is

	type Storage_Element is mod 2 ** System.Storage_Unit;
	for Storage_Element'Size use System.Storage_Unit;
	type Storage_Offset is range -(2 ** (System.Word_Size - 1)) ..  +(2 ** (System.Word_Size - 1)) - 1;
	subtype Storage_Count is Storage_Offset range 0 .. Storage_Offset'Last;

	subtype Object_Byte      is Storage_Element;
	subtype Object_Character is Wide_Character;
	subtype Object_String    is Wide_String;

	type Object_Kind is (Pointer_Object, Character_Object, Byte_Object);
	for Object_Kind use (Pointer_Object => 0, Character_Object => 1, Byte_Object => 2);

	type Object_Record;
	type Object_Pointer is access all Object_Record;

	type Byte_Array is array (Storage_Count range <>) of Object_Byte;
	type Character_Array is array (Storage_Count range <>) of Object_Character;
	type Pointer_Array is array (Storage_Count range <>) of Object_Pointer;

	type Object_Record (Kind: Object_Kind; Size: Storage_Count) is record
		Flags: Standard.Integer range 0 .. 3;-- := 0;
		Extra: Standard.Integer range 0 .. 1;-- := 0;
		Unit:  Standard.Integer range 0 .. 4;-- := 0;
		Class: Object_Pointer;-- := null;

		case Kind is
			when Pointer_Object =>
				Pointer_Slot: Pointer_Array (1 .. Size);-- := (Others => null);
			when Character_Object =>
				Character_Slot: Character_Array (0 .. Size);-- := (Others => Object_Character'Val(0));
			when Byte_Object =>
				Byte_Slot: Byte_Array (1 .. Size);-- := (Others => 0);
		end case;
	end record;
	for Object_Record use record
		Kind  at 0 range 0 .. 7;
		Flags at 1 range 0 .. 2;
		Extra at 1 range 3 .. 3;
		Unit  at 1 range 4 .. 7;
	end record;

	subtype Empty_Object_Record is Object_Record (Byte_Object, 0);
	-- the number of bytes in an object header. this is fixed in size
	OBJECT_HEADER_BYTES: constant Storage_Count := Empty_Object_Record'Max_Size_In_Storage_Elements;
	-- the largest number of bytes that an object can hold after the header
	OBJECT_DATA_BYTES: constant Storage_Count := Storage_Count'Last - OBJECT_HEADER_BYTES;

	subtype Pointer_Object_Size is Storage_Count range
		Storage_Count'First .. (OBJECT_DATA_BYTES / (Object_Pointer'Max_Size_In_Storage_Elements * System.Storage_Unit));
	--	Storage_Count'First .. Storage_Count'Last;

	procedure Alloc_Pointer_Object (Size: in Pointer_Object_Size; Result: out Object_Pointer) is
		subtype Pointer_Object_Record is Object_Record (Pointer_Object, Size);
		type Pointer_Object_Pointer is access Pointer_Object_Record;
		function To_Object_Pointer is new Ada.Unchecked_Conversion (Pointer_Object_Pointer, Object_Pointer);
		Ptr: Pointer_Object_Pointer;
	begin
   		--Ada.Text_IO.Put_Line (Pointer_Object_Record'Size'Img);
   		--Ada.Text_IO.Put_Line (Pointer_Object_Record'Max_Size_In_Storage_Elements'Img);
		Ptr := new Pointer_Object_Record'( 
			Kind => Pointer_Object, 
			Size => Size,
			Flags => 0,
			Extra => 0,
			Unit => 0,
			Class => null,
			Pointer_Slot => (others=>null)
		);

		Result := To_Object_Pointer (Ptr);
	exception
		when others =>
			Result := null;
	end Alloc_Pointer_Object;


	ObjPtr: Object_Pointer;
begin

	Alloc_Pointer_Object (10, ObjPtr);
	if ObjPtr = null then
		Ada.Text_IO.Put_Line ("1. Allocation Failed");
	else
		Ada.Text_IO.Put_Line ("1. Kind: " & Object_Kind'Image(ObjPtr.Kind)  & " Size: " & Storage_Count'Image(ObjPtr.Size));
	end if;

	Alloc_Pointer_Object (Pointer_Object_Size'Last, ObjPtr);
	if ObjPtr = null then
		Ada.Text_IO.Put_Line ("2. Allocation Failed");
	else
		Ada.Text_IO.Put_Line ("2. Kind: " & Object_Kind'Image(ObjPtr.Kind)  & " Size: " & Storage_Count'Image(ObjPtr.Size));
	end if;

end X1;

Re: GNAT Allocation of a very large record

[Per Sandberg]

Tried 4 different GCC:s and one failed:
gcc 4.4.7 20120313 (Red Hat 4.4.7-3) (GCC) x86 ok
gcc 4.6.3 20120306 (Red Hat 4.6.3-2) (GCC) x86_64 ok
gcc 4.7.3 20130318 for GNAT Pro 7.2.0w (20130317) (GCC) x86_64 ok
gcc 4.4.5 (Debian 4.4.5-8) arm (raspberry-pi) fail

It seems as there is a bug in your particular instance of GCC as well
as the GCC for the pi.


/Per




On Tue, 13 Aug 2013 20:50:21 -0700 (PDT)
hyunghwan.chung@gmail.com wrote:

> Hi, 
> 
> The program at the bottom of this message, when compiled with GNAT
> 4.6 on Ubuntu12/x86_64, seems to corrupt memory, ending up with a
> malloc error message.
> 
> $ ./x1
> 1. Kind: POINTER_OBJECT Size:  10
> x1: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr)
> (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct
> malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size)
> >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk,
> >fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 *
> >(sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned
> >long)old_end & pagemask) == 0)' failed. ^C
> 
> On some other platforms (i.e. debian/armv5tel, gcc/gnat 4.4.5), just
> segmentation fault.
> 
> I expect an output like this, meaning that the second call to 'new'
> should raise an exception like storage_error.
> 
> $ ./x1
> 1. Kind: POINTER_OBJECT Size:  10
> 2. Allocation Failed
> 
> You'd get the result like above if you changed the upper bound of the
> range of Pointer_Obje



ct_Size from Storage_Count'Last to
> (OBJECT_DATA_BYTES / (Object_Pointer'Max_Size_In_Storage_Elements *
> System.Storage_Unit)).
> 
> The new upper bound has been defined so that
> Pointer_Object_Record'Size don't exceed 2 ** 63 - 1 (System.Max_Int).
> On an 32-bit platform, the upper bound seems to be able to reach
> OBJECT_DATA_BYTES / Object_Pointer'Max_Size_In_Storage_Elements. With
> this upper bound on the 32-bit platform, Pointer_Object_Record'Size
> may go beyond 2 ^ 31 - 1 but
> Pointer_Object_Record'Max_Size_In_Storage_Elements falls below 2 ^ 31
> - 1. 
> 
> Anyway, when the upper bound is set to Storage_Count'Last,
> Pointer_Object_Record'Max_Size_In_Storage_Elements and
> Pointer_Object_Record'Size seem to wrap around to an undesired value
> 
> The problem is that neither Constraint_Error nor Storage_Error is
> raised when the size of a record is very large. I'd like to have a
> graceful way to catch a ridiculously large size given to the function
> and handle the exception properly without ending up with segmentation
> fault or something similar.
> 
> Is it a bug of the GNU Ada compiler or am i doing anything wrong?
> 
> Thanks,
> Hyung-Hwan
> 
> ------------------------------------------------------------------------------- 
> with Ada.Text_IO; 
> with System;
> with Ada.Unchecked_Conversion;
> 
> procedure X1 is
> 
> 	type Storage_Element is mod 2 ** System.Storage_Unit;
> 	for Storage_Element'Size use System.Storage_Unit;
> 	type Storage_Offset is range -(2 ** (System.Word_Size -
> 1)) ..  +(2 ** (System.Word_Size - 1)) - 1; subtype Storage_Count is
> Storage_Offset range 0 .. Storage_Offset'Last;
> 
> 	subtype Object_Byte      is Storage_Element;
> 	subtype Object_Character is Wide_Character;
> 	subtype Object_String    is Wide_String;
> 
> 	type Object_Kind is (Pointer_Object, Character_Object,
> Byte_Object); for Object_Kind use (Pointer_Object => 0,
> Character_Object => 1, Byte_Object => 2);
> 
> 	type Object_Record;
> 	type Object_Pointer is access all Object_Record;
> 
> 	type Byte_Array is array (Storage_Count range <>) of
> Object_Byte; type Character_Array is array (Storage_Count range <>)
> of Object_Character; type Pointer_Array is array (Storage_Count range
> <>) of Object_Pointer;
> 
> 	type Object_Record (Kind: Object_Kind; Size: Storage_Count)
> is record Flags: Standard.Integer range 0 .. 3;-- := 0;
> 		Extra: Standard.Integer range 0 .. 1;-- := 0;
> 		Unit:  Standard.Integer range 0 .. 4;-- := 0;
> 		Class: Object_Pointer;-- := null;
> 
> 		case Kind is
> 			when Pointer_Object =>
> 				Pointer_Slot: Pointer_Array (1 ..
> Size);-- := (Others => null); when Character_Object =>
> 				Character_Slot: Character_Array (0 ..
> Size);-- := (Others => Object_Character'Val(0)); when Byte_Object =>
> 				Byte_Slot: Byte_Array (1 ..
> Size);-- := (Others => 0); end case;
> 	end record;
> 	for Object_Record use record
> 		Kind  at 0 range 0 .. 7;
> 		Flags at 1 range 0 .. 2;
> 		Extra at 1 range 3 .. 3;
> 		Unit  at 1 range 4 .. 7;
> 	end record;
> 
> 	subtype Empty_Object_Record is Object_Record (Byte_Object, 0);
> 	-- the number of bytes in an object header. this is fixed in
> size OBJECT_HEADER_BYTES: constant Storage_Count :=
> Empty_Object_Record'Max_Size_In_Storage_Elements; -- the largest
> number of bytes that an object can hold after the header
> OBJECT_DATA_BYTES: constant Storage_Count := Storage_Count'Last -
> OBJECT_HEADER_BYTES;
> 
> 	subtype Pointer_Object_Size is Storage_Count range
> 		Storage_Count'First .. (OBJECT_DATA_BYTES /
> (Object_Pointer'Max_Size_In_Storage_Elements * System.Storage_Unit));
> --	Storage_Count'First .. Storage_Count'Last;
> 
> 	procedure Alloc_Pointer_Object (Size: in Pointer_Object_Size;
> Result: out Object_Pointer) is subtype Pointer_Object_Record is
> Object_Record (Pointer_Object, Size); type Pointer_Object_Pointer is
> access Pointer_Object_Record; function To_Object_Pointer is new
> Ada.Unchecked_Conversion (Pointer_Object_Pointer, Object_Pointer);
> Ptr: Pointer_Object_Pointer; begin
>    		--Ada.Text_IO.Put_Line
> (Pointer_Object_Record'Size'Img); --Ada.Text_IO.Put_Line
> (Pointer_Object_Record'Max_Size_In_Storage_Elements'Img); Ptr := new
> Pointer_Object_Record'( Kind => Pointer_Object, 
> 			Size => Size,
> 			Flags => 0,
> 			Extra => 0,
> 			Unit => 0,
> 			Class => null,
> 			Pointer_Slot => (others=>null)
> 		);
> 
> 		Result := To_Object_Pointer (Ptr);
> 	exception
> 		when others =>
> 			Result := null;
> 	end Alloc_Pointer_Object;
> 
> 
> 	ObjPtr: Object_Pointer;
> begin
> 
> 	Alloc_Pointer_Object (10, ObjPtr);
> 	if ObjPtr = null then
> 		Ada.Text_IO.Put_Line ("1. Allocation Failed");
> 	else
> 		Ada.Text_IO.Put_Line ("1. Kind: " &
> Object_Kind'Image(ObjPtr.Kind)  & " Size: " &
> Storage_Count'Image(ObjPtr.Size)); end if;
> 
> 	Alloc_Pointer_Object (Pointer_Object_Size'Last, ObjPtr);
> 	if ObjPtr = null then
> 		Ada.Text_IO.Put_Line ("2. Allocation Failed");
> 	else
> 		Ada.Text_IO.Put_Line ("2. Kind: " &
> Object_Kind'Image(ObjPtr.Kind)  & " Size: " &
> Storage_Count'Image(ObjPtr.Size)); end if;
> 
> end X1;

Re: GNAT Allocation of a very large record

[Anh Vo]

On Wednesday, August 14, 2013 12:32:10 PM UTC-7, Per Sandberg wrote:
> Tried 4 different GCC:s and one failed:
> 
> gcc 4.4.7 20120313 (Red Hat 4.4.7-3) (GCC) x86 ok 
> gcc 4.6.3 20120306 (Red Hat 4.6.3-2) (GCC) x86_64 ok 
> gcc 4.7.3 20130318 for GNAT Pro 7.2.0w (20130317) (GCC) x86_64 ok 
> gcc 4.4.5 (Debian 4.4.5-8) arm (raspberry-pi) fail

It works fine with GNAT-GPL-2013 on Windows and GNATPro 7.1.1 on Red Hat x86.

A. Vo

Re: GNAT Allocation of a very large record

[Hyung-Hwan Chung]

On Thursday, August 15, 2013 5:08:32 AM UTC+9, Anh Vo wrote:
> On Wednesday, August 14, 2013 12:32:10 PM UTC-7, Per Sandberg wrote:
> 
> > Tried 4 different GCC:s and one failed:
> > 
> > gcc 4.4.7 20120313 (Red Hat 4.4.7-3) (GCC) x86 ok 
> > gcc 4.6.3 20120306 (Red Hat 4.6.3-2) (GCC) x86_64 ok 
> > gcc 4.7.3 20130318 for GNAT Pro 7.2.0w (20130317) (GCC) x86_64 ok 
> > gcc 4.4.5 (Debian 4.4.5-8) arm (raspberry-pi) fail
> 
> It works fine with GNAT-GPL-2013 on Windows and GNATPro 7.1.1 on Red Hat x86.
>  
> A. Vo


I tried gcc 4.4.7 20120313 (Red Hat 4.4.7-2) (GCC) x86_64 and it seemed to work find originally.  Soon later, I wanted to confirm this at a slightly lower-level and realized that it gave me the illusion that it worked.

It raised the STORAGE_ERROR exception. But the key here is that how it was raised.

Strace reveals that there was a segmentation fault. You can guess that the program catches SIGSEGV from rt_sigaction (SIGSEGV,...) at the beginning.

$ strace ./x1
--<snip>--
rt_sigaction(SIGSEGV, {0x41ac90, [], SA_RESTORER|SA_STACK|SA_RESTART|SA_NODEFER|SA_SIGINFO, 0x7fd6ba93a920}, NULL, 8) = 0
fstat(2, {st_mode=S_IFREG|0664, st_size=2182, ...}) = 0
fstat(0, {st_mode=S_IFCHR|0620, st_rdev=makedev(4, 2), ...}) = 0
fstat(1, {st_mode=S_IFCHR|0620, st_rdev=makedev(4, 2), ...}) = 0
brk(0)                                  = 0xd23000
brk(0xd44000)                           = 0xd44000
write(1, "1. Kind: POINTER_OBJECT Size:  1"..., 34) = 34
--- SIGSEGV (Segmentation fault) @ 0 (0) ---
brk(0xd65000)                           = 0xd65000
write(1, "2. Allocation Failed\n", 21)  = 21
exit_group(0)                           = ?
 
Ltrace reveals that the second attempt to 'new' that translated to malloc eventually requested 16 bytes only. You can also print the Pointer_Object_Record'Max_Size_In_Storage_Elements from within the program. I used ltrace to see the real value  passed to the malloc function. This comes down to the overflow/wrap-around issue.

$ ltrace ./x1
-- <snip> --
malloc(104)                                      = 0x18cd010
memcpy(0x7fff0c80a6d0, "1. Kind: ", 9)           = 0x7fff0c80a6d0
memcpy(0x7fff0c80a6d9, "POINTER_OBJECT", 14)     = 0x7fff0c80a6d9
memcpy(0x7fff0c80a6e7, " Size: ", 7)             = 0x7fff0c80a6e7
memcpy(0x7fff0c80a6ee, " 10", 3)                 = 0x7fff0c80a6ee
memcpy(0x6302a8, "1. Kind: POINTER_OBJECT Size:  1"..., 33) = 0x6302a8
memcpy(0x7fff0c80a790, "1. Kind: POINTER_OBJECT Size:  1"..., 33) = 0x7fff0c80a790
fwrite("1. Kind: POINTER_OBJECT Size:  1"..., 34, 1, 0x7f630f8ad780) = 1
malloc(16)                                       = 0x18cd080
--- SIGSEGV (Segmentation fault) ---
malloc(64)                                       = 0x18ee010
-- <snip> --

The assembly listing for the 'new' allocation part is shown here. It was produced with the command 'gcc -S x1.adb'. You can see that it initializes the record after memory allocation. I assume that __gnat_malloc contains the check for the actual malloc failure. For this case, malloc was given 16 so I'm sure it has reached the part after 'call __gnat_malloc'. The line 'movq $0, 16(%rax)' violates the memory access for the first time as it's accessing beyond the 16 byte block allocated. I believe that the line is for 'Class => Null'. If it carried on, it would execute the code after the label .L53 starting from a few lines above it in a loop for 'Pointer_Slot => (others => null)'. That would also violate the memory access.

     --<snip>--
     call __gnat_malloc
     movb $0, (%rax)
     movq -840(%rbp), %rdx
     movq %rdx, 8(%rax)
     movzbl    1(%rax), %edx
     andl $-8, %edx
     movb %dl, 1(%rax)
     movzbl    1(%rax), %edx
     andl $-9, %edx
     movb %dl, 1(%rax)
     movzbl    1(%rax), %edx
     andl $15, %edx
     movb %dl, 1(%rax)
     movq $0, 16(%rax) 
     movq -840(%rbp), %rdx
     movq %rdx, %rcx
     testq     %rcx, %rcx
     jle  .L52
     movl $1, %edx
.L53:
     movq %rdx, %rbx
     addq $1, %rbx
     movq $0, 8(%rax,%rbx,8)
     cmpq %rdx, %rcx
     je   .L52
     addq $1, %rdx
     jmp  .L53
     --<snip>--

Valgrind fails more miserably with this program. Any comments?

Thanks again,
Hyung-Hwan

Re: GNAT Allocation of a very large record

[Georg Bauhaus]

On 14.08.13 05:50, hyunghwan.chung@gmail.com wrote:
> Hi,
>
> The program at the bottom of this message, when compiled with GNAT 4.6 on Ubuntu12/x86_64, seems to corrupt memory, ending up with a malloc error message.
>
> $ ./x1
> 1. Kind: POINTER_OBJECT Size:  10
> x1: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed.
> ^C
>
> On some other platforms (i.e. debian/armv5tel, gcc/gnat 4.4.5), just segmentation fault.
>
> I expect an output like this, meaning that the second call to 'new' should raise an exception like storage_error.
>
> $ ./x1
> 1. Kind: POINTER_OBJECT Size:  10
> 2. Allocation Failed
>

For the record only, On Mac OS X, using both GNAT GPL 2012 and GCC 4.8.1
I get

$ ./x1
1. Kind: POINTER_OBJECT Size:  10
x1(3471) malloc: *** mmap(size=576460752303427584) failed (error code=12)
*** error: can't allocate region
*** set a breakpoint in malloc_error_break to debug
2. Allocation Failed


Or, with some tracing statements inserted, some in a local copy
of s-memory.adb,

$ ./x1
Size: 10
Alloc: Size as S.CRTL.size_t  104, Size 104, Actual_Size 104
1. Kind: POINTER_OBJECT Size:  10
Size: 72057594037927935
Alloc: Size as S.CRTL.size_t  576460752303423504, Size 576460752303423504, Actual_Size 576460752303423504
x1(13497) malloc: *** mmap(size=576460752303427584) failed (error code=12)
*** error: can't allocate region
*** set a breakpoint in malloc_error_break to debug
Alloc: Size as S.CRTL.size_t  656, Size 656, Actual_Size 656
Exception name: STORAGE_ERROR
Message: heap exhausted

2. Allocation Failed



Doing as the ***-lines suggest, in the debugger, the stack frames said
that System.Memory.Alloc had been called (line 92 of s-memory.adb),

  Result := c_malloc (System.CRTL.size_t (Actual_Size));


After learning about libgmalloc, I tried

$ DYLD_INSERT_LIBRARIES=/usr/lib/libgmalloc.dylib ./x1
GuardMalloc[x1-6126]: Allocations will be placed on 16 byte boundaries.
GuardMalloc[x1-6126]:  - Some buffer overruns may not be noticed.
GuardMalloc[x1-6126]:  - Applications using vector instructions (e.g., SSE) should work.
GuardMalloc[x1-6126]: version 25
1. Kind: POINTER_OBJECT Size:  10
GuardMalloc[x1-6126]: Attempting excessively large memory allocation:  576460752303423504 bytes
GuardMalloc[x1-6126]: If you really wanted to allocate so much memory, launch your executable with the environment variable MALLOC_PERMIT_INSANE_REQUESTS set to any value to circumvent this check.
GuardMalloc[x1-6126]: If you run under the debugger, it will automatically break here.

^C

The following little C program triggers the same message from the
mallocs:

#include <stdio.h>
#include <stdlib.h>

int main()
{
   size_t what = 576460752303423504;  /* 0x800000000000010 */
   void *bytes;

   fprintf(stdout, "d %ld u %lu x %lx\n", what, what, what);
   bytes = malloc(what);
   return 0;
}

The C program starts working properly on this (4GB, OS X 10.7) machine at
  what = 576460752303423504 /(1<<13).

Re: GNAT Allocation of a very large record

[Robert A Duff]

hyunghwan.chung@gmail.com writes:

> Anyway, when the upper bound is set to Storage_Count'Last,
> Pointer_Object_Record'Max_Size_In_Storage_Elements and
> Pointer_Object_Record'Size seem to wrap around to an undesired value

Do you have overflow checks enabled (-gnato)?

- Bob

Re: GNAT Allocation of a very large record

[Hyung-Hwan Chung]

On Thursday, August 15, 2013 11:27:41 PM UTC+9, Robert A Duff wrote:
> 
> > Anyway, when the upper bound is set to Storage_Count'Last,
> > Pointer_Object_Record'Max_Size_In_Storage_Elements and
> > Pointer_Object_Record'Size seem to wrap around to an undesired value
> 
> Do you have overflow checks enabled (-gnato)?
> 
> - Bob

Yes, I did. That didn't make any difference. 

GNAT, however, is able to detect it if a predictable size is in play.

The following raises the Storage_Error exception with the message "object too large". No compile time warning is given.

--<snip>--
begin
     ObjPtr := new Object_Record'(
               Kind => Pointer_Object,
               Size => Pointer_Object_Size'Last,
               Flags => 0,
               Extra => 0,
               Unit => 0,
               Class => null,
               Pointer_Slot => (others=>null)
          );
--<snip>--
end X1; 

The following raises the CONSTRAINT_ERROR exception. A compile-time warning is also issued - "warning: "Constraint_Error" will be raised at run time".

--<snip>--
     subtype Dummy_Object_Record is Object_Record (Pointer_Object, Pointer_Object_Size'Last);
begin
     Ada.Text_IO.Put_Line (Storage_Count'Image(Dummy_Object_Record'Max_Size_In_Storage_Elements));
--<snip>--
end X1;

In my case, the type definition is rather dynamic as it uses the Size parameter passed to the function.

I put this line in Alloc_Pointer_Object before 'Ptr := new Pointer_Object_Record'...'.

 Ada.Text_IO.Put_Line (Pointer_Object_Record'Max_Size_In_Storage_Elements'Img);

and generated the assembly listing, of course with -gnato enabled.

     --<snip>--
     call system__img_lli__image_long_long_integer
     movl %eax, %edx
     testl     %edx, %edx
     movl %eax, %edx
     testl     %edx, %edx
     movl %eax, %edx
     testl     %edx, %edx
     leaq -160(%rbp), %rdx
     movq %rdx, -112(%rbp)
     movl $1, -96(%rbp)
     movl %eax, -92(%rbp)
     leaq -96(%rbp), %rax
     movq %rax, -104(%rbp)
     movq -112(%rbp), %rdx
     movq -104(%rbp), %rax
     movq %rdx, %rdi
     movq %rax, %rsi
     call ada__text_io__put_line__2
     movq -808(%rbp), %rax
     salq $3, %rax
     addq $7, %rax
     andq $-8, %rax
     addq $7, %rax
     andq $-8, %rax
     addq $31, %rax
     andq $-8, %rax
     movq %rax, %rdi
     call __gnat_malloc
     --<snip>--

There is no call to the range check routines. GNAT seems to produce calls to __gnat_rcheck_XX when it performs the range check for normal cases.

     call __gnat_rcheck_00
     call __gnat_rcheck_02
     and so on.

Cheers,
Hyung-Hwan