Bug in GNAT? - Max_size_in_storage_elements insufficient

Bug in GNAT? - Max_size_in_storage_elements insufficient

[Vratislav Podzimek]

When trying to set Storage_Size for tasks or records I'm getting a crash 
indicating that the Max_size_in_storage_elements attribute doesn't seem 
to provide the correct value. Here are the simple reproducers:

1) task type

with Ada.Text_Io; use Ada.Text_Io;

procedure Task_Storage_Size is
   task type My_Task (Id : Natural);    --  works just fine without the 
discriminant
   task body My_Task is
      X : Natural := Id;
   begin
      X := X ** 10;

      Put_Line ("ID : " & Natural'Image (Id));
   end;

   type My_Task_Access is access My_Task
     with Storage_Size => 4 * My_Task'Max_size_in_storage_elements; --  
Crashes
--     with Storage_Size => 6 * My_Task'Max_size_in_storage_elements; -- 
Doesn't crash.

   type My_Task_Array is array (Natural range <>) of My_Task_Access;
   Tasks : My_Task_Array (1..4);
begin
   --  raises STORAGE_ERROR : s-poosiz.adb:259 explicit raise
   for I in Tasks'Range loop
      Tasks (I) := new My_Task (I);
   end loop;
end;


2) record type:

procedure Record_Storage_Size is
   type My_Record is
      record
         X : Natural := 0;
      end record;

   type My_Record_Access is access My_Record
     with Storage_Size => 4 * My_Record'Max_size_in_storage_elements; -- 
crashes
     -- with Storage_Size => 8 * My_Record'Max_size_in_storage_elements; 
-- doesn't crash

   type My_Record_Array is array (Natural range <>) of My_Record_Access;
   Records : My_Record_Array (1..4);
begin
   --  raises STORAGE_ERROR : s-poosiz.adb:108 explicit raise
   for I in Records'Range loop
      Records (I) := new My_Record;
   end loop;
end;


To safe us from a potential discussion that has already happened on IRC, 
here's the related chunk from the RM:
"If Storage_Size is specified for an access type, then the Storage_Size 
of this pool is at least that requested, and the storage for the pool is 
reclaimed when the master containing the declaration of the access type 
is left"

So Storage_Size specification for an access type specifies the amount of 
space for the allocated objects of the accessed type. However, it can 
also be used for a task type (not access to a task type) to specify the 
amount of space for a task's stack.

Or am I missing something?

Re: Bug in GNAT? - Max_size_in_storage_elements insufficient

[Dmitry A. Kazakov]

On 18/10/2016 13:17, Vratislav Podzimek wrote:

> Or am I missing something?

A storage pool requires additional space to organize its structure and 
maintain its state. Depending on the method it is usually impossible to 
estimate the exact number of object  the pool may hold. It depends on 
the individual sizes of the objects, the history of object's allocation 
and deallocation, the history of claiming the memory for the parts of 
the pool from the OS (a pool can be segmented).

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: Bug in GNAT? - Max_size_in_storage_elements insufficient

[Vratislav Podzimek]

On Tue, 18 Oct 2016 14:35:15 +0200, Dmitry A. Kazakov wrote:

> On 18/10/2016 13:17, Vratislav Podzimek wrote:
> 
>> Or am I missing something?
> 
> A storage pool requires additional space to organize its structure and
> maintain its state. Depending on the method it is usually impossible to
> estimate the exact number of object  the pool may hold. It depends on
> the individual sizes of the objects, the history of object's allocation
> and deallocation, the history of claiming the memory for the parts of
> the pool from the OS (a pool can be segmented).

Well, then it's at least a bug in [Barnes] because there's an example 
like the above ones just with a simpler type. But I still think the idea 
here is/was to provide enough space for N items of some type. That's what 
the 'Max_size_in_storage_elements' attribute should be good for. And 
there's no fragmentation nor history of allocation/deallocation in my 
case. If I specify I want space for 4 records, I should get space for 4 
records. Not slightly less due to some hidden, unpredictable overhead.

Re: Bug in GNAT? - Max_size_in_storage_elements insufficient

[Dmitry A. Kazakov]

On 2016-10-18 17:26, Vratislav Podzimek wrote:
> On Tue, 18 Oct 2016 14:35:15 +0200, Dmitry A. Kazakov wrote:
>
>> On 18/10/2016 13:17, Vratislav Podzimek wrote:
>>
>>> Or am I missing something?
>>
>> A storage pool requires additional space to organize its structure and
>> maintain its state. Depending on the method it is usually impossible to
>> estimate the exact number of object  the pool may hold. It depends on
>> the individual sizes of the objects, the history of object's allocation
>> and deallocation, the history of claiming the memory for the parts of
>> the pool from the OS (a pool can be segmented).
>
> Well, then it's at least a bug in [Barnes] because there's an example
> like the above ones just with a simpler type. But I still think the idea
> here is/was to provide enough space for N items of some type.

I am not sure if that were technically possible. Considering an 
implementation of, for example, a marked-margins pool, the penalty to 
have a guarantee would be extremely high.

> That's what
> the 'Max_size_in_storage_elements' attribute should be good for. And
> there's no fragmentation nor history of allocation/deallocation in my
> case.

You don't know the implementation of. A segmented pool may request 
system memory on demand adapting the size of segments to the allocation 
requests: Max (Chunk size, Requested size).

> If I specify I want space for 4 records, I should get space for 4
> records. Not slightly less due to some hidden, unpredictable overhead.

If you know the pool size in advance and the objects in it, why don't 
you use a custom pool backed by an array? An arena pool would be exactly 
for this and trivial to implement in Ada.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: Bug in GNAT? - Max_size_in_storage_elements insufficient

[Luke A. Guest]

Dmitry A. Kazakov <mailbox@dmitry-kazakov.de> wrote:

>> Well, then it's at least a bug in [Barnes] because there's an example
>> like the above ones just with a simpler type. But I still think the idea
>> here is/was to provide enough space for N items of some type.
> 
> I am not sure if that were technically possible. Considering an 
> implementation of, for example, a marked-margins pool, the penalty to 
> have a guarantee would be extremely high.

Surely that's what a static language can supply, the actual amount of space
required?

Re: Bug in GNAT? - Max_size_in_storage_elements insufficient

[Dmitry A. Kazakov]

On 19/10/2016 03:39, Luke A. Guest wrote:
> Dmitry A. Kazakov <mailbox@dmitry-kazakov.de> wrote:
>
>>> Well, then it's at least a bug in [Barnes] because there's an example
>>> like the above ones just with a simpler type. But I still think the idea
>>> here is/was to provide enough space for N items of some type.
>>
>> I am not sure if that were technically possible. Considering an
>> implementation of, for example, a marked-margins pool, the penalty to
>> have a guarantee would be extremely high.
>
> Surely that's what a static language can supply, the actual amount of space
> required?

In the mentioned above method you allocate a "margin" in front and in 
the end of each block. The margin contains an indicator if the block is 
free and the block size. When a block is freed the margins of adjacent 
blocks are inspected and the block is merged with its free neighbors. 
The memory overhead depends on the number of blocks, which in turn 
depends on the number of allocated objects and on how their blocks are 
distributed. In the worst case scenario you have a gap between each two 
allocated blocks. Which gives 2 * (2*N - 1) - 2 margins to keep in the pool.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: Bug in GNAT? - Max_size_in_storage_elements insufficient

[Randy Brukardt]

"Luke A. Guest" <laguest@archeia.com> wrote in message 
news:647636100.498533813.258228.laguest-archeia.com@nntp.aioe.org...
> Dmitry A. Kazakov <mailbox@dmitry-kazakov.de> wrote:
>
>>> Well, then it's at least a bug in [Barnes] because there's an example
>>> like the above ones just with a simpler type. But I still think the idea
>>> here is/was to provide enough space for N items of some type.
>>
>> I am not sure if that were technically possible. Considering an
>> implementation of, for example, a marked-margins pool, the penalty to
>> have a guarantee would be extremely high.
>
> Surely that's what a static language can supply, the actual amount of 
> space
> required?

Depends on the implementation of the storage pool, of course. The amount of 
memory used by objects can be figured out at compile-time (assuming that the 
subtype is definite and there aren't any discriminant-dependent components - 
it's possible to declare such objects non-contiguously), but the pool will 
have some overhead.

Besides, that's irrelevant. S'Max_Storage_Size_in_Elements is not a static 
attribute and thus doesn't have to be calculatable at compile-time. The OP's 
example is legal because the value of Storage_Size doesn't have to be 
static, either. Of course, that means that whatever the OP is trying to do 
probably isn't going to happen here.

Turning to the OP's question:

I would never expect to be able say anything useful about the size of a task 
object - it's going to have to be allocated in pieces all over the memory 
map (there's limits on where the stack can be allocated, for instance). No 
attribute (Size, Object_Size, Max_Storage_Size_in_Elements) can have 
anything useful to say about non-contiguous objects.

Note: I've always interpreted Max_Storage_Size_in_Elements to apply to a 
single call to Allocate; there might be multiple calls to Allocate to 
allocate a single object (Janus/Ada works this way; there is a separate call 
to Allocate for each contiguous part of an object). Indeed, I wrote the 
definition of the attribute with this model in mind. Whether any other Ada 
lawyers would agree with me is unclear. :-)

                          Randy.

Re: Bug in GNAT? - Max_size_in_storage_elements insufficient

[Eryndlia Mavourneen]

On Tuesday, October 18, 2016 at 10:26:28 AM UTC-5, Vratislav Podzimek wrote:
> Well, then it's at least a bug in [Barnes] because there's an example 
> like the above ones just with a simpler type. But I still think the idea 
> here is/was to provide enough space for N items of some type. That's what 
> the 'Max_size_in_storage_elements' attribute should be good for. And 
> there's no fragmentation nor history of allocation/deallocation in my 
> case. If I specify I want space for 4 records, I should get space for 4 
> records. Not slightly less due to some hidden, unpredictable overhead.

There is another use for the attribute Max_Size_In_Storage_Elements.  That is to allow a dynamic allocation on the stack for an object whose size and subtype you may not know until run-time.  This can occur when receiving messages over a communications link or when reading from a loosely-defined file.  This type of thing, otherwise, is not good design and is questionable even in this case.

-- Eryndlia

Re: Bug in GNAT? - Max_size_in_storage_elements insufficient

[Eryndlia Mavourneen]

On Tuesday, October 18, 2016 at 10:26:28 AM UTC-5, Vratislav Podzimek wrote:
> Well, then it's at least a bug in [Barnes] because there's an example 
> like the above ones just with a simpler type. But I still think the idea 
> here is/was to provide enough space for N items of some type. That's what 
> the 'Max_size_in_storage_elements' attribute should be good for. And 
> there's no fragmentation nor history of allocation/deallocation in my 
> case. If I specify I want space for 4 records, I should get space for 4 
> records. Not slightly less due to some hidden, unpredictable overhead.

There is another use for the attribute Max_Size_In_Storage_Elements.  That is to allow a dynamic allocation on the stack for an object whose size and subtype you may not know until run-time and for mapping using along with using the 'Address attribute for mapping these subtypes.  This can occur when receiving messages over a communications link or when reading from a loosely-defined file.  This type of thing, otherwise, is not good design and is questionable even in this case. 

-- Eryndlia

Re: Bug in GNAT? - Max_size_in_storage_elements insufficient

[Egil H H]

Using Max_Size_In_Storage_Elements in this way is asking for trouble, and not at all portable, as the actual storage needed may depend on Alignment and fragmentation.
However, looking at GNAT's implementation (GNATPro 7.4.1), there are a couple quirks;

First off, the storage pool used by GNAT in this case ends up in a buffer on the stack. Unallocated parts of the buffer is implemented as a kind of overlayed linked list of chunks. This list always needs at least one root node, with a size field and pointer to the next node (on my 64 bit machine, the node is a total of 16 storage_elements), so that's several bytes worth of extra space needed, which 'Max_Size_In_Storage_Elements does not (and can not) take into account. I'll leave it up to AdaCore to answer whether this is the intended behaviour.

Secondly, when allocating an object in the pool, there is a search for a large enough chunk. This search checks if the chunk size is less than (not less than or equal to) the requested size + alignment. So the actual chunk selected must be at least one byte larger than requested, which must be a bug...

Knowing this, I got your code code working on my machine by doing
Storage_Size => (Num_Tasks * My_Task'Max_Size_In_Storage_Elements) + 17
This is of course not at all portable across compilers, not even across GNAT versions... And certainly not portable across machine architectures. 

So, one oddity and a likely bug... You may as well report it...

-- 
~egilhh