Re: Large arrays (again), problem case for GNAT

["Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de>]

On 14 Apr 2005 17:19:31 -0400, Robert A Duff wrote:

> "Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> writes:
> 
>> On 14 Apr 2005 11:18:22 -0400, Robert A Duff wrote:
>> 
>>> The reason I like allocate-on-write is that it is very useful to
>>> allocate a huge array, and then only use some part of it.  That way, the
>>> program can deal with enormous input data, without paying the cost when
>>> the input data is small.  Physical memory is 1000 times more expensive
>>> than address space.
>> 
>> I think this approach is flawed. The contract of malloc is to return memory
>> not the address space. 
> 
> Well, I'm not an expert on malloc.  Where is it documented that this is
> the contract?

OK, let's call it an implied contract. The result of malloc is either 0 or
a valid pointer as long as C does not distinguish pointers and addresses.

> Apparently the Linux folks don't think that's the contract.

Yes I know. I participated in discussions on this issue 10+ years ago in
Linux groups. Nothing has changed since then. It is still badly wrong.

> I *am* an expert on Ada, and I know that "new" does not necessarily
> reserve real memory -- it might just reserve address space.

I think that this should be changed. Consider a tagged object.
Storage_Error could be raised in Initialize! Even worse, if the constructor
does not touch all bits, then you might get a properly constructed object
which is potentially unusable. It is a disaster. This is what people have
with that large arrays.

[ BTW, as a palliative, what about the attribute X'Valid, where X is a
pointer. X'Valid should be false in all cases when X.all is not properly
allocated (may result in Storage_Error.) ]

>>...That must be just another system call.
> 
> I agree that there *should* be two separate calls: one reserves address
> space, and the other reserves real memory.  Or one call with a Boolean
> parameter.
> 
>> Consider an application that needs contiguous address space. User storage
>> pool is an excellent example. The application must request the space from
>> OS telling something like: "reserve me 100GB and allocate n bytes starting
>> from the first address." After that it could allocate memory *within* that
>> space by pieces as needed. For this it would simply use realloc instead of
>> malloc.
> 
> No, I don't want to do realloc or anything like that.  I just want it to
> allocate physical memory (and backing store) when I write upon that
> memory.

You cannot write into memory you don't have. Moreover, either in Ada or in
any other higher level language one simply cannot define the term "memory
write" in a way which would allow any reasonable handling of "write
errors". But in the case of the storage pool your program *knows* that it
does not have this memory. So it can allocate it using realloc without any
assumptions about OS behavior which might appear wrong.

[ Here I mean imaginary malloc and realloc and not Linux ones. The contract
or realloc should be that if I call it on a pointer which memory block is
followed by free address space, then the result will be the same pointer if
the additional memory is allocated and fits into the space ]

> Apparently, malloc on Linux is exactly the "reserve me 100GB
> and allocate n bytes starting from the first address" primitive, except
> there's no need for the "allocate n bytes starting from the first
> address" part.  And it does exactly what I want in the storage-pool
> example.

But your storage pool will be simply unusable for many applications. And
that is independent on the fact that Storage_Error cannot be handled.
Consider an application that allocates something in your pool and then
starts series of transactions with a data base. In the middle of this
business it will crash leaving data in some unpredictable state! I wouldn't
equalize this scenario to memory check-sum error etc.

> I understand that other examples would prefer to prevent Storage_Error
> happening willy-nilly all over the place.  Ada does not provide that
> semantics, and apparently Linux does not either, unless you set some
> global flag.

I think that Ada should differentiate Storage_[Access]_Error and
Storage_[Allocation]_Error.

>>... No need to break contracts.
> 
> I don't think Linux (or GNAT upon Linux) is breaking any contract.
> You just don't like what the contract defines! Fair enough.
>
> - Bob
> 
> P.S. Nobody has challenged my remark that Storage_Error handlers are
> impossible.  ;-)

Maybe because you are right! (:-))

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de