Large arrays passed to arithmetic operators overflows GNAT stack

Large arrays passed to arithmetic operators overflows GNAT stack

[Jerry]

For what it's worth, passing large numerical arrays to the arithmetic
operators in GNAT can cause the stack to overflow (segfault) even when
the arrays are allocated from the heap. I suppose that indicates that
they are being copied rather than being passed by reference.

For example, on OS X which has a default stack size of 8192 KB, the
following program segfaults when N is greater than about 1_048_138
(about 8 MB per Long_Float array) but runs OK when it is somewhat less
than that number.

with
    Ada.Numerics.Long_Real_Arrays;
use
    Ada.Numerics.Long_Real_Arrays;
procedure array_test is
    type Real_Vector_Access    is access Real_Vector;
    N : Integer := 1_048_130;
    t_Ptr : Real_Vector_Access := new Real_Vector(0 .. N);
      t : Real_Vector renames t_Ptr.all;
    t_Diff_Ptr : Real_Vector_Access := new Real_Vector(0 .. N - 1);
      t_Diff : Real_Vector renames t_Diff_Ptr.all;
begin
    for i in t'range loop
        t(i) := 1.0;
    end loop;
    t_Diff := t(1 .. N) - t(0 .. N - 1);
end array_test;

The quick fix (in my case) is to increase the stack size from the
shell:

ulimit -s 65532

or whatever is appropriate for your machine--65532 is the hard limit
set by OS X.

Another way to expand the stack without invoking shell commands, as
noted by Björn Persson on this thread

http://groups.google.com/group/comp.lang.ada/browse_thread/thread/ae395e5c11de7bc9/bda8d61bd3a66ee9?hl=en&q=Jerry+stack&lnk=nl&

is to call getrlimit and/or setrlimit from within the program, linking
to these POSIX C routines.

Another fix would be to write operator procedures that specifically
take pointers as arguments; that would possibly be the only fix for
arrays that still overflow the stack when it is maxed out.

As usual, taking the dumb-guy approach, this seems like an unnecessary
nuisance.

Jerry

Re: Large arrays passed to arithmetic operators overflows GNAT stack

[Vinzent Hoefler]

Jerry wrote:

> with
>     Ada.Numerics.Long_Real_Arrays;
> use
>     Ada.Numerics.Long_Real_Arrays;
> procedure array_test is
>     type Real_Vector_Access    is access Real_Vector;
>     N : Integer := 1_048_130;
[...]
> begin
>     for i in t'range loop
>         t(i) := 1.0;
>     end loop;
>     t_Diff := t(1 .. N) - t(0 .. N - 1);
> end array_test;

It's probably not the operator call that crashes, but the creation of
the temporary arrays t(...). Couple of days ago, I stumbled upon a
similar problem with aggegrate assignments:

    --  Do array initialization in a loop.
    --  If it's done with an aggregate assignment, GNAT raises Storage_Error
    --  during elaboration.

> Another fix would be to write operator procedures that specifically
> take pointers as arguments; that would possibly be the only fix for
> arrays that still overflow the stack when it is maxed out.

This wouldn't help here, I suppose.


Vinzent.

-- 
Beaten by the odds since 1974.

Re: Large arrays passed to arithmetic operators overflows GNAT stack

[Jeffrey Creem]

On 12/4/2010 1:32 AM, Jerry wrote:
> For what it's worth, passing large numerical arrays to the arithmetic
> operators in GNAT can cause the stack to overflow (segfault) even when
> the arrays are allocated from the heap. I suppose that indicates that
> they are being copied rather than being passed by reference.
>

In general, GNAT (and most other compilers) don't copy arguments like 
large arrays on the parameters to subprograms. The problem here is 
almost certainly in the return value associated with the subtraction. I 
am not sure I'd say it is impossible but I don't think I have seen any 
Ada implementation that can avoid the creation of the temporary based on 
the internally defined local variable that is generally required to 
construct code the returns an array.

Even if certain cases could be found by the optimizer and made to work, 
there are all sorts of cases where users would end up being surprised 
when temporaries had to be created.

In cases where array slices, assignment and the math involved 'destroy' 
the future values as the current values are being read, the compiler 
would have no choice (given the semantics of the operator) but to 
introduce some temporaries and it would be quite a feat for a compiler 
to figure out how much of a mini-slice temp it would need.

I really think it is a shame that Generic_Real_Arrays was defined the 
way that it was as the pretty code one gets when being able to use infix 
notation is nice but the overhead involved in the resulting copies 
renders the operators useless for all but the smallest of array vectors.

Re: Large arrays passed to arithmetic operators overflows GNAT stack

[Peter C. Chapin]

On 2010-12-04 01:32, Jerry wrote:

> with
>     Ada.Numerics.Long_Real_Arrays;
> use
>     Ada.Numerics.Long_Real_Arrays;
> procedure array_test is
>     type Real_Vector_Access    is access Real_Vector;
>     N : Integer := 1_048_130;
>     t_Ptr : Real_Vector_Access := new Real_Vector(0 .. N);
>       t : Real_Vector renames t_Ptr.all;

I don't think you need this renaming. If you do t_Ptr'Range or t_Ptr(i)
the compiler will forward those operations directly to the object.

>     t_Diff_Ptr : Real_Vector_Access := new Real_Vector(0 .. N - 1);
>       t_Diff : Real_Vector renames t_Diff_Ptr.all;
> begin
>     for i in t'range loop
>         t(i) := 1.0;
>     end loop;
>     t_Diff := t(1 .. N) - t(0 .. N - 1);

I believe a temporary object (on the stack) needs to be created here to
hold the result of the difference before it gets assigned to t_Diff. So
despite the fact that the result is ultimately stored on the heap it
ends up passing through the stack on its way. If I'm correct it's not
the subtraction operator itself that is causing the problem but rather
what is happening with the result.

In short "-" is returning the array by value and not an access to the
array. The compiler has to figure out what to do with that value before
assigning it to t_Diff and it is using a stack temporary.

Peter

Re: Large arrays passed to arithmetic operators overflows GNAT stack

[Jerry]

On Dec 4, 7:17 am, "Peter C. Chapin" <PCha...@vtc.vsc.edu> wrote:
> On 2010-12-04 01:32, Jerry wrote:
>
> > with
> >     Ada.Numerics.Long_Real_Arrays;
> > use
> >     Ada.Numerics.Long_Real_Arrays;
> > procedure array_test is
> >     type Real_Vector_Access    is access Real_Vector;
> >     N : Integer := 1_048_130;
> >     t_Ptr : Real_Vector_Access := new Real_Vector(0 .. N);
> >       t : Real_Vector renames t_Ptr.all;
>
> I don't think you need this renaming. If you do t_Ptr'Range or t_Ptr(i)
> the compiler will forward those operations directly to the object.
>  ...
> Peter

That is true but the renaming is still convenient for other reasons.
For example,
the lines

t      : Real_Vector_Access := new Real_Vector(0 .. N);
t_Diff : Real_Vector_Access := new Real_Vector(0 .. N - 1);
...
[Line 14] t_Diff := t(1 .. N);

causes the complaints

expected type "Real_Vector_Access" defined at line 14
found type "Ada.Numerics.Generic_Real_Arrays.Real_Vector" from
instance at a-nlrear.ads:18

And passing t to a vector cosine (e.g.) function that expects to see a
Real_Vector will also fail. Of course, with two like pointers t and x,
assignment  t := x assigns the pointers, not the array contents, and
with three like pointers t, x, y, t := x - y fails to find an
applicable operator function.

This clever renaming trick was mentioned by Brian Drummond in this
thread:
http://groups.google.com/group/comp.lang.ada/browse_thread/thread/ae395e5c11de7bc9/bda8d61bd3a66ee9?hl=en&q=Jerry+stack&lnk=nl&

Jerry