Storage_Size in a Simple Program

Storage_Size in a Simple Program

[Charles H. Sampson]

I've got a conceptually simple program that uses a lot of memory. It is
highly recursive (using a lot of stack) and also puts a lot of stuff on
the heap. Is there any way to specify that a lot of memory is needed
other than pragma Storage_Size?

As it is, I have three totally artificial tasks hidden in packages. The
packages' entry routines are simply pass-throughs to their embedded
task's entries. There are no concurrency issues because the simple
program is single-threaded at heart.

Is that it? That's a lot of baggage just to give permission to use more
memory, particularly when there's a lot of memory lying around now.

Thanks.

Charlie
-- 
Nobody in this country got rich on his own.  You built a factory--good.
But you moved your goods on roads we all paid for.  You hired workers we
all paid to educate. So keep a big hunk of the money from your factory.
But take a hunk and pay it forward.  Elizabeth Warren (paraphrased)

Re: Storage_Size in a Simple Program

[Simon Wright]

csampson@inetworld.net (Charles H. Sampson) writes:

> I've got a conceptually simple program that uses a lot of memory. It is
> highly recursive (using a lot of stack) and also puts a lot of stuff on
> the heap. Is there any way to specify that a lot of memory is needed
> other than pragma Storage_Size?
>
> As it is, I have three totally artificial tasks hidden in packages. The
> packages' entry routines are simply pass-throughs to their embedded
> task's entries. There are no concurrency issues because the simple
> program is single-threaded at heart.
>
> Is that it? That's a lot of baggage just to give permission to use more
> memory, particularly when there's a lot of memory lying around now.

You can increase the maximum stack size (up to some system-set limit,
berween 32 MB and 64 MB here) by ulimit -s <kbytes>.

ulimit -a says the max memory size and the virtual memory are both
unlimited, but of course there has to be a limit (2**64 - 1!).

https://ss64.com/osx/ulimit.html
https://serverfault.com/questions/15564/where-are-the-default-ulimits-specified-on-os-x-10-5
https://wilsonmar.github.io/maximum-limits/

Re: Storage_Size in a Simple Program

[Jacob Sparre Andersen]

csampson@inetworld.net (Charles H. Sampson) writes:

> I've got a conceptually simple program that uses a lot of memory. It
> is highly recursive (using a lot of stack) and also puts a lot of
> stuff on the heap. Is there any way to specify that a lot of memory is
> needed other than pragma Storage_Size?

Usually programs have "unlimited" access to the heap.

Most operating systems put a limit on the stack size, but the default
limit is most likely adjustable.  On Linux I let some Ada programs run
with 4 Gb stack.

The command to change the default stack size is:

   limit stacksize 4G

(the setting is local to the shell you run the command in)

Greetings,

Jacob
-- 
"There is nothing worse than having only one drunk head."

Re: Storage_Size in a Simple Program

[joakimds]

> The command to change the default stack size is:
> 
>    limit stacksize 4G
> 

I guess a character ('u') got lost, it should be:

     ulimit stacksize 4G

Best regards,
Joakim

Re: Storage_Size in a Simple Program

[Egil H H]

On Monday, October 22, 2018 at 1:39:33 PM UTC+2, joak...@kth.se wrote:
> > The command to change the default stack size is:
> > 
> >    limit stacksize 4G
> > 
> 
> I guess a character ('u') got lost, it should be:
> 
>      ulimit stacksize 4G
> 

Depends on which shell you're using

Re: Storage_Size in a Simple Program

[Simon Wright]

Egil H H <ehh.public@gmail.com> writes:

> On Monday, October 22, 2018 at 1:39:33 PM UTC+2, joak...@kth.se wrote:
>> > The command to change the default stack size is:
>> > 
>> >    limit stacksize 4G
>> > 
>> 
>> I guess a character ('u') got lost, it should be:
>> 
>>      ulimit stacksize 4G
>> 
>
> Depends on which shell you're using

And which OS

Re: Storage_Size in a Simple Program

[Charles H. Sampson]

Charles H. Sampson <csampson@inetworld.net> wrote:

> I've got a conceptually simple program that uses a lot of memory. It is
> highly recursive (using a lot of stack) and also puts a lot of stuff on
> the heap. Is there any way to specify that a lot of memory is needed
> other than pragma Storage_Size?
> 
> As it is, I have three totally artificial tasks hidden in packages. The
> packages' entry routines are simply pass-throughs to their embedded
> task's entries. There are no concurrency issues because the simple
> program is single-threaded at heart.
> 
> Is that it? That's a lot of baggage just to give permission to use more
> memory, particularly when there's a lot of memory lying around now.

All the responses about limit/ulimit are certainly solutions, but
they're not quite what I was asking about. I was looking for a solution
written in Ada. I've been around Ada long enough to remember that a goal
was to be able to completely express programs in Ada itself without
having to worry about outside influences. (I write this knowing that the
semantics of pragma Storage_Size are not at all tight.)

To my knowledge, the current specification of Ada comes closer to that
goal than any other language.

Charlie
-- 
Nobody in this country got rich on his own.  You built a factory--good.
But you moved your goods on roads we all paid for.  You hired workers we
all paid to educate. So keep a big hunk of the money from your factory.
But take a hunk and pay it forward.  Elizabeth Warren (paraphrased)

Re: Storage_Size in a Simple Program

[Niklas Holsti]

On 18-10-24 12:07 , Charles H. Sampson wrote:
> Charles H. Sampson <csampson@inetworld.net> wrote:
>
>> I've got a conceptually simple program that uses a lot of memory. It is
>> highly recursive (using a lot of stack) and also puts a lot of stuff on
>> the heap. Is there any way to specify that a lot of memory is needed
>> other than pragma Storage_Size?
>>
>> As it is, I have three totally artificial tasks hidden in packages. The
>> packages' entry routines are simply pass-throughs to their embedded
>> task's entries. There are no concurrency issues because the simple
>> program is single-threaded at heart.
>>
>> Is that it? That's a lot of baggage just to give permission to use more
>> memory, particularly when there's a lot of memory lying around now.
>
> All the responses about limit/ulimit are certainly solutions, but
> they're not quite what I was asking about. I was looking for a solution
> written in Ada. I've been around Ada long enough to remember that a goal
> was to be able to completely express programs in Ada itself without
> having to worry about outside influences. (I write this knowing that the
> semantics of pragma Storage_Size are not at all tight.)

If you need a big stack:

- All Ada compilers I have used have had some way of defining the size 
of the stack for the environment task. But usually not by pragma 
Storage_Size, unfortunately.

- If you don't want to use the compiler-specific method to size the 
environment task stack, put all of the code into one task, with a large 
Storage_Size.

- If that doesn't work, because your shell or O/S doesn't let you have 
such a large stack, use limit/ulimit to tell the shell or O/S what you need.

- If you don't want to use limit/ulimit, and the stack is still too 
small, you are trying to run the program on a "machine" with too little 
memory resources, and no language can help.

That said, I would welcome a standard Ada way to specify the stack size 
for the environment task. Perhaps Storage_Size could be specified for 
the main subprogram, in the same way as the Priority for the environment 
task.

-- 
Niklas Holsti
Tidorum Ltd
niklas holsti tidorum fi
       .      @       .

Re: Storage_Size in a Simple Program

[joakimds]

> That said, I would welcome a standard Ada way to specify the stack size 
> for the environment task. Perhaps Storage_Size could be specified for 
> the main subprogram, in the same way as the Priority for the environment 
> task.

At "http://www.adaic.org/resources/add_content/standards/05aarm/html/AA-D-1.html" one can read: Likewise, the priority value is associated with the environment task if the pragma appears in the declarative_part of the main subprogram.

Therefore, specifying Storage_Size for the main subprogram as can be done with Priority should be technically possible.

Re: Storage_Size in a Simple Program

[Simon Wright]

joakimds@kth.se writes:

>> That said, I would welcome a standard Ada way to specify the stack
>> size for the environment task. Perhaps Storage_Size could be
>> specified for the main subprogram, in the same way as the Priority
>> for the environment task.
>
> At
> "http://www.adaic.org/resources/add_content/standards/05aarm/html/AA-D-1.html"
> one can read: Likewise, the priority value is associated with the
> environment task if the pragma appears in the declarative_part of the
> main subprogram.
>
> Therefore, specifying Storage_Size for the main subprogram as can be
> done with Priority should be technically possible.

On the other hand, I just tried

   with Ada.Text_IO; use Ada.Text_IO;
   with Interfaces;
   procedure Main_Stack is
      Size : Natural := 16;
      type Arr is array (Natural range <>) of Interfaces.Unsigned_8;
   begin
      loop
         Put_Line ("trying size" & Size'Image);
         declare
            A : Arr (1 .. Size) := (others => 0);
         begin
            A (A'Last) := 42;
         end;
         Size := Size * 2;
      end loop;
   end Main_Stack;

on this Mac (High Sierra, 8GB RAM) and it failed at 8 Mb. (ulimit -s
says 8192k).

If I say ulimit -s 16384, it fails at 16 Mb.

Since the main program's stack limit is determined by the OS, I don't
see how an Ada aspect/pragma can affect it.

Re: Storage_Size in a Simple Program

[Niklas Holsti]

On 18-10-25 18:56 , Simon Wright wrote:
> joakimds@kth.se writes:
>
>>> That said, I would welcome a standard Ada way to specify the stack
>>> size for the environment task. Perhaps Storage_Size could be
>>> specified for the main subprogram, in the same way as the Priority
>>> for the environment task.
>>
>> At
>> "http://www.adaic.org/resources/add_content/standards/05aarm/html/AA-D-1.html"
>> one can read: Likewise, the priority value is associated with the
>> environment task if the pragma appears in the declarative_part of the
>> main subprogram.
>>
>> Therefore, specifying Storage_Size for the main subprogram as can be
>> done with Priority should be technically possible.
>
> On the other hand, I just tried
>
>    with Ada.Text_IO; use Ada.Text_IO;
>    with Interfaces;
>    procedure Main_Stack is
>       Size : Natural := 16;
>       type Arr is array (Natural range <>) of Interfaces.Unsigned_8;
>    begin
>       loop
>          Put_Line ("trying size" & Size'Image);
>          declare
>             A : Arr (1 .. Size) := (others => 0);
>          begin
>             A (A'Last) := 42;
>          end;
>          Size := Size * 2;
>       end loop;
>    end Main_Stack;
>
> on this Mac (High Sierra, 8GB RAM) and it failed at 8 Mb. (ulimit -s
> says 8192k).
>
> If I say ulimit -s 16384, it fails at 16 Mb.
>
> Since the main program's stack limit is determined by the OS, I don't
> see how an Ada aspect/pragma can affect it.

In bare-machine systems, the environment task's stack size is usually 
set in the linker command file, or possibly by a linker option, and 
there is no OS to veto it.

The desire here is to have a standard Ada way of saying (in the Ada 
program itself) what the size _should_ be for the program to work. 
Whether or not the OS agrees to provide that size, or whether the OS has 
to be told to provide it (with ulimit) is a secondary issue.

If some linker/OS combo makes it impossible for a program to define its 
own main-task stack size (within the permitted limit), perhaps the Ada 
program should just check, at start-up, that it _has_ been given enough 
space (that is, at least as much as specified in the program) and if 
not, the program should fail immediately with Storage_Error, say.

At present, there is an Ada way to specify, in the Ada program, how much 
heap memory a certain access type needs, and how much stack a certain 
task needs -- except for the environment task.

-- 
Niklas Holsti
Tidorum Ltd
niklas holsti tidorum fi
       .      @       .

Re: Storage_Size in a Simple Program

[joakimds]

> Since the main program's stack limit is determined by the OS, I don't
> see how an Ada aspect/pragma can affect it.

Thinking more about his I can't help but wonder if it is possible to circumvent using limit/ulimit by taking advantage of getrlimit/setrlimit:
https://stackoverflow.com/questions/2279052/increase-stack-size-in-linux-with-setrlimit
The Ada run-time could perhaps take advantage of getrlimit/setrlimit on Linux to modify the size of the stack as the first step of elaboration.

On Windows it may be possible to specify the size of stack at link time. Consider for example the Windows linker that has a flag /Stack to be able to set the size of the stack in bytes: https://msdn.microsoft.com/en-us/library/y0zzbyt4.aspx

Best regards,
Joakim

Re: Storage_Size in a Simple Program

[Anh Vo]

On Wednesday, October 24, 2018 at 2:07:55 AM UTC-7, Charles H. Sampson wrote:
> Charles H. Sampson <csampson@inetworld.net> wrote:
> 
> > I've got a conceptually simple program that uses a lot of memory. It is
> > highly recursive (using a lot of stack) and also puts a lot of stuff on
> > the heap. Is there any way to specify that a lot of memory is needed
> > other than pragma Storage_Size?
> > 
> > As it is, I have three totally artificial tasks hidden in packages. The
> > packages' entry routines are simply pass-throughs to their embedded
> > task's entries. There are no concurrency issues because the simple
> > program is single-threaded at heart.
> > 
> > Is that it? That's a lot of baggage just to give permission to use more
> > memory, particularly when there's a lot of memory lying around now.
> 
> All the responses about limit/ulimit are certainly solutions, but
> they're not quite what I was asking about. I was looking for a solution
> written in Ada. I've been around Ada long enough to remember that a goal
> was to be able to completely express programs in Ada itself without
> having to worry about outside influences. (I write this knowing that the
> semantics of pragma Storage_Size are not at all tight.)
> 
> To my knowledge, the current specification of Ada comes closer to that
> goal than any other language.
 
How about use task wrapper and pragma Storage_Size (...) combo.

Anh Vo

Re: Storage_Size in a Simple Program

[Randy Brukardt]

<joakimds@kth.se> wrote in message 
news:2635fb12-5836-4713-998d-e8179b801500@googlegroups.com...
>> Since the main program's stack limit is determined by the OS, I don't
>> see how an Ada aspect/pragma can affect it.
...
> On Windows it may be possible to specify the size of stack at link time. 
> Consider for example the Windows linker that has a flag /Stack to be able 
> to set the size of the stack in bytes: 
> https://msdn.microsoft.com/en-us/library/y0zzbyt4.aspx

That's the only way I know for Windows. Janus/Ada has a binder option that 
sets the stack size in the generated .OBJ file, and the Windows linker then 
puts that in the executable. I believe that GNAT has something similar on 
Windows (haven't had to try that to date).

                                        Randy.

Re: Storage_Size in a Simple Program

[Niklas Holsti]

On 18-10-29 22:58 , Randy Brukardt wrote:
> <joakimds@kth.se> wrote in message
> news:2635fb12-5836-4713-998d-e8179b801500@googlegroups.com...
>>> Since the main program's stack limit is determined by the OS, I don't
>>> see how an Ada aspect/pragma can affect it.
> ...
>> On Windows it may be possible to specify the size of stack at link time.
>> Consider for example the Windows linker that has a flag /Stack to be able
>> to set the size of the stack in bytes:
>> https://msdn.microsoft.com/en-us/library/y0zzbyt4.aspx
>
> That's the only way I know for Windows. Janus/Ada has a binder option that
> sets the stack size in the generated .OBJ file, and the Windows linker then
> puts that in the executable. I believe that GNAT has something similar on
> Windows (haven't had to try that to date).

That sounds as if the environment stack size _could_ theoretically be 
specified in the Ada source, for the Janus/Ada compiler to export to the 
Janus/Ada binder, which would then set it in the .OBJ for the Windws 
linker. Still we would need a standard way (a pragma or aspect, no 
doubt) to specify the size in the Ada source. That way is currently 
missing, AFAIK.

-- 
Niklas Holsti
Tidorum Ltd
niklas holsti tidorum fi
       .      @       .

Re: Storage_Size in a Simple Program

[Randy Brukardt]

"Niklas Holsti" <niklas.holsti@tidorum.invalid> wrote in message 
news:g3rprtFfc9dU1@mid.individual.net...
> On 18-10-29 22:58 , Randy Brukardt wrote:
>> <joakimds@kth.se> wrote in message
>> news:2635fb12-5836-4713-998d-e8179b801500@googlegroups.com...
>>>> Since the main program's stack limit is determined by the OS, I don't
>>>> see how an Ada aspect/pragma can affect it.
>> ...
>>> On Windows it may be possible to specify the size of stack at link time.
>>> Consider for example the Windows linker that has a flag /Stack to be 
>>> able
>>> to set the size of the stack in bytes:
>>> https://msdn.microsoft.com/en-us/library/y0zzbyt4.aspx
>>
>> That's the only way I know for Windows. Janus/Ada has a binder option 
>> that
>> sets the stack size in the generated .OBJ file, and the Windows linker 
>> then
>> puts that in the executable. I believe that GNAT has something similar on
>> Windows (haven't had to try that to date).
>
> That sounds as if the environment stack size _could_ theoretically be 
> specified in the Ada source, for the Janus/Ada compiler to export to the 
> Janus/Ada binder, which would then set it in the .OBJ for the Windws 
> linker. Still we would need a standard way (a pragma or aspect, no doubt) 
> to specify the size in the Ada source. That way is currently missing, 
> AFAIK.

Yup. Sounds like a reasonable enhancement request (too late for Ada 2020, 
unfortunately, but possibly in a future Corrigendum or revision). I suggest 
sending it and a use-case to Ada-Comment.

                       Randy.