dynamic multithreading

dynamic multithreading

[Artiom Ivanov]

I have a question to all who had been working closely with multithreaded
programming.

As you know in many applications when using threads, we sometimes need to
create a "tasks" dynamically (I mean at run-time).

For example when you need to create a server for a client applications (when
there can be more then one client application) you need to create tasks when
it's needed at run-time and after finishing processing destruct them.

It will be very cool if you could explain me a way to proceed for dynamic
multithreading and post me some examples.

Thanks in advance

Re: dynamic multithreading

[David Marceau]

Artiom Ivanov wrote:
> 
> I have a question to all who had been working closely with multithreaded
> programming.
> 
> As you know in many applications when using threads, we sometimes need to
> create a "tasks" dynamically (I mean at run-time).
> 
> For example when you need to create a server for a client applications (when
> there can be more then one client application) you need to create tasks when
> it's needed at run-time and after finishing processing destruct them.
> 
> It will be very cool if you could explain me a way to proceed for dynamic
> multithreading and post me some examples.
> 
> Thanks in advance
Ok well I'm no expert with ada concurrency issues but I would recommend:
1)reading the ada rationale 
2)reading what Cohen has to say in Cohen's Ada book, and 
3)"Concurrency in Ada".   From what I understand this is the better book
for what you want.
http://titles.cambridge.org/catalogue.asp?isbn=0521414717
The following quote taken from
http://portal.acm.org/citation.cfm?id=211392&dl=ACM&coll=portal#
"Burns and Wellings and their associates were major contributors to the
movement that called for the revision of Ada 83 and its subsequent
upgrade to Ada 95. This book turns their spotlight squarely on the new
language, Ada 95."
4)IMHO You'll find a good way of using multhreaded tasks in the ada web
server sources.  Search for the keyword "task" in their sources and
you'll find what you want.  Be warned you'll need to understand all the
intricacies of "select" statements also.
http://libre.act-europe.fr/aws/

I hope this helps and good luck :)

Sant� bonheur,
David Marceau

Re: dynamic multithreading

[Björn Lundin]

Artiom Ivanov wrote:
cut
> It will be very cool if you could explain me a way to proceed for dynamic
> multithreading and post me some examples.
cut

http://www.rfc1149.net/devel/adasockets

If I remember correctly, there's an example of
a process action as server, listening for new connections.
For each new connection, a new task is created

Listener.adb was the name
(At least it was in 0.1.14)

/Bj�rn

Re: dynamic multithreading

[David C. Hoos]

----- Original Message -----
From: "Artiom Ivanov" <rarelang@ua.fm>
Newsgroups: comp.lang.ada
To: <comp.lang.ada@ada.eu.org>
Sent: Thursday, November 14, 2002 7:12 AM
Subject: dynamic multithreading


> I have a question to all who had been working closely with multithreaded
> programming.
>
> As you know in many applications when using threads, we sometimes need to
> create a "tasks" dynamically (I mean at run-time).
>
> For example when you need to create a server for a client applications
(when
> there can be more then one client application) you need to create tasks
when
> it's needed at run-time and after finishing processing destruct them.
>
> It will be very cool if you could explain me a way to proceed for dynamic
> multithreading and post me some examples.
>
On my FTP site you will find the following file:

ftp.ada95.com/pub/TCP_Servers.zip

In the body of the CI.TCP.Servers package you will see the task Connect
which listens for connection requests,  When a connection is requested and
granted, a new instance of the task Agent is created to service the
connection.

Re: dynamic multithreading

[Thierry Lelegard]

Bj�rn Lundin wrote:
> If I remember correctly, there's an example of
> a process action as server, listening for new connections.
> For each new connection, a new task is created

This is generally a bad design.

- First, there is performance issue. Creating a new task each time
  a new connection comes in is a pain on heavily loaded servers.

- Second, you must absolutely unchecked_deallocate each task after
  it is terminated (not always trivial to synchronize on actual
  termination of a task). Otherwise, you have a memory leak.

As an alternative, never let a service task die. Once a task has
finished servicing a connection, make it wait and reuse it for a
later connection. You may create a finite number of service tasks
at startup or create new tasks only when no service task is idle
and a new connection arrives.

This is not so difficult to design and much more robust.
____________________________________________________________________________

Thierry Lelegard
CANAL+ Technologies, 34 place Raoul Dautry, 75906 Paris Cedex 15, France
____________________________________________________________________________

Re: dynamic multithreading

[Jeffrey Carter]

Artiom Ivanov wrote:
> As you know in many applications when using threads, we sometimes need to
> create a "tasks" dynamically (I mean at run-time).

Ada has 2 ways to dynamically create tasks:

1. Declare tasks locally

declare
    T : Some_Task_Type;
begin
    ...
end;

-- execution will not reach here until T finishes.

If necessary, the task definition can also be local. Subprograms may 
declare task objects which are local to the subprogram; the subprogram 
will not return until its local tasks finish.

2. Use access to task values

task type Whatever is ...

type Whatever_Ptr is access Whatever;

T : Whatever_Ptr;

...

T := new Whatever;

The scope in which the access type is declared will not be left until 
all tasks created using new have finished. The access values can be 
passed around, assigned, returned from functions, and so on.

-- 
Jeff Carter
"I spun around, and there I was, face to face with a
six-year-old kid. Well, I just threw my guns down and
walked away. Little bastard shot me in the ass."
Blazing Saddles

Re: dynamic multithreading

[tmoran]

> - First, there is performance issue. Creating a new task each time
> ...
> - Second, you must absolutely unchecked_deallocate each task after
> ...
  Some systems I've used still have a memory leak when a task is
deallocated.  (Task Control Block?)

> As an alternative, never let a service task die. Once a task has
> ...
> This is not so difficult to design and much more robust.
> ...
  For a simple example, download www.adapower.com/reuse/clawweb.zip
and look at Find_A_Worker.

Re: dynamic multithreading

[Robert A Duff]

"Artiom Ivanov" <rarelang@ua.fm> writes:

> As you know in many applications when using threads, we sometimes need to
> create a "tasks" dynamically (I mean at run-time).

You create tasks in Ada just like you create variables of any other
type.  If you have a task type:

    task type T;

    task body T is
    begin
        ... 
    end T;

then you can create local tasks:

    X: T;

or you can create them in the heap:

    type T_Ptr is access T;

    X: T_Ptr := new T;

You can also have record components of type T, and create local records,
or records in the heap.

- Bob

Re: dynamic multithreading

[Robert A Duff]

tmoran@acm.org writes:

>   Some systems I've used still have a memory leak when a task is
> deallocated.  (Task Control Block?)

An Ada run-time system that leaks memory (out of control of the Ada
programmer) has a bug.  Don't just meekly avoid using heap-allocated
tasks -- report the bug to your vendor!

- Bob

Re: dynamic multithreading

[Jean-Pierre Rosen]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 599 bytes --]


<tmoran@acm.org> a �crit dans le message news:
IoSA9.21803$NH2.261@sccrnsc01...
> > - First, there is performance issue. Creating a new task each time
> > ...
> > - Second, you must absolutely unchecked_deallocate each task after
> > ...
>   Some systems I've used still have a memory leak when a task is
> deallocated.  (Task Control Block?)
>
Must have been in Ada83. The problem (known as the "Rosen's pathology ;-) is
now fixed in Ada95.

--
---------------------------------------------------------
           J-P. Rosen (rosen@adalog.fr)
Visit Adalog's web site at http://www.adalog.fr

Re: dynamic multithreading

[tmoran]

> It's much much cleaner to just let the task die and start from scratch.
  It's hard to get much simpler than:
  task body t is
    -- don't declare stuff here
  begin
    loop
      declare
        --declare it here instead
      begin
        accept do_something(what_to_do);
        -- code
      exception
        when others =>
          -- handle any unhandled exceptions,
          -- or this can just be "null;"
      end;
      -- everthing about the last run is forgotten, start fresh
    end loop;
  end t;

> you simply don't trust your run-time system to even de-allocate the stack)
  e.g. aborting a thread (TerminateThread) on MS Windows

Re: dynamic multithreading

[Simon Wright]

"AG" <ang@xtra.co.nz> writes:

> I'd disagree. It is much easier to design a system which
> always starts from scratch. There is also a lot less potential
> for errors since you don't need to keep or handle previous
> states. So how can it be less robust? (Unless, as stated above,
> you simply don't trust your run-time system to even de-allocate
> the stack)

With GNAT it seems to be the task control block that doesn't get
deallocated.

Re: dynamic multithreading

[AG]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 2024 bytes --]


"Thierry Lelegard" <thierry.lelegard@canal-plus.fr> wrote in message
news:3DD3D841.CD16A4CB@canal-plus.fr...
> Bj�rn Lundin wrote:
> > If I remember correctly, there's an example of
> > a process action as server, listening for new connections.
> > For each new connection, a new task is created
>
> This is generally a bad design.

That's arguable.

>
> - First, there is performance issue. Creating a new task each time
>   a new connection comes in is a pain on heavily loaded servers.

Well, servers come and go, tech impoves or not but design stays.

>
> - Second, you must absolutely unchecked_deallocate each task after
>   it is terminated (not always trivial to synchronize on actual
>   termination of a task). Otherwise, you have a memory leak.

Which means you don't trust the run-time system to do some
elementary GC? If you are *that* paranoid (or have a good
reason to be) perhaps it's time to abandon Ada and switch
back to the good old Assembly language.

>
> As an alternative, never let a service task die. Once a task has
> finished servicing a connection, make it wait and reuse it for a
> later connection.

Yuck. That means the task needs to be able to reset to a clean
state anytime it's finished with a connection. It would also need
to handle all possible error conditions and abnormal ends to be
able to come back on line "up and ready". It's much much cleaner
to just let the task die and start from scratch. (Hey, how many
projects agonized like that for years instead of just restarting
at a good time?) That does not include the safety-critical systems,
of course, just the common "run of the mill" things.

> This is not so difficult to design and much more robust.

I'd disagree. It is much easier to design a system which
always starts from scratch. There is also a lot less potential
for errors since you don't need to keep or handle previous
states. So how can it be less robust? (Unless, as stated above,
you simply don't trust your run-time system to even de-allocate
the stack)

Re: dynamic multithreading

[Thierry Lelegard]

> > - First, there is performance issue. Creating a new task each time
> >   a new connection comes in is a pain on heavily loaded servers.
> 
> Well, servers come and go, tech impoves or not but design stays.

Just be pragmatic. When a customer has a performance requirement,
the application should be designed in such a way that it matches
the requirement. This is, _by_definition_, a good design. And a
design with prevents the application from matching the performance
requirement, given the current technologies at the time of the
application release is, _by_definition_, a bad design.

And currently, creating/deleting dozens of tasks per second (provided
that this is what you want) is an actual pain.

> > - Second, you must absolutely unchecked_deallocate each task after
> >   it is terminated (not always trivial to synchronize on actual
> >   termination of a task). Otherwise, you have a memory leak.
> 
> Which means you don't trust the run-time system to do some
> elementary GC? If you are *that* paranoid (or have a good
> reason to be) perhaps it's time to abandon Ada and switch
> back to the good old Assembly language.

No need to argue, just try the following program with and without
the "Free". If you find one Ada95 compiler one one platform where
it does not eat up all the memory, please let us know.

With GNAT on Solaris, VMS, Linux, HP-UX and Windows, I already know
the answer...

-Thierry

with Ada.Text_IO;
with Ada.Unchecked_Deallocation;
with System;

procedure Crazy_Tasks is

    task type T is
        pragma Priority (System.Priority'Last);
        entry Foo;
    end T;

    type Access_T is access T;

    task body T is
    begin
        select
            accept Foo;
        else
            null;
        end select;
    end T;

    procedure Free is new Ada.Unchecked_Deallocation (T, Access_T);

    type Counter is mod 2**32;

    Count : Counter := 0;
    TV    : Access_T;

begin

    loop
        TV := new T; -- should terminate immediately
        while not TV.all'Terminated loop
            delay 0.0;
        end loop;
        -- Free (TV);
        Count := Count + 1;
        if Count mod 50_000 = 0 then
            Ada.Text_IO.Put_Line (Counter'Image (Count) & " tasks");
        end if;
    end loop;

end Crazy_Tasks;

____________________________________________________________________________

Thierry Lelegard, CANAL+ Technologies, Paris, France
____________________________________________________________________________

Re: dynamic multithreading

[Grein, Christoph]

Of course this program _must_ consume all memory.

You explicitly allocate on the heap, but never deallocate. That's what 
Unchecked_Deallocation is for.

Or do you mean even with Free, it consumes all memory?

> No need to argue, just try the following program with and without
> the "Free". If you find one Ada95 compiler one one platform where
> it does not eat up all the memory, please let us know.
> 
> With GNAT on Solaris, VMS, Linux, HP-UX and Windows, I already know
> the answer...
> 
> -Thierry
> 
> with Ada.Text_IO;
> with Ada.Unchecked_Deallocation;
> with System;
> 
> procedure Crazy_Tasks is
> 
>     task type T is
>         pragma Priority (System.Priority'Last);
>         entry Foo;
>     end T;
> 
>     type Access_T is access T;
> 
>     task body T is
>     begin
>         select
>             accept Foo;
>         else
>             null;
>         end select;
>     end T;
> 
>     procedure Free is new Ada.Unchecked_Deallocation (T, Access_T);
> 
>     type Counter is mod 2**32;
> 
>     Count : Counter := 0;
>     TV    : Access_T;
> 
> begin
> 
>     loop
>         TV := new T; -- should terminate immediately
>         while not TV.all'Terminated loop
>             delay 0.0;
>         end loop;
>         -- Free (TV);
>         Count := Count + 1;
>         if Count mod 50_000 = 0 then
>             Ada.Text_IO.Put_Line (Counter'Image (Count) & " tasks");
>         end if;
>     end loop;
> 
> end Crazy_Tasks;

Re: dynamic multithreading

[Dmitry A. Kazakov]

On Mon, 18 Nov 2002 10:11:08 +0100, Thierry Lelegard
<thierry.lelegard@canal-plus.fr> wrote:

>No need to argue, just try the following program with and without
>the "Free". If you find one Ada95 compiler one one platform where
>it does not eat up all the memory, please let us know.
>
>With GNAT on Solaris, VMS, Linux, HP-UX and Windows, I already know
>the answer...

Alas.

[Ooch, I will add your example to our compiler validation site]

>with Ada.Text_IO;
>with Ada.Unchecked_Deallocation;
>with System;
>
>procedure Crazy_Tasks is
>
>    task type T is
>        pragma Priority (System.Priority'Last);
>        entry Foo;
>    end T;
>
>    type Access_T is access T;
>
>    task body T is
>    begin
>        select
>            accept Foo;
>        else
>            null;
>        end select;
>    end T;
>
>    procedure Free is new Ada.Unchecked_Deallocation (T, Access_T);
>
>    type Counter is mod 2**32;
>
>    Count : Counter := 0;
>    TV    : Access_T;
>
>begin
>
>    loop
>        TV := new T; -- should terminate immediately
>        while not TV.all'Terminated loop
>            delay 0.0;
>        end loop;
>        -- Free (TV);
>        Count := Count + 1;
>        if Count mod 50_000 = 0 then
>            Ada.Text_IO.Put_Line (Counter'Image (Count) & " tasks");
>        end if;
>    end loop;
>
>end Crazy_Tasks;

This does not leak in ObjectAda 7.2 under Windows. [Trumpets!]

---
Regards,
Dmitry Kazakov
www.dmitry-kazakov.de

Re: dynamic multithreading

[Grein, Christoph]

> >with Ada.Text_IO;
> >with Ada.Unchecked_Deallocation;
> >with System;
> >
> >procedure Crazy_Tasks is
> >
> >    task type T is
> >        pragma Priority (System.Priority'Last);
> >        entry Foo;
> >    end T;
> >
> >    type Access_T is access T;
> >
> >    task body T is
> >    begin
> >        select
> >            accept Foo;
> >        else
> >            null;
> >        end select;
> >    end T;
> >
> >    procedure Free is new Ada.Unchecked_Deallocation (T, Access_T);
> >
> >    type Counter is mod 2**32;
> >
> >    Count : Counter := 0;
> >    TV    : Access_T;
> >
> >begin
> >
> >    loop
> >        TV := new T; -- should terminate immediately
> >        while not TV.all'Terminated loop
> >            delay 0.0;
> >        end loop;
> >        -- Free (TV);
> >        Count := Count + 1;
> >        if Count mod 50_000 = 0 then
> >            Ada.Text_IO.Put_Line (Counter'Image (Count) & " tasks");
> >        end if;
> >    end loop;
> >
> >end Crazy_Tasks;
> 
> This does not leak in ObjectAda 7.2 under Windows. [Trumpets!]

I do not understand... Even with Free commented out doesn't it leak?

Re: dynamic multithreading

[Thierry Lelegard]

> Of course this program _must_ consume all memory.
> 
> You explicitly allocate on the heap, but never deallocate. That's what
> Unchecked_Deallocation is for.
>
> Or do you mean even with Free, it consumes all memory?

We agree, that's the whole point. Using Free does not leak.

But, in the previous note, the author seems to think that there should
be not memory leak, even without unchecked_deallocation:

> > - Second, you must absolutely unchecked_deallocate each task after
> >   it is terminated (not always trivial to synchronize on actual
> >   termination of a task). Otherwise, you have a memory leak.
> 
> Which means you don't trust the run-time system to do some
> elementary GC? If you are *that* paranoid (or have a good
> reason to be) perhaps it's time to abandon Ada and switch
> back to the good old Assembly language.

-Thierry Lelegard

Re: dynamic multithreading

[Dmitry A. Kazakov]

On Mon, 18 Nov 2002 13:25:39 +0100, Thierry Lelegard
<thierry.lelegard@canal-plus.fr> wrote:

>> Of course this program _must_ consume all memory.
>> 
>> You explicitly allocate on the heap, but never deallocate. That's what
>> Unchecked_Deallocation is for.
>>
>> Or do you mean even with Free, it consumes all memory?
>
>We agree, that's the whole point. Using Free does not leak.
>
>But, in the previous note, the author seems to think that there should
>be not memory leak, even without unchecked_deallocation:

It should not leak in cases:

1. Stack allocation

   declare
      TV : T;
   begin
      ...
   end;

2. Heap allocation

   TV_Ptr := new T;
   ...
   Free (TV_Ptr);

3. Aggregation

   type Server is ... record
       ...
       TV : T;
   end record;

I don't think he meant GC in the sense of "garbage collection". He
probably meant "elementary collection of the garbage the compiler has
produced to support your tasks". (:-))

1-3 shall not leak. If it does, then it is a compiler bug. The only
case when a task pool could be useful is 3. If Server is controlled,
then Finalize is not called till TV is runing. But it is another
story.

>> > - Second, you must absolutely unchecked_deallocate each task after
>> >   it is terminated (not always trivial to synchronize on actual
>> >   termination of a task). Otherwise, you have a memory leak.
>> 
>> Which means you don't trust the run-time system to do some
>> elementary GC? If you are *that* paranoid (or have a good
>> reason to be) perhaps it's time to abandon Ada and switch
>> back to the good old Assembly language.

---
Regards,
Dmitry Kazakov
www.dmitry-kazakov.de

Re: dynamic multithreading

[Dmitry A. Kazakov]

On Mon, 18 Nov 2002 13:13:28 +0100 (MET), "Grein, Christoph"
<christoph.grein@eurocopter.com> wrote:

>> >with Ada.Text_IO;
>> >with Ada.Unchecked_Deallocation;
>> >with System;
>> >
>> >procedure Crazy_Tasks is
>> >
>> >    task type T is
>> >        pragma Priority (System.Priority'Last);
>> >        entry Foo;
>> >    end T;
>> >
>> >    type Access_T is access T;
>> >
>> >    task body T is
>> >    begin
>> >        select
>> >            accept Foo;
>> >        else
>> >            null;
>> >        end select;
>> >    end T;
>> >
>> >    procedure Free is new Ada.Unchecked_Deallocation (T, Access_T);
>> >
>> >    type Counter is mod 2**32;
>> >
>> >    Count : Counter := 0;
>> >    TV    : Access_T;
>> >
>> >begin
>> >
>> >    loop
>> >        TV := new T; -- should terminate immediately
>> >        while not TV.all'Terminated loop
>> >            delay 0.0;
>> >        end loop;
>> >        -- Free (TV);
>> >        Count := Count + 1;
>> >        if Count mod 50_000 = 0 then
>> >            Ada.Text_IO.Put_Line (Counter'Image (Count) & " tasks");
>> >        end if;
>> >    end loop;
>> >
>> >end Crazy_Tasks;
>> 
>> This does not leak in ObjectAda 7.2 under Windows. [Trumpets!]
>
>I do not understand... Even with Free commented out doesn't it leak?

Of course not. It does not leak when Free is called.

I thought that the poster meant that it will leak *no matter* whether
Free is called or not. If it would be the case, then I could
understand his argumentation for a task pool.

---
Regards,
Dmitry Kazakov
www.dmitry-kazakov.de

Re: dynamic multithreading

[Stephen Leake]

"AG" <ang@xtra.co.nz> writes:

> "Thierry Lelegard" <thierry.lelegard@canal-plus.fr> wrote in message
> news:3DD3D841.CD16A4CB@canal-plus.fr...
> > - Second, you must absolutely unchecked_deallocate each task after
> >   it is terminated (not always trivial to synchronize on actual
> >   termination of a task). Otherwise, you have a memory leak.
> 
> Which means you don't trust the run-time system to do some
> elementary GC? 

Well, since the Ada Standard says that anything allocated with "new"
will _not_ be "garbage collected", I do _not_ expect the "run-time
system" to do "free" for me.

Nothing wrong with doing "free" myself. If, as Thierry hinted, it is
not easy to determine when the "free" is safe, then why would it be
any easier for some run-time system to determine this?

> If you are *that* paranoid (or have a good reason to be) perhaps
> it's time to abandon Ada and switch back to the good old Assembly
> language.

Name one Assembly language that has garbage collection! 

Paranoia has nothing to do with it; this is about reading the standard
and understanding what it says.

> > As an alternative, never let a service task die. Once a task has
> > finished servicing a connection, make it wait and reuse it for a
> > later connection.
> 
> Yuck. That means the task needs to be able to reset to a clean
> state anytime it's finished with a connection. 

Yes. Why is that hard? If the state is not "clean", then there are
loose ends that should be cleaned up. Why is it ok to "just kill"
something with loose ends? That's a sloppy design.

> It would also need to handle all possible error conditions and
> abnormal ends to be able to come back on line "up and ready". It's
> much much cleaner to just let the task die and start from scratch.

Apparently you believe the operation of "let the task die" will clean
up all the loose ends and restore a clean state. Can you quote the Ada
Standard to justify that?

How, exactly, do you "let the task die"?

> (Hey, how many projects agonized like that for years instead of just
> restarting at a good time?)

Interesting analogy.

> That does not include the safety-critical systems, of course, just
> the common "run of the mill" things.

If it makes money for me, it is _not_ "run of the mill" :).

> > This is not so difficult to design and much more robust.
> 
> I'd disagree. It is much easier to design a system which
> always starts from scratch. 

That's arguable :).

> There is also a lot less potential for errors since you don't need
> to keep or handle previous states. 

You are assuming the operating system does it for you. Not a safe
assumption. 

> So how can it be less robust? 

Knowing precisely what is going on is always more robust than hoping
some undocumented feature will save you.

> (Unless, as stated above, you simply don't trust your run-time
> system to even de-allocate the stack)

I don't trust the run-time system to do something it explicitly says
it will _not_ do!

-- 
-- Stephe

Re: dynamic multithreading

[Pascal Obry]

Dmitry A. Kazakov <mailbox@dmitry-kazakov.de> writes:

> >    loop
> >        TV := new T; -- should terminate immediately
> >        while not TV.all'Terminated loop
> >            delay 0.0;
> >        end loop;
> >        -- Free (TV);
> >        Count := Count + 1;
> >        if Count mod 50_000 = 0 then
> >            Ada.Text_IO.Put_Line (Counter'Image (Count) & " tasks");
> >        end if;
> >    end loop;
> >
> >end Crazy_Tasks;
> 
> This does not leak in ObjectAda 7.2 under Windows. [Trumpets!]

!!!!! How is this possible ? A "new" without a "Free" and you claim no memory
leak !!!!!

Pascal.

-- 

--|------------------------------------------------------
--| Pascal Obry                           Team-Ada Member
--| 45, rue Gabriel Peri - 78114 Magny Les Hameaux FRANCE
--|------------------------------------------------------
--|         http://perso.wanadoo.fr/pascal.obry
--| "The best way to travel is by means of imagination"
--|
--| gpg --keyserver wwwkeys.pgp.net --recv-key C1082595

Re: dynamic multithreading

[Pascal Obry]

Dmitry A. Kazakov <mailbox@dmitry-kazakov.de> writes:

> It should not leak in cases:
> 
> 1. Stack allocation
> 
>    declare
>       TV : T;
>    begin
>       ...
>    end;
> 
> 2. Heap allocation
> 
>    TV_Ptr := new T;
>    ...
>    Free (TV_Ptr);
> 
> 3. Aggregation
> 
>    type Server is ... record
>        ...
>        TV : T;
>    end record;

Ok ! This is indeed what is expected (new + Free).

Pascal.

-- 

--|------------------------------------------------------
--| Pascal Obry                           Team-Ada Member
--| 45, rue Gabriel Peri - 78114 Magny Les Hameaux FRANCE
--|------------------------------------------------------
--|         http://perso.wanadoo.fr/pascal.obry
--| "The best way to travel is by means of imagination"
--|
--| gpg --keyserver wwwkeys.pgp.net --recv-key C1082595

Re: dynamic multithreading

[Simon Wright]

Lutz Donnerhacke <lutz@iks-jena.de> writes:

> * Pascal Obry wrote:
> > !!!!! How is this possible ? A "new" without a "Free" and you claim no memory
> > leak !!!!!
> 
> Simple. The access variable has a local scope and all objects associated
> with it are automatically freed when the type goes out of scope. You may
> limit the pool size of this access type ...

If you look at the code in question you will see that the type is not
local (to the loop). It is local to the given subprogram, but the loop
is endless in the subprogram.

I agree that on a modern hosted OS like Linux or Windows, when you
terminate the main program the memory it has allocated will be
restored. But that will not do you much good if your os is eg VxWorks.

Re: dynamic multithreading

[Lutz Donnerhacke]

* Pascal Obry wrote:
> !!!!! How is this possible ? A "new" without a "Free" and you claim no memory
> leak !!!!!

Simple. The access variable has a local scope and all objects associated
with it are automatically freed when the type goes out of scope. You may
limit the pool size of this access type ...

Re: dynamic multithreading

[Preben Randhol]

Lutz Donnerhacke wrote:
> 
> Simple. The access variable has a local scope and all objects associated
> with it are automatically freed when the type goes out of scope. You may
> limit the pool size of this access type ...

So ObjectAda has included a GC ?

-- 
Preben Randhol ------------------------ http://www.pvv.org/~randhol/ --
�There are three things you can do to a woman. You can love her, suffer
 for her, or turn her into literature.�  - Justine, by Lawrence Durrell

Re: dynamic multithreading

[Lutz Donnerhacke]

* Preben Randhol wrote:
> Lutz Donnerhacke wrote:
>> Simple. The access variable has a local scope and all objects associated
>> with it are automatically freed when the type goes out of scope. You may
>> limit the pool size of this access type ...
>
> So ObjectAda has included a GC ?

Not necessary. This kind of "Garbage collection" is easy to implement: Free
all allocated data assigned to variables of a specific type if this type
goes out of space.

Re: dynamic multithreading

[Preben Randhol]

Lutz Donnerhacke wrote:
> Not necessary. This kind of "Garbage collection" is easy to implement: Free
> all allocated data assigned to variables of a specific type if this type
> goes out of space.

But it doesn't make sense, because you can have two access variables. If
the inner variable goes out of scope it should not free up the rest of
the objects. I can understand if what you are talking about is a type
with Finalize, but then you need to call Free in the Finalize function.
But still an access to such a variable won't be detected by the Adajust
function so how can you know if there are no more pointers to the
object.


-- 
Preben Randhol ------------------------ http://www.pvv.org/~randhol/ --
�There are three things you can do to a woman. You can love her, suffer
 for her, or turn her into literature.�  - Justine, by Lawrence Durrell

Re: dynamic multithreading

[Lutz Donnerhacke]

* Preben Randhol wrote:
> Lutz Donnerhacke wrote:
>> Not necessary. This kind of "Garbage collection" is easy to implement: Free
>> all allocated data assigned to variables of a specific type if this type
>> goes out of space.
>
> But it doesn't make sense, because you can have two access variables.

The current variables can be ignored. I write only about the type of them.

> If the inner variable goes out of scope it should not free up the rest of
> the objects.

  declare
     type A_Access is access A;
     a : A_Access := new A;        -- Instance 1.
  begin
     a := new A;                   -- Instance 2: Leaks memory (Instance 1)
     declare
       b : A_Access := a;
     begin
       b := new A;                 -- Instance 3.
       a := b;                     -- Leaks memory (Instance 2)
     end;
  end;                             -- Frees all three instances.

> I can understand if what you are talking about is a type with Finalize,
> but then you need to call Free in the Finalize function.

No, this not the Ada approach.

Re: dynamic multithreading

[David C. Hoos]

"Stephen Leake" <stephen.a.leake.1@gsfc.nasa.gov> wrote in message
news:uzns7t0ft.fsf@gsfc.nasa.gov...
> "AG" <ang@xtra.co.nz> writes:
>
> > "Thierry Lelegard" <thierry.lelegard@canal-plus.fr> wrote in message
> > news:3DD3D841.CD16A4CB@canal-plus.fr...
> > > - Second, you must absolutely unchecked_deallocate each task after
> > >   it is terminated (not always trivial to synchronize on actual
> > >   termination of a task). Otherwise, you have a memory leak.
> >
> > Which means you don't trust the run-time system to do some
> > elementary GC?
>
> Well, since the Ada Standard says that anything allocated with "new"
> will _not_ be "garbage collected", I do _not_ expect the "run-time
> system" to do "free" for me.

Can you cite the section that states this?  My copy of the Ada LRM
says in 4.8 (15)  "Implementations are permitted, but not required, to
provide garbage collection (see 13.11.3)."
<snip>

Re: dynamic multithreading

[Preben Randhol]

Lutz Donnerhacke wrote:
>> But it doesn't make sense, because you can have two access variables.
> 
> The current variables can be ignored. I write only about the type of them.
> 
>> If the inner variable goes out of scope it should not free up the rest of
>> the objects.
> 
>   declare
>      type A_Access is access A;
>      a : A_Access := new A;        -- Instance 1.
>   begin
>      a := new A;                   -- Instance 2: Leaks memory (Instance 1)
>      declare
>        b : A_Access := a;
>      begin
>        b := new A;                 -- Instance 3.
>        a := b;                     -- Leaks memory (Instance 2)
>      end;
>   end;                             -- Frees all three instances.

Well try this:

declare
   type A_Access is access A;
   a : A_Access := new A;        -- Instance 1.
begin
   loop
      a := new A;                   -- Instance 2: Leaks memory (Instance 1)
      declare
        b : A_Access := a;
      begin
        b := new A;                 -- Instance 3.
        a := b;                     -- Leaks memory (Instance 2)
      end;
   end loop;
end;                             -- Frees all three instances.

-- 
Preben Randhol ------------------------ http://www.pvv.org/~randhol/ --
�There are three things you can do to a woman. You can love her, suffer
 for her, or turn her into literature.�  - Justine, by Lawrence Durrell

Re: dynamic multithreading

[Preben Randhol]

Lutz Donnerhacke wrote:
> No, this not the Ada approach.

http://www.adapower.com/alg/smartp.html

-- 
Preben Randhol ------------------------ http://www.pvv.org/~randhol/ --
�There are three things you can do to a woman. You can love her, suffer
 for her, or turn her into literature.�  - Justine, by Lawrence Durrell

Re: dynamic multithreading

[Grein, Christoph]

> Well, since the Ada Standard says that anything allocated with "new"
> will _not_ be "garbage collected", I do _not_ expect the "run-time
> system" to do "free" for me.

Where, please, does it say that? There is nothing in the standard that prevents a GC. Fact is however that no current compiler implements one because of 
induced problems in real-time systems by unpredictable timing.

Note that there is a pragma Controlled that prevents automatic GC. So far this 
pragma is void for current compilers.

So at least your statement is correct that you should clean up things yourself.

Re: dynamic multithreading

[Grein, Christoph]

From: Lutz Donnerhacke <lutz@iks-jena.de>
> 
> * Pascal Obry wrote:
> > !!!!! How is this possible ? A "new" without a "Free" and you claim no 
memory
> > leak !!!!!
> 
> Simple. The access variable has a local scope and all objects associated
> with it are automatically freed when the type goes out of scope. You may
> limit the pool size of this access type ...

This is not true in general!

Every access type has an assciated storage pool. If this pool is not defined 
upon declaration, a default storage pool is used.

There is nothing in the standard that says that all allocated objects of an 
access type are freed when the access type goes out of scope. How could it, 
since this might be a global default storage pool.

There is however RM 13.11(18): If Storage_Size is defined for an access type, 
all storage is reclaimed if the (master of) the type goes out of scope.

So you have to limit the pool if you want such a kind of garbage collection.

Re: dynamic multithreading

[Grein, Christoph]

From: Lutz Donnerhacke <lutz@iks-jena.de>
> > If the inner variable goes out of scope it should not free up the rest of
> > the objects.
> 
>   declare
>      type A_Access is access A;
>      a : A_Access := new A;        -- Instance 1.
>   begin
>      a := new A;                   -- Instance 2: Leaks memory (Instance 1)
>      declare
>        b : A_Access := a;
>      begin
>        b := new A;                 -- Instance 3.
>        a := b;                     -- Leaks memory (Instance 2)
>      end;
>   end;                             -- Frees all three instances.

Not true, see me other mail.

You need 'Storag_Size on A_Access for this statement to become true.

Re: dynamic multithreading

[Grein, Christoph]

> http://www.adapower.com/alg/smartp.html

There is another implementation of referece counted pointers, Safe_Pointers, in my home 
page. This has been published in Ada Letters some years ago.

See http://home.T-Online.de/home/Christ-Usch.Grein/Ada/

Re: dynamic multithreading

[Dmitry A. Kazakov]

On 18 Nov 2002 17:18:10 +0100, Pascal Obry <p.obry@wanadoo.fr> wrote:

>
>Dmitry A. Kazakov <mailbox@dmitry-kazakov.de> writes:
>
>> >    loop
>> >        TV := new T; -- should terminate immediately
>> >        while not TV.all'Terminated loop
>> >            delay 0.0;
>> >        end loop;
>> >        -- Free (TV);
>> >        Count := Count + 1;
>> >        if Count mod 50_000 = 0 then
>> >            Ada.Text_IO.Put_Line (Counter'Image (Count) & " tasks");
>> >        end if;
>> >    end loop;
>> >
>> >end Crazy_Tasks;
>> 
>> This does not leak in ObjectAda 7.2 under Windows. [Trumpets!]
>
>!!!!! How is this possible ? A "new" without a "Free" and you claim no memory
>leak !!!!!

Sorry!! It was misleading! I could not imagine that someone would
require the above to work with Free commented out. MY FAULT.

To clarify everything, of course, it does not leak ONLY if Free is
called. Without Free, Object Ada "leaks". But it is not a leak, it is
just a program design fault.

---
Regards,
Dmitry Kazakov
www.dmitry-kazakov.de

Re: dynamic multithreading

[Lutz Donnerhacke]

* Simon Wright wrote:
> Lutz Donnerhacke <lutz@iks-jena.de> writes:
>> Simple. The access variable has a local scope and all objects associated
>> with it are automatically freed when the type goes out of scope. You may
>> limit the pool size of this access type ...
> 
> If you look at the code in question you will see that the type is not
> local (to the loop). It is local to the given subprogram, but the loop
> is endless in the subprogram.

I saw this and therefore noted the limitation of the pool.

> I agree that on a modern hosted OS like Linux or Windows, when you
> terminate the main program the memory it has allocated will be
> restored. But that will not do you much good if your os is eg VxWorks.

I do not talk about termination.

Re: dynamic multithreading

[Lutz Donnerhacke]

* Preben Randhol wrote:
> Well try this:
> 
> declare
>    type A_Access is access A;
>    a : A_Access := new A;        -- Instance 1.
> begin
>    loop
>       a := new A;                   -- Instance 2: Leaks memory (Instance 1)
>       declare
>         b : A_Access := a;
>       begin
>         b := new A;                 -- Instance 3.
>         a := b;                     -- Leaks memory (Instance 2)
>       end;
>    end loop;
> end;                             -- Frees all three instances.

This will leak unlimited memory. But add:
  for A_Access'Storage_Size use A'Max_Size_In_Storage_Elements * 2;

This will limit the leakage.

Re: dynamic multithreading

[Lutz Donnerhacke]

* Preben Randhol wrote:
> Lutz Donnerhacke wrote:
>> No, this not the Ada approach.
>
> http://www.adapower.com/alg/smartp.html

Of course, you can implement such thing, but IMHO Ada takes the way to
storage pools.

Re: dynamic multithreading

[Lutz Donnerhacke]

* Grein, Christoph wrote:
> From: Lutz Donnerhacke <lutz@iks-jena.de>
>> Simple. The access variable has a local scope and all objects associated
>> with it are automatically freed when the type goes out of scope. You may
>> limit the pool size of this access type ...
[...]
> There is however RM 13.11(18): If Storage_Size is defined for an access
> type, all storage is reclaimed if the (master of) the type goes out of
> scope.

Thanks, I missed that.

Re: dynamic multithreading

[Preben Randhol]

Lutz Donnerhacke wrote:
> Of course, you can implement such thing, but IMHO Ada takes the way to
> storage pools.

Also for a linked list?

-- 
Preben Randhol ------------------------ http://www.pvv.org/~randhol/ --
�There are three things you can do to a woman. You can love her, suffer
 for her, or turn her into literature.�  - Justine, by Lawrence Durrell

Re: dynamic multithreading

[Lutz Donnerhacke]

* Preben Randhol wrote:
> Lutz Donnerhacke wrote:
>> Of course, you can implement such thing, but IMHO Ada takes the way to
>> storage pools.
>
> Also for a linked list?

A linked list is something more persitent than locally scoped access types.
Therefore you have to manage the storage over the long term. And you will do
this by a combination of the following methods:
 a  Storage Pool for access-to-item.
 b  Locally scoped generic instantiation.
 c  Private data types with specialized interface to the package.

a and b does guarantee that the memory leak is bounded and will disappear
when the list type is not longer needed.

c ensures definite access to the allocated memory and therefore it provides
a infrastructure for correct implementation of memory management. You might
free the data or manage your own pool of allocated objects.

Re: dynamic multithreading

[Georg Bauhaus]

Lutz Donnerhacke <lutz@iks-jena.de> wrote:
:  declare
:     type A_Access is access A;
:     a : A_Access := new A;        -- Instance 1.
:  begin

If you really want to try this out, it might be helpful to
not chose a and A for a variable and its type. :-)

-- georg

Re: dynamic multithreading

[Simon Wright]

Lutz Donnerhacke <lutz@iks-jena.de> writes:

> * Simon Wright wrote:
> > Lutz Donnerhacke <lutz@iks-jena.de> writes:
> >> Simple. The access variable has a local scope and all objects associated
> >> with it are automatically freed when the type goes out of scope. You may
> >> limit the pool size of this access type ...
> > 
> > If you look at the code in question you will see that the type is not
> > local (to the loop). It is local to the given subprogram, but the loop
> > is endless in the subprogram.
> 
> I saw this and therefore noted the limitation of the pool.

Oh, sorry, misunderstanding: did you mean "If you want to achieve this
effect, you must give the access type a local scope [and set
'Storage_Size] and then all objects associated with it are
automatically freed when the type goes out of scope."? I thought you
were discussing the posted code ..

> > I agree that on a modern hosted OS like Linux or Windows, when you
> > terminate the main program the memory it has allocated will be
> > restored. But that will not do you much good if your os is eg VxWorks.
> 
> I do not talk about termination.

Some of us have no choice.

Re: dynamic multithreading

[Matthew Heaney]

"Grein, Christoph" <christoph.grein@eurocopter.com> wrote in message news:<mailman.1037689022.2833.comp.lang.ada@ada.eu.org>...
> > http://www.adapower.com/alg/smartp.html
> 
> There is another implementation of referece counted pointers, Safe_Pointers, in my home 
> page. This has been published in Ada Letters some years ago.
> 
> See http://home.T-Online.de/home/Christ-Usch.Grein/Ada/

The package Charles.Access_Control provides an abstraction very
similar to an auto_ptr in C++.

The idea is that someone always "owns" a pointer, and it's up to the
owner to Free the object.

In the case of Access_Control, the Pointer_Type can be used as an
owner, presumably in a temporary scope.  This means that no matter how
you exit the scope (say, an unhandled exception), you have a guarantee
that the memory get free'd.

For example:

function Factory_Function return T_Access is

   Pointer : T_Access_Control.Pointer_Type;
   
begin
   
   Initialize (Pointer, new T);

   --do some stuff that might raise an exception, etc

   --ok, we're all done, transfer ownership to caller
   return Release (Pointer);

end Factory_Function;


I put up a new version (20021119) of Charles last night (I added a
bounded vector container, and generalized the Access_Control package):

http://home.earthlink.net/~matthewjheaney/charles/index.html

Re: dynamic multithreading

[John English]

"Grein, Christoph" wrote:
> 
> > http://www.adapower.com/alg/smartp.html
> 
> There is another implementation of referece counted pointers, Safe_Pointers, in my home
> page. This has been published in Ada Letters some years ago.
> 
> See http://home.T-Online.de/home/Christ-Usch.Grein/Ada/

There are no doubt many others too, including this:
   http://www.it.bton.ac.uk/staff/je/adacraft/ch16.htm#16.3
(first published in 1996).

-----------------------------------------------------------------
 John English              | mailto:je@brighton.ac.uk
 Senior Lecturer           | http://www.it.bton.ac.uk/staff/je
 Dept. of Computing        | ** NON-PROFIT CD FOR CS STUDENTS **
 University of Brighton    |    -- see http://burks.bton.ac.uk
-----------------------------------------------------------------