Re: API design problem - buffer scatter I/O

[Brad Moore <brad.moore@shaw.ca>]

Robert A Duff wrote:
> Maciej Sobczak <see.my.homepage@gmail.com> writes:
>> Still, in both of these solutions, there is a problem of scope - the
>> type of access value is defined by the library API (well, at the
>> library level), whereas user-provided buffers are likely to be defined
>> locally.
> 
> Right.  You have to use Local'Unchecked_Access, and take care to avoid
> dangling pointers.

I have been working on a set of general purpose buffers, and was
curious to see how I could incorporate a scatter buffer implementation
into my buffer framework. I ended up with a design that seems to satisfy 
the requirements stated in the original post. My experimentation led to 
some results that might be of interest, and I have a compiled test 
driver that reads and writes data to a scatter buffer.

1) The design I am using is an expansion of the idea discussed earlier 
in this thread. That is, a record type with a discriminant is used to
define the size of the array. In addition, the package is a generic
where the type of object to be stored in the buffer is passed in as a
generic formal parameter.

2) As part of the instantiation for a scatter buffer, I pass in the
array of buffers as a generic formal parameter. Passing in these
parameters has other benefits to my design, which I wont bother
trying to explain here.

generic

    type Scatterbuffer_Array_Type is
      array (Count_Type range <>) of access Buffer_Interface'Class;

    Scatter_Buffers : Scatterbuffer_Array_Type;

package Buffers.Scattered
    pragma Remote_Types;
...


3) Note that I am using anonymous access types in the generic formal
Scatterbuffer_Array_Type

I didn't have any issues with using anonymous access types. In my
main test procedure, I was able to use 'Access in a nested scope
without having to use 'Unchecked_Access. In fact, my implementation
requires anonymous access types, and I don't believe I could have
got this to work without using Ada 2005's anonymous access feature.
I actually tried, (after commenting out the pragma Remote_Types),
and using named access types, but the compiler wouldn't compile the code.
I suspect this is a compiler bug, but since I need the pragma 
Remote_Types, I didn't pursue that any further.

The need to use anonymous access types has more to do with the fact that 
I allow buffers to be accessed
remotely, but I need the buffers in the buffer list to be anonymous so 
that they are treated as local buffers, not remote ones.

The main test procedure looks like;

with Buffers.Simple_Bounded;
with Buffers.Simple_Unbounded;
with Buffers.Simple_Scattered;
with Buffers.Passive_Bounded;

procedure Main_Test is

    -- Create a root buffer framework for storing text (Character data)
    package My_Buffer is new Buffers
      (Element_Index_Type => Natural,
       Element_Type => Character,
       Element_Vector_Type => String);

    -- Instantiate some differing buffer types
    package My_Simple_Bounded_Buffer is new My_Buffer.Simple_Bounded;
    package My_Simple_Unbounded_Buffer is new My_Buffer.Simple_Unbounded;
    package My_Passive_Bounded_Buffer is new My_Buffer.Passive_Bounded;

   -- Declare buffers of different types and different sizes
   My_Bounded : aliased My_Simple_Bounded_Buffer.Buffer
      (Maximum_Capacity => 100);
   My_Unbounded : aliased My_Simple_Unbounded_Buffer.Buffer
      (Maximum_Capacity => 5000);
   My_Passive : aliased My_Passive_Bounded_Buffer
      (Maximum_Capacity => 33);

   -- Declare the type needed by Scatter Buffer
   type Scatterbuffer_Array
      is array (Ada.Containers.Count_Type range <>) of access
         My_Buffer.Buffer_Interface'Class;

    -- Declare the list of buffers to be used by the scatter buffer

    -- Note using the 'Access in a nested scope. Works fine.
    Scatterbuffers : constant Scatterbuffer_Array := Scatterbuffer_Array'
      (1 => My_Bounded'Access,
       2 => My_Unbounded'Access,
       3 => My_Passive'Access);  -- Synchronous Buffer based on Protected

    -- Instantiate the scatter buffer
    package My_Simple_Scatter_Buffer is new
      My_Buffer.Simple_Scattered
        (Scatterbuffer_List => Scatterbuffer_Array,
         Scatter_Buffers => Scatterbuffers);

    -- Declare the Scatter Buffer
    My_Scattered : My_Simple_Scatter_Buffer.Buffer
      (Maximum_Capacity => 5133);
begin
    -- Each write request typically gets written to a different buffer
    -- or can be split across buffers, if a targetted buffer does
    -- not have the available space
    My_Scattered.Write ("Hello World");

    -- Each read request reads from a separate buffer
    -- or can be split across buffers, if a buffer does not
    -- have the available data
    Put_Line ("Text Read=" & My_Scattered.Read (Maximum_Count => 5));
  -- Reads a string

end Main_Test;

The use of Ada 2005 interfaces is also helpful here.
Some of the buffers in the scatter list are synchronous buffer types,
based on a task or a protected type. Others are intended only for
non-synchronous use.

Scatter buffers can in fact appear in the list of buffers in another 
scatter buffer list.

If a scatter buffer is acting as a producer, then there can be separate 
consumers for each of the buffers in the buffer list.

Likewise, if a scatter buffer is acting as a consumer, then there can be 
separate producers for each of the buffers in the buffer list.

It is possible to have one scatter buffer instance as a producer, and 
another scatter buffer instance as a consumer, for the same list of buffers.

My framework also gives me synchronous scatter buffer forms where the
scatter buffer itself can be synchronized to allow multiple readers/writers.

I am thinking of releasing the source under some license similar to the 
GNAT license, but haven't sorted out the details of how to go about this 
yet.

- Brad

> 
>> This brings the old problem of having an access value of global type
>> pointing to local object (GNAT says: "non-local pointer cannot point
>> to local object").
>> Unrestricted_Access hides the issue,...
> 
> Don't use Unrestricted_Access here (which is not standard Ada).
> Use Unchecked_Access.  Yes, it's dangerous -- you need to ensure that
> Scatter_Input_Data doesn't save pointers to locals into globals,
> along with comments explaining what clients can do.
> There's no better solution.
> 
>> ... but I know from Rationale that
>> anonymous access types were supposed to solve some of that mess.
> 
> "Supposed to", maybe.  But they don't.  I suggest you stay away from
> them.  They are confusing, and don't help here.
> 
>> ...The
>> problem is that I cannot find any way to benefit from them.
> 
> Right.  Neither can I.
> [snipped discussion of failed attempts to use anonymous
> access types]
> 
> By the way, ARG is discussing some ideas for improving anonymous
> access types.  My own opinion is that this will not work out,
> but others do not agree with me.
> 
> - Bob