C bindings, Interfaces.C.Pointers etc.

C bindings, Interfaces.C.Pointers etc.

[Dr. Adrian Wrigley]

Hi folks!

I have a problem creating a binding to a C library.

The library makes extensive use of arrays of arrays of structs.
Both levels of arrays are of variable length.

 struct node { int index; double value; };
 struct problem { node **x; };

When I attempt to define Ada types for this type of thing,
I find I can't use access types to the arrays with the
necessary C convention.
"warning: this access type does not correspond to C pointer"
is the compiler error.

I have tried using Interfaces.C.Pointers, but I have run
into (soluble?) problems.

1)  How to I allocate an array (in Ada) and assign it to a
   C pointer? (in a single expression?)

2)  How do I free the allocaed array from the C pointer?
   Do I need to keep track of the return values from "new"
   separately from the pointers?

3)  How do I access the n'th element of the Array, given the
   C pointer.

4)  Is there some convenient shortcut to avoid using I.C.P?
   perhaps by declaring an Ada constrained array much larger than
   intended, and using a subset.
   Using I.C.P seems massively verbose (10x) for this application.

As I understand it, if the array is unconstrained, an access type
to the array contains the dope information, and can't be used
as a C pointer.

I have given an example of my problem code below (without I.C.P).

Any ideas?
--
Adrian Wrigley, Cambridge, England


procedure Problem is

-- C code is:
--
-- struct node
-- {
--      int index;
--      double value;
-- };

-- struct problem
-- {
--      node **x;
-- };

-- x is a pointer to an array of pointers to arrays of nodes (both variable size)

   type Node_T is record
      Index : Interfaces.C.Int;
      Value : Interfaces.C.Double;
   end record;
   pragma Convention (C, Node_T);

   type NodeArray_T is array (Interfaces.C.Size_T range <>) of Node_T;
   pragma Convention (C, NodeArray_T);
   type NodeArray_A is access all NodeArray_T;
   pragma Convention (C, NodeArray_A);

   type NodeArrayArray_T is array (Interfaces.C.Size_T range <>) of NodeArray_A;
   pragma Convention (C, NodeArrayArray_T);
   type NodeArrayArray_A is access all NodeArrayArray_T;
   pragma Convention (C, NodeArrayArray_A);

   type Problem_T is record
      X : NodeArrayArray_A;
   end record;
   pragma Convention (C, Problem_T);

   MyProblem : Problem_T;
   MyNode : Node_T;

begin

   MyProblem := (X => new NodeArrayArray_T (1 .. 100));

   for I in MyProblem.X'range loop
      MyProblem.X (I) := new NodeArray_T (1 .. I);
   end loop;

   MyNode := MyProblem.X (10)(5);

end Problem;

Re: C bindings, Interfaces.C.Pointers etc.

[tmoran]

>4)  Is there some convenient shortcut to avoid using I.C.P?
>    perhaps by declaring an Ada constrained array much larger than
>    intended, and using a subset.
  That's what I'd do.  You clearly can't expect Ada to bounds check an
array that has no bounds, ie, a C array.  If C is doing all the allocating
there's no memory penalty.  If Ada is doing the allocating, you might want
to tell Ada you have a (fixed size) one dimensional array of pointers to
elements, rather than to arrays, and let C assume a pointer to a first
element is a pointer to an array.

Re: C bindings, Interfaces.C.Pointers etc.

[Jeffrey Carter]

Dr. Adrian Wrigley wrote:

> Hi folks!
> 
> I have a problem creating a binding to a C library.
> 
> The library makes extensive use of arrays of arrays of structs.
> Both levels of arrays are of variable length.
> 
>  struct node { int index; double value; };
>  struct problem { node **x; };
> 
> When I attempt to define Ada types for this type of thing,
> I find I can't use access types to the arrays with the
> necessary C convention.
> "warning: this access type does not correspond to C pointer"
> is the compiler error.

    -- struct node { int index; double value; };
    type Node_Info is record
       Index : Interfaces.C.Int;
       Value : Interfaces.C.Double;
    end record;
    pragma Convention (C, Node_Info);

    -- struct problem { node **x; };
    type Node_Ptr     is access all Node_Info;
    pragma Convention (C, Node_Ptr);
    type Node_Ptr_Ptr is access all Node_Ptr;
    pragma Convention (C, Node_Ptr_Ptr);
    type Problem_Info is record
       X : Node_Ptr_Ptr;
    end record;
    pragma Convention (C, Problem_Info);

This compiles fine. This is an exact duplicate of the C declarations. To 
deal with this using a higher level Ada abstraction, you'd wrap 
something around this to provide that abstraction.

-- 
Jeff Carter
"Blessed is just about anyone with a vested interest in the status quo."
Monty Python's Life of Brian
73

Re: C bindings, Interfaces.C.Pointers etc.

[Simon Wright]

Jeffrey Carter <spam@spam.com> writes:

> Dr. Adrian Wrigley wrote:
> 
> > Hi folks!
> > I have a problem creating a binding to a C library.
> > The library makes extensive use of arrays of arrays of structs.
> > Both levels of arrays are of variable length.
> >  struct node { int index; double value; };
> >  struct problem { node **x; };
> > When I attempt to define Ada types for this type of thing,
> > I find I can't use access types to the arrays with the
> > necessary C convention.
> > "warning: this access type does not correspond to C pointer"
> > is the compiler error.
> 
>     -- struct node { int index; double value; };
>     type Node_Info is record
>        Index : Interfaces.C.Int;
>        Value : Interfaces.C.Double;
>     end record;
>     pragma Convention (C, Node_Info);
> 
>     -- struct problem { node **x; };
>     type Node_Ptr     is access all Node_Info;
>     pragma Convention (C, Node_Ptr);
>     type Node_Ptr_Ptr is access all Node_Ptr;
>     pragma Convention (C, Node_Ptr_Ptr);
>     type Problem_Info is record
>        X : Node_Ptr_Ptr;
>     end record;
>     pragma Convention (C, Problem_Info);
> 
> This compiles fine. This is an exact duplicate of the C
> declarations.

But not, I think, of the implications. Where is the equivalent of 

  problem p;
  node *np = *p.x;
  np++;               // <<<<

I've used constrained subtypes in the past:

  type Node_Array is array (Integer range <>) of Node_Info;
  
then in a subprogram taking a Node_Array parameter Nodes,

  subtype Specific_Node_Array is Node_Array (Nodes'First .. Nodes'Last);
  type Node_Array_Access is access Specific_Node_Array;
  pragma Convention (C, Node_Array_Access);

then declare a local binding to the C operation.

Same again for Node_Ptr_Array, though the Ada side of this is going to
be rather difficult since the Node_Ptrs in this are really
Node_Array_Accesses and the Node_Arrays are not all of the same
length.

Perhaps the simplest way would be to use System.Address:

   (Arr (Arr'First))'Address

>               To deal with this using a higher level Ada
> abstraction, you'd wrap something around this to provide that
> abstraction.



-- 
Simon Wright                               100% Ada, no bugs.

Re: C bindings, Interfaces.C.Pointers etc.

[Dr. Adrian Wrigley]

On Tue, 11 May 2004 18:48:57 +0000, Jeffrey Carter wrote:

>     -- struct node { int index; double value; };
>     type Node_Info is record
>        Index : Interfaces.C.Int;
>        Value : Interfaces.C.Double;
>     end record;
>     pragma Convention (C, Node_Info);
> 
>     -- struct problem { node **x; };
>     type Node_Ptr     is access all Node_Info;
>     pragma Convention (C, Node_Ptr);
>     type Node_Ptr_Ptr is access all Node_Ptr;
>     pragma Convention (C, Node_Ptr_Ptr);
>     type Problem_Info is record
>        X : Node_Ptr_Ptr;
>     end record;
>     pragma Convention (C, Problem_Info);
> 
> This compiles fine. This is an exact duplicate of the C declarations. To 
> deal with this using a higher level Ada abstraction, you'd wrap 
> something around this to provide that abstraction.

I hadn't thought of doing this - it doesn't seem to provide easy answers
for allocation and indexing though.  In this application, I am building
data structures for use by a C/C++ library.  The wrapper you're
suggesting (I guess) is for accessing elements of the arrays,
and allocating/freeing them.  I'd have to check that the additional
code didn't impose excessive overheads.

The solution I have adopted in the end is based on my idea (4),
mentioned by tmoran and Martin Chrischik (see other messages).

Thanks!
-- 
Adrian

Re: C bindings, Interfaces.C.Pointers etc.

[Martin Krischik]

Dr. Adrian Wrigley wrote:

> Hi folks!
> 
> I have a problem creating a binding to a C library.
> 
> The library makes extensive use of arrays of arrays of structs.
> Both levels of arrays are of variable length.
> 
>  struct node { int index; double value; };
>  struct problem { node **x; };
> 
> When I attempt to define Ada types for this type of thing,
> I find I can't use access types to the arrays with the
> necessary C convention.
> "warning: this access type does not correspond to C pointer"
> is the compiler error.
> 
> I have tried using Interfaces.C.Pointers, but I have run
> into (soluble?) problems.
> 
> 1)  How to I allocate an array (in Ada) and assign it to a
>    C pointer? (in a single expression?)

Ada_Array : <some array>

C_Pointer  :  constant C_Pointers.Pointer := Ada_Array (0)'Unchecked_Access;

Depending on how Ada_Array is defined it might work without Unchecked_.
 
> 2)  How do I free the allocaed array from the C pointer?
>    Do I need to keep track of the return values from "new"
>    separately from the pointers?

Since the Pointer are for use with C you might consider:

   function malloc (Size : size_t) return System.Address;
   pragma Import (C, malloc, "malloc");
   procedure free (Ptr : System.Address);
   pragma Import (C, free, "free");

You can either combine it with package System.Address_To_Access_Conversions
or a special Storrage_Pool.
 
> 3)  How do I access the n'th element of the Array, given the
>    C pointer.

Depending on actual use You can either convert the C array to an Ada array
or use:

Access_Pointer := Base_Pointer + n;

... Base_Pointer.all ...

> 4)  Is there some convenient shortcut to avoid using I.C.P?
>    perhaps by declaring an Ada constrained array much larger than
>    intended, and using a subset.
>    Using I.C.P seems massively verbose (10x) for this application.

From you example down I suggest "aliased" Elements

type NodeArray_T is array (Interfaces.C.Size_T range <>) of  aliased Node_T;
type NodeArrayArray_T is array (Interfaces.C.Size_T range <>) of aliased 
NodeArray_A;

then you can use I.C.P.Value and I.C.P.Copy_Array as alternative to pure
Pointer use. Giving you more flexibility.
 
> As I understand it, if the array is unconstrained, an access type
> to the array contains the dope information, and can't be used
> as a C pointer.
> 
> I have given an example of my problem code below (without I.C.P).

Well, it does look Ok to me. Apart from the missing "aliased". All elements
in an C array need an address while in Ada they don't. (Look for pragma
Pack and the attributes 'Size and 'Component_Size for details).

With Regards

Martin
-- 
mailto://krischik@users.sourceforge.net
http://www.ada.krischik.com

Re: C bindings, Interfaces.C.Pointers etc.

[Dr. Adrian Wrigley]

Thanks Martin for this suggestion.

I have given (an approximation of) my code below.

It doesn't use the I.C.Pointers package, and allows
Ada programs to access elements of the arrays using
standard array notation.  The main problem is that
the bound Ada uses are over-sized, and simple use
of 'range and 'last will fall off the ends.

I have used a generic package to create the appropriate
types needed, and wrap the "malloc" and "free".
This way, the C library can intermix allocations and
deallocations with those of the Ada code.

I put in the "aliased" - something I often forget
until the compiler complains at me!

Overall this solution meets my needs very well, and
if I accidentally use the invalid array (upper) bound,
the program crashes very rapidly (like it would have
done in C with a similar bug!).

I'm a little surprised that I.C.Pointers doesn't seem
to provide a convenient and efficient solution to this
problem - using C pointers directly in array access.

The next problem I am thinking about with this project
is how to abort a call into the C/C++ library safely.
Sometimes the library takes *much* too long to compute
a result. (If the parameters are unsuitable, it can
take hours vs seconds with good parameters).
If the thread executing the library call is aborted,
memory will hemmorage from the application.  This problem
will probably need a hacked version of the (Open Source)
library.  I'm not sure how to communicate with the
call as it executes telling it to clean up and exit
immediately.  Shared memory flags? Signals?

Thanks for the input on these problems!
--
Adrian


generic

   type Element_T is private;
   IndexBase : Integer := 0;

package CArrayUtils is

   type ElementArray_T is array (0 .. Interfaces.C.Size_T'Last) of aliased Element_T;
   pragma Convention (C, ElementArray_T);
   type ElementArray_A is access ElementArray_T;
   pragma Convention (C, ElementArray_A);

   function CAlloc (N : Integer) return ElementArray_A;
   procedure CFree (X : in out ElementArray_A);

end CArrayUtils;

package body CArrayUtils is

   function CAlloc (N : Integer) return ElementArray_A is
      function Malloc (A : Interfaces.C.Size_T) return ElementArray_A;
      pragma Import (C, Malloc, "malloc");
   begin
-- Is the size in bytes calculated correctly???  Probably not!
      return Malloc (Interfaces.C.Size_T (N * Element_T'Size / 8 + 1));
   end CAlloc;

   procedure CFree (X : in out ElementArray_A) is
      procedure Free (X : ElementArray_A);
      pragma Import (C, Free, "free");
   begin
      Free (X);
      X := null;
   end CFree;

end CArrayUtils;

with Interfaces.C;

package Blob is

  type Node_T is record
      Index : Interfaces.C.Int;
      Value : Interfaces.C.Double;
   end record;
   pragma Convention (C, Node_T);

   package NodeArray_P is new CArrayUtils (Element_T => Node_T);
   subtype NodeArray_A is NodeArray_P.ElementArray_A;

   package NodeArrayArray_P is new CArrayUtils (Element_T => NodeArray_A);
   subtype NodeArrayArray_A is NodeArrayArray_P.ElementArray_A;

   package DoubleArray_P is new CArrayUtils (Element_T => Interfaces.C.Double);
   subtype DoubleArray_A is DoubleArray_P.ElementArray_A;


   type SVMProblem_T is record
      L  : Interfaces.C.Int;
      Y  : DoubleArray_A;       -- Pointer to array of double
      X  : NodeArrayArray_A; -- (pointer to array of (pointer to array of SVMNode))
   end record;
   pragma Convention (C, Problem_T);

end Blob;

Re: C bindings, Interfaces.C.Pointers etc.

[Martin Krischik]

Dr. Adrian Wrigley wrote:

> Thanks Martin for this suggestion.
> 
> I have given (aninvolvedimation of) my code below.
> 
> It doesn't use the I.C.Pointers package, and allows
> Ada programs to access elements of the arrays using
> standard array notation.  The main problem is that
> the bound Ada uses are over-sized, and simple use
> of 'range and 'last will fall off the ends.

There is one more way but only usable when you C and Ada compiler use the
same memory layout for variables. The example works for Strings:

   type C_Array is array (Natural range <>) of aliased Character;

   for C_Array'Component_Size use Interfaces.C.char'Size;

   pragma Convention (
             Convention => C,
             Entity     => C_Array);

   package C_Pointers
   is new
      Interfaces.C.Pointers (
         Index              => Natural,
         Element            => Character,
         Element_Array      => C_Array,
         Default_Terminator => Character'First);

   type C_String
   is record
      --  Address of some C Array
      Data : C_Pointers.Pointer;
      --  Size of the Array.
      Size : Interfaces.C.size_t;
   end record;

Ada_String  : String (1 .. Natural (Some_C_String.Size));

for Ada_String 'Address use Some_C_String.Data.all'Address;

The trick is in the last line. This is of corse an Unchecked_Convertion
without actually moving the Data. Use at your own risk. 

You can probably reduce the code involved by using an normal access to
Character for C_String.Data.

> Overall this solution meets my needs very well, and
> if I accidentally use the invalid array (upper) bound,
> the program crashes very rapidly (like it would have
> done in C with a similar bug!).

With the trick above Ada will check the array bounds - when Size containts
the right value.
 
> I'm a little surprised that I.C.Pointers doesn't seem
> to provide a convenient and efficient solution to this
> problem - using C pointers directly in array access.

I.C.Pointers is supposed to all the checked convertions needed and will work
when C and Ada use different memory layout - at least that's the theorie.
 
With Regards

Martin.

-- 
mailto://krischik@users.sourceforge.net
http://www.ada.krischik.com

Re: C bindings, Interfaces.C.Pointers etc.

[Jeffrey Carter]

Dr. Adrian Wrigley wrote:
> Thanks Martin for this suggestion.
> 
> I have given (an approximation of) my code below.
> 
> It doesn't use the I.C.Pointers package, and allows
> Ada programs to access elements of the arrays using
> standard array notation.  The main problem is that
> the bound Ada uses are over-sized, and simple use
> of 'range and 'last will fall off the ends.

Once you have an acceptable "oversized" Ada array, you can slice it to 
obtain a usable Ada array of the correct size.

-- 
Jeff Carter
"Sir Robin the-not-quite-so-brave-as-Sir-Lancelot,
who had nearly fought the Dragon of Angnor,
who nearly stood up to the vicious Chicken of Bristol,
and who had personally wet himself at the
Battle of Badon Hill."
Monty Python & the Holy Grail
68