Thick bindings to a C library and gnattest: suggestions?

Thick bindings to a C library and gnattest: suggestions?

[ziotom78]

Hi to everybody,

  I am a novice Ada programmer which is trying to convert a C++ code
into an Ada program. The C++ code relies on a C library which reads
and writes vectors of numeric data in binary format (CFITSIO), for
which I am developing a set of "thick" Ada bindings, named AdaFITS.
Overall my experience is great: I feel Ada orders of magnitude more
secure and "nicer" than C++ (with its odd syntax and complex
constructs).

There are a few things for which I do not know what is the best
"Ada"-ish way to do them, and I would like to ask for the advice of
some experts.

First question: the vectors used by the CFITSIO library are sometimes
huge (millions of elements), sometimes very small (~ 10 elements).
I decided to always allocate them on the heap, using declarations like
these:

   subtype Double is Interfaces.C.double;
   type Double_Array is array (Positive range <>)
     of Interfaces.C.double;
   pragma Convention (C, Double_Array);
   type Double_Array_Ptr is access Double_Array;

(similar declarations exist for arrays of integers/long...), and then
using "new Double_Array" whenever I need to allocate room for an
array. Every binding function only uses Double_Array_Ptr. Is this the best
way to do this in Ada? Every text I read about Ada says to stay away
from objects allocated on the heap, but I do not see any other
solution here.

Second question. I run gnattest periodically to keep the list of tests
updated with the functions I am implementing. Gnattest is smart enough
not to overwrite the existing implementations of my tests, however it
does keep rewriting two things:

1. The project "gnattest/harness/test_driver.gpr", where I put the
"-lcfitsio" linker switch to link the C library. Understandably, not
having such a switch causes a lot of undefined references during the
linking phase.

2. The list of "with ..." clauses at the beginning of the .adb file
implementing the tests (in the subdir "gnattest/tests"). I need at
least to include the library I am testing (AdaFITS), as well as a few
packages I implemented to ease comparisons between objects like the
Double_Array_Ptr above and other things like Ada.Directories and
Ada.Strings.Unbounded.

I am sure there is some clever way to solve these two minor points,
but so far I have not been able to find it. I tried e.g. to put
"-lcfitsio" in the project file of the AdaFITS library, but with no
success. (If Ada works like C++, I believe that things like
"-lcfitsio" are used when a *program* is built, not a library.)

Thanks to everybody for any advice you will give me,
  Maurizio.

Re: Thick bindings to a C library and gnattest: suggestions?

[Dmitry A. Kazakov]

On Mon, 1 Jul 2013 02:02:34 -0700 (PDT), ziotom78@gmail.com wrote:

> First question: the vectors used by the CFITSIO library are sometimes
> huge (millions of elements), sometimes very small (~ 10 elements).
> I decided to always allocate them on the heap, using declarations like
> these:

Why should bindings care about that?

>    subtype Double is Interfaces.C.double;
>    type Double_Array is array (Positive range <>)
>      of Interfaces.C.double;
>    pragma Convention (C, Double_Array);
>    type Double_Array_Ptr is access Double_Array;

Using unconstrained arrays could be troublesome because they contain
bounds. You may consider the package B.3.2

   Interfaces.C.Pointers

which does this.

But normally you do not need array pointers in bindings except for the
cases when the C library stores the pointer to keep it after returning from
the subprogram.

> (similar declarations exist for arrays of integers/long...), and then
> using "new Double_Array" whenever I need to allocate room for an
> array. Every binding function only uses Double_Array_Ptr. Is this the best
> way to do this in Ada? Every text I read about Ada says to stay away
> from objects allocated on the heap, but I do not see any other
> solution here.

Why don't you simply pass the array down to the C subprogram? You can do
something like:

   type Double_Array is array (Positive range <>)
      of aliased Interfaces.C.double;
   pragma Convention (C, Double_Array);
   procedure Foo (A : Double_Array);

Implementation:

   type Double_Ptr is access all Interfaces.C.double;
   pragma Convention (C, Double_Ptr);

   procedure Foo (A : Double_Array) is
   --
   -- Assuming foo's signature in C:
   --
   --    foo (double * a, unsigned n);
   --
      procedure Internal (A : Double_Ptr; N : Interfaces.C.unsigned);
      pragma Import (C, Internal, "foo");
   begin
      Internal (A (A'First)'Access, A'Length);
   end Foo;

> I am sure there is some clever way to solve these two minor points,
> but so far I have not been able to find it. I tried e.g. to put
> "-lcfitsio" in the project file of the AdaFITS library, but with no
> success.

Make a library project file for cfitsio instead. "with" it from your
project. GNAT knows how to handle it and will add appropriate linker
switches to any project using it directly or indirectly. A library project
file could look like:

project cfitsio is
   for Externally_Built use "true"; -- Do not bother to compile me
   for Source_Files use (); -- No sources
   for Library_Dir use ".";   -- Where .llb, .a, .dll, .so are
   for Library_Name use "cfitsio"; -- Without "lib" prefix!
   for Library_Kind use "dynamic"; -- A DLL
end cfitsio;

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: Thick bindings to a C library and gnattest: suggestions?

[Maurizio Tomasi]

Hi Dmitry,

  Many thanks for your suggestions. You gave me a lot of things to think about.  I have a few questions about what you wrote.

> > First question: the vectors used by the CFITSIO library are sometimes
> > huge (millions of elements), sometimes very small (~ 10 elements).
> > I decided to always allocate them on the heap, using declarations like
> > these:
> 
> Why should bindings care about that?

Shouldn't they care? Perhaps I am missing something regarding the Ada language, in fact C++ does not care about whether an array is allocated on the stack or on the heap. But if I declare a function like the following:

function Read_Double_Array_From_FITS (File_Name : String) return Double_Array;

and I implement it in the following way:

--  Call the C functions that returns the number of rows in the file
Num_Of_Elements := Get_Number_Of_Elements(FITS_File);
declare
  Vector_To_Return : Double_Array (1 .. Num_Of_Elements);
begin
  --  Call the C function that fills Vector_To_Return
  ...
  return Vector_To_Return;
end;

then every time I read some huge array I will get a STORAGE_ERROR near the declaration of "Vector_To_Return". This is the reason why I declared the return type of "Read_Data_From_FITS" as a Double_Array_Ptr instead of a Double_Array, and implemented it as:

Num_Of_Elements := Get_Number_Of_Elements(FITS_File);
declare
  Vector_To_Return : Double_Array_Ptr;
begin
  Vector_To_Return := new Double_Array (1 .. Num_Of_Elements);
  --  Call the C function that fills Vector_To_Return
  ...
  return Vector_To_Return;
end;

Your answer makes me think there is a smarter way of avoiding the distinction between Double_Array_Ptr and Double_Array. (This would in fact make Ada much more like C++, where this distinction does not exist.) Is this the reason why in your example you declared Double_Array an array of *aliased* doubles? Or was your point just to use the address of its first member?

> Using unconstrained arrays could be troublesome because they contain
> bounds. You may consider the package B.3.2
> 
>    Interfaces.C.Pointers
> 
> But normally you do not need array pointers in bindings except for the
> cases when the C library stores the pointer to keep it after returning from
> the subprogram.

I need the C library only in those cases where I need to save/retrieve arrays of data from disk in binary format. But 100% of the calculations on the values stored in these arrays will be done using Ada code (e.g., loops...). The point of rewriting my C++ program in Ada is because my codes need to work with very large arrays (I am part of a large astrophysics project), and I find Ada arrays much more appealing than C++ vectors. The fact that Ada arrays can have arbitrary bounds whom they carry is one of the things that made me interested towards Ada at the beginning. Why did you say this might be "troublesome"?

Given this context, is your suggestion of using Interfaces.C.Pointers still valid?

> Why don't you simply pass the array down to the C subprogram? You can do
> something like:
> 
>    type Double_Array is array (Positive range <>)
>       of aliased Interfaces.C.double;
>    pragma Convention (C, Double_Array);
>    procedure Foo (A : Double_Array);
> 
> Implementation:
> 
>    type Double_Ptr is access all Interfaces.C.double;
>    pragma Convention (C, Double_Ptr);
> 
>    procedure Foo (A : Double_Array) is
>    --
>    -- Assuming foo's signature in C:
>    --
>    --    foo (double * a, unsigned n);
>    --
>       procedure Internal (A : Double_Ptr; N : Interfaces.C.unsigned);
>       pragma Import (C, Internal, "foo");
>    begin
>       Internal (A (A'First)'Access, A'Length);
>    end Foo;

But what if A'Length is so large that the array does not fit into the stack? Increasing the stack to a good value would be better than allocating objects on the heap? I would have problems in determining what size to use, as my code should work on a variety of systems (my laptop, supercomputing facilities...) and the amount of data it needs to load depends on the context.

> > I am sure there is some clever way to solve these two minor points,
> > but so far I have not been able to find it. I tried e.g. to put
> > "-lcfitsio" in the project file of the AdaFITS library, but with no
> > success.
> 
> Make a library project file for cfitsio instead. "with" it from your
> project. GNAT knows how to handle it and will add appropriate linker
> switches to any project using it directly or indirectly. A library project
> file could look like:
> 
> project cfitsio is
>    for Externally_Built use "true"; -- Do not bother to compile me
>    for Source_Files use (); -- No sources
>    for Library_Dir use ".";   -- Where .llb, .a, .dll, .so are
>    for Library_Name use "cfitsio"; -- Without "lib" prefix!
>    for Library_Kind use "dynamic"; -- A DLL
> end cfitsio;

This is really a good idea, I did not think about this! There are only two problems with this approach:

1. The CFITSIO library is often compiled using ad-hoc flags in supercomputing facilities, in order to make it work better with the storage systems. I need to use the library provided by the system, not my own. But...

2. ...the system library is not always available as a dynamic .so file: in some cases I must statically link CFITSIO (the libcfitsio.so library is not available on every node of the cluster: when I discovered this, the system admin told me to link statically).

Is there no other option? I searched the "gnattest" documentation for some way to prevent it from overwriting some parts of the files, but with no success.

Again, many thanks for your answers.
  Maurizio.

Re: Thick bindings to a C library and gnattest: suggestions?

[Simon Wright]

Maurizio Tomasi <ziotom78@gmail.com> writes:

> 2. ...the system library is not always available as a dynamic .so
> file: in some cases I must statically link CFITSIO (the libcfitsio.so
> library is not available on every node of the cluster: when I
> discovered this, the system admin told me to link statically).

This should do the trick:

   for Library_Kind use "static";

(

Re: Thick bindings to a C library and gnattest: suggestions?

[Maurizio Tomasi]

Many thanks Simon, my problem was that I cannot decide "a priori" when to link statically and when dynamically, but you helped me in devise a solution:

project cfitsio is 
  for Externally_Built use "true";
  for Source_Files use ();
  for Library_Dir use external("CFITSIO_LIB_DIR");
  for Library_Name use "cfitsio";
  for Library_Kind use external("CFITSIO_LINKING");
end cfitsio; 


On my laptop running Linux Mint I have both the static and dynamic versions of CFITSIO, and now I am able to compile them using these commands:

CFITSIO_LIB_DIR=/usr/lib/x86_64-linux-gnu CFITSIO_LINKING=dynamic gprbuild test_driver.gpr

CFITSIO_LIB_DIR=/usr/lib/ CFITSIO_LINKING=static gprbuild test_driver.gpr

So, one of my problems is fully solved!
  Maurizio.

Re: Thick bindings to a C library and gnattest: suggestions?

[Dmitry A. Kazakov]

On Mon, 1 Jul 2013 04:11:55 -0700 (PDT), Maurizio Tomasi wrote:

>>> First question: the vectors used by the CFITSIO library are sometimes
>>> huge (millions of elements), sometimes very small (~ 10 elements).
>>> I decided to always allocate them on the heap, using declarations like
>>> these:
>> 
>> Why should bindings care about that?
> 
> Shouldn't they care? Perhaps I am missing something regarding the Ada
> language, in fact C++ does not care about whether an array is allocated on
> the stack or on the heap. But if I declare a function like the following:
> 
> function Read_Double_Array_From_FITS (File_Name : String) return Double_Array;
> 
> and I implement it in the following way:
> 
> --  Call the C functions that returns the number of rows in the file
> Num_Of_Elements := Get_Number_Of_Elements(FITS_File);
> declare
>   Vector_To_Return : Double_Array (1 .. Num_Of_Elements);
> begin
>   --  Call the C function that fills Vector_To_Return
>   ...
>   return Vector_To_Return;
> end;
> 
> then every time I read some huge array I will get a STORAGE_ERROR near the
> declaration of "Vector_To_Return". This is the reason why I declared the
> return type of "Read_Data_From_FITS" as a Double_Array_Ptr instead of a
> Double_Array, and implemented it as:
> 
> Num_Of_Elements := Get_Number_Of_Elements(FITS_File);
> declare
>   Vector_To_Return : Double_Array_Ptr;
> begin
>   Vector_To_Return := new Double_Array (1 .. Num_Of_Elements);
>   --  Call the C function that fills Vector_To_Return
>   ...
>   return Vector_To_Return;
> end;
> 
> Your answer makes me think there is a smarter way of avoiding the
> distinction between Double_Array_Ptr and Double_Array. (This would in fact
> make Ada much more like C++, where this distinction does not exist.) Is
> this the reason why in your example you declared Double_Array an array of
> *aliased* doubles? Or was your point just to use the address of its first member?

The first point is that it is not the objective of bindings to manage
memory. Of course, there could be bindings which do that, in which case you
would allocate objects transparently to the caller and have some garbage
collection schema behind opaque handles to the objects. This is a possible
design but it is not what you probably wanted. So let us take for granted
that it is the client's responsibility to allocate objects. In this case
the bindings shall work for any kind of objects allocated in any possible
memory pool, stack included.

Now, how would you do that? There are many ways.

1. Prior to Ada 2005, the usual method was one you find in
Ada.Text_IO.Get_Line. You use a procedure and a parameter telling how much
elements were written:

   procedure Read_Double_Array_From_FITS
       (A : in out Double_Array; Last : out Positive);

Here Last indicates the last element of A containing data. The
implementation would raise End_Error when there are more than A'Length
elements in A. Get_Line, for example, returns Last = A'Last meaning that
there is more to read. The caller uses A (A'First..Last) in further calls.

In my libraries I am using a slightly more universal approach:

   procedure Read_Double_Array_From_FITS
       (A : in out Double_Array; Pointer : in out Positive);

Here A (Pointer..A'Last) is where the result is stored and then Pointer is
advanced to the first element following the input. So the result is between
old pointer and new pointer - 1.

2. With Ada 2005 you can use return statement

   procedure Read_Double_Array_From_FITS return Double_Array is
   begin
       return Result : Double_Array (1..Get_Number_Of_Elements) do
           -- Fill Result here
       end return;
   end Read_Double_Array_From_FITS;

The caller is free to use this function with the allocator new:

   A : access Double_Array :=
           new Double_Array'(Read_Double_Array_From_FITS);

Theoretically the compiler could optimize temp object away. (You should
check if GNAT really does this)

Dealing with huge arrays I would prefer the approach #1. I would probably
allocate some scratch buffer and reuse it all over again.

Another approach is using #1 or #2 with some custom storage pool organized
as a stack or arena in order to minimize memory management overhead.

In any case, it is not the bindings' business.

> The fact that Ada arrays can have arbitrary bounds whom they
> carry is one of the things that made me interested towards Ada at the
> beginning. Why did you say this might be "troublesome"?

Because C arrays have none. When you want to pass an Ada array to C you
must flatten it. One way is to declare a subtype:

   procedure Bar (A : Double_Array) is
      subtype Flat is A (A'Range);
      B : Flat := ...;
   begin
      --- B does not have bounds and can be passed around as-is

Some pass pointer to the first element. After all, C's arrays are a
fiction.

Some use addresses. E.g. GtkAda bindings pass System.Address for any C
objects sparing headache of proper types. Purists would consider this
approach rather being sloppy.

> Given this context, is your suggestion of using Interfaces.C.Pointers still valid?

Yes.
 
>> Why don't you simply pass the array down to the C subprogram? You can do
>> something like:
>> 
>>    type Double_Array is array (Positive range <>)
>>       of aliased Interfaces.C.double;
>>    pragma Convention (C, Double_Array);
>>    procedure Foo (A : Double_Array);
>> 
>> Implementation:
>> 
>>    type Double_Ptr is access all Interfaces.C.double;
>>    pragma Convention (C, Double_Ptr);
>> 
>>    procedure Foo (A : Double_Array) is
>>    --
>>    -- Assuming foo's signature in C:
>>    --
>>    --    foo (double * a, unsigned n);
>>    --
>>       procedure Internal (A : Double_Ptr; N : Interfaces.C.unsigned);
>>       pragma Import (C, Internal, "foo");
>>    begin
>>       Internal (A (A'First)'Access, A'Length);
>>    end Foo;
> 
> But what if A'Length is so large that the array does not fit into the
> stack?

It is a client's problem. 

>>> I am sure there is some clever way to solve these two minor points,
>>> but so far I have not been able to find it. I tried e.g. to put
>>> "-lcfitsio" in the project file of the AdaFITS library, but with no
>>> success.
>> 
>> Make a library project file for cfitsio instead. "with" it from your
>> project. GNAT knows how to handle it and will add appropriate linker
>> switches to any project using it directly or indirectly. A library project
>> file could look like:
>> 
>> project cfitsio is
>>    for Externally_Built use "true"; -- Do not bother to compile me
>>    for Source_Files use (); -- No sources
>>    for Library_Dir use ".";   -- Where .llb, .a, .dll, .so are
>>    for Library_Name use "cfitsio"; -- Without "lib" prefix!
>>    for Library_Kind use "dynamic"; -- A DLL
>> end cfitsio;
> 
> This is really a good idea, I did not think about this! There are only two problems with this approach:
> 
> 1. The CFITSIO library is often compiled using ad-hoc flags in
> supercomputing facilities, in order to make it work better with the
> storage systems. I need to use the library provided by the system, not my
> own.

This is why Externally_Built is set true.

> 2. ...the system library is not always available as a dynamic .so file: in
> some cases I must statically link CFITSIO (the libcfitsio.so library is
> not available on every node of the cluster: when I discovered this, the
> system admin told me to link statically).

Then Library_Kind is "static". You can even have a scenario variable to
select if you wanted to link it statically or dynamically.
 
> Is there no other option?

There are other options, but they are incredibly intrusive, especially when
your bindings are themselves a library. Linker options are ignored for
library projects. They are not "transitive". I.e. each client project will
need to specify ever changing linker switches.

There is also a pragma for specify linker switches in the source code.
Which is obviously the worst maintenance nightmare one could ever
imagine...

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: Thick bindings to a C library and gnattest: suggestions?

[Maurizio Tomasi]

On Monday, July 1, 2013 2:32:27 PM UTC+2, Dmitry A. Kazakov wrote:
> The first point is that it is not the objective of bindings to manage
> memory. Of course, there could be bindings which do that, in which case you
> would allocate objects transparently to the caller and have some garbage
> collection schema behind opaque handles to the objects. This is a possible
> design but it is not what you probably wanted. So let us take for granted
> that it is the client's responsibility to allocate objects. In this case
> the bindings shall work for any kind of objects allocated in any possible
> memory pool, stack included.

Thanks a lot for your long and detailed answer, Dmitry! Your suggestion of letting the client do all the memory management stuff is very reasonable and flexible. I'll try to do that.

Cheers,
  Maurizio.

Re: Thick bindings to a C library and gnattest: suggestions?

[Dmitry A. Kazakov]

On Mon, 1 Jul 2013 05:00:50 -0700 (PDT), Maurizio Tomasi wrote:

> Many thanks Simon, my problem was that I cannot decide "a priori" when to
> link statically and when dynamically, but you helped me in devise a
> solution:
> 
> project cfitsio is 
>   for Externally_Built use "true";
>   for Source_Files use ();
>   for Library_Dir use external("CFITSIO_LIB_DIR");
>   for Library_Name use "cfitsio";
>   for Library_Kind use external("CFITSIO_LINKING");
> end cfitsio; 
> 
> On my laptop running Linux Mint I have both the static and dynamic
> versions of CFITSIO, and now I am able to compile them using these
> commands:
> 
> CFITSIO_LIB_DIR=/usr/lib/x86_64-linux-gnu CFITSIO_LINKING=dynamic gprbuild test_driver.gpr
> 
> CFITSIO_LIB_DIR=/usr/lib/ CFITSIO_LINKING=static gprbuild test_driver.gpr
> 
> So, one of my problems is fully solved!

Use scenario variables instead of environment variables:

   type Library_Type is ("static", "dynamic");
   Linkage : Library_Type := external ("Linkage", "static");

   case Linkage is         -- You could do it directly without case, but
       when "static" => -- case illustrates how powerful this method is
          for Library_Kind use "static";
       when "dynamic"
          for Library_Kind use "dynamic";
   end case;

Scenario variables appear in GPS and can be selected there on the fly.

When compiled using gnatmake or gprbuild you specify them this way:

gnatmake -Pmy_project.gpr -XLinkage="dynamic"

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: Thick bindings to a C library and gnattest: suggestions?

[Simon Wright]

"Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> writes:

> There are other options, but they are incredibly intrusive, especially
> when your bindings are themselves a library. Linker options are
> ignored for library projects. They are not "transitive". I.e. each
> client project will need to specify ever changing linker switches.

In TASH[1], the GPR has

  for Library_Name use "tash";
  for Library_Kind use "static";
  for Library_Dir use "../../lib/tash/lib-static";
  for Source_Dirs use ("../../include/tash");
  for Externally_Built use "true";

  package Linker is
     for Linker_Options use Tash_Options.Linker_Options;
  end Linker;

where tash_options.gpr is generated in the configure stage for the
library; here it contains

   Linker_Options :=
     (
      "-L/usr/lib",
      "-ltk8.5",
      "-ltcl8.5"
     );

and to use it I just 'with "tash";' in my own GPR, and the linker
options get applied as you would hope.

[1] https://sourceforge.net/projects/tcladashell/

Re: Thick bindings to a C library and gnattest: suggestions?

[Jeffrey Carter]

On 07/01/2013 02:02 AM, ziotom78@gmail.com wrote:
>
> First question: the vectors used by the CFITSIO library are sometimes
> huge (millions of elements), sometimes very small (~ 10 elements).
> I decided to always allocate them on the heap, using declarations like
> these:
>
>     subtype Double is Interfaces.C.double;
>     type Double_Array is array (Positive range <>)
>       of Interfaces.C.double;
>     pragma Convention (C, Double_Array);
>     type Double_Array_Ptr is access Double_Array;
>
> (similar declarations exist for arrays of integers/long...), and then
> using "new Double_Array" whenever I need to allocate room for an
> array. Every binding function only uses Double_Array_Ptr. Is this the best
> way to do this in Ada? Every text I read about Ada says to stay away
> from objects allocated on the heap, but I do not see any other
> solution here.

One should not use heap allocation unnecessarily, as it requires memory 
management, which is often complex and error prone, but that does not mean one 
should never use heap allocation. When it is necessary, there is no alternative. 
What one should not do is have visible access types in public package specs.

As others have pointed out, generally the application is what knows the sizes of 
the objects it works on, and that decides whether heap allocation is required.

I would not use "_Array" in type names. We know it's an array because the type 
declaration says so. The type name should indicate its intended meaning or use, 
not its implementation.

Another approach not mentioned is the use of Ada.Containers.Vectors. This 
implements unbounded arrays. Internally, the arrays are allocated on the heap 
and memory managed, but this complication is hidden from the user. Ada 12 
provides the ability to use them like an array (use of those features currently 
limits you to a single compiler, though).

(Aside: in a world in which GB of RAM are common, it seems odd for a language to 
require explicit heap allocation, and the associated memory management, for 
objects that fit in memory, but not in the stack. Surely it would be better for 
the language to allow the developer to simply declare the object, and for the 
compiler to decide where it will fit, and allocate it and manage its memory 
appropriately. A means to disable automatic heap allocation would be needed for 
systems that disallow heap allocation.)

-- 
Jeff Carter
"Hold your temper. Count ten.... Now let 'er go.
You got a good aim."
Never Give a Sucker an Even Break
105

Re: Thick bindings to a C library and gnattest: suggestions?

[Simon Wright]

"Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> writes:

> When compiled using gnatmake or gprbuild you specify them this way:
>
> gnatmake -Pmy_project.gpr -XLinkage="dynamic"

Though you can use env vars if you like and if it suits your build
environment: in bash,

   $ Linkage=dynamic gnatmake -P my_project.gpr

Re: Thick bindings to a C library and gnattest: suggestions?

[Jerry]

On Monday, July 1, 2013 2:02:34 AM UTC-7, Maurizio Tomasi wrote:
> Hi to everybody,
> snip
>   Maurizio.

This is maybe a little bit off-topic but handy when allocating things from the heap. See

https://groups.google.com/forum/?hl=en#!topic/comp.lang.ada/QZhk7ZHMk30

and search the page for "renames" for a neat trick by Brian Drummond. You'll be glad you did.

Jerry

Re: Thick bindings to a C library and gnattest: suggestions?

[Shark8]

On Monday, July 1, 2013 9:16:22 PM UTC-6, Jerry wrote:
> On Monday, July 1, 2013 2:02:34 AM UTC-7, Maurizio Tomasi wrote:
> 
> > Hi to everybody,
> 
> > snip
> 
> >   Maurizio.
> 
> 
> 
> This is maybe a little bit off-topic but handy when allocating things from the heap. See
> 
> 
> 
> https://groups.google.com/forum/?hl=en#!topic/comp.lang.ada/QZhk7ZHMk30
> 
> 
> 
> and search the page for "renames" for a neat trick by Brian Drummond. You'll be glad you did.

That *IS* a pretty good story; thanks for remembering/sharing it.

Re: Thick bindings to a C library and gnattest: suggestions?

[Randy Brukardt]

"Jeffrey Carter" <spam.jrcarter.not@spam.not.acm.org> wrote in message 
news:kqsd6d$1ik$2@dont-email.me...
...
> (Aside: in a world in which GB of RAM are common, it seems odd for a 
> language to require explicit heap allocation, and the associated memory 
> management, for objects that fit in memory, but not in the stack. Surely 
> it would be better for the language to allow the developer to simply 
> declare the object, and for the compiler to decide where it will fit, and 
> allocate it and manage its memory appropriately. A means to disable 
> automatic heap allocation would be needed for systems that disallow heap 
> allocation.)

Amen! I've been saying that for 30+ years; perhaps someday it will really be 
true in commonly used compilers.

                             Randy.

Re: Thick bindings to a C library and gnattest: suggestions?

[Shark8]

On Monday, July 1, 2013 10:24:57 PM UTC-6, Randy Brukardt wrote:
> "Jeffrey Carter" <spam.jrcarter.not@spam.not.acm.org> wrote in message 
> 
> > (Aside: in a world in which GB of RAM are common, it seems odd for a 
> > language to require explicit heap allocation, and the associated memory 
> > management, for objects that fit in memory, but not in the stack. Surely 
> > it would be better for the language to allow the developer to simply 
> > declare the object, and for the compiler to decide where it will fit, and 
> > allocate it and manage its memory appropriately. A means to disable 
> > automatic heap allocation would be needed for systems that disallow heap 
> > allocation.)
> 
> 
> Amen! I've been saying that for 30+ years; perhaps someday it will really be 
> true in commonly used compilers.

True; that would be a nice feature..

I think though that maybe FORTH is close to that; AIUI it's odd combination of ultra low-level (words resolve to either more words or some chunk of machine-code) and ultra high-level (you can define words in such a way as to form grammars [i.e. you don't use raw FORTH, you use a DSL that happens to be written/implemented in FORTH]). [And, of course, it's a stack based language itself.]

Re: Thick bindings to a C library and gnattest: suggestions?

[tmoran]

> > (Aside: in a world in which GB of RAM are common, it seems odd for a
> > language to require explicit heap allocation, and the associated memory
> > management, for objects that fit in memory, but not in the stack. Surely
> > it would be better for the language to allow the developer to simply
> > declare the object, and for the compiler to decide where it will fit, and
> > allocate it and manage its memory appropriately. A means to disable
> > automatic heap allocation would be needed for systems that disallow heap
> > allocation.)
>
> Amen! I've been saying that for 30+ years; perhaps someday it will really be
> true in commonly used compilers.
>
>                              Randy.
  I notice that when I tried the OP's declare with the Janus Ada compiler
it ran for up to a little over 800MB for the array, then above that it
said:
** Unhandled STORAGE_ERROR
   Heap Overflow
On Line Number 16 In TRYBIG.DOIT
Called from line number 27 In TRYBIG
suggesting that the compiler was
> > decide where it will fit, and
> > allocate it and manage its memory appropriately.

Re: Thick bindings to a C library and gnattest: suggestions?

[Maurizio Tomasi]

On Monday, July 1, 2013 11:45:04 AM UTC+2, Dmitry A. Kazakov wrote:
> Why don't you simply pass the array down to the C subprogram? You can do
> something like:
> 
>    type Double_Array is array (Positive range <>)
>       of aliased Interfaces.C.double;
>    pragma Convention (C, Double_Array);
>    procedure Foo (A : Double_Array);
> 
> Implementation:
> 
>    type Double_Ptr is access all Interfaces.C.double;
>    pragma Convention (C, Double_Ptr);
> 
>    procedure Foo (A : Double_Array) is
>    --
>    -- Assuming foo's signature in C:
>    --
>    --    foo (double * a, unsigned n);
>    --
>       procedure Internal (A : Double_Ptr; N : Interfaces.C.unsigned);
>       pragma Import (C, Internal, "foo");
>    begin
>       Internal (A (A'First)'Access, A'Length);
>    end Foo;

Hi Dmitry,

  I like a lot this solution, but apparently I am not able even to implement this correctly. I wrote a dummy package to make my problem clearer:

with Interfaces.C;

procedure Test_Package is
   
   type Vector is array (Positive range <>) of aliased Natural;
   
   type Pointer is access all Natural;
   pragma Convention (C, Pointer); 
   
   type Const_Pointer is access constant Natural;
   pragma Convention (C, Const_Pointer); 

   procedure Read_Vector_From_File (A : out Vector) is 
      
      procedure Internal (A : Pointer; N : Interfaces.C.unsigned); 
      pragma Import (C, Internal, "read_vector"); 
      
   begin 
      Internal (A (A'First)'Access, A'Length); 
   end Read_Vector_From_File; 
   
   procedure Write_Vector_To_File (A : Vector) is 
      
      procedure Internal (A : Const_Pointer; N : Interfaces.C.unsigned); 
      pragma Import (C, Internal, "write_vector"); 
      
   begin 
      Internal (A (A'First)'Access, A'Length); 
   end Write_Vector_To_File; 
   
   A : Vector (1 .. 3);
   
begin
   Read_Vector_From_File (A);
   Write_Vector_To_File (A);
end Test_Package;

When I compile this program, both occurrences of "A (A'First)'Access" in Read_Vector and Write_Vector make GNAT complain:

test_package.adb:19:17: non-local pointer cannot point to local object
test_package.adb:28:17: non-local pointer cannot point to local object

I do not understand what causes this problem: in both cases the address of A's first element is not stored in a global variable but used as the argument to a procedure call. So how can this be a "non-local pointer"?

Maurizio.

Re: Thick bindings to a C library and gnattest: suggestions?

[Georg Bauhaus]

On 01.07.13 13:11, Maurizio Tomasi wrote:
> Hi Dmitry,

>> Why don't you simply pass the array down to the C subprogram? You can do
>> something like:

>
> But what if A'Length is so large that the array does not fit into the stack?

Avoid pointers on the Ada side and you need not worry about
the stack. This is true insofar as Ada language definition
says that Ada arrays will be passed as pointers to the C world,
automatically. (See Interfacing to C)

Indeed, just do what is natural on both sides:
You can pass C-array variables (pointers) on the C side and
expect Ada to handle plain Ada-array variables. No pointers
needed. In fact, they complicate things due to doubled
indirections.

The following seems to work on my system.
The Ada side "imports" data and exports its subprograms.


#include <stdlib.h>
#include <stdio.h>

double call_ada (size_t n)
{
   extern void ada_side_takes_vector (double*, size_t);

   double *thing = malloc(n * sizeof(double));
   for (int k = 0; k < (int)n; ++k) {
     thing[k] = k;
   }
   ada_side_takes_vector(thing, n);
   return thing[n/2];
}

int main()
{
   extern void adainit(void);
   extern void adafinal(void);
   double result;
#define M ((2<<20)/sizeof(double))

   adainit();
   result = call_ada(500 * M);
   adafinal();
   printf("result is %f\n", result);
   return 0;
}

with Interfaces.C; use Interfaces;

package Bigimport is
    
    pragma Pure (Bigimport);
    
    subtype Dbl is C.Double;
    subtype Zint is C.ptrdiff_t range 0 .. C.ptrdiff_t'Last;
    
    type Lots_Of_Numbers is array (Zint) of Dbl;
    pragma Convention (C, Lots_Of_Numbers);
    
    procedure Takes_Vector
      (V : in out Lots_Of_Numbers;
       N : C.size_t);
    pragma Export (C, Takes_Vector, "ada_side_takes_vector");
    
end Bigimport;

package body Bigimport is
    
    procedure Takes_Vector
      (V : in out Lots_Of_Numbers; N : C.size_t)
    is
       use type Dbl, C.size_t;
    begin
       for K in Zint range 0 .. Zint(N-1) loop
          V(K) := V(K) / 2.0;
       end loop;
    end Takes_Vector;
    
end Bigimport;

Re: Thick bindings to a C library and gnattest: suggestions?

[Dmitry A. Kazakov]

On Tue, 2 Jul 2013 01:33:04 -0700 (PDT), Maurizio Tomasi wrote:

> When I compile this program, both occurrences of "A (A'First)'Access" in
> Read_Vector and Write_Vector make GNAT complain:

Replace

   A (A'First)'Access

with

   A (A'First)'Unchecked_Access

> I do not understand what causes this problem: in both cases the address of
> A's first element is not stored in a global variable but used as the
> argument to a procedure call. So how can this be a "non-local pointer"?

The type Pointer is declared in the scope outer to the object (indexed
subprogram's argument) the pointer is supposed to point.

The rule is that the scope of a pointer type should not be wider than the
scope of the object, which statically prevents dangling pointers. In the
cases like yours, you know better than the compiler, so you just tell him
to back off. This is what Unchecked_Access is for.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: Thick bindings to a C library and gnattest: suggestions?

[Robert A Duff]

Maurizio Tomasi <ziotom78@gmail.com> writes:

> I do not understand what causes this problem: in both cases the
> address of A's first element is not stored in a global variable but
> used as the argument to a procedure call. So how can this be a
> "non-local pointer"?

If you pass a pointer as a parameter to some separately compiled
procedure (written in Ada or C or whatever), the compiler can't
very well know that the callee doesn't copy it into a global.

Even locally, the compiler isn't required to track how you copy
pointers around the place -- the rule is conservative, and based
on the _type_.

- Bob

Re: Thick bindings to a C library and gnattest: suggestions?

[Jeffrey Carter]

On 07/02/2013 01:33 AM, Maurizio Tomasi wrote:
>     type Vector is array (Positive range <>) of aliased Natural;
>
>     type Pointer is access all Natural;
>     pragma Convention (C, Pointer);
>
>     procedure Read_Vector_From_File (A : out Vector) is
>
>        procedure Internal (A : Pointer; N : Interfaces.C.unsigned);
>        pragma Import (C, Internal, "read_vector");
>
>     begin
>        Internal (A (A'First)'Access, A'Length);
>     end Read_Vector_From_File;

There is no need for access types in Ada to do this. See ARM B.3 (70):

"An Ada parameter of an access-to-subprogram type is passed as a pointer to a C 
function whose prototype corresponds to the designated subprogram's specification."

In other words, you can write

type Vector is array (Positive range <>) of Natural;

procedure Read_Vector_From_File (A : out Vector) is
    procedure Internal (A : out Vector; N : in Interfaces.C.Unsigned);
    pragma Import (C, Internal, "read_vector");
begin -- Read_Vector_From_File
    Internal (A => A, N => A'Length);
end Read_Vector_From_File;

and the compiler takes care of passing a C pointer to the 1st component of A. No 
need for the components of Vector to be aliased, and no problems with 
accessibility checks.

-- 
Jeff Carter
"That was the most fun I've ever had without laughing."
Annie Hall
43

Re: Thick bindings to a C library and gnattest: suggestions?

[Randy Brukardt]

<tmoran@acm.org> wrote in message news:kqtn1d$2ua$1@speranza.aioe.org...
...
>> Amen! I've been saying that for 30+ years; perhaps someday it will really 
>> be
>> true in commonly used compilers.
>>
>>                              Randy.
>  I notice that when I tried the OP's declare with the Janus Ada compiler
> it ran for up to a little over 800MB for the array, then above that it
> said:
> ** Unhandled STORAGE_ERROR
>   Heap Overflow
> On Line Number 16 In TRYBIG.DOIT
> Called from line number 27 In TRYBIG
> suggesting that the compiler was

Janus/Ada unfortunately doesn't do this consistently. This is a hard 
problem, because the compiler doesn't know how big the stack will be at 
runtime, and adding the level of indirection costs a bit of performance. So 
you could run into trouble if the program uses a bunch of medium-sized but 
statically-sized objects that collectively overflow the stack.

In the OP's program, however, all of the objects are dynamically-sized; in 
that case, Janus/Ada always uses a level of indirection and only uses the 
stack for very small objects, so one gets the automatic heap-based 
allocation (and deallocation) seen here.

                                                    Randy.

Re: Thick bindings to a C library and gnattest: suggestions?

[Jacob Sparre Andersen]

Jeffrey Carter wrote:

> (Aside: in a world in which GB of RAM are common, it seems odd for a
> language to require explicit heap allocation, and the associated
> memory management, for objects that fit in memory, but not in the
> stack. Surely it would be better for the language to allow the
> developer to simply declare the object, and for the compiler to decide
> where it will fit, and allocate it and manage its memory
> appropriately. A means to disable automatic heap allocation would be
> needed for systems that disallow heap allocation.)

I like to set the stack size for my (non-embedded) systems to an
indecently large value (1, 2 or 4 Gb) as it only is virtual memory I use
that way anyway.

But yes, it would be nice, if the compiler would do the worrying about
which objects to allocate where.

Greetings,

Jacob
-- 
"No!  The universe is ours."