Dispatching callback handed over to C

Dispatching callback handed over to C

[Natasha Kerensikova]

Hello,

I'm wondering whether I'm calling on you too much, so please tell me
when I'm abusing the resource that comp.lang.ada is and/or your time.

The problem I'm facing currently is binding to a very common idiom for
callback in C. There a setup function that expects a pointer to a
function and a pointer to opaque data, that will be given as-is to the
pointed function.

On the Ada side, it seemed to be a perfect use case for tagged types. I
still haven't decided between a dedicated interface with a single
method, or add an abstract method in the tagged type that handles the C
structure where the callback reference is stored.

It would be wonderful if there was a way to "pre-dispatch" the call,
i.e. giving to the C library the address of the actual subprogram
called. The information has to be available at this point, since it's
the address called if I went for a real dispatching call at this point.
However I have no idea what the syntax would look like, or how it could
be type-checked (since one of the parameters of that subprogram would be
of the same as the actual type of the class-wide variable, which is not
known at compile-time). But since we're interfacing with C, we're
expecting all sorts of type-check trouble anyway.

So, is there any way to get the address where a dispatching call would
lead?



I tried to work around it, by having a single dedicated callback
subprogram given to the C library, and interfacing C « void* » with
access Event_Callback'Class. So that the body of the dedicated callback
is only a dispatching call. It's a waste of code lines and calling
sequence, but I guess bindings are never optimal.

I was stopped by the following warning from GNAT:
        >>> warning: "Event_New.Arg" involves a tagged type which does
not correspond to any C type
And I can't decipher it. I tried with various access-type naming and
pragma Convention, but I couldn't make it disappear.

Of course, it's only a warning, so I could just ignore it. However
I'm comfortable with ignoring warnings only when I actually know they
don't warn about any real danger in the particular situation. This is
far from being the case, since I don't really understand what is going
on.



On top of all that, I wanted to add some safety mechanisms, to prevent
the callback from disappearing while still being referred to.

One possibility is to make the callback part of the type that handles
the C structure where it is stored. That structure is freed in
Finalize, so references to the callback object are cleaned up when the
both are destroyed.

However, since it deals with opaque C pointers that cannot be
duplicated, it has to be limited. This makes a lot of things unnecessary
difficult for the callback part.

Moreover, some applications might want to attach the same callback to
multiple structure (since the C structure is tied to a lot of other
thins, like a socket FD and an event mask, so it might make sense to
have several of these for the same underlying callback code and state).

So I thought of access discriminants: if I understand correctly, there
is some compile-time check to ensure the lifetime of the accessed object
(i.e. the callback) is at least as long as that of the discriminated
object (i.e. the C structure).

However, I can't seem to properly hand it over to the C function:
when I use an anonymous access type, I get a convention error, and when
I use a named access type (with pragma Convention), I cannot convert
the discriminant to it.



I'm really surprised to find so little information about this problem,
considering how often I saw the idiom during my C days. I also tried a
few existing bindings, like Claw, but couldn't find any precedent either
(I admit I have not looked in Qt or Gtk bindings, but I'm afraid it
would take too much time to understand what is going on before I could
mimic it (I have never coded for Qt or Gtk in my life)).


Thanks in advance for your help, and sorry to so annoying for you,
Natasha

Re: Dispatching callback handed over to C

[Brian Drummond]

On Tue, 03 Apr 2012 07:19:56 +0000, Natasha Kerensikova wrote:

> Hello,
> 
> I'm wondering whether I'm calling on you too much, so please tell me
> when I'm abusing the resource that comp.lang.ada is and/or your time.

Absolutely not! Your questions - this one included - are very good and 
your examples are useful to learn from - I enjoy these conversations and 
follow them with great interest and even though I rarely have anything 
useful to add.

- Brian

Re: Dispatching callback handed over to C

[Thomas Locke]

On 04/03/2012 11:04 AM, Brian Drummond wrote:
> Absolutely not! Your questions - this one included - are very good and
> your examples are useful to learn from - I enjoy these conversations and
> follow them with great interest and even though I rarely have anything
> useful to add.


I second that!


-- 
Thomas Løcke | thomas@12boo.net | http://12boo.net

Re: Dispatching callback handed over to C

[Maciej Sobczak]

On 3 Kwi, 09:19, Natasha Kerensikova <lithium...@gmail.com> wrote:

> The problem I'm facing currently is binding to a very common idiom for
> callback in C.

[...]

I had to solve this poblem in the context of the YAMI4 poject and I
managed to get the essential parts of the solution here:

http://inspirel.com/articles/Polymorphic_Callbacks_Ada_Cpp.html

The callback, as seen by Ada, is polymorphic (the user sees it as a
dipatching call). The solution does not involve any compiler-specific
magic any more than what could be expected, so should be quite
portable.

Note that this article presents complete solution for Ada/C++ binding,
but since there is a C intermediary layer involved anyway, you might
consider it to be a Ada/C solution as well and use that part only if
this is what you need.

--
Maciej Sobczak * http://www.inspirel.com

Re: Dispatching callback handed over to C

[Natasha Kerensikova]

On 2012-04-03, Maciej Sobczak <see.my.homepage@gmail.com> wrote:
> On 3 Kwi, 09:19, Natasha Kerensikova <lithium...@gmail.com> wrote:
>
>> The problem I'm facing currently is binding to a very common idiom for
>> callback in C.
>
> [...]
>
> I had to solve this poblem in the context of the YAMI4 poject and I
> managed to get the essential parts of the solution here:
>
> http://inspirel.com/articles/Polymorphic_Callbacks_Ada_Cpp.html

Thanks a lot for the solution.

It turns out this is almost what my tentative solution looked like, the
only step I missed was to use System.Address to carry the data part,
instead of some access type.

But still, I can't help but think there is one needless subprogram call
in that solution. With a way to obtain the address of the actual
subprogram the dispatching call would dispatch to, the address could be
handed over to C code so that it can call it directly.

> The callback, as seen by Ada, is polymorphic (the user sees it as a
> dipatching call). The solution does not involve any compiler-specific
> magic any more than what could be expected, so should be quite
> portable.

Actually it uses the assumption that System.Address and void* have the
same representation.

I wonder how reliable the assumption is.



Thanks for your insights,
Natasha

Re: Dispatching callback handed over to C

[Maciej Sobczak]

On 3 Kwi, 14:02, Natasha Kerensikova <lithium...@gmail.com> wrote:

> But still, I can't help but think there is one needless subprogram call
> in that solution. With a way to obtain the address of the actual
> subprogram the dispatching call would dispatch to, the address could be
> handed over to C code so that it can call it directly.

Then the target operation would need to have a C calling convention.
Since one of the reasons to do OO is to allow users to extend the
system with new operations (new implementations), this requirement
would be a limitation.
The indirection allows to completely isolate the user from such
details.

> Actually it uses the assumption that System.Address and void* have the
> same representation.
>
> I wonder how reliable the assumption is.

Whenever Ada and C (or C++) components are linked together, a set of
"friendly" compilers is needed, so that basic binary compatibility can
be guaranted - otherwise nothing is going to work at all, not even
Interfaces.C.int. :-)
This assumption is pretty safe.

--
Maciej Sobczak * http://www.inspirel.com

Re: Dispatching callback handed over to C

[Randy Brukardt]

"Maciej Sobczak" <see.my.homepage@gmail.com> wrote in message 
news:85d1ad51-c02b-44fa-87b6-02aa1d8ba1b2@x17g2000vba.googlegroups.com...
> On 3 Kwi, 14:02, Natasha Kerensikova <lithium...@gmail.com> wrote:
...
>> Actually it uses the assumption that System.Address and void* have the
>> same representation.
>>
>> I wonder how reliable the assumption is.
>
> Whenever Ada and C (or C++) components are linked together, a set of
> "friendly" compilers is needed, so that basic binary compatibility can
> be guaranted - otherwise nothing is going to work at all, not even
> Interfaces.C.int. :-)
> This assumption is pretty safe.

I disagree. It doesn't work in the current Janus/Ada compiler, for instance. 
The basic idea is that System.Address ought to be able to access any memory 
on the target that might have hardware located at it. Thus, that address 
might need to include segments or equivalent information along with the 
"address" itself. Matching the C compiler is irrelevant; we have access 
types and convention C for that purpose (and you can't match both of two 
possibly different requirements).

Indeed, using System.Address for any purpose other than interfacing directly 
to hardware is Ada 83 thinking, in my view. (Not everyone agrees with this 
view.) Using System.Address loses type-safety, and prevents the compiler 
optimizer for working at all (it has to assume the worst about any object in 
your program, unless it is willing to completely trash your program).

The whole idea of the void pointer is so anti-Ada that it is best to get rid 
of it as early as you can. (There is no such thing as "untyped data" in Ada, 
and, as Dmitry likes to point out, not in real life, either. There is only 
data for which the type is not known at compile-time, and that sort of data 
is a problem - best to avoid it.) In Claw, we tried to avoid void by simply 
defining the interfaces for the types we cared about. For instance, if we 
needed to read an array of stream elements, we'd define the API using that 
type (rather than void). In other cases, we defined a void type (as access 
to something) and used Unchecked_Conversions as needed to make it 
crystal-clear this is wildly unsafe. Obviously, you have to do something to 
interface to mindless APIs, but you have to hide them as quickly as 
possible.

[That said, it is possible to go too far for your target. For instance, 
Windows can't use segments at all, so having them in System.Address is more 
likely to cause problems than fixes. The fact that our Windows compiler 
still has segments is actually a bug -- it supposedly was changed years 
ago -- it's in the change logs -- but somehow it disappeared from the actual 
definitions. Will get changed someday.]

                                           Randy.

Re: Dispatching callback handed over to C

[Georg Bauhaus]

On 03.04.12 22:20, Randy Brukardt wrote:

> The whole idea of the void pointer is so anti-Ada that it is best to get rid
> of it as early as you can. (There is no such thing as "untyped data" in Ada,
> and, as Dmitry likes to point out, not in real life, either. There is only
> data for which the type is not known at compile-time, and that sort of data
> is a problem - best to avoid it.)

Just to mention that there is more to (void*) than "untyped data"!
The C standard guarantees certain properties of objects when
the programmer applies C's type conversion rules in many
directions. (void*) is not a demolition operation.

There is also a ptrdiff_t both in C and in Ada, the ptrdiff_t rules
again implying anything but untyped data. And the definitions of
either language's type system seem to join here and there.

Unchecked_Union works for discriminated records, but it does not
extend to the union of tag-discriminated records of some R'Class.

I see that Maciej's solution has two hooks in it that I would hope are
hooks for addressing the concerns. They both have body visibility only.
One declares subtype Void_Ptr is System.Address, so
System.Address can be replaced with something that better matches O-O
arguments of type (void*). And the solution has a pointer translation
mechanism involving an instance of System.Address_To_Access_Conversions.

What else can we do to enable O-O mechanics in existing C libraries?

(If there isn't anything Ada can do, then I-can-emulate-all-your-language
in my not-that-many-CVE-generating C programs, so why don't you just make
your great Ada language work with today's libraries after only 30 years...
;-)

Re: Dispatching callback handed over to C

[Natasha Kerensikova]

On 2012-04-03, Randy Brukardt <randy@rrsoftware.com> wrote:
> "Maciej Sobczak" <see.my.homepage@gmail.com> wrote in message 
> news:85d1ad51-c02b-44fa-87b6-02aa1d8ba1b2@x17g2000vba.googlegroups.com...
>> On 3 Kwi, 14:02, Natasha Kerensikova <lithium...@gmail.com> wrote:
> ...
>>> Actually it uses the assumption that System.Address and void* have the
>>> same representation.
>>>
>>> I wonder how reliable the assumption is.
>>
>> Whenever Ada and C (or C++) components are linked together, a set of
>> "friendly" compilers is needed, so that basic binary compatibility can
>> be guaranted - otherwise nothing is going to work at all, not even
>> Interfaces.C.int. :-)
>> This assumption is pretty safe.
>
> I disagree. It doesn't work in the current Janus/Ada compiler, for instance. 
> The basic idea is that System.Address ought to be able to access any memory 
> on the target that might have hardware located at it. Thus, that address 
> might need to include segments or equivalent information along with the 
> "address" itself. Matching the C compiler is irrelevant; we have access 
> types and convention C for that purpose (and you can't match both of two 
> possibly different requirements).

That was roughly what I was thinking (though not exactly with these
words).

In my bindings, I'm currently the following types :

   type Opaque_Data is null record;
   pragma Convention (C, Opaque_Data);

   type Opaque_Pointer is access all Opaque_Data;
   pragma Convention (C, Opaque_Pointer);

The library I'm currently binding provides pointers as handles to its
internal structures, but even in C the types exposed are incomplete. So
I'm using types derived from Opaque_Pointer to deal with them from Ada.

It feels to me that Interfaces.C.int is explicitly requesting something
that maps correctly to int in C, and that my Opaque_Pointer is
explicitly something that maps correctly to C data pointers.
While the RM seems to offer no such connection between System.Address
and pointers.

Now the problem is that I have a solution with System.Address, that
works at least on my machine. Despite all my attempts I'm still unable
to reach that point with C-convention-ed accesses.

> Indeed, using System.Address for any purpose other than interfacing directly 
> to hardware is Ada 83 thinking, in my view. (Not everyone agrees with this 
> view.) Using System.Address loses type-safety,

Well, considering I'm writing a C binding, there is not much more
type safety that can be reached even without System.Address.
It even seems I'm fighting type-safety that would have made sense if the
subprogram weren't imported, but that is moot since the import itself
forsakes any kind of type-safety.

>                                                and prevents the compiler 
> optimizer for working at all (it has to assume the worst about any object in 
> your program, unless it is willing to completely trash your program).

That was another question I had in mind with that mechanism: I'm giving
to C a reference to Ada data in one call (namely Event_New) but it gets
used later, in another call (to Event_Base_Dispatch) which seeming does
not involve the data at all.

Should go to some extra steps (but which?) to warn the compiler that my
Event object can change during Event_Base_Dispatch call despite that
call not involving any Event object?
Or maybe just marking the Event object as aliased is enough?

> The whole idea of the void pointer is so anti-Ada that it is best to get rid 
> of it as early as you can. (There is no such thing as "untyped data" in Ada, 
> and, as Dmitry likes to point out, not in real life, either. There is only 
> data for which the type is not known at compile-time, and that sort of data 
> is a problem - best to avoid it.)

I feel you are being a bit unfairly harsh towards void pointers here: it
looks to me that you are looking at them in a purely Ada context, which
is about as incorrect as coding in Ada with another language in mind and
complaining about being forced to fight the compiler.

In C, void pointer is not really about untyped data, but data whose type
is actively treated with agnosticism.
Transposed to Ada world, it's like saying it does not make sense to
have a record whose components are unknown, and in real life there is no
such thing as a record with unknown components. "private" is only about
preventing client from depending on component information rather than
"void"ing components.

>                                   In Claw, we tried to avoid void by simply 
> defining the interfaces for the types we cared about. For instance, if we 
> needed to read an array of stream elements, we'd define the API using that 
> type (rather than void). In other cases, we defined a void type (as access 
> to something) and used Unchecked_Conversions as needed to make it 
> crystal-clear this is wildly unsafe. Obviously, you have to do something to 
> interface to mindless APIs, but you have to hide them as quickly as 
> possible.

That's exactly what I'm trying to do. Void pointers are indeed anti-Ada,
because Ada rather uses generic formal types or tagged types or
interfaces. The whole art of designing bindings is, in my opinion, to
expose an interface as Ada-ish as possible despite the underlying
anti-Ada stuff.

Still, I cannot avoid void pointer without avoiding writing the binding
altogether.



Thanks for your insights,
Natasha

Re: Dispatching callback handed over to C

[Maciej Sobczak]

On 3 Kwi, 22:20, "Randy Brukardt" <ra...@rrsoftware.com> wrote:

> The basic idea is that System.Address ought to be able to access any memory
> on the target that might have hardware located at it.

Interestingly, this is also the intent of void*.

> Matching the C compiler is irrelevant; we have access
> types and convention C for that purpose

This is almost true.
The problem is that AARM gives no provisions whatsoever to pass access
to class-wide types (only types that have "corresponding C types" are
covered) to or from C and the ability to do it seems to be essential
in the OO context. That is, there is no direct way to be standard-
compliant here.

Note that in my solution the access to class-wide is passed to C++ and
back only for storage - the actual use (the dispatching call) is at
the Ada side, so the mechanics used for that storage need to only
ensure that the physical representation of access value is retained.
System.Address with void* do it properly on my platform.
Using a string representation might be a possible solution, too, even
if potentially slower.

That it does not work on your compiler?
Count it as my feature request, then. ;-)

--
Maciej Sobczak * http://www.inspirel.com

Re: Dispatching callback handed over to C

[Randy Brukardt]

"Maciej Sobczak" <see.my.homepage@gmail.com> wrote in message 
news:62246369-6b9f-4a96-9c67-fd1774c02e78@k24g2000yqe.googlegroups.com...
> On 3 Kwi, 22:20, "Randy Brukardt" <ra...@rrsoftware.com> wrote:
>
>> The basic idea is that System.Address ought to be able to access any 
>> memory
>> on the target that might have hardware located at it.
>
> Interestingly, this is also the intent of void*.

But that's impossible. In general, data memory and code memory are different 
things and require different access. They are totally separated in 
Janus/Ada, with separate code pointer and data pointer types. (Some targets 
map these as the same, of course, but many don't.) An additional concern is 
callability: code pointers have enough information to support calls, and 
data pointers don't. (In particular, the U2200 hardware requires a static 
link as part of a code pointer. You can't call a bare address under any 
circumstances.)

void * is a data pointer, and ought to be treated that way. It is not 
necessarily going to have access to the code of a program (nor should there 
be any need to access the code from an data pointer). System.Address, OTOH, 
does need access to the code portions and that often will require a 
different, larger representation.

>> Matching the C compiler is irrelevant; we have access
>> types and convention C for that purpose
>
> This is almost true.
> The problem is that AARM gives no provisions whatsoever to pass access
> to class-wide types (only types that have "corresponding C types" are
> covered) to or from C and the ability to do it seems to be essential
> in the OO context. That is, there is no direct way to be standard-
> compliant here.

It doesn't even make sense to pass "access-to-class-wide types" to C (as 
opposed to C++). C doesn't know what to do with them.

> Note that in my solution the access to class-wide is passed to C++ and
> back only for storage - the actual use (the dispatching call) is at
> the Ada side, so the mechanics used for that storage need to only
> ensure that the physical representation of access value is retained.
> System.Address with void* do it properly on my platform.
> Using a string representation might be a possible solution, too, even
> if potentially slower.

If all you want to do is store them, just stick them in the record as 
"access-to-class-wide" and don't even worry about having a C counterpart. 
You need some c-compatible type in the C-side, but it is just a 
"bucket-of-bits".

For instance, that's how we find the Ada object in Claw; we 
unchecked-convert access-to-class-wide to a DWord in Windows, and then when 
we need to use it, we unchecked-convert back. If I needed to make this 
totally portable and had sufficient control over the API, I'd replace the 
DWord by a locally-declared modular type of which I could control the size 
to match the target. But there is little point in going further than that.

> That it does not work on your compiler?
> Count it as my feature request, then. ;-)

System.Address is the wrong type to use for a "bucket-of-bits". I agree that 
it would be nice to be able to directly put the access-to-class-wide in the 
C convention record (and most compilers will in fact allow this, after 
giving some warnings), but it doesn't seem to make that much difference.

                                   Randy.

Re: Dispatching callback handed over to C

[Randy Brukardt]

"Natasha Kerensikova" <lithiumcat@gmail.com> wrote in message 
news:slrnjnnvk6.1lme.lithiumcat@sigil.instinctive.eu...
...
>>                                                and prevents the compiler
>> optimizer for working at all (it has to assume the worst about any object 
>> in
>> your program, unless it is willing to completely trash your program).
>
> That was another question I had in mind with that mechanism: I'm giving
> to C a reference to Ada data in one call (namely Event_New) but it gets
> used later, in another call (to Event_Base_Dispatch) which seeming does
> not involve the data at all.
>
> Should go to some extra steps (but which?) to warn the compiler that my
> Event object can change during Event_Base_Dispatch call despite that
> call not involving any Event object?
> Or maybe just marking the Event object as aliased is enough?

Aliased should be enough (Ada 95 suggests that it be used for this purpose). 
Some compilers don't insist on this (mostly because of Ada 83 code 
compatibility concerns), but I presume that they figure this out via some 
other mechanism (probably the same one that C compilers use, which typically 
is fiendishly complicated and rather prone to bugs).

>> The whole idea of the void pointer is so anti-Ada that it is best to get 
>> rid
>> of it as early as you can. (There is no such thing as "untyped data" in 
>> Ada,
>> and, as Dmitry likes to point out, not in real life, either. There is 
>> only
>> data for which the type is not known at compile-time, and that sort of 
>> data
>> is a problem - best to avoid it.)
>
> I feel you are being a bit unfairly harsh towards void pointers here: it
> looks to me that you are looking at them in a purely Ada context, which
> is about as incorrect as coding in Ada with another language in mind and
> complaining about being forced to fight the compiler.

Well, we're talking about Ada here, and what we have to do to interface to 
some foreign language API. I'm not interested in writing C - in any syntax!

> In C, void pointer is not really about untyped data, but data whose type
> is actively treated with agnosticism.
> Transposed to Ada world, it's like saying it does not make sense to
> have a record whose components are unknown, and in real life there is no
> such thing as a record with unknown components. "private" is only about
> preventing client from depending on component information rather than
> "void"ing components.

The void pointer is C is about data where the C type system isn't strong 
enough to describe it. So it simply throws away all of the type information, 
leaving you with none. The data itself surely has a type, but you can't use 
any knowledge about it. OTOH, the richer Ada type system often *can* 
describe this data, and in such a case, you will want to do that. (Yes, 
there are cases where Ada can't describe it, either - but I generally think 
those require a redesign, not a hack.)

An obvious example is the access-to-class-wide that Mariej was talking 
about. Clearly Ada can describe this, so you want to avoid using "void" in 
your (Ada) interface.

>>                                   In Claw, we tried to avoid void by 
>> simply
>> defining the interfaces for the types we cared about. For instance, if we
>> needed to read an array of stream elements, we'd define the API using 
>> that
>> type (rather than void). In other cases, we defined a void type (as 
>> access
>> to something) and used Unchecked_Conversions as needed to make it
>> crystal-clear this is wildly unsafe. Obviously, you have to do something 
>> to
>> interface to mindless APIs, but you have to hide them as quickly as
>> possible.
>
> That's exactly what I'm trying to do. Void pointers are indeed anti-Ada,
> because Ada rather uses generic formal types or tagged types or
> interfaces. The whole art of designing bindings is, in my opinion, to
> expose an interface as Ada-ish as possible despite the underlying
> anti-Ada stuff.

Absolutely. The really low-level stuff is likely to be grungy and not 100% 
portable. But that's OK, so long as it is localized as much as possible. 
(Pragmatism has to trump goodness -- if the binding doesn't work, it doesn't 
matter how "good" it is!)

                                        Randy.

Re: Dispatching callback handed over to C

[Simon Wright]

Natasha Kerensikova <lithiumcat@gmail.com> writes:

> That was another question I had in mind with that mechanism: I'm giving
> to C a reference to Ada data in one call (namely Event_New) but it gets
> used later, in another call (to Event_Base_Dispatch) which seeming does
> not involve the data at all.
>
> Should go to some extra steps (but which?) to warn the compiler that my
> Event object can change during Event_Base_Dispatch call despite that
> call not involving any Event object?
> Or maybe just marking the Event object as aliased is enough?

Volatile, perhaps?

Re: Dispatching callback handed over to C

[Maciej Sobczak]

On 4 Kwi, 21:16, "Randy Brukardt" <ra...@rrsoftware.com> wrote:

> > Interestingly, this is also the intent of void*.
>
> But that's impossible. In general, data memory and code memory are different
> things and require different access.

But we are talking about using void* as a data pointer here (a class-
wide object).

> If all you want to do is store them, just stick them in the record as
> "access-to-class-wide" and don't even worry about having a C counterpart.
> You need some c-compatible type in the C-side, but it is just a
> "bucket-of-bits".

I use void* as bit bucket with the indication that what is inside is,
well, a pointer.
Ironically, contrary to your insistence of using type information
correctly, I find my use of void* as a bit bucket for a pointer value
somewhat more typed (at least in the sense of being self-documenting)
than your DWORD, which is a bit bucket completely devoid of any self
description and even exposing wrong set of operations (arithmetic? bit
shifting? silent integer conversion? is it a set of flags or
something?).

Sorry, you are not convincing. I'll stick to void* when pointers are
involved.

> System.Address is the wrong type to use for a "bucket-of-bits".

It is a proper type to use for a "bucket-of-pointer-bits" in the sense
that I have no doubts about the intent when I see it later in code.

--
Maciej Sobczak * http://www.inspirel.com

Re: Dispatching callback handed over to C

[Natasha Kerensikova]

On 2012-04-04, Randy Brukardt <randy@rrsoftware.com> wrote:
> "Natasha Kerensikova" <lithiumcat@gmail.com> wrote in message 
>> Should go to some extra steps (but which?) to warn the compiler that my
>> Event object can change during Event_Base_Dispatch call despite that
>> call not involving any Event object?
>> Or maybe just marking the Event object as aliased is enough?
>
> Aliased should be enough (Ada 95 suggests that it be used for this purpose). 
> Some compilers don't insist on this (mostly because of Ada 83 code 
> compatibility concerns), but I presume that they figure this out via some 
> other mechanism (probably the same one that C compilers use, which typically 
> is fiendishly complicated and rather prone to bugs).

I was wondering between that and volatile (thanks to Simon Wright for
the suggestion). Volatile is also "can change behind compiler's back",
but it is much more stringent. Actually if the C library was calling
back to Ada in another thread of its, I would have no doubt about
volatile (or even protected or something) would have been necessary.
One the other hand here there is no concurrency, only the C library
calling Ada without the compiler being able to guess so.

Thinking again about though made me conclude that it is actually only
breaking a very strong assumption, which is that an external call does
not change anything reachable by Ada code except its direct arguments.
So I would guess no optimizer actually makes such an assumption,
especially on aliased objects (I guess the assumption makes sense on
local non-aliased objects).

Now about aliasing, I read somewhere (but can't find anymore where) that
arguments whose type is tagged is automatically aliased. Is it enough
aliasing, or should the original object be declared as aliased too?

>>> The whole idea of the void pointer is so anti-Ada that it is best to get 
>>> rid
>>> of it as early as you can. (There is no such thing as "untyped data" in 
>>> Ada,
>>> and, as Dmitry likes to point out, not in real life, either. There is 
>>> only
>>> data for which the type is not known at compile-time, and that sort of 
>>> data
>>> is a problem - best to avoid it.)
>>
>> I feel you are being a bit unfairly harsh towards void pointers here: it
>> looks to me that you are looking at them in a purely Ada context, which
>> is about as incorrect as coding in Ada with another language in mind and
>> complaining about being forced to fight the compiler.
>
> Well, we're talking about Ada here, and what we have to do to interface to 
> some foreign language API. I'm not interested in writing C - in any syntax!

Fair enough. Your text seemed to attack void pointers in general, which
is what I objected to. I completely agree that anything like a void
pointer in pure Ada is to be avoided.

And indeed, the Ada side of my binding uses interface or tagged objects
(I still haven't firmly decided which one), and the void pointer (with
its associated function pointer) is only on the C side of the binding.


Thanks for your insights,
Natasha

Re: Dispatching callback handed over to C

[Natasha Kerensikova]

On 2012-04-04, Randy Brukardt <randy@rrsoftware.com> wrote:
> For instance, that's how we find the Ada object in Claw; we 
> unchecked-convert access-to-class-wide to a DWord in Windows, and then when 
> we need to use it, we unchecked-convert back. If I needed to make this 
> totally portable and had sufficient control over the API, I'd replace the 
> DWord by a locally-declared modular type of which I could control the size 
> to match the target. But there is little point in going further than that.
>
> System.Address is the wrong type to use for a "bucket-of-bits". I agree that 
> it would be nice to be able to directly put the access-to-class-wide in the 
> C convention record (and most compilers will in fact allow this, after 
> giving some warnings), but it doesn't seem to make that much difference.

So for my binding, what about something like :

   type Opaque_Data is null record;
   pragma Convention (C, Opaque_Data);

   type Opaque_Pointer is access all Opaque_Data;
   pragma Convention (C, Opaque_Pointer);

   type Callback_Access is access all Event_Callback'Class;
   pragma Convention (C, Callback_Access);

   procedure Set_C_Callback (<various arguments>;
                             Data : Opaque_Pointer);
   pragma Import (C, Set_C_Callback, "set_callback");


   procedure Called_From_C (<various arguments>;
                            Data : Opaque_Pointer)
   is
      package To_Ada is new Ada.Unchecked_Conversion
        (Source => Opaque_Pointer, Target => Callback_Access);

      Callback : Callback_Access := To_Ada (Data);
   begin
      Callback.all.Handle (<...>);
   end Called_From_C;

   pragma Convention (C, Called_From_C);


   procedure Set_Callback (<various arguments>;
                           Callback : in out Event_Callback'Class)
   is
      package To_C is new Ada.Unchecked_Conversion
        (Source => Callback_Access, Target => Opaque_Pointer);
   begin
      Set_C_Callback (<...>, To_C (Callback'Access));
   end Set_Callback;

As far as I cen tell, Opaque_Pointer refers only to types compatibles
with C, so the imports goes well and without warning, while
Opaque_Pointer and Callback_Access, being both access types with the
same convention, ensures they can be safely (in terms of keeping the
bit pattern intact, of course not type-safety) converted back and forth.

Or is there some trap in the above code that I'm missing?


Thanks for your help,
Natasha

Re: Dispatching callback handed over to C

[Randy Brukardt]

"Natasha Kerensikova" <lithiumcat@gmail.com> wrote in message 
news:slrnjnqnng.1lme.lithiumcat@sigil.instinctive.eu...
> On 2012-04-04, Randy Brukardt <randy@rrsoftware.com> wrote:
...
> Now about aliasing, I read somewhere (but can't find anymore where) that
> arguments whose type is tagged is automatically aliased. Is it enough
> aliasing, or should the original object be declared as aliased too?

It probably doesn't hurt to declare the original object as aliased, but as 
always it depends on the compiler. If you're actually passing these to the C 
code, then of course the assumption that the (by-reference) parameters can 
change will cover you in any case. If they're passed some other way, then 
marking them as aliased (or volatile as you noted elsewhere) would be a good 
idea.

                                          Randy.

Re: Dispatching callback handed over to C

[Randy Brukardt]

Your code looks OK to me. (But I didn't try to compiler or use it, so there 
may be something I missed.)

                               Randy.

"Natasha Kerensikova" <lithiumcat@gmail.com> wrote in message 
news:slrnjnqoga.1lme.lithiumcat@sigil.instinctive.eu...
> On 2012-04-04, Randy Brukardt <randy@rrsoftware.com> wrote:
>> For instance, that's how we find the Ada object in Claw; we
>> unchecked-convert access-to-class-wide to a DWord in Windows, and then 
>> when
>> we need to use it, we unchecked-convert back. If I needed to make this
>> totally portable and had sufficient control over the API, I'd replace the
>> DWord by a locally-declared modular type of which I could control the 
>> size
>> to match the target. But there is little point in going further than 
>> that.
>>
>> System.Address is the wrong type to use for a "bucket-of-bits". I agree 
>> that
>> it would be nice to be able to directly put the access-to-class-wide in 
>> the
>> C convention record (and most compilers will in fact allow this, after
>> giving some warnings), but it doesn't seem to make that much difference.
>
> So for my binding, what about something like :
>
>   type Opaque_Data is null record;
>   pragma Convention (C, Opaque_Data);
>
>   type Opaque_Pointer is access all Opaque_Data;
>   pragma Convention (C, Opaque_Pointer);
>
>   type Callback_Access is access all Event_Callback'Class;
>   pragma Convention (C, Callback_Access);
>
>   procedure Set_C_Callback (<various arguments>;
>                             Data : Opaque_Pointer);
>   pragma Import (C, Set_C_Callback, "set_callback");
>
>
>   procedure Called_From_C (<various arguments>;
>                            Data : Opaque_Pointer)
>   is
>      package To_Ada is new Ada.Unchecked_Conversion
>        (Source => Opaque_Pointer, Target => Callback_Access);
>
>      Callback : Callback_Access := To_Ada (Data);
>   begin
>      Callback.all.Handle (<...>);
>   end Called_From_C;
>
>   pragma Convention (C, Called_From_C);
>
>
>   procedure Set_Callback (<various arguments>;
>                           Callback : in out Event_Callback'Class)
>   is
>      package To_C is new Ada.Unchecked_Conversion
>        (Source => Callback_Access, Target => Opaque_Pointer);
>   begin
>      Set_C_Callback (<...>, To_C (Callback'Access));
>   end Set_Callback;
>
> As far as I cen tell, Opaque_Pointer refers only to types compatibles
> with C, so the imports goes well and without warning, while
> Opaque_Pointer and Callback_Access, being both access types with the
> same convention, ensures they can be safely (in terms of keeping the
> bit pattern intact, of course not type-safety) converted back and forth.
>
> Or is there some trap in the above code that I'm missing?
>
>
> Thanks for your help,
> Natasha 

Re: Dispatching callback handed over to C

[Randy Brukardt]

"Maciej Sobczak" <see.my.homepage@gmail.com> wrote in message 
news:a236ccdb-e8b8-4b13-9e93-a2596f54ddcb@l7g2000vbw.googlegroups.com...
> On 4 Kwi, 21:16, "Randy Brukardt" <ra...@rrsoftware.com> wrote:
>
>> > Interestingly, this is also the intent of void*.
>>
>> But that's impossible. In general, data memory and code memory are 
>> different
>> things and require different access.
>
> But we are talking about using void* as a data pointer here (a class-
> wide object).

That's the point. "void *" is a data pointer (only). System.Address has to 
be both a code and data pointer. So it may not have the same representation 
as "void *".

>> If all you want to do is store them, just stick them in the record as
>> "access-to-class-wide" and don't even worry about having a C counterpart.
>> You need some c-compatible type in the C-side, but it is just a
>> "bucket-of-bits".
>
> I use void* as bit bucket with the indication that what is inside is,
> well, a pointer.
> Ironically, contrary to your insistence of using type information
> correctly, I find my use of void* as a bit bucket for a pointer value
> somewhat more typed (at least in the sense of being self-documenting)
> than your DWORD, which is a bit bucket completely devoid of any self
> description and even exposing wrong set of operations (arithmetic? bit
> shifting? silent integer conversion? is it a set of flags or
> something?).

We used DWord in CLAW because that's what Win32 used for that parameter. 
Otherwise, I think we would have done something more in line with the 
"Opaque_Access" suggestion made elsewhere in this thread, which is #2 here.]

But we also could have removed the arithmetic operations, if we were using 
this as a stand-alone solution, as they're harmful in this case. Indeed, 
it's often the case that modular types are not used as numeric, and it's 
annoying that they are that way by default. (Making them private doesn't 
really solve anything here, because these are already only used in the 
private part.)

> Sorry, you are not convincing. I'll stick to void* when pointers are
> involved.

You can't write "void *" in Ada! And this is the wrong forum to care at all 
about how to write C.

In particular, we're talking about the mapping of C to Ada here, and not the 
underlying design of the C API (which is out of our control anyway). The 
question is how to map "void *" to Ada?

1) Some convention C access type designating an appropriate Ada type.

2) Some convention C access type designating a dummy Ada type (use 
unchecked_conversion as needed on the real types).

3) A constrained array of stream elements (also using U_C).

4) An explicitly declared modular type with the arithmetic operations 
removed (also using U_C).

...

674) System.Address, using Access_to_Address_Conversions to get to/from the 
real type.

675) System.Address, using U_C to get to/from the real type.

>> System.Address is the wrong type to use for a "bucket-of-bits".
>
> It is a proper type to use for a "bucket-of-pointer-bits" in the sense
> that I have no doubts about the intent when I see it later in code.

Well, I doubt that clarity of intent extends to other Ada programmers. 
Perhaps not to you, either, years in the future.

                                            Randy.

Re: Dispatching callback handed over to C

[Maciej Sobczak]

On 5 Kwi, 11:13, Natasha Kerensikova <lithium...@gmail.com> wrote:

> So for my binding, what about something like :
>
>    type Opaque_Data is null record;
>    pragma Convention (C, Opaque_Data);
>
>    type Opaque_Pointer is access all Opaque_Data;
>    pragma Convention (C, Opaque_Pointer);
[...]

Looks good!

> As far as I cen tell, Opaque_Pointer refers only to types compatibles
> with C,

> Or is there some trap in the above code that I'm missing?

That according to AARM, the C counterpart needs to correspond to
Opaque_Data, that is, a pointer to an empty struct with the same
alignment requirements.

But in general, this seems to be the most convincng solution.

--
Maciej Sobczak * http://www.inspirel.com

Re: Dispatching callback handed over to C

[Simon Wright]

Maciej Sobczak <see.my.homepage@gmail.com> writes:

> On 5 Kwi, 11:13, Natasha Kerensikova <lithium...@gmail.com> wrote:
>
>> So for my binding, what about something like :
>>
>>    type Opaque_Data is null record;
>>    pragma Convention (C, Opaque_Data);
>>
>>    type Opaque_Pointer is access all Opaque_Data;
>>    pragma Convention (C, Opaque_Pointer);
> [...]
>
> Looks good!
>
>> As far as I cen tell, Opaque_Pointer refers only to types compatibles
>> with C,
>
>> Or is there some trap in the above code that I'm missing?
>
> That according to AARM, the C counterpart needs to correspond to
> Opaque_Data, that is, a pointer to an empty struct with the same
> alignment requirements.

My approach in TASH[1] was to write a script in Tcl to write (at
installation) a C program which #include's tcl.h etc to generate an Ada
spec[2] containing eg

   --  Size macros defined in tcl.h.

   NUM_STATIC_TOKENS : constant := 20;

   TCL_DSTRING_STATIC_SIZE : constant := 200;

   --  Sizes of structs defined in tcl.h.

   Tcl_Interp_Size : constant := 24;
   Tcl_Interp_Alignment : constant := 8;

but this is only needed where the user has to declare an instance of an
object. Otherwise, and this must be especially true where the C type is
void *, the library gives you back a handle and your only responsibility
is to give it back when called for.

Typically I wrote

   type Tcl_Interp_Rec (<>) is private;
   type Tcl_Interp is access all Tcl_Interp_Rec;
   pragma Convention (C, Tcl_Interp);

private

   type Tcl_Interp_Rec is new Interfaces.C.char_array
     (0 .. Tcl_Record_Sizes.Tcl_Interp_Size - 1);
   for Tcl_Interp_Rec'Alignment
     use Tcl_Record_Sizes.Tcl_Interp_Alignment;

I think I could easily have got away with 

   type Tcl_Interp_Rec is null record;

and left out all the size/alignment stuff!

[1] https://sourceforge.net/projects/tcladashell/
[2] this may have been a step further than necessary!

Re: Dispatching callback handed over to C

[Natasha Kerensikova]

On 2012-04-05, Randy Brukardt <randy@rrsoftware.com> wrote:
> In particular, we're talking about the mapping of C to Ada here, and not the 
> underlying design of the C API (which is out of our control anyway). The 
> question is how to map "void *" to Ada?
>
> 1) Some convention C access type designating an appropriate Ada type.
>
> 2) Some convention C access type designating a dummy Ada type (use 
> unchecked_conversion as needed on the real types).

What I'm still struggling with is why do I have to settle for #2, or
what is preventing me from going for #1?

   type Callback_Access is access all Event_Callback'Class;
   pragma Convention (C, Callback_Access);

Using Callback_Access in a subprogram imported from C trigged the
following GNAT warning:

        >>> warning: "Imported_Subprogram.Arg" involves a tagged type
which does not correspond to any C type

I guess since it's "only" a warning, I could just silence. However I
can't bring myself to silence something I don't understand.

If it's only about the fact that Callback_Access is a C pointer to an
object with which C subprograms has no way for dealing properly, then
fine (since in that particular case C code considers it as an opaque
pointer).

However it's worded like "warning: you're mixing C and Ada", to which
I'm tempted to answer "breaking news ! writing a binding involves mixing
two languages", but that would be rude and there is no point in telling
a compiler things in a human language anyway.

On the other hand, solution #2 allows the import package to not know
about Event_Callback type, and potentially be used with other Ada
callback implementation.


Thanks for your insights,
Natasha

Re: Dispatching callback handed over to C

[Simon Wright]

Natasha Kerensikova <lithiumcat@gmail.com> writes:

> On 2012-04-05, Randy Brukardt <randy@rrsoftware.com> wrote:
>> In particular, we're talking about the mapping of C to Ada here, and
>> not the underlying design of the C API (which is out of our control
>> anyway). The question is how to map "void *" to Ada?
>>
>> 1) Some convention C access type designating an appropriate Ada type.
>>
>> 2) Some convention C access type designating a dummy Ada type (use 
>> unchecked_conversion as needed on the real types).
>
> What I'm still struggling with is why do I have to settle for #2, or
> what is preventing me from going for #1?
>
>    type Callback_Access is access all Event_Callback'Class;
>    pragma Convention (C, Callback_Access);
>
> Using Callback_Access in a subprogram imported from C trigged the
> following GNAT warning:
>
>         >>> warning: "Imported_Subprogram.Arg" involves a tagged type
> which does not correspond to any C type
>
> I guess since it's "only" a warning, I could just silence. However I
> can't bring myself to silence something I don't understand.
>
> If it's only about the fact that Callback_Access is a C pointer to an
> object with which C subprograms has no way for dealing properly, then
> fine (since in that particular case C code considers it as an opaque
> pointer).

I think that must be it. I did wonder whether a classwide pointer might
be a fat pointer, but one would hope that in that case the compiler
would complain that pointer-to-classwide doesn't fit in a standard C
pointer.

I do seem to remember that, in a similar context - possibly context
parameters to Xt callbacks - a GNAT compiler managed to squeeze what was
normally a fat pointer into a thin one. It was a long time ago.

However, in this case GNAT uses thin (64-bit) pointers for both Ada and
C convention pointers (GCC 4.6.0, x86_64-apple-darwin10).

Re: Dispatching callback handed over to C

[Robert A Duff]

Simon Wright <simon@pushface.org> writes:

> Natasha Kerensikova <lithiumcat@gmail.com> writes:

> I think that must be it.

That's right.  If you're not doing anything with these things
on the C side, except copying the pointers around, then it's
fine to silence the warning.  Use the form of pragma Warnings(Off/On)
with a string around that one line of code, so you only silence
the one warning.

There's nothing wrong with silencing warnings when they are
false alarms, although it's a good idea to have a comment
explaining why.

>...I did wonder whether a classwide pointer might
> be a fat pointer, ...

GNAT never uses fat pointers for class-wide.  Fat pointers
are used only when the designated type is an unconstrained
array (and there are ways to get thin pointers for arrays).
I am ignoring access-to-subprogram, here.

- Bob

Re: Dispatching callback handed over to C

[Maciej Sobczak]

On 6 Kwi, 13:41, Simon Wright <si...@pushface.org> wrote:

> My approach in TASH[1] was to write a script in Tcl to write (at
> installation) a C program which #include's tcl.h etc to generate an Ada
> spec[2] containing eg

>    --  Sizes of structs defined in tcl.h.
>
>    Tcl_Interp_Size : constant := 24;
>    Tcl_Interp_Alignment : constant := 8;
>
> but this is only needed where the user has to declare an instance of an
> object.

I have implemented it dynamically - you can export a similar constant
from C and read it from Ada in order to know the size of memory block
to allocate.
This can have an impact on some object-allocation language constructs,
which will become runtime dependent (returning object from a function,
for example), which is incompatible with the No_Implicit_Allocation
pragma (part of Ravenscar), but in general-purpose code it works fine
without the need to complicate the installation process.

--
Maciej Sobczak * http://www.inspirel.com


 Otherwise, and this must be especially true where the C type is
> void *, the library gives you back a handle and your only responsibility
> is to give it back when called for.
>
> Typically I wrote
>
>    type Tcl_Interp_Rec (<>) is private;
>    type Tcl_Interp is access all Tcl_Interp_Rec;
>    pragma Convention (C, Tcl_Interp);
>
> private
>
>    type Tcl_Interp_Rec is new Interfaces.C.char_array
>      (0 .. Tcl_Record_Sizes.Tcl_Interp_Size - 1);
>    for Tcl_Interp_Rec'Alignment
>      use Tcl_Record_Sizes.Tcl_Interp_Alignment;
>
> I think I could easily have got away with
>
>    type Tcl_Interp_Rec is null record;
>
> and left out all the size/alignment stuff!
>
> [1]https://sourceforge.net/projects/tcladashell/
> [2] this may have been a step further than necessary!