Passing C-style flags to C subprograms

Passing C-style flags to C subprograms

[Natasha Kerensikova]

Hello,

I have been trying to write an Ada binding to libevent2. There are all
sorts of issues and I have only started, so I'm not sure it will ever
lead to anything interesting, except for my own personal enlightenment.
One of these issues is passing and retrieving "C-style" flags, and I
would love to see sharing of insights on this topic.

I guess "C-style flags" could be defined as a set of boolean values
represented on the least significant of a machine word, and manipulated
as "int" values (or sometimes "unsigned" or "unsigned long").

The pattern in libevent2 is to declare an enum type for the flag set
objects (which allows some form of type checking), and enumeration
elements are powers of two, so that can be or'ed together to form the
final flag integer.

So the problem is, what is the best way to deal with such flags from
Ada?

Of course "best" depends on how a solution is evaluated. Usually my
first priority is client code readability and maintenability, followed
by library code readability and maintanability, followed by
"efficiency". However sometimes (in critical parts identified through
profiling and/or other a-posteriori analysis) time and/or space
efficiency jumps as the first concern, and having a solution for these
cases might be useful too (especially for bindings of stuff like
libevent, which is advertised for its scalability).

I found a solution in Florist. I don't whether it's specific to Florist
or standard (I haven't been able to find a reference for POSIX for Ada).
It defines a private Option_Set type, which is actually a modular
integer, and operations on it implemented as bitwise arithmetic.

Then binding C-style flags looks like this:

   type These_Flags is new POSIX.Option_Set;

   Allow_Stuff, Do_Otherwise, Special_Feature : constant These_Flags;

   procedure Set_These_Flags (Flags : These_Flags);
   pragma Import (C, Set_These_Flags, "set_these_flags");

   function Get_Flags return These_Flags;
   pragma Import (C, Get_Flags, "get_flags");

private
   Allow_Stuff : constant These_Flags := These_Flags (POSIX.Option_1);
   Do_Otherwise : constant These_Flags := These_Flags (POSIX.Option_2);
   Special_Feature : constant These_Flags := These_Flags (POSIX.Option_3);



And then in the client:

   Set_These_Flags (Allow_Stuff + Do_Otherwise);



On the other hand, while experimenting with my bindings, I came up with
a completely different approach, based on record of booleans and
representation clauses, so that it can be passed-by-copy and be
interpreted correctly as an integer.

Here is how it looks like:

   type These_Flags is record
      Allow_Stuff : Boolean := False;
      Do_Otherwise : Boolean := False;
      Special_Feature : Boolean := False;
   end record;
   for These_Flags use record
      Allow_Stuff at 0 range 0 .. 0;
      Do_Otherwise at 0 range 1 .. 1;
      Special_Feature at 0 range 2 .. Interfaces.C.int'Size;
   end record;
   for These_Flags'Size use Interfaces.C.int'Size;
   pragma Convention (C_Pass_By_Copy, These_Flags);

   procedure Set_These_Flags (Flags : These_Flags);
   pragma Import (C, Set_These_Flags, "set_these_flags");

   function Get_Flags return These_Flags;
   pragma Import (C, Get_Flags, "get_flags");

And then

   Set_These_Flags (These_Flags'(Allow_Stuff => True,
                                 Do_Otherwise => True,
                                 Special_Feature => False));


I find the record-based approach feels more "Ada-ish" while Florist
approach feels much more C-ish, it doesn't clutter the namespace, and it
allows for 2-value enumerations that don't map to boolean (e.g.
switching between two different modes depending on the bit value), so
I generally prefer the former. However, the declaration is much bulkier
and uglier and there is no code reuse.

However, a much more pressing question is: does the record-based
approach actually work?

The fact that it run successfully on my machine does not mean it would
run successfully anywhere else. Even worse, it might run but silently
produce incorrect results.

If I understand the RM correctly, it is guaranteed that
Boolean'Pos (False) = 0 and Boolean'Pos (True) = 1.
But does that mean a 1-bit representation of a boolean necessarily match
the position value?
Should I add some representation clauses somewhere to fix that?

Can I trust the extended range of the last flag to not cause any
trouble? (I used it rather than "range 2 .. 2" to avoid a GNAT warning,
and it seemed less ugly than a pragma Warnings or bother the client with
an explicit padding field).

Could endianness break the solution? (I understood that that "at 0" part
avoids it by looking at bit rank in a machine word, so with the same
mapping as a C int, but I might be wrong).

Are there any other issues I have missed?


Thanks in advance for your insights,
Natasha

Re: Passing C-style flags to C subprograms

[Randy Brukardt]

"Natasha Kerensikova" <lithiumcat@gmail.com> wrote in message 
news:slrnjnbu7o.1lme.lithiumcat@sigil.instinctive.eu...
> I have been trying to write an Ada binding to libevent2. There are all
> sorts of issues and I have only started, so I'm not sure it will ever
> lead to anything interesting, except for my own personal enlightenment.
> One of these issues is passing and retrieving "C-style" flags, and I
> would love to see sharing of insights on this topic.
...
> I found a solution in Florist. I don't whether it's specific to Florist
> or standard (I haven't been able to find a reference for POSIX for Ada).
> It defines a private Option_Set type, which is actually a modular
> integer, and operations on it implemented as bitwise arithmetic.
...
This is essentilly what Claw does for (most) Windows flags.

...
> On the other hand, while experimenting with my bindings, I came up with
> a completely different approach, based on record of booleans and
> representation clauses, so that it can be passed-by-copy and be
> interpreted correctly as an integer.
...

We considered this and used it for a few sets of flags in Claw, but didn't 
do this in general for a number of reasons:

(1) We needed the flag sets to be extensible without reprogramming. That is, 
if Microsoft adds a new flag value, we want people to be able to use it even 
before Claw is modified to add the flag. That works well with a numeric 
solution (just add a new constant); it doesn't work at all with a bit 
solution (unused bits stay unused).
(2) Not all of the flag sets are pure bit-maps. Some include combinations so 
that one bit doesn't necessarily represent one flag.
(3) Ada doesn't have defaults for aggregates; every component has to be 
mentioned (or an others clause used). That means that the aggregates 
representing flags are wordy; and the use of an others clause makes them 
less safe against modification than typical Ada code. (It's about the same 
safety as the original bit-flags; it's not a detrement to using records, but 
it means that there is no advantage in this way, either.)

That is, compare:
          Style => WS_Border + WS_Sizable,
to
          Style => (WS_Border => True, WS_Sizable => True, others => False),

Since uses are much more common than the set of declarations, we wanted the 
uses to be easier.

(Note that for very commonly used flags, we broke those out and gave them 
their own parameters to the routines, which can be properly defaulted and 
thus not mentioned at all in calls.)

> I find the record-based approach feels more "Ada-ish" while Florist
> approach feels much more C-ish, it doesn't clutter the namespace, and it
> allows for 2-value enumerations that don't map to boolean (e.g.
> switching between two different modes depending on the bit value), so
> I generally prefer the former. However, the declaration is much bulkier
> and uglier and there is no code reuse.

Right. As noted above, however, the declaration is close to irrelevant; the 
real issue is that the *uses* are much bulkier.

> However, a much more pressing question is: does the record-based
> approach actually work?

Yes, assuming appropriate representation clauses. Either that or your 
compiler should reject it (but that won't happen for typical cases).

> The fact that it run successfully on my machine does not mean it would
> run successfully anywhere else. Even worse, it might run but silently
> produce incorrect results.
>
> If I understand the RM correctly, it is guaranteed that
> Boolean'Pos (False) = 0 and Boolean'Pos (True) = 1.
> But does that mean a 1-bit representation of a boolean necessarily match
> the position value?
> Should I add some representation clauses somewhere to fix that?

The representation of an enumeration is the same as the position numbers 
unless you specify otherwise. This is a requirement -- see 13.4(8).

> Can I trust the extended range of the last flag to not cause any
> trouble? (I used it rather than "range 2 .. 2" to avoid a GNAT warning,
> and it seemed less ugly than a pragma Warnings or bother the client with
> an explicit padding field).

I think it is OK, but I've always used a padding field in such records. The 
main reason being that when the next version of the underlying thing comes 
out with two new flags, you won't have a problem with invalid codes in that 
field. (This matters a lot if you can read these flags as well as set them, 
as in Windows.)

> Could endianness break the solution? (I understood that that "at 0" part
> avoids it by looking at bit rank in a machine word, so with the same
> mapping as a C int, but I might be wrong).

Endianness only matters for items larger than a byte, so for most simple 
flags it's irrelevant. Moreover, endianness only matters if you need to do 
something that has to work on a machine with different endianness than the 
one you are running on. (The default bit numbering works "properly" on 
either endianness so long as the endianness is the same for all uses of the 
type, as it will be in a single program.)

[But note, I've never done this on a big-endian machine, so there may be 
some issue that I've missed.]

> Are there any other issues I have missed?

Not that I know of.

                                  Randy.

Re: Passing C-style flags to C subprograms

[Natasha Kerensikova]

Hello,

On 2012-03-30, Randy Brukardt <randy@rrsoftware.com> wrote:
> "Natasha Kerensikova" <lithiumcat@gmail.com> wrote in message 
> news:slrnjnbu7o.1lme.lithiumcat@sigil.instinctive.eu...
>> I have been trying to write an Ada binding to libevent2. There are all
>> sorts of issues and I have only started, so I'm not sure it will ever
>> lead to anything interesting, except for my own personal enlightenment.
>> One of these issues is passing and retrieving "C-style" flags, and I
>> would love to see sharing of insights on this topic.
> ...
>> I found a solution in Florist. I don't whether it's specific to Florist
>> or standard (I haven't been able to find a reference for POSIX for Ada).
>> It defines a private Option_Set type, which is actually a modular
>> integer, and operations on it implemented as bitwise arithmetic.
> ...
> This is essentilly what Claw does for (most) Windows flags.

Thanks a lot for sharing your experience. I was about to go for the
other solution, so you saved me the time to switch away from it when I
would have realized by myself that it had been a mistake.

One thing I still don't completely understand though is the choice of
">=" operator for testing output flags. Could you explain the rationale
behind it?

The only idea I have found it to consider flag type as representing
subsets of flags, and then consider ">=" as the equivalent of inclusion,
which is an order in the set of subsets. But in that case, wouldn't it
make more sense to use set-ish operators like "or" and "and" instead of
"+" and "-"?  Also "+" and "-" feel like operators on numbers, so having
a ">=" besides feels like an total order, while inclusion is only a
partial order.

Unless I'm missing something, that's the rational behind operators on
Ada.Strings.Maps.Character_Set. So I would have expected a Bit_Set or
Flag_Set (to include multi-bit flags) to have a similar interface. Why
isn't it so?



And as a side question, for situations when time efficiency is critical,
what are the ways to help a compiler use intrinsic operators for setting
and testing flags?

I guess publicly deriving specific flag types from Interfaces.C.unsigned
is a way to inherit implicit operators that are intrinsic, but then I
also allow operators that make no sense for a flag type, like "**" or
"/".

However, wouldn't hiding derivation in private part also turn public
operators into non-intrinsic operations?

And is there a way to allow "+" to be implemented with intrinsic "or" on
C.unsigned? If I understand 8.5.4.5 (from Ada 2005 RM) correctly,
renaming cannot do that. Or would a pragma Inline be enough to insert
the expression (with its private intrinsic "or") into the caller code
and end up with the same (assembly) result?

Or should all the above be just dismissed as premature optimization? (I
don't think so, because I believe it is never wrong to question, and
premature optimization is a wrong answer rather than a wrong question;
but I might be wrong).


Thanks again for your insights,
Natasha

Re: Passing C-style flags to C subprograms

[Randy Brukardt]

"Natasha Kerensikova" <lithiumcat@gmail.com> wrote in message 
news:slrnjnj0gk.1lme.lithiumcat@sigil.instinctive.eu...
> Hello,
>
> On 2012-03-30, Randy Brukardt <randy@rrsoftware.com> wrote:
>> "Natasha Kerensikova" <lithiumcat@gmail.com> wrote in message
>> news:slrnjnbu7o.1lme.lithiumcat@sigil.instinctive.eu...
>>> I have been trying to write an Ada binding to libevent2. There are all
>>> sorts of issues and I have only started, so I'm not sure it will ever
>>> lead to anything interesting, except for my own personal enlightenment.
>>> One of these issues is passing and retrieving "C-style" flags, and I
>>> would love to see sharing of insights on this topic.
>> ...
>>> I found a solution in Florist. I don't whether it's specific to Florist
>>> or standard (I haven't been able to find a reference for POSIX for Ada).
>>> It defines a private Option_Set type, which is actually a modular
>>> integer, and operations on it implemented as bitwise arithmetic.
>> ...
>> This is essentilly what Claw does for (most) Windows flags.
>
> Thanks a lot for sharing your experience. I was about to go for the
> other solution, so you saved me the time to switch away from it when I
> would have realized by myself that it had been a mistake.
>
> One thing I still don't completely understand though is the choice of
> ">=" operator for testing output flags. Could you explain the rationale
> behind it?

The comments in Claw explain this: this defines "inclusion":

        function ">=" (Left, Right : in Window_Style_Type) return Boolean;
            -- Read  A >= B   as   A includes B
            -- That is, each style set in B is also set in A.

I also should note that we made these type private (so no predefined 
operators are inherited visibly). Instead, we defined a batch of constants 
and a pair of conversion operators (so that values that we didn't anticipate 
could be handled).

> The only idea I have found it to consider flag type as representing
> subsets of flags, and then consider ">=" as the equivalent of inclusion,
> which is an order in the set of subsets. But in that case, wouldn't it
> make more sense to use set-ish operators like "or" and "and" instead of
> "+" and "-"?  Also "+" and "-" feel like operators on numbers, so having
> a ">=" besides feels like an total order, while inclusion is only a
> partial order.

I think the idea was that "+" is "adding" a style to the set, and "-" is 
removing a style from the set. We definitely did not want anyone to think of 
any of the these operators as numeric.

The problem with "or" is that it doesn't have an obvious relationship to a 
set operator. It's (at best) an operator on bits, and we did not want to 
talk about bits. We wanted to talk solely about operations on the set of 
styles (adding or removing styles, etc.). [Admittedly, the "escape hatch" of 
using numbers via the conversion functions somewhat dilutes that approach, 
but we thought that most users would never need to worry about those "escape 
hatches".] We could have tried to use functions like "Union" or "Add" 
instead, but those would have gotten wordy: the "+" notation is shorter.

(Note that I don't remember this clearly and I may have misremembered why we 
did some of the things we did.)

Of course, YMMV; every problem is different.

                       Randy.

Re: Passing C-style flags to C subprograms

[BrianG]

On 03/30/2012 03:57 PM, Randy Brukardt wrote:
> "Natasha Kerensikova"<lithiumcat@gmail.com>  wrote in message
> news:slrnjnbu7o.1lme.lithiumcat@sigil.instinctive.eu...
> ...
>
> We considered this and used it for a few sets of flags in Claw, but didn't
> do this in general for a number of reasons:
>
> (1) We needed the flag sets to be extensible without reprogramming. That is,
> if Microsoft adds a new flag value, we want people to be able to use it even
> before Claw is modified to add the flag. That works well with a numeric
> solution (just add a new constant); it doesn't work at all with a bit
> solution (unused bits stay unused).
> (2) Not all of the flag sets are pure bit-maps. Some include combinations so
> that one bit doesn't necessarily represent one flag.
> (3) Ada doesn't have defaults for aggregates; every component has to be
> mentioned (or an others clause used). That means that the aggregates
> representing flags are wordy; and the use of an others clause makes them
> less safe against modification than typical Ada code. (It's about the same
> safety as the original bit-flags; it's not a detrement to using records, but
> it means that there is no advantage in this way, either.)
>
You could also add
(4) Some users may be used to using the "Option_A + Option_B" type 
format.  While I consider this trivial, in building a library or 
binding, you need to worry about your users' priorities, not your own. 
It can also make porting existing apps easier.  (Of course, you could 
always define your own "+" or whatever - and I have done this.)


-- 
---
BrianG
000
@[Gee, Mail!]
.com

Re: Passing C-style flags to C subprograms

[Jeffrey Carter]

On 03/30/2012 11:18 AM, Natasha Kerensikova wrote:
>
>     type These_Flags is record
>        Allow_Stuff : Boolean := False;
>        Do_Otherwise : Boolean := False;
>        Special_Feature : Boolean := False;
>     end record;
>     for These_Flags use record
>        Allow_Stuff at 0 range 0 .. 0;
>        Do_Otherwise at 0 range 1 .. 1;
>        Special_Feature at 0 range 2 .. Interfaces.C.int'Size;
>     end record;
>     for These_Flags'Size use Interfaces.C.int'Size;

I don't see how this could work. You've specified Interfaces.C.Int'Size + 1 bits 
in These_Flags, and then specified that These_Flags is Interfaces.C.Int'Size 
bits in size.

It's probably better to have 3 Boolean fields and a padding field that takes up 
the additional bits. That will work as long as you don't move to a big-endian 
machine.

If you're worried about portability, the modular type approach is probably 
better. Brukardt has pointed out some pros and cons of both approaches.

If you're really worried about portability, then you should make sure you pass 
the same kind to C that C expects. If the C functions take an int, then you 
should pass an Interfaces.C.Int to them. This doesn't prevent using either of 
the approaches, but does involve an unchecked conversion to or from 
Interfaces.C.Int hidden in the lowest level of your code.

There are at least 2 parts to your problem: How you make this appear to the rest 
of your Ada code, and how you implement the low-level interface to C. There 
could also be a 3rd part that sits in between those, something more Ada-like 
than the low-level interface but less application-specific than the interface to 
this specific application.

-- 
Jeff Carter
"Friends don't let friends program in C++."
Ludovic Brenta
114