Interfaces.C.Strings chars_ptr memory management strategy

Interfaces.C.Strings chars_ptr memory management strategy

[NiGHTS]

I am creating a binding to a C library that requires me to repeat the same function but with a string parameter that changes on each call. I don't want to have to keep creating and destroying string memory for all of these function calls. I would like to create the memory for the string once, allocate enough space so it doesn't need to grow, and reuse that memory for the function call every time I need to pass a new string to it.

The trick here is that whatever strategy I use must be compatible with C, so for instance using a storage pool would not be directly compatible with the C binding.

Here is just a quick and sloppy idea I had on how to tackle my problem.

str : chars_ptr := New_String ("                                     ");
...
Update (Item => str, Offset => 0, Str => "Some Param");
...
Update (Item => str, Offset => 0, Str => "Some Other Param");
...
Free (str);

I find the first line quite ugly. I'm sure there is an easier way to create a large empty string but I can't seem to come up with an elegant way to do it.

As far as the Update commands, will it act like strcpy() in C? If so I'd guess that this is an efficient technique.

Thanks for your help!

Re: Interfaces.C.Strings chars_ptr memory management strategy

[Shark8]

On Friday, May 25, 2018 at 4:22:13 PM UTC-6, NiGHTS wrote:
> 
> Here is just a quick and sloppy idea I had on how to tackle my problem.
> 
> str : chars_ptr := New_String ("                                     ");
> 
> I find the first line quite ugly. I'm sure there is an easier way to create a large empty string but I can't seem to come up with an elegant way to do it.


Something like this?
Big_String : chars_ptr := New_String( (1..200 => ' ') );

Re: Interfaces.C.Strings chars_ptr memory management strategy

[NiGHTS]

On Friday, May 25, 2018 at 10:52:55 PM UTC-4, Shark8 wrote:
> On Friday, May 25, 2018 at 4:22:13 PM UTC-6, NiGHTS wrote:
> > 
> > Here is just a quick and sloppy idea I had on how to tackle my problem.
> > 
> > str : chars_ptr := New_String ("                                     ");
> > 
> > I find the first line quite ugly. I'm sure there is an easier way to create a large empty string but I can't seem to come up with an elegant way to do it.
> 
> 
> Something like this?
> Big_String : chars_ptr := New_String( (1..200 => ' ') );

Thanks, It was on the tip of my brain I just for some reason couldn't put it together quite right.

Re: Interfaces.C.Strings chars_ptr memory management strategy

[Shark8]

On Saturday, May 26, 2018 at 6:45:01 AM UTC-6, NiGHTS wrote:
> On Friday, May 25, 2018 at 10:52:55 PM UTC-4, Shark8 wrote:
> > On Friday, May 25, 2018 at 4:22:13 PM UTC-6, NiGHTS wrote:
> > > 
> > > Here is just a quick and sloppy idea I had on how to tackle my problem.
> > > 
> > > str : chars_ptr := New_String ("                                     ");
> > > 
> > > I find the first line quite ugly. I'm sure there is an easier way to create a large empty string but I can't seem to come up with an elegant way to do it.
> > 
> > 
> > Something like this?
> > Big_String : chars_ptr := New_String( (1..200 => ' ') );
> 
> Thanks, It was on the tip of my brain I just for some reason couldn't put it together quite right.

It happens.
If you're going to be using this a lot I'd recommend a generic-wrapper something like:

GENERIC
  Max_Size : Natural;
FUNCTION Buffer_String Return Chars_Ptr;
FUNCTION Buffer_String Return Chars_Ptr IS
( New_String( (1..Max_Size => ' ') ) );

Or maybe a package.

Re: Interfaces.C.Strings chars_ptr memory management strategy

[Alejandro R. Mosteo]

On 26/05/2018 00:22, NiGHTS wrote:
> I am creating a binding to a C library that requires me to repeat the same function but with a string parameter that changes on each call. I don't want to have to keep creating and destroying string memory for all of these function calls. I would like to create the memory for the string once, allocate enough space so it doesn't need to grow, and reuse that memory for the function call every time I need to pass a new string to it.

I'm currently using this:

https://github.com/mosteo/cstrings

which is not what you want since it allocates on every instance (although 
on the stack). Still, it might give you some ideas.

I'm still unsure if that's 100% guaranteed to be safe; for the experts 
out here, the question is, in a call to a C function like this:

Call_To_C_Function
    (Function_That_Returns_A_Limited_Tagged_Type (...)
       .Subprogram_That_Returns_A_C_Pointer_To_Data_In_The_Tagged_Type);

Is the in-place built limited tagged type guaranteed to live during the 
call to the C function? (In other words, is the pointer safe (as long as 
the C side does not make a copy, of course)?

My suspicion is that once the subprogram returns the pointer, the limited 
type can be optimized away before the call to the C side. It's not what 
I'm seeing now, but I don't want to depend on an erroneous assumption.

Alex.

> 
> The trick here is that whatever strategy I use must be compatible with C, so for instance using a storage pool would not be directly compatible with the C binding.
> 
> Here is just a quick and sloppy idea I had on how to tackle my problem.
> 
> str : chars_ptr := New_String ("                                     ");
> ...
> Update (Item => str, Offset => 0, Str => "Some Param");
> ...
> Update (Item => str, Offset => 0, Str => "Some Other Param");
> ...
> Free (str);
> 
> I find the first line quite ugly. I'm sure there is an easier way to create a large empty string but I can't seem to come up with an elegant way to do it.
> 
> As far as the Update commands, will it act like strcpy() in C? If so I'd guess that this is an efficient technique.
> 
> Thanks for your help!
> 

Re: Interfaces.C.Strings chars_ptr memory management strategy

[Randy Brukardt]

"Alejandro R. Mosteo" <alejandro@mosteo.com> wrote in message 
news:pem7s0$go4$1@dont-email.me...
> On 26/05/2018 00:22, NiGHTS wrote:
>> I am creating a binding to a C library that requires me to repeat the 
>> same function but with a string parameter that changes on each call. I 
>> don't want to have to keep creating and destroying string memory for all 
>> of these function calls. I would like to create the memory for the string 
>> once, allocate enough space so it doesn't need to grow, and reuse that 
>> memory for the function call every time I need to pass a new string to 
>> it.
>
> I'm currently using this:
>
> https://github.com/mosteo/cstrings
>
> which is not what you want since it allocates on every instance (although 
> on the stack). Still, it might give you some ideas.
>
> I'm still unsure if that's 100% guaranteed to be safe; for the experts out 
> here, the question is, in a call to a C function like this:
>
> Call_To_C_Function
>    (Function_That_Returns_A_Limited_Tagged_Type (...)
>       .Subprogram_That_Returns_A_C_Pointer_To_Data_In_The_Tagged_Type);
>
> Is the in-place built limited tagged type guaranteed to live during the 
> call to the C function? (In other words, is the pointer safe (as long as 
> the C side does not make a copy, of course)?

That depends on the master of the parameter. I believe that the master of a 
parameter is that of the call (each call being it's own master for the 
parameters) -- you'd have to look in 7.6.1 to be sure. So they stay around 
as long as the call.

If that wasn't true, passing an aggregate could have the temporary object 
freed/finalized before the call ended, which would be a disaster.

> My suspicion is that once the subprogram returns the pointer, the limited 
> type can be optimized away before the call to the C side. It's not what 
> I'm seeing now, but I don't want to depend on an erroneous assumption.

 Don't think this is a problem in general -- and the result of a function 
does *not* belong to the master of the call but rather the enclosing one 
(else you couldn't use it before it went away). You might be able to get it 
with nested calls:

     Foo (Bar (Ugh (...))

The result of Ugh is finalized when the call to Bar ends, so if it is 
somehow in the result of Bar, you could get trouble in Foo. You can avoid 
that by declaring the parameter "aliased" (those belong to the *result* of 
the function, so they stick around longer).

                                                          Randy.

Re: Interfaces.C.Strings chars_ptr memory management strategy

[Alejandro R. Mosteo]

On 30/05/2018 21:56, Randy Brukardt wrote:
> "Alejandro R. Mosteo" <alejandro@mosteo.com> wrote in message
> news:pem7s0$go4$1@dont-email.me...
>> On 26/05/2018 00:22, NiGHTS wrote:
>>> I am creating a binding to a C library that requires me to repeat the
>>> same function but with a string parameter that changes on each call. I
>>> don't want to have to keep creating and destroying string memory for all
>>> of these function calls. I would like to create the memory for the string
>>> once, allocate enough space so it doesn't need to grow, and reuse that
>>> memory for the function call every time I need to pass a new string to
>>> it.
>>
>> I'm currently using this:
>>
>> https://github.com/mosteo/cstrings
>>
>> which is not what you want since it allocates on every instance (although
>> on the stack). Still, it might give you some ideas.
>>
>> I'm still unsure if that's 100% guaranteed to be safe; for the experts out
>> here, the question is, in a call to a C function like this:
>>
>> Call_To_C_Function
>>     (Function_That_Returns_A_Limited_Tagged_Type (...)
>>        .Subprogram_That_Returns_A_C_Pointer_To_Data_In_The_Tagged_Type);
>>
>> Is the in-place built limited tagged type guaranteed to live during the
>> call to the C function? (In other words, is the pointer safe (as long as
>> the C side does not make a copy, of course)?
> 
> That depends on the master of the parameter. I believe that the master of a
> parameter is that of the call (each call being it's own master for the
> parameters) -- you'd have to look in 7.6.1 to be sure. So they stay around
> as long as the call.

Might that be 6.4.1? 7 deals with packages (in 2012).

Although it was too dense for me anyway :(

> If that wasn't true, passing an aggregate could have the temporary object
> freed/finalized before the call ended, which would be a disaster.
> 
>> My suspicion is that once the subprogram returns the pointer, the limited
>> type can be optimized away before the call to the C side. It's not what
>> I'm seeing now, but I don't want to depend on an erroneous assumption.
> 
>   Don't think this is a problem in general -- and the result of a function
> does *not* belong to the master of the call but rather the enclosing one
> (else you couldn't use it before it went away). You might be able to get it
> with nested calls:
> 
>       Foo (Bar (Ugh (...))
> 
> The result of Ugh is finalized when the call to Bar ends, so if it is
> somehow in the result of Bar, you could get trouble in Foo. You can avoid
> that by declaring the parameter "aliased" (those belong to the *result* of
> the function, so they stick around longer).

The parameter is actually aliased already. So I hope your impressions are 
right and I'm on firm ground then, a pleasant surprise.

Thanks,
Alex.

> 
>                                                            Randy.
> 
> 

Re: Interfaces.C.Strings chars_ptr memory management strategy

[Randy Brukardt]

"Alejandro R. Mosteo" <alejandro@mosteo.com> wrote in message 
news:peoj3f$8ti$1@dont-email.me...
> On 30/05/2018 21:56, Randy Brukardt wrote:
...
>>> Is the in-place built limited tagged type guaranteed to live during the
>>> call to the C function? (In other words, is the pointer safe (as long as
>>> the C side does not make a copy, of course)?
>>
>> That depends on the master of the parameter. I believe that the master of 
>> a
>> parameter is that of the call (each call being it's own master for the
>> parameters) -- you'd have to look in 7.6.1 to be sure. So they stay 
>> around
>> as long as the call.
>
> Might that be 6.4.1? 7 deals with packages (in 2012).

No. The rules for masters are defined with finalization in 7.6, and 
specifically in 7.6.1(3/2). The single middle sentence in that paragraph (a 
classic RM run-on sentence) defines completely where every object in an Ada 
program is finalized -- and also defines large parts of the accessibility 
and tasking models (which mainly follow the same master rules).

> Although it was too dense for me anyway :(

That's why I didn't want to give you a definitive answer. It takes a lot of 
mental effort to do that, and I need to save that effort for things people 
pay me to do. ;-)

...
>>       Foo (Bar (Ugh (...))
>>
>> The result of Ugh is finalized when the call to Bar ends, so if it is
>> somehow in the result of Bar, you could get trouble in Foo. You can avoid
>> that by declaring the parameter "aliased" (those belong to the *result* 
>> of
>> the function, so they stick around longer).
>
> The parameter is actually aliased already. So I hope your impressions are 
> right and I'm on firm ground then, a pleasant surprise.

Turns out I was wrong: 7.6.1(3/2) says that only the outer function call is 
a master. So there is no problem in even the case I suggested.

                                                   Randy.

Re: Interfaces.C.Strings chars_ptr memory management strategy

[ytomino]

Perhaps, malloc is better than New_String in this case.

 function malloc (s : Interfaces.C.size_t) return Interfaces.C.Strings.chars_ptr
    with Import, Convention => C;

Re: Interfaces.C.Strings chars_ptr memory management strategy

[Dmitry A. Kazakov]

On 2018-06-03 20:31, ytomino wrote:
> Perhaps, malloc is better than New_String in this case.
> 
>   function malloc (s : Interfaces.C.size_t) return Interfaces.C.Strings.chars_ptr
>      with Import, Convention => C;

I had a case when that caused the application crashed.

I guess it was because of mixed Visual Studio and GCC run-times. The 
pointer returned by the malloc from one was freed in a third-party C 
library by another.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: Interfaces.C.Strings chars_ptr memory management strategy

[ytomino]

On Monday, June 4, 2018 at 4:33:20 AM UTC+9, Dmitry A. Kazakov wrote:
> On 2018-06-03 20:31, ytomino wrote:
> > Perhaps, malloc is better than New_String in this case.
> > 
> >   function malloc (s : Interfaces.C.size_t) return Interfaces.C.Strings.chars_ptr
> >      with Import, Convention => C;
> 
> I had a case when that caused the application crashed.
> 
> I guess it was because of mixed Visual Studio and GCC run-times. The 
> pointer returned by the malloc from one was freed in a third-party C 
> library by another.
> 
> -- 
> Regards,
> Dmitry A. Kazakov
> http://www.dmitry-kazakov.de

What!?

New_String calls malloc in the end, too, in mingw runtime.
(It calls Memory_Alloc, Memory_Alloc is _gnat_malloc, and __gnat_malloc calls malloc.)
https://gcc.gnu.org/svn/gcc/trunk/gcc/ada/libgnat/i-cstrin.adb
https://gcc.gnu.org/svn/gcc/trunk/gcc/ada/libgnat/s-parame.ads
https://gcc.gnu.org/svn/gcc/trunk/gcc/ada/libgnat/s-memory__mingw.adb

If Interfaces.C.Strings.Free (malloc) is crashed, New_String would be same.

Re: Interfaces.C.Strings chars_ptr memory management strategy

[ytomino]

On Monday, June 4, 2018 at 4:33:20 AM UTC+9, Dmitry A. Kazakov wrote:
> On 2018-06-03 20:31, ytomino wrote:
> > Perhaps, malloc is better than New_String in this case.
> > 
> >   function malloc (s : Interfaces.C.size_t) return Interfaces.C.Strings.chars_ptr
> >      with Import, Convention => C;
> 
> I had a case when that caused the application crashed.
> 
> I guess it was because of mixed Visual Studio and GCC run-times. The 
> pointer returned by the malloc from one was freed in a third-party C 
> library by another.
> 
> -- 
> Regards,
> Dmitry A. Kazakov
> http://www.dmitry-kazakov.de

By my intuition, the crash is caused by not free but Update.
Because malloc does not set NUL.
So maybe is the length-checking in Update crashed?
If that is so, "Check => False" should be inserted into the first Update.

 Update (..., Check => False);

Re: Interfaces.C.Strings chars_ptr memory management strategy

[Dmitry A. Kazakov]

On 2018-06-03 10:03 PM, ytomino wrote:
> On Monday, June 4, 2018 at 4:33:20 AM UTC+9, Dmitry A. Kazakov wrote:
>> On 2018-06-03 20:31, ytomino wrote:
>>> Perhaps, malloc is better than New_String in this case.
>>>
>>>    function malloc (s : Interfaces.C.size_t) return Interfaces.C.Strings.chars_ptr
>>>       with Import, Convention => C;
>>
>> I had a case when that caused the application crashed.
>>
>> I guess it was because of mixed Visual Studio and GCC run-times. The
>> pointer returned by the malloc from one was freed in a third-party C
>> library by another.
>>
>> -- 
>> Regards,
>> Dmitry A. Kazakov
>> http://www.dmitry-kazakov.de
> 
> What!?
> 
> New_String calls malloc in the end, too, in mingw runtime.
> (It calls Memory_Alloc, Memory_Alloc is _gnat_malloc, and __gnat_malloc calls malloc.)
> https://gcc.gnu.org/svn/gcc/trunk/gcc/ada/libgnat/i-cstrin.adb
> https://gcc.gnu.org/svn/gcc/trunk/gcc/ada/libgnat/s-parame.ads
> https://gcc.gnu.org/svn/gcc/trunk/gcc/ada/libgnat/s-memory__mingw.adb
> 
> If Interfaces.C.Strings.Free (malloc) is crashed, New_String would be same.

No, not this free but the one called from the third-party library 
because the pointer was passed there to handle.

Of course it is safe to call malloc-free or New_String-Free pairs. Other 
combinations can be unsafe.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: Interfaces.C.Strings chars_ptr memory management strategy

[ytomino]

On Monday, June 4, 2018 at 4:06:56 PM UTC+9, Dmitry A. Kazakov wrote:
> On 2018-06-03 10:03 PM, ytomino wrote:
> > On Monday, June 4, 2018 at 4:33:20 AM UTC+9, Dmitry A. Kazakov wrote:
> >> On 2018-06-03 20:31, ytomino wrote:
> >>> Perhaps, malloc is better than New_String in this case.
> >>>
> >>>    function malloc (s : Interfaces.C.size_t) return Interfaces.C.Strings.chars_ptr
> >>>       with Import, Convention => C;
> >>
> >> I had a case when that caused the application crashed.
> >>
> >> I guess it was because of mixed Visual Studio and GCC run-times. The
> >> pointer returned by the malloc from one was freed in a third-party C
> >> library by another.
> >>
> >> -- 
> >> Regards,
> >> Dmitry A. Kazakov
> >> http://www.dmitry-kazakov.de
> > 
> > What!?
> > 
> > New_String calls malloc in the end, too, in mingw runtime.
> > (It calls Memory_Alloc, Memory_Alloc is _gnat_malloc, and __gnat_malloc calls malloc.)
> > https://gcc.gnu.org/svn/gcc/trunk/gcc/ada/libgnat/i-cstrin.adb
> > https://gcc.gnu.org/svn/gcc/trunk/gcc/ada/libgnat/s-parame.ads
> > https://gcc.gnu.org/svn/gcc/trunk/gcc/ada/libgnat/s-memory__mingw.adb
> > 
> > If Interfaces.C.Strings.Free (malloc) is crashed, New_String would be same.
> 
> No, not this free but the one called from the third-party library 
> because the pointer was passed there to handle.
> 
> Of course it is safe to call malloc-free or New_String-Free pairs. Other 
> combinations can be unsafe.
> 
> -- 
> Regards,
> Dmitry A. Kazakov
> http://www.dmitry-kazakov.de

> Of course it is safe to call malloc-free or New_String-Free pairs. Other 
> combinations can be unsafe.

That is only talking on the standard, probably is not the cause of the crash.

Re: Interfaces.C.Strings chars_ptr memory management strategy

[Alejandro R. Mosteo]

On 01/06/2018 00:25, Randy Brukardt wrote:
> "Alejandro R. Mosteo" <alejandro@mosteo.com> wrote in message
> news:peoj3f$8ti$1@dont-email.me...
>> On 30/05/2018 21:56, Randy Brukardt wrote:
> ...
>>>> Is the in-place built limited tagged type guaranteed to live during the
>>>> call to the C function? (In other words, is the pointer safe (as long as
>>>> the C side does not make a copy, of course)?
>>>
>>> That depends on the master of the parameter. I believe that the master of
>>> a
>>> parameter is that of the call (each call being it's own master for the
>>> parameters) -- you'd have to look in 7.6.1 to be sure. So they stay
>>> around
>>> as long as the call.
>>
>> Might that be 6.4.1? 7 deals with packages (in 2012).
> 
> No. The rules for masters are defined with finalization in 7.6, and
> specifically in 7.6.1(3/2). The single middle sentence in that paragraph (a
> classic RM run-on sentence) defines completely where every object in an Ada
> program is finalized -- and also defines large parts of the accessibility
> and tasking models (which mainly follow the same master rules).

I was a bit thrown off by the use of finalization in the non-controlled 
sense.

>> Although it was too dense for me anyway :(
> 
> That's why I didn't want to give you a definitive answer. It takes a lot of
> mental effort to do that, and I need to save that effort for things people
> pay me to do. ;-)
> 
> ...
>>>        Foo (Bar (Ugh (...))
>>>
>>> The result of Ugh is finalized when the call to Bar ends, so if it is
>>> somehow in the result of Bar, you could get trouble in Foo. You can avoid
>>> that by declaring the parameter "aliased" (those belong to the *result*
>>> of
>>> the function, so they stick around longer).
>>
>> The parameter is actually aliased already. So I hope your impressions are
>> right and I'm on firm ground then, a pleasant surprise.
> 
> Turns out I was wrong: 7.6.1(3/2) says that only the outer function call is
> a master. So there is no problem in even the case I suggested.

Thanks again. And for doing it for free ;-)

Alex.

> 
>                                                     Randy.
> 
>