Execute an OS command and capture output

Execute an OS command and capture output

[Victor Porton]

I need a (preferably portable, but only Linux would be well) way to execute 
an OS command with some input in stdin which I provide and capture the 
output from stdout.

Note that stdin and stdout may possibly pass NUL characters.

I am interested in both shell commands for different shells and in direct 
(such as execve()) execution of a command.

-- 
Victor Porton - http://portonvictor.org

Re: Execute an OS command and capture output

[Victor Porton]

Victor Porton wrote:

> I need a (preferably portable, but only Linux would be well) way to

Also all Ada implementations is desirable, but only GNAT is acceptable.

> execute an OS command with some input in stdin which I provide and capture
> the output from stdout.
> 
> Note that stdin and stdout may possibly pass NUL characters.
> 
> I am interested in both shell commands for different shells and in direct
> (such as execve()) execution of a command.
> 
-- 
Victor Porton - http://portonvictor.org

Re: Execute an OS command and capture output

[Dmitry A. Kazakov]

On 2017-11-13 17:38, Victor Porton wrote:
> I need a (preferably portable, but only Linux would be well) way to execute
> an OS command with some input in stdin which I provide and capture the
> output from stdout.

It is a vague question because there are numerous methods of how the 
input, output and error pipes are handled. E.g. by 0, 1, 2 tasks.

Anyway see:

1. GNAT's System.OS_Lib.Spawn, System.OS_Lib.Non_Blocking_Spawn.

2. A more comfortable method would be to use GTK bindings (GLib actually):

    http://www.dmitry-kazakov.de/ada/gtkada_contributions.htm#10

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: Execute an OS command and capture output

[Victor Porton]

Dmitry A. Kazakov wrote:

> On 2017-11-13 17:38, Victor Porton wrote:
>> I need a (preferably portable, but only Linux would be well) way to
>> execute an OS command with some input in stdin which I provide and
>> capture the output from stdout.
> 
> It is a vague question because there are numerous methods of how the
> input, output and error pipes are handled. E.g. by 0, 1, 2 tasks.
> 
> Anyway see:
> 
> 1. GNAT's System.OS_Lib.Spawn, System.OS_Lib.Non_Blocking_Spawn.

Where are docs for this? I sought Google but not found.

> 2. A more comfortable method would be to use GTK bindings (GLib actually):
> 
>     http://www.dmitry-kazakov.de/ada/gtkada_contributions.htm#10

I don't want my software to depend on GTK as it is Web based.

Is it possible to extract this code and use it separately?

-- 
Victor Porton - http://portonvictor.org

Re: Execute an OS command and capture output

[Victor Porton]

Dmitry A. Kazakov wrote:

> On 2017-11-13 17:38, Victor Porton wrote:
>> I need a (preferably portable, but only Linux would be well) way to
>> execute an OS command with some input in stdin which I provide and
>> capture the output from stdout.
> 
> It is a vague question because there are numerous methods of how the
> input, output and error pipes are handled. E.g. by 0, 1, 2 tasks.

I need synchronous processing of input/output pipes. That is I need just 
provide a String variable for the input and receive a String value with the 
output after the command ends.

However, I will most probably need also to terminate the spawned command 
after a timeout (not allow it to run more than 20 secs, for example). But 
this "timeout" functionality may be added later.

> Anyway see:
> 
> 1. GNAT's System.OS_Lib.Spawn, System.OS_Lib.Non_Blocking_Spawn.
> 
> 2. A more comfortable method would be to use GTK bindings (GLib actually):
> 
>     http://www.dmitry-kazakov.de/ada/gtkada_contributions.htm#10
> 
-- 
Victor Porton - http://portonvictor.org

Re: Execute an OS command and capture output

[Victor Porton]

Victor Porton wrote:

> Dmitry A. Kazakov wrote:
> 
>> On 2017-11-13 17:38, Victor Porton wrote:
>>> I need a (preferably portable, but only Linux would be well) way to
>>> execute an OS command with some input in stdin which I provide and
>>> capture the output from stdout.
>> 
>> It is a vague question because there are numerous methods of how the
>> input, output and error pipes are handled. E.g. by 0, 1, 2 tasks.
> 
> I need synchronous processing of input/output pipes. That is I need just
> provide a String variable for the input and receive a String value with
> the output after the command ends.

Hm, in a future version I may want to spawn asynchronously to reduce the 
number of tasks.

> However, I will most probably need also to terminate the spawned command
> after a timeout (not allow it to run more than 20 secs, for example). But
> this "timeout" functionality may be added later.
> 
>> Anyway see:
>> 
>> 1. GNAT's System.OS_Lib.Spawn, System.OS_Lib.Non_Blocking_Spawn.
>> 
>> 2. A more comfortable method would be to use GTK bindings (GLib
>> actually):
>> 
>>     http://www.dmitry-kazakov.de/ada/gtkada_contributions.htm#10
>> 
-- 
Victor Porton - http://portonvictor.org

Re: Execute an OS command and capture output

[Dmitry A. Kazakov]

On 2017-11-13 19:52, Victor Porton wrote:
> Dmitry A. Kazakov wrote:
> 
>> On 2017-11-13 17:38, Victor Porton wrote:
>>> I need a (preferably portable, but only Linux would be well) way to
>>> execute an OS command with some input in stdin which I provide and
>>> capture the output from stdout.
>>
>> It is a vague question because there are numerous methods of how the
>> input, output and error pipes are handled. E.g. by 0, 1, 2 tasks.
>>
>> Anyway see:
>>
>> 1. GNAT's System.OS_Lib.Spawn, System.OS_Lib.Non_Blocking_Spawn.
> 
> Where are docs for this? I sought Google but not found.

Only package specification AFAIK. But it is quite easy to understand 
because Spawn and Non_Blocking_Spawn are very low-level.

>> 2. A more comfortable method would be to use GTK bindings (GLib actually):
>>
>>      http://www.dmitry-kazakov.de/ada/gtkada_contributions.htm#10
> 
> I don't want my software to depend on GTK as it is Web based.

GTK is not Web-based. It is written in C.

> Is it possible to extract this code and use it separately?

Sources of GLib are freely available.

But it would be too complicated, especially under Windows were the GLib 
implementation uses a helper process.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: Execute an OS command and capture output

[Victor Porton]

Dmitry A. Kazakov wrote:

> On 2017-11-13 19:52, Victor Porton wrote:
>> Dmitry A. Kazakov wrote:
>> 
>>> On 2017-11-13 17:38, Victor Porton wrote:
>>>> I need a (preferably portable, but only Linux would be well) way to
>>>> execute an OS command with some input in stdin which I provide and
>>>> capture the output from stdout.
>>>
>>> It is a vague question because there are numerous methods of how the
>>> input, output and error pipes are handled. E.g. by 0, 1, 2 tasks.
>>>
>>> Anyway see:
>>>
>>> 1. GNAT's System.OS_Lib.Spawn, System.OS_Lib.Non_Blocking_Spawn.
>> 
>> Where are docs for this? I sought Google but not found.
> 
> Only package specification AFAIK. But it is quite easy to understand
> because Spawn and Non_Blocking_Spawn are very low-level.
> 
>>> 2. A more comfortable method would be to use GTK bindings (GLib
>>> actually):
>>>
>>>      http://www.dmitry-kazakov.de/ada/gtkada_contributions.htm#10
>> 
>> I don't want my software to depend on GTK as it is Web based.
> 
> GTK is not Web-based. It is written in C.

I meant that my software (not GTK) is Web based.

>> Is it possible to extract this code and use it separately?
> 
> Sources of GLib are freely available.
> 
> But it would be too complicated, especially under Windows were the GLib
> implementation uses a helper process.

It seems we need a helper process on Linux/Unix too:

From https://codelabs.ch/spawn-manager/index.html

The Ada spawn manager implements the concept described in the GNAT GPL 
package System.OS_Lib. It is used to solve the problem of spawning processes 
at arbitrary times from multitasking Ada programs:

If the parent is using tasking, and needs to spawn subprocesses at arbitrary 
times, one technique is for the parent to spawn (very early) a particular 
spawn-manager subprocess whose job is to spawn other processes. The spawn-
manager avoids tasking. The parent sends messages to the spawn-manager 
requesting it to spawn processes, using whatever inter-process communication 
mechanism you like, such as sockets.

SYSTEM.OS_LIB specification
— GNAT COMPILER COMPONENTS

So I am going to write patches to Ada Spawn Manager instead of using an 
existing solution, because my software is going to be multi-tasking.

In your opinion, is it really necessary? or can I use an existing solution?

-- 
Victor Porton - http://portonvictor.org

Re: Execute an OS command and capture output

[Dmitry A. Kazakov]

On 2017-11-13 20:07, Victor Porton wrote:
> Dmitry A. Kazakov wrote:
> 
>> On 2017-11-13 17:38, Victor Porton wrote:
>>> I need a (preferably portable, but only Linux would be well) way to
>>> execute an OS command with some input in stdin which I provide and
>>> capture the output from stdout.
>>
>> It is a vague question because there are numerous methods of how the
>> input, output and error pipes are handled. E.g. by 0, 1, 2 tasks.
> 
> I need synchronous processing of input/output pipes. That is I need just
> provide a String variable for the input and receive a String value with the
> output after the command ends.

You cannot receive string as you don't known its length.

It is tricky. The GLib Ada binding based implementation uses a callback 
when a portion of the output pipe is read. From the callback you can 
write a string-backed stream, e.g.:

http://www.dmitry-kazakov.de/ada/strings_edit.htm#Strings_Edit.Streams

Upon completion the stream contents can be retrieved as a single String.

Though normally one would process output instead of accumulating it. 
E.g. one would pass chunks of the output straight to the parser, etc.

As I said, in real-life applications (sanely written) it is usually 
complicated.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: Execute an OS command and capture output

[Dmitry A. Kazakov]

On 2017-11-13 21:16, Victor Porton wrote:
> Dmitry A. Kazakov wrote:
> 
>> GTK is not Web-based. It is written in C.
> 
> I meant that my software (not GTK) is Web based.

Well, Web-based as browser side or web-based as server side. On the 
server side you can do anything on the browser side basically nothing.

>>> Is it possible to extract this code and use it separately?
>>
>> Sources of GLib are freely available.
>>
>> But it would be too complicated, especially under Windows were the GLib
>> implementation uses a helper process.
> 
> It seems we need a helper process on Linux/Unix too:
> 
>  From https://codelabs.ch/spawn-manager/index.html
> 
> The Ada spawn manager implements the concept described in the GNAT GPL
> package System.OS_Lib. It is used to solve the problem of spawning processes
> at arbitrary times from multitasking Ada programs:

I have no idea why spawning a process should be a problem from a task.

> In your opinion, is it really necessary?

No idea. It is not necessary under with Win32 API. As for Linux I don't 
know if Posix calls must be done from a dedicated task. I somewhat doubt 
that.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: Execute an OS command and capture output

[Victor Porton]

Dmitry A. Kazakov wrote:

> On 2017-11-13 21:16, Victor Porton wrote:
>> Dmitry A. Kazakov wrote:
>> 
>>> GTK is not Web-based. It is written in C.
>> 
>> I meant that my software (not GTK) is Web based.
> 
> Well, Web-based as browser side or web-based as server side. On the
> server side you can do anything on the browser side basically nothing.

I mean server-based.

If it were browser-based, I would write to a JavaScript group instead of Ada 
group.

>>>> Is it possible to extract this code and use it separately?
>>>
>>> Sources of GLib are freely available.
>>>
>>> But it would be too complicated, especially under Windows were the GLib
>>> implementation uses a helper process.
>> 
>> It seems we need a helper process on Linux/Unix too:
>> 
>>  From https://codelabs.ch/spawn-manager/index.html
>> 
>> The Ada spawn manager implements the concept described in the GNAT GPL
>> package System.OS_Lib. It is used to solve the problem of spawning
>> processes at arbitrary times from multitasking Ada programs:
> 
> I have no idea why spawning a process should be a problem from a task.
> 
>> In your opinion, is it really necessary?
> 
> No idea. It is not necessary under with Win32 API. As for Linux I don't
> know if Posix calls must be done from a dedicated task. I somewhat doubt
> that.

I suspect, you doubt in vain. It is got from a very reputable source 
(SYSTEM.OS_LIB specification from GNAT COMPILER COMPONENTS):

[[
If the parent is using tasking, and needs to spawn subprocesses at 
arbitrary 
times, one technique is for the parent to spawn (very early) a particular 
spawn-manager subprocess whose job is to spawn other processes. The spawn-
manager avoids tasking. The parent sends messages to the spawn-manager 
requesting it to spawn processes, using whatever inter-process 
communication 
mechanism you like, such as sockets.
]]

-- 
Victor Porton - http://portonvictor.org

Re: Execute an OS command and capture output

[briot.emmanuel]

> 1. GNAT's System.OS_Lib.Spawn, System.OS_Lib.Non_Blocking_Spawn.
> 
> 2. A more comfortable method would be to use GTK bindings (GLib actually):
> 
>     http://www.dmitry-kazakov.de/ada/gtkada_contributions.htm#10

In addition, GNAT.Expect is meant for exactly this purpose. The list is 1 will
not let you input data interactively to a running process.

Re: Execute an OS command and capture output

[Dmitry A. Kazakov]

On 13/11/2017 22:09, Victor Porton wrote:
> Dmitry A. Kazakov wrote:
> 
>> On 2017-11-13 21:16, Victor Porton wrote:
>>> Dmitry A. Kazakov wrote:
>>>
>>>> GTK is not Web-based. It is written in C.
>>>
>>> I meant that my software (not GTK) is Web based.
>>
>> Well, Web-based as browser side or web-based as server side. On the
>> server side you can do anything on the browser side basically nothing.
> 
> I mean server-based.

Therefore using GLib is no problem.

> I suspect, you doubt in vain. It is got from a very reputable source
> (SYSTEM.OS_LIB specification from GNAT COMPILER COMPONENTS):
> 
> [[
> If the parent is using tasking, and needs to spawn subprocesses at
> arbitrary
> times, one technique is for the parent to spawn (very early) a particular
> spawn-manager subprocess whose job is to spawn other processes. The spawn-
> manager avoids tasking. The parent sends messages to the spawn-manager
> requesting it to spawn processes, using whatever inter-process
> communication
> mechanism you like, such as sockets.
> ]]

This is no explanation why such process is needed. There is no problem 
statement and no description of its solution.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: Execute an OS command and capture output

[A. Cervetti]

Il giorno lunedì 13 novembre 2017 20:08:00 UTC+1, Victor Porton ha scritto:

> I need synchronous processing of input/output pipes. That is I need just 
> provide a String variable for the input and receive a String value with the 
> output after the command ends.

See the example http://rosettacode.org/wiki/Get_system_command_output#Ada

It uses GNAT.Expect.Get_Command_Output that does exactly what you need.

You have just to define a String for the Input parameter to the Get_Command_Output function.

A.

Re: Execute an OS command and capture output

[Randy Brukardt]

"Victor Porton" <porton@narod.ru> wrote in message 
news:ouchqn$1vo1$1@gioia.aioe.org...
>I need a (preferably portable, but only Linux would be well) way to execute
> an OS command with some input in stdin which I provide and capture the
> output from stdout.

There is no portable solution, as Ada does not provide any sort of OS 
access. You'll have to use something either compiler-specific (like the GNAT 
Spawn mentioned elsewhere or the Janus/Ada Prog_Call), or perhaps something 
in a library (most of which only work with GNAT anyway).

We (the ARG) once tried to define a portable OS access. We discussed it for 
the better part of an hour and ended up arguing about a portable problem 
statement -- we couldn't even agree on how to portably describe the problem 
we were trying to solve. We eventually decided that we could make better use 
of our time working on something else (pretty much anything else!). So it's 
unlikely that Ada will ever get such a facility; it is very target-specific. 
It would be nice if all Windows compilers supported the same mechanism, but 
that would require herding cats (implementers) and probably would be a long 
shot. [Implementers don't even seem to support file information portably, 
even though the specification of Ada.Directories.Information for Windows and 
for Linux is provided in the AARM. For instance, my understanding is that 
GNAT doesn't implement either of these packages as specified there.]

                                     Randy.