Buffered I/O with GNAT on Linux

Buffered I/O with GNAT on Linux

[Markus Kuhn]

The GNAT 3.10p reference manual claims that the Ada.Text_IO is
performed over the stream buffering provided by the C library.
However if I execute under Linux (RH 4.1 and 5.0) the test program

-----------------------------------------------
with Ada.Text_IO; use Ada.Text_IO;

procedure Strange is
begin
   for I in 1 .. 10 loop
      Put("Hello ");
   end loop;
   New_Line;
end Strange;
-----------------------------------------------

and start it with strace to log the system calls, then I get

$ strace ./strange >/dev/null
[...]
write(1, "Hello ", 6)                   = 6
write(1, "Hello ", 6)                   = 6
write(1, "Hello ", 6)                   = 6
write(1, "Hello ", 6)                   = 6
write(1, "Hello ", 6)                   = 6
write(1, "Hello ", 6)                   = 6
write(1, "Hello ", 6)                   = 6
write(1, "Hello ", 6)                   = 6
write(1, "Hello ", 6)                   = 6
write(1, "Hello ", 6)                   = 6
write(1, "\n", 1)                       = 1
_exit(0)                                = ?

which clearly shows that there is no buffering going on and
every single Ada.Text_IO call seems to result in a write system
call.

C is much more efficient here:

$ cat notstrange.c
#include <stdio.h>

int main()
{
  int i;

  for (i = 0; i < 10; i++)
    printf("Hello ");
  printf("/n");
  return 0;
}

$ strace ./notstrange >/dev/null
[...]
write(1, "Hello Hello Hello Hello Hello He"..., 62) = 62
munmap(0x400ac000, 4096)                = 0
_exit(0)                                = ?


The lack of buffering is especially troubelsome for Linux users,
because Linux 2.0 does not have any NFS buffering in the kernel.
Every single write() results immediately in a remote procedure call
to the NFS server. If the NFS server does in addition synchronous
writes (like the DEC one I have to use here does), then a single Put
instruction under Ada requires several hundred milliseconds to
execute. This causes programs such as gnatbind and gnatlink to
execute over 50 times slower if the accessed files are Linux NFS
mounted. Compiling the above test program with gnatmake takes
around 55 seconds in my NFS mounted home directory, but only
around one second in /tmp, which is a local harddisk that enjoys
kernel block buffering.

Is there any trick to get GNAT 3.10p to really use the libc stream
buffering, especially for gnatlink and gnatbind?

Markus

-- 
Markus G. Kuhn, Security Group, Computer Lab, Cambridge University, UK
email: mkuhn at acm.org,  home page: <http://www.cl.cam.ac.uk/~mgk25/>

Re: Buffered I/O with GNAT on Linux

[Robert Dewar]

Markus comments on the buffering in GNAT. This is totally under control
of the user, as documented in the GNAT manual. THe default for a non-regular
file is buffering off, which is what most people want as a default!

If you want buffering for stdout, turn it on!

Re: Buffered I/O with GNAT on Linux

[Markus Kuhn]

Robert Dewar wrote:
> Markus comments on the buffering in GNAT. This is totally under control
> of the user, as documented in the GNAT manual. THe default for a non-regular
> file is buffering off, which is what most people want as a default!
> 
> If you want buffering for stdout, turn it on!

I tried several things along the lines of

----------------------------------------------------------------------
with Ada.Text_IO;           use Ada.Text_IO;
with System;                use System;
with Interfaces.C_Streams;  use Interfaces.C_Streams;
with Ada.Text_IO.C_Streams; use Ada.Text_IO.C_Streams;

procedure Strange is
begin
   if setvbuf(C_Stream(Current_Output), Null_Address, IOFBF, 8092) /= 0 then
      Put_Line("setvbuf failed");
   end if;
   for X in 1..10 loop
      Put("Hello ");
   end loop;
   New_Line;
end Strange;
----------------------------------------------------------------------

but with no success. I still can't get buffered output.

In addition, if normal files are really buffered, then why are
gnatbind and gnatlink so incredibly slow (factor 50) on Linux
NFS mounted filesystems? It really looks like these two programs
use very inefficient buffered I/O. For instance, look at the system
calls when gnatbind is writing the C main program (recorded by
strace):

...
open("b_strange.c", O_WRONLY|O_CREAT|O_TRUNC, 0666) = 3
write(3, "extern int gnat_argc;", 21)   = 21
write(3, "\n", 1)                       = 1
write(3, "extern char **gnat_argv;", 24) = 24
write(3, "\n", 1)                       = 1
write(3, "extern char **gnat_envp;", 24) = 24
write(3, "\n", 1)                       = 1
write(3, "extern int gnat_exit_status;", 28) = 28
write(3, "\n", 1)                       = 1
write(3, "void adafinal ();", 17)       = 17
...

And each of these write() system calls triggers a full blown
remote procedure call under Linux with NFS. Same with gnatlink.

Markus

-- 
Markus G. Kuhn, Security Group, Computer Lab, Cambridge University, UK
email: mkuhn at acm.org,  home page: <http://www.cl.cam.ac.uk/~mgk25/>

Re: Buffered I/O with GNAT on Linux

[Tarjei T. Jensen]

Markus Kuhn wrote:
> 
> I tried several things along the lines of
> 
> ----------------------------------------------------------------------
> with Ada.Text_IO;           use Ada.Text_IO;
> with System;                use System;
> with Interfaces.C_Streams;  use Interfaces.C_Streams;
> with Ada.Text_IO.C_Streams; use Ada.Text_IO.C_Streams;
> 
> procedure Strange is
> begin
>    if setvbuf(C_Stream(Current_Output), Null_Address, IOFBF, 8092) /= 0 then
>       Put_Line("setvbuf failed");
>    end if;
>    for X in 1..10 loop
>       Put("Hello ");
>    end loop;
>    New_Line;
> end Strange;
> ----------------------------------------------------------------------
> 
> but with no success. I still can't get buffered output.

I belive there is supposed to be a Ada level procedure that turns on
buffering. Whether it works or not I cannot tell you.

Greetings,

-- 
// Tarjei T. Jensen 
//    tarjei@online.no || voice +47 51 62 85 58
//   Support you local rescue centre: GET LOST!

Re: Buffered I/O with GNAT on Linux

[Tarjei T. Jensen]

Markus Kuhn wrote:
> 
> I tried several things along the lines of
> 
> ----------------------------------------------------------------------
> with Ada.Text_IO;           use Ada.Text_IO;
> with System;                use System;
> with Interfaces.C_Streams;  use Interfaces.C_Streams;
> with Ada.Text_IO.C_Streams; use Ada.Text_IO.C_Streams;
> 
> procedure Strange is
> begin
>    if setvbuf(C_Stream(Current_Output), Null_Address, IOFBF, 8092) /= 0 then
>       Put_Line("setvbuf failed");
>    end if;
>    for X in 1..10 loop
>       Put("Hello ");
>    end loop;
>    New_Line;
> end Strange;
> ----------------------------------------------------------------------
> 
> but with no success. I still can't get buffered output.

I belive there is supposed to be a Ada level procedure that turns on
buffering. Whether it works or not I cannot tell you.

Greetings,

-- 
// Tarjei T. Jensen 
//    tarjei@online.no || voice +47 51 62 85 58
//   Support you local rescue centre: GET LOST!

Re: Buffered I/O with GNAT on Linux

[ak]

dewar@merv.cs.nyu.edu (Robert Dewar) writes:

> Markus comments on the buffering in GNAT. This is totally under control
> of the user, as documented in the GNAT manual. THe default for a non-regular
> file is buffering off, which is what most people want as a default!

This looks like a lousy default. How about doing a isatty(1)[1] first like
C stdio does?  If it is a terminal turn buffering off (or turn it into
line buffering), when it is no terminal turn it on. 

This would make writing unix-style filters in GNAT more efficient.

-Andi

[1] Actually glibc 2.0 under Linux does fstat(1, ..) and checks st_rdev,
but that should be equivalent.