Re: File output and buffering

["Steve" <nospam_steved94@comcast.net>]

"Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> wrote in message 
news:yiw2f938342v.xzb47swyx5h4$.dlg@40tude.net...
> On Fri, 22 Aug 2008 14:41:18 -0700 (PDT), Maciej Sobczak wrote:
>
>> On 22 Sie, 15:22, "Dmitry A. Kazakov" <mail...@dmitry-kazakov.de>
>> wrote:
>>
>>> One last try. In your example, when
>>> characters of a message are sent *synchronously* (assuming E-mail as the
>>> transport layer, no back doors, etc) then each single character has to 
>>> be
>>> sent as a reply to the answer to the earlier mail.
>>
>> Then we have a different notion of "synchronously".
>> When I write something to the file, the operation is synchronous when
>> the program *waits* for the transfer to complete.
>
> The transfer of the group, not the transfers of the individual items of.
>

Dmitry,

I have read enough of your posts on this newsgroup to know you're not a 
troll, but is sure hard to tell from reading this thread.

In my experience (theory aside) sending one character a time to an OS is 
considerably slower than buffering the data and sending blocks of data.

Several years ago I rewrote a driver on one of our system that we used to 
communicate serially (using RS232) with a PLC (Programmable Logic 
Controller).  The driver was originally written to make separate calls to 
the OS for each character sent to the PLC.  The original implementation 
utilized approximately 15% of the CPU.  When I re-wrote the driver to buffer 
the characters into blocks of up to 128 characters (defined by the PLC 
protocol) and make one OS call for the buffered data, the CPU utilization 
dropt to less than 1% of the CPU.

This behavior makes perfect sense to me because for each call to the OS a 
buffer is allocated containing the data to be transmitted and placed in a 
queue for the OS.  The buffer itself contains more than just the data to be 
sent, it includes some overhead, sometimes significant in size.  The 
addition of the buffer to the OS queue often includes considerable overhead, 
context switches, mutexes, etc.  When the number of characters in the buffer 
is increased the overhead is not significantly increased.

Sure, if you're talking directly to hardware hardware that only handles one 
character at a time then buffering and unbuffering data adds overhead.  But 
it is rare in these days to talk directly with the hardware.  Even the 
simpler systems often use a kernel or OS that makes buffering worthwhile.

If you're using TCP/IP to send data, if you're going to send a bunch of data 
at a time it would be silly to send one byte at a time.  IP has considerable 
overhead for each block.  You should try to include as much data as possible 
to minimize the number of packets sent and minimize the overhead.

I find it interesting that you seem to be arguing that it is always better 
to not bufffer, when the original poster has indicated that he has tried 
both buffered and unbuffered approaches and observed thant unbuffered was 
considerably slower on his system.  Either you are miscommunicating or you 
are just plain wrong.

Regards,
Steve

> -- 
> Regards,
> Dmitry A. Kazakov
> http://www.dmitry-kazakov.de