Palatable Windows IO using Ada

Palatable Windows IO using Ada

[Le]

I are looking for Ada package(s) which make Windows serial IO more
palatable for varying length asynchronous inputs.

My team is laying a large legacy Ada application on top of Windows XP.
There is a fair amount of serial IO which follows a VMS flavor -
pending reads while writes stimulate responses back to the reading
tasks & sets of terminator characters to delimit messages. Wrappers we
have seen for the Win32 API simply export the ReadFile and ReadFileEx
which appear to be blocking by their very nature and demand that the
reader have apriori knowledge of the length of input expected at any
particular instance.

Of course one can deal with the varying length by doing all reads one
character at a time, but this is quite waseful in the face of Windows
already daunting CPU requirements.

So, if you know of a somewhat thicker wrapper for Windows external IO
that makes it a little more (may I say) (x)nix like (or some other
alternative) I'd really appreciate a pointer or copy.

Re: Palatable Windows IO using Ada

[Dmitry A. Kazakov]

On 6 Apr 2006 17:21:44 -0700, Le wrote:

> Wrappers we
> have seen for the Win32 API simply export the ReadFile and ReadFileEx
> which appear to be blocking by their very nature and demand that the
> reader have apriori knowledge of the length of input expected at any
> particular instance.

which is true for all kinds of any I/O in any OS. You should know the size
of the byte, block, record, line before you start to read it.

Windows ReadFile isn't always blocking, it can be asynchronous, see
"overlapping" I/O in MSDN description of ReadFile.
 
> Of course one can deal with the varying length by doing all reads one
> character at a time, but this is quite waseful in the face of Windows
> already daunting CPU requirements.

Windows does buffering in background, so it isn't that wasteful. The
knowledge of how many items can be read at once depends solely on the
application.

As for serial I/O, there are further settings to keep in mind. See Windows
API SetCommTimeouts.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: Palatable Windows IO using Ada

[Stephen Leake]

"Le" <snelson1@llnl.gov> writes:

> I are looking for Ada package(s) which make Windows serial IO more
> palatable for varying length asynchronous inputs.

http://www.toadmail.com/~ada_wizard/ada/win32_com_ports.zip

provides a simple low-level binding to read and write serial ports,
including asynchronous. I haven't tried it on Windows XP.

-- 
-- Stephe

Re: Palatable Windows IO using Ada

[Le]

Your attempt at a tutorial is in error in a couple of places. Stating
that an application "should know" what will be returned is
presumptuous. Note this is a legacy application, so obviously there are
some devices out there which give varying responses. These are raw byte
sequences - no blocks or lines. Those esternal device ICDs are not
typically in the control of a system designer.

The ReadFile interface *is* blocking.  Overlap allows an app to come
back and see if it is done.  If not, assuming timeout wasn't used, one
must then cancel the IO or some other cleanup.

With respect to buffering, my point was that doing byte at a time IO to
get around the blocking nature of the API defeats the efficiencies
presumably provided by the driver's internal buffering.

Re: Palatable Windows IO using Ada

[Dmitry A. Kazakov]

On 7 Apr 2006 09:30:23 -0700, Le wrote:

> Your attempt at a tutorial is in error in a couple of places. Stating
> that an application "should know" what will be returned is
> presumptuous. Note this is a legacy application, so obviously there are
> some devices out there which give varying responses. These are raw byte
> sequences - no blocks or lines.

If you mean a byte stream, then it is *byte* you (or application) should
know of. You cannot read any device not knowing the size of input. You
might have a higher level communication protocol which build something out
of these bytes, for example, "a valid Ada program encoded in ASCII." But
how an OS driver might know about it?

> The ReadFile interface *is* blocking.  Overlap allows an app to come
> back and see if it is done.

Then it is rather a strange definition of "blocking." In my world blocking
operation means that the execution thread gets blocked until its completes.
The thread cannot go anywhere it that state.

> With respect to buffering, my point was that doing byte at a time IO to
> get around the blocking nature of the API defeats the efficiencies
> presumably provided by the driver's internal buffering.

How so? Byte-wise reading just does not do any extra and unnecessary
buffering, because OS already did it. One could argue that OS API are
expensive to call. This a different issue. I don"t have any figures
concerning Windows, but in my experience that was never a problem. At least
when reading from sockets and serial devices.

As for overlapping ReadFile, your argument does not apply either, because
it gets *all available* bytes the buffer provided in its argument can take.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: Palatable Windows IO using Ada

[Le]

>You cannot read any device not knowing the size of input.

You referred to blocks & lines previously indicating that you
understood what was being referred to. I was speaking of byte sequences
which vary in *length*.

>> The ReadFile interface *is* blocking.  Overlap allows an app to come
>> back and see if it is done.

>Then it is rather a strange definition of "blocking." In my world blocking
>operation means that the execution thread gets blocked until its completes.
>The thread cannot go anywhere it that state.

Same universe that I am in... ReadFile blocks or times out.

In the future if you don't have an answer to the heart of the question
please resist the urge to wax pedagogical.

Re: Palatable Windows IO using Ada

[Dmitry A. Kazakov]

On 7 Apr 2006 12:21:11 -0700, Le wrote:

>>You cannot read any device not knowing the size of input.
> 
> You referred to blocks & lines previously indicating that you
> understood what was being referred to.

No, I didn't. Blocks were mentioned having block-oriented devices in mind.
Lines were for record-oriented I/O existed, for example, in RSX-11.

> I was speaking of byte sequences which vary in *length*.

which length is determined by what?

> ... ReadFile blocks or times out.

This is plain wrong. Here is the citation from MSDN "Synchronization and
Overlapped Input and Output"

"The WriteFile, ReadFile, DeviceIoControl, WaitCommEvent, ConnectNamedPipe,
and TransactNamedPipe functions can be performed either synchronously or
asynchronously."

and further:

"... Functions called for overlapped operation can return immediately, even
though the operation has not been completed. This enables a time-consuming
I/O operation to be executed in the background while the calling thread is
free to perform other tasks. For example, a single thread can perform
simultaneous I/O operations on different handles, or even simultaneous read
and write operations on the same handle."

From MSDN "ReadFile"

"... If hFile is opened with FILE_FLAG_OVERLAPPED and lpOverlapped is not
NULL, the read operation starts at the offset that is specified in the
OVERLAPPED structure, and ReadFile may return before the read operation is
complete. In this scenario, ReadFile returns FALSE and the GetLastError
function returns ERROR_IO_PENDING, which allows the calling process to
continue while the read operation completes."

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: Palatable Windows IO using Ada

[John]

Le wrote:
> I are looking for Ada package(s) which make Windows serial IO more
> palatable for varying length asynchronous inputs.

I am aware of a body of work called "Ada Terminal Emulator" that is
signed by Ross Higson.  rosshigson@optusnet.com.au

It's just part of my packrat's nest, so I can't easily say where I got
it, but it might be that some portions of its implementation will be
interesting.  References in its documentation might lead to something
as well.

> My team is laying a large legacy Ada application on top of Windows XP.
> There is a fair amount of serial IO which follows a VMS flavor -
> pending reads while writes stimulate responses back to the reading
> tasks & sets of terminator characters to delimit messages.

Twenty or so years ago I worked for a year while between assignments on
a product then called Silas.  I wonder if you're dealing with one his
grandsons.

John Woodruff
retired software engineer.

Re: Palatable Windows IO using Ada

[Steve]

"Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> wrote in message 
news:15qxrbngb2kll.wkigwit0g1tn.dlg@40tude.net...
> On 6 Apr 2006 17:21:44 -0700, Le wrote:
>
>> Wrappers we
>> have seen for the Win32 API simply export the ReadFile and ReadFileEx
>> which appear to be blocking by their very nature and demand that the
>> reader have apriori knowledge of the length of input expected at any
>> particular instance.
>
> which is true for all kinds of any I/O in any OS. You should know the size
> of the byte, block, record, line before you start to read it.
>

I disagree.

I have written a several high level drivers that communicate with serial 
devices.  In my experience, knowing the size of the byte, block, record or 
line before you start to read is the exception rather than the rule.

In my experience it is much more common to detect the end of input by the 
presence of special marker characters in the data or by the time between 
characters exceeding some threshold.

In some cases an upper limit on the number of characters that will be 
available to read is known, but not always.  In these cases multiple reads 
may be required.

> Windows ReadFile isn't always blocking, it can be asynchronous, see
> "overlapping" I/O in MSDN description of ReadFile.
>
I've been through the windows overlapped I/O for serial communications.  In 
my opinion it's a royal pain in the ass.  I think NT was the fourth OS I 
used to implement a bi-directional communications driver.  I was very 
suprised to learn that you had to do something special to have one task 
reading a serial port at the same time a second task writes to the port in 
order to make things work.  In the other systems I used, it just worked.

>> Of course one can deal with the varying length by doing all reads one
>> character at a time, but this is quite waseful in the face of Windows
>> already daunting CPU requirements.
>
> Windows does buffering in background, so it isn't that wasteful. The
> knowledge of how many items can be read at once depends solely on the
> application.
>

You're kidding... right?  Every transaction with a driver in NT involves a 
significant amount of overhead.  If you read one character at a time you're 
incurring the overhead for each character.  If you read a bunch of 
characters at once, there is just one transaction with the driver... less 
overhead.

If you have a nice library that buffers things for you, you don't have to 
worry about it.  But for serial I/O you'll have to build that on top of the 
call to ReadFile.

Steve
(The Duck)

BTW: I sent the original poster a copy of a hight level driver for XP that 
should meet their needs.

> As for serial I/O, there are further settings to keep in mind. See Windows
> API SetCommTimeouts.
>
> -- 
> Regards,
> Dmitry A. Kazakov
> http://www.dmitry-kazakov.de 

Re: Palatable Windows IO using Ada

[Dmitry A. Kazakov]

On Fri, 7 Apr 2006 19:07:35 -0700, Steve wrote:

> "Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> wrote in message 
> news:15qxrbngb2kll.wkigwit0g1tn.dlg@40tude.net...
>> On 6 Apr 2006 17:21:44 -0700, Le wrote:
>>
>>> Wrappers we
>>> have seen for the Win32 API simply export the ReadFile and ReadFileEx
>>> which appear to be blocking by their very nature and demand that the
>>> reader have apriori knowledge of the length of input expected at any
>>> particular instance.
>>
>> which is true for all kinds of any I/O in any OS. You should know the size
>> of the byte, block, record, line before you start to read it.
>>
> I have written a several high level drivers that communicate with serial 
> devices.  In my experience, knowing the size of the byte, block, record or 
> line before you start to read is the exception rather than the rule.
> 
> In my experience it is much more common to detect the end of input by the 
> presence of special marker characters in the data or by the time between 
> characters exceeding some threshold.

This is a higher level protocol. The point is that when the OS does not
natively support a device-specific application layer protocol, and it
normally does not, then there is no other place to do it as in the
application. So the communication item becomes one byte. Blocking for more
than one byte at once is looking for troubles. Even if the protocol has
things like bytes counts. Practically none of multiple serial devices we
dealt with worked according to the specification. Using bytes stream full
duplex communication model has saved us a lot of time, because we could
easily adapt our software and overcome bugs in the hardware.

>> Windows ReadFile isn't always blocking, it can be asynchronous, see
>> "overlapping" I/O in MSDN description of ReadFile.
>>
> I've been through the windows overlapped I/O for serial communications.  In 
> my opinion it's a royal pain in the ass.

Yes. In my opinion it is overkill to use overlapped I/O for serial
connections. Though if you have 1000 devices to handle simultaneously,
there might be no other choice. In RSX and VAX, where the idea of
asynchronous I/O seems was borrowed from, it worked well. I also think that
overlapped I/O and asynchronous system trap could nicely fit to Ada's
protected objects. The interface to overlapped I/O is so clumsy because API
is in C, it should be in Ada! (:-))

>  I think NT was the fourth OS I 
> used to implement a bi-directional communications driver.  I was very 
> suprised to learn that you had to do something special to have one task 
> reading a serial port at the same time a second task writes to the port in 
> order to make things work.  In the other systems I used, it just worked.

?

>>> Of course one can deal with the varying length by doing all reads one
>>> character at a time, but this is quite waseful in the face of Windows
>>> already daunting CPU requirements.
>>
>> Windows does buffering in background, so it isn't that wasteful. The
>> knowledge of how many items can be read at once depends solely on the
>> application.
> 
> You're kidding... right?  Every transaction with a driver in NT involves a 
> significant amount of overhead.  If you read one character at a time you're 
> incurring the overhead for each character.

I didn't saw any significant difference when using synchronous vs.
asynchronous read for serial devices. Taking into account serial connection
speed I doubt one could notice it. A more interesting case is socket I/O.
Our commercial software reads from sockets using 1 byte buffer and blocking
read. Recently we did elaborated time measurements under Windows XP and
Server 2003 for stream uni- and multicast sockets. It was all but easy,
because of quite flawed design of Windows time services. But that's a
different story. Anyway, our experiments have shown that the bottleneck
wasn't communication. We tested publisher / subscriber services of our
product and have noticed that the communication layer by margin outperforms
other (local) parts. Roughly, global mutexes and context switches are much
worse than I/O APIs, even if the former are used per data block and the
later per byte.

> If you have a nice library that buffers things for you, you don't have to 
> worry about it.  But for serial I/O you'll have to build that on top of the 
> call to ReadFile.

There is no other choice. The transport layer of serial connection is byte
stream. It is a medical fact. (:-))

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: Palatable Windows IO using Ada

[Steve]

"Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> wrote in message 
news:14sfx2j059hst$.1x6sfev1savpw.dlg@40tude.net...
> On Fri, 7 Apr 2006 19:07:35 -0700, Steve wrote:
>
[snip]
>
> This is a higher level protocol. The point is that when the OS does not
> natively support a device-specific application layer protocol, and it
> normally does not, then there is no other place to do it as in the
> application. So the communication item becomes one byte. Blocking for more
> than one byte at once is looking for troubles. Even if the protocol has
> things like bytes counts. Practically none of multiple serial devices we
> dealt with worked according to the specification. Using bytes stream full
> duplex communication model has saved us a lot of time, because we could
> easily adapt our software and overcome bugs in the hardware.
>

It is interesting the opinions that are formed based on one's experiences. 
Mine is at least in part formed because in the past I have had to rewrite 
drivers that handled one character at a time to process as many as possible 
at a time due to performance issues.

In one case, when communicating with an Allen Bradley PLC using DF1 
protocol, I found that the driver that handled one character at a time used 
up a significant part of the available CPU time.  Enough that the computer 
was observed to be very sluggish.

The DF1 protocol uses markers (DLE) to mark key locations in the data 
stream.  By using a "read until character" function instead of reading one 
character at a time, the CPU overhead was significantly reduced, and the 
system was no longer sluggish.

Most of the applications I have worked on make heavy use of the CPU and are 
time critical (but not safety critical).  Minimizing the processor overhead 
used by communication is often a goal.

>>> Windows ReadFile isn't always blocking, it can be asynchronous, see
>>> "overlapping" I/O in MSDN description of ReadFile.
>>>
>> I've been through the windows overlapped I/O for serial communications. 
>> In
>> my opinion it's a royal pain in the ass.
>
> Yes. In my opinion it is overkill to use overlapped I/O for serial
> connections. Though if you have 1000 devices to handle simultaneously,
> there might be no other choice. In RSX and VAX, where the idea of
> asynchronous I/O seems was borrowed from, it worked well. I also think 
> that
> overlapped I/O and asynchronous system trap could nicely fit to Ada's
> protected objects. The interface to overlapped I/O is so clumsy because 
> API
> is in C, it should be in Ada! (:-))

On NT/XP for full duplex communication, you really have two choices:
  Use non-blocking I/O (poll)
  Use overlapped I/O

The clumsy interface has no relation to the choice of language of the API. 
And it shouldn't.  It is based on the design of the driver.  In the good (or 
bad?) old days, the OS interfaces used to be described in a language neutral 
manner... as it should be.

When I had to make duplex serial communications work right on NT, I found an 
article descrbing how great it was that NT could do overlapped I/O, and how 
it was something new and wonderful.  I was left with the impression that the 
developer had only done the D part of R&D and thought this was a great new 
feature.  I found it to be a clumsy feature that was working around a poor 
driver implementation.  I had the same impression when Microsoft "invented" 
multi-threaded programming... after I had already been using multi-tasking 
systems for years.

[snip]
>>> Windows does buffering in background, so it isn't that wasteful. The
>>> knowledge of how many items can be read at once depends solely on the
>>> application.
>>
>> You're kidding... right?  Every transaction with a driver in NT involves 
>> a
>> significant amount of overhead.  If you read one character at a time 
>> you're
>> incurring the overhead for each character.
>
> I didn't saw any significant difference when using synchronous vs.
> asynchronous read for serial devices. Taking into account serial 
> connection
> speed I doubt one could notice it. A more interesting case is socket I/O.
> Our commercial software reads from sockets using 1 byte buffer and 
> blocking
> read. Recently we did elaborated time measurements under Windows XP and
> Server 2003 for stream uni- and multicast sockets. It was all but easy,
> because of quite flawed design of Windows time services. But that's a
> different story. Anyway, our experiments have shown that the bottleneck
> wasn't communication. We tested publisher / subscriber services of our
> product and have noticed that the communication layer by margin 
> outperforms
> other (local) parts. Roughly, global mutexes and context switches are much
> worse than I/O APIs, even if the former are used per data block and the
> later per byte.
>

Since you're using Windows XP and Server 2003, you're probably using recent 
hardware, which is fast enough that you may not see an overall time 
difference.  If you use a high resolution performance counter to do you're 
timing you are much more likely to find a difference.

I started working with networking on NT on Pentium 166 machines (or 
slower... it's been long enough that I can't remember).  On those machines 
the time difference was more significant.

Steve
(The Duck)

>> If you have a nice library that buffers things for you, you don't have to
>> worry about it.  But for serial I/O you'll have to build that on top of 
>> the
>> call to ReadFile.
>
> There is no other choice. The transport layer of serial connection is byte
> stream. It is a medical fact. (:-))
>
> -- 
> Regards,
> Dmitry A. Kazakov
> http://www.dmitry-kazakov.de 

Re: Palatable Windows IO using Ada

[Dmitry A. Kazakov]

On Sun, 9 Apr 2006 06:50:23 -0700, Steve wrote:

> It is interesting the opinions that are formed based on one's experiences. 
> Mine is at least in part formed because in the past I have had to rewrite 
> drivers that handled one character at a time to process as many as possible 
> at a time due to performance issues.
>
> In one case, when communicating with an Allen Bradley PLC using DF1 
> protocol, I found that the driver that handled one character at a time used 
> up a significant part of the available CPU time.  Enough that the computer 
> was observed to be very sluggish.
> 
> The DF1 protocol uses markers (DLE) to mark key locations in the data 
> stream.  By using a "read until character" function instead of reading one 
> character at a time, the CPU overhead was significantly reduced, and the 
> system was no longer sluggish.

How would you tell that to the driver? Do you mean a true driver here or
merely a program that uses an OS serial driver? I meant the later. Talking
about drivers, you are absolutely right. Interrupts are very expensive on
wintel platform. I had to redesign one driver (an ISA transputer link
driver) for a similar reason. The card produced an interrupt per byte at up
to 1Mbaud...

> Most of the applications I have worked on make heavy use of the CPU and are 
> time critical (but not safety critical).  Minimizing the processor overhead 
> used by communication is often a goal.
> 
>>>> Windows ReadFile isn't always blocking, it can be asynchronous, see
>>>> "overlapping" I/O in MSDN description of ReadFile.
>>>>
>>> I've been through the windows overlapped I/O for serial communications. 
>>> In
>>> my opinion it's a royal pain in the ass.
>>
>> Yes. In my opinion it is overkill to use overlapped I/O for serial
>> connections. Though if you have 1000 devices to handle simultaneously,
>> there might be no other choice. In RSX and VAX, where the idea of
>> asynchronous I/O seems was borrowed from, it worked well. I also think 
>> that
>> overlapped I/O and asynchronous system trap could nicely fit to Ada's
>> protected objects. The interface to overlapped I/O is so clumsy because 
>> API
>> is in C, it should be in Ada! (:-))
> 
> On NT/XP for full duplex communication, you really have two choices:
>   Use non-blocking I/O (poll)
>   Use overlapped I/O

I prefer two threads, one does blocking read (incoming bytes stream),
another blocking write (outgoing packets), it works well. Overlapped I/O is
too clumsy, then the read buffer cannot be accessed anyway when the
operation is pending.

> The clumsy interface has no relation to the choice of language of the API. 
> And it shouldn't.  It is based on the design of the driver.  In the good (or 
> bad?) old days, the OS interfaces used to be described in a language neutral 
> manner... as it should be.

How can you do it language neutral? The problem is that asynchronous I/O
inherently requires concurrency. If the language does not support
concurrency, you simply cannot comprehensively describe what's going on.

> When I had to make duplex serial communications work right on NT, I found an 
> article descrbing how great it was that NT could do overlapped I/O, and how 
> it was something new and wonderful.  I was left with the impression that the 
> developer had only done the D part of R&D and thought this was a great new 
> feature.  I found it to be a clumsy feature that was working around a poor 
> driver implementation.  I had the same impression when Microsoft "invented" 
> multi-threaded programming... after I had already been using multi-tasking 
> systems for years.

I started on RSX-11 and VAX-11, so to me it was a shock to see for the
first time how many processes were running on an idle Unix machine. Slowly
I adapted to the idea that each mouse button needs a separate process to
deal with (:-)) Windows is somewhere between. It is unfair to blame alone
Microsoft for all mess. It was Unix first, which destroyed the market of
operating systems. We have what we have deserved.

>> I didn't saw any significant difference when using synchronous vs.
>> asynchronous read for serial devices. Taking into account serial 
>> connection
>> speed I doubt one could notice it. A more interesting case is socket I/O.
>> Our commercial software reads from sockets using 1 byte buffer and 
>> blocking
>> read. Recently we did elaborated time measurements under Windows XP and
>> Server 2003 for stream uni- and multicast sockets. It was all but easy,
>> because of quite flawed design of Windows time services. But that's a
>> different story. Anyway, our experiments have shown that the bottleneck
>> wasn't communication. We tested publisher / subscriber services of our
>> product and have noticed that the communication layer by margin 
>> outperforms
>> other (local) parts. Roughly, global mutexes and context switches are much
>> worse than I/O APIs, even if the former are used per data block and the
>> later per byte.
> 
> Since you're using Windows XP and Server 2003, you're probably using recent 
> hardware, which is fast enough that you may not see an overall time 
> difference.  If you use a high resolution performance counter to do you're 
> timing you are much more likely to find a difference.
> 
> I started working with networking on NT on Pentium 166 machines (or 
> slower...

Once I managed to install Windows NT 4.0 on a 486DX 66MHz / 12Mb. It took
about one day. The system booted in 10 minutes or so. I still remember VMS,
which served 6 interactive users having 4Mb memory. It had DEC Ada too!
(:-)) 

> it's been long enough that I can't remember).  On those machines 
> the time difference was more significant.

Actually it worked well on a Pentium 133 under Windows NT 4.0. COM port
isn't fast enough to break it down. In fact, in one case we had to slow
down the PC side, because the device crashed when PC had written too fast.

I'm too lazy to look into Kernel32.dll, but I guess, that an ReadFile call
isn't that expensive. I suppose that there is an OS-allocated buffer in the
process space, so ReadFile does nothing extraordinary. Because the input is
buffered, it makes little sense to do it again. 

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: Palatable Windows IO using Ada

[Steve]

"Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> wrote in message 
news:o2i2brhcp49x$.1lmgnwrqgqzf7$.dlg@40tude.net...
> On Sun, 9 Apr 2006 06:50:23 -0700, Steve wrote:
[snip]
>>
>> In one case, when communicating with an Allen Bradley PLC using DF1
>> protocol, I found that the driver that handled one character at a time 
>> used
>> up a significant part of the available CPU time.  Enough that the 
>> computer
>> was observed to be very sluggish.
>>
>> The DF1 protocol uses markers (DLE) to mark key locations in the data
>> stream.  By using a "read until character" function instead of reading 
>> one
>> character at a time, the CPU overhead was significantly reduced, and the
>> system was no longer sluggish.
>
> How would you tell that to the driver? Do you mean a true driver here or
> merely a program that uses an OS serial driver? I meant the later...

So did I.  On the systems I have used there have been systems calls "read 
until character or timeout" or at least a mechnism of implementing that 
functionality.

> Talking
> about drivers, you are absolutely right. Interrupts are very expensive on
> wintel platform. I had to redesign one driver (an ISA transputer link
> driver) for a similar reason. The card produced an interrupt per byte at 
> up
> to 1Mbaud...
>
[snip]

>> On NT/XP for full duplex communication, you really have two choices:
>>   Use non-blocking I/O (poll)
>>   Use overlapped I/O
>
> I prefer two threads, one does blocking read (incoming bytes stream),
> another blocking write (outgoing packets), it works well. Overlapped I/O 
> is
> too clumsy, then the read buffer cannot be accessed anyway when the
> operation is pending.
>

I prefer two threads as well.  It is my understanding that the only way to 
do this on NT is using overlapped I/O.  I have this working in my high level 
interface to NT's serial port.

>> The clumsy interface has no relation to the choice of language of the 
>> API.
>> And it shouldn't.  It is based on the design of the driver.  In the good 
>> (or
>> bad?) old days, the OS interfaces used to be described in a language 
>> neutral
>> manner... as it should be.
>
> How can you do it language neutral? The problem is that asynchronous I/O
> inherently requires concurrency. If the language does not support
> concurrency, you simply cannot comprehensively describe what's going on.
>

Sure you can.  You just have to be explicit about how concurrency is 
handled, and can't take anything for granted.  This leads to platform 
dependent code, which I usually handle by implementing a high level 
interface inside of a platform dependent package implementation.

I believe that programming languages that handle concurrency are the 
exception rather than the rule.  Which is one of the reasons I prefer Ada 
over those languages, but it certainly doesn't keep multi-tasking programs 
from being implemented in those languages.

The majority of development I have done in the past has not been in Ada. 
When developing systems non-Ada systems I always face an OS learning curve 
and a programming language learning curve.  The programming language 
learning curve is to learn the local dialect of the language, since the 
standard language doesn't include everything needed to do concurrent work.

Ada covers enough bases I only have to learn a small part of working with 
the OS.  This comes back to my claim that you can develop a system in almost 
any programming language, given enough time and money... but if you want to 
minimize either, use Ada.

>> When I had to make duplex serial communications work right on NT, I found 
>> an
>> article descrbing how great it was that NT could do overlapped I/O, and 
>> how
>> it was something new and wonderful.  I was left with the impression that 
>> the
>> developer had only done the D part of R&D and thought this was a great 
>> new
>> feature.  I found it to be a clumsy feature that was working around a 
>> poor
>> driver implementation.  I had the same impression when Microsoft 
>> "invented"
>> multi-threaded programming... after I had already been using 
>> multi-tasking
>> systems for years.
>
> I started on RSX-11 and VAX-11, so to me it was a shock to see for the
> first time how many processes were running on an idle Unix machine. Slowly
> I adapted to the idea that each mouse button needs a separate process to
> deal with (:-)) Windows is somewhere between. It is unfair to blame alone
> Microsoft for all mess. It was Unix first, which destroyed the market of
> operating systems. We have what we have deserved.
>
[snip]
>
> Once I managed to install Windows NT 4.0 on a 486DX 66MHz / 12Mb. It took
> about one day. The system booted in 10 minutes or so. I still remember 
> VMS,
> which served 6 interactive users having 4Mb memory. It had DEC Ada too!
> (:-))
>
>> it's been long enough that I can't remember).  On those machines
>> the time difference was more significant.
>
> Actually it worked well on a Pentium 133 under Windows NT 4.0. COM port
> isn't fast enough to break it down. In fact, in one case we had to slow
> down the PC side, because the device crashed when PC had written too fast.
>
> I'm too lazy to look into Kernel32.dll, but I guess, that an ReadFile call
> isn't that expensive. I suppose that there is an OS-allocated buffer in 
> the
> process space, so ReadFile does nothing extraordinary. Because the input 
> is
> buffered, it makes little sense to do it again.

I think this is the case for networked communication, but not for serial 
communication.

Steve
(The Duck)

>
> -- 
> Regards,
> Dmitry A. Kazakov
> http://www.dmitry-kazakov.de 

Re: Palatable Windows IO using Ada

[Dmitry A. Kazakov]

On Mon, 10 Apr 2006 06:50:53 -0700, Steve wrote:

> "Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> wrote in message 
> news:o2i2brhcp49x$.1lmgnwrqgqzf7$.dlg@40tude.net...

>> How would you tell that to the driver? Do you mean a true driver here or
>> merely a program that uses an OS serial driver? I meant the later...
> 
> So did I.  On the systems I have used there have been systems calls "read 
> until character or timeout" or at least a mechnism of implementing that 
> functionality.

Then it should be "until delimiter or timeout or buffer is full." This is a
too specific, IMO. For some devices you would need a pattern matching
rather than simple delimiter. I don't think that it is worth to have
something like that in OS. If protected objects were supported, one could
just give a protected ring buffer to the driver. 

> I prefer two threads as well.  It is my understanding that the only way to 
> do this on NT is using overlapped I/O.  I have this working in my high level 
> interface to NT's serial port.

We used both overlapped and synchronous I/O over Windows serial port, with
no noticeable difference. In almost all cases the communication was without
either hardware handshake or parity, which should be a quite hard test, if
OS wouldn't buffer. In my experience, a correct use of SetCommTimeouts is
more delicate issue.

>>> The clumsy interface has no relation to the choice of language of the 
>>> API.
>>> And it shouldn't.  It is based on the design of the driver.  In the good 
>>> (or
>>> bad?) old days, the OS interfaces used to be described in a language 
>>> neutral
>>> manner... as it should be.
>>
>> How can you do it language neutral? The problem is that asynchronous I/O
>> inherently requires concurrency. If the language does not support
>> concurrency, you simply cannot comprehensively describe what's going on.
> 
> Sure you can.  You just have to be explicit about how concurrency is 
> handled, and can't take anything for granted.

But you just cannot be explicit. You have to document, to comment etc,
rather than to code. It is like types, you don't need typed languages. Just
carefully document each memory cell. The only problem is the abstraction
level.

[...]
> Ada covers enough bases I only have to learn a small part of working with 
> the OS.  This comes back to my claim that you can develop a system in almost 
> any programming language, given enough time and money... but if you want to 
> minimize either, use Ada.

Actually there is a physical limit of complexity at which the amount of
money becomes infinite. Though, long before that it will be bigger than the
national product. With time it is even worse. I wouldn't even try to
understand the Assembler code I wrote 20 years ago... (:-))

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Re: Palatable Windows IO using Ada

[Randy Brukardt]

"Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> wrote in message
news:o2i2brhcp49x$.1lmgnwrqgqzf7$.dlg@40tude.net...
> Once I managed to install Windows NT 4.0 on a 486DX 66MHz / 12Mb. It took
> about one day. The system booted in 10 minutes or so.

Our main server is a 90 MHZ Pentium running Windows NT 4.0 Server. (It's
fast enough, as it only has to manage a dozen user accounts and less than
100 e-mail messages per day.) It takes about 90 seconds to boot; it's faster
than our Windows 95 test machine (which is a 250 MHZ machine).

The "bad old days" were not really *that* bad. :-)

                       Randy.