* Palatable Windows IO using Ada @ 2006-04-07 0:21 Le 2006-04-07 7:29 ` Dmitry A. Kazakov ` (2 more replies) 0 siblings, 3 replies; 15+ messages in thread From: Le @ 2006-04-07 0:21 UTC (permalink / raw) I are looking for Ada package(s) which make Windows serial IO more palatable for varying length asynchronous inputs. My team is laying a large legacy Ada application on top of Windows XP. There is a fair amount of serial IO which follows a VMS flavor - pending reads while writes stimulate responses back to the reading tasks & sets of terminator characters to delimit messages. Wrappers we have seen for the Win32 API simply export the ReadFile and ReadFileEx which appear to be blocking by their very nature and demand that the reader have apriori knowledge of the length of input expected at any particular instance. Of course one can deal with the varying length by doing all reads one character at a time, but this is quite waseful in the face of Windows already daunting CPU requirements. So, if you know of a somewhat thicker wrapper for Windows external IO that makes it a little more (may I say) (x)nix like (or some other alternative) I'd really appreciate a pointer or copy. ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Palatable Windows IO using Ada 2006-04-07 0:21 Palatable Windows IO using Ada Le @ 2006-04-07 7:29 ` Dmitry A. Kazakov 2006-04-07 16:30 ` Le 2006-04-08 2:07 ` Steve 2006-04-07 10:55 ` Stephen Leake 2006-04-07 21:19 ` John 2 siblings, 2 replies; 15+ messages in thread From: Dmitry A. Kazakov @ 2006-04-07 7:29 UTC (permalink / raw) On 6 Apr 2006 17:21:44 -0700, Le wrote: > Wrappers we > have seen for the Win32 API simply export the ReadFile and ReadFileEx > which appear to be blocking by their very nature and demand that the > reader have apriori knowledge of the length of input expected at any > particular instance. which is true for all kinds of any I/O in any OS. You should know the size of the byte, block, record, line before you start to read it. Windows ReadFile isn't always blocking, it can be asynchronous, see "overlapping" I/O in MSDN description of ReadFile. > Of course one can deal with the varying length by doing all reads one > character at a time, but this is quite waseful in the face of Windows > already daunting CPU requirements. Windows does buffering in background, so it isn't that wasteful. The knowledge of how many items can be read at once depends solely on the application. As for serial I/O, there are further settings to keep in mind. See Windows API SetCommTimeouts. -- Regards, Dmitry A. Kazakov http://www.dmitry-kazakov.de ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Palatable Windows IO using Ada 2006-04-07 7:29 ` Dmitry A. Kazakov @ 2006-04-07 16:30 ` Le 2006-04-07 18:51 ` Dmitry A. Kazakov 2006-04-08 2:07 ` Steve 1 sibling, 1 reply; 15+ messages in thread From: Le @ 2006-04-07 16:30 UTC (permalink / raw) Your attempt at a tutorial is in error in a couple of places. Stating that an application "should know" what will be returned is presumptuous. Note this is a legacy application, so obviously there are some devices out there which give varying responses. These are raw byte sequences - no blocks or lines. Those esternal device ICDs are not typically in the control of a system designer. The ReadFile interface *is* blocking. Overlap allows an app to come back and see if it is done. If not, assuming timeout wasn't used, one must then cancel the IO or some other cleanup. With respect to buffering, my point was that doing byte at a time IO to get around the blocking nature of the API defeats the efficiencies presumably provided by the driver's internal buffering. ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Palatable Windows IO using Ada 2006-04-07 16:30 ` Le @ 2006-04-07 18:51 ` Dmitry A. Kazakov 2006-04-07 19:21 ` Le 0 siblings, 1 reply; 15+ messages in thread From: Dmitry A. Kazakov @ 2006-04-07 18:51 UTC (permalink / raw) On 7 Apr 2006 09:30:23 -0700, Le wrote: > Your attempt at a tutorial is in error in a couple of places. Stating > that an application "should know" what will be returned is > presumptuous. Note this is a legacy application, so obviously there are > some devices out there which give varying responses. These are raw byte > sequences - no blocks or lines. If you mean a byte stream, then it is *byte* you (or application) should know of. You cannot read any device not knowing the size of input. You might have a higher level communication protocol which build something out of these bytes, for example, "a valid Ada program encoded in ASCII." But how an OS driver might know about it? > The ReadFile interface *is* blocking. Overlap allows an app to come > back and see if it is done. Then it is rather a strange definition of "blocking." In my world blocking operation means that the execution thread gets blocked until its completes. The thread cannot go anywhere it that state. > With respect to buffering, my point was that doing byte at a time IO to > get around the blocking nature of the API defeats the efficiencies > presumably provided by the driver's internal buffering. How so? Byte-wise reading just does not do any extra and unnecessary buffering, because OS already did it. One could argue that OS API are expensive to call. This a different issue. I don"t have any figures concerning Windows, but in my experience that was never a problem. At least when reading from sockets and serial devices. As for overlapping ReadFile, your argument does not apply either, because it gets *all available* bytes the buffer provided in its argument can take. -- Regards, Dmitry A. Kazakov http://www.dmitry-kazakov.de ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Palatable Windows IO using Ada 2006-04-07 18:51 ` Dmitry A. Kazakov @ 2006-04-07 19:21 ` Le 2006-04-07 20:49 ` Dmitry A. Kazakov 0 siblings, 1 reply; 15+ messages in thread From: Le @ 2006-04-07 19:21 UTC (permalink / raw) >You cannot read any device not knowing the size of input. You referred to blocks & lines previously indicating that you understood what was being referred to. I was speaking of byte sequences which vary in *length*. >> The ReadFile interface *is* blocking. Overlap allows an app to come >> back and see if it is done. >Then it is rather a strange definition of "blocking." In my world blocking >operation means that the execution thread gets blocked until its completes. >The thread cannot go anywhere it that state. Same universe that I am in... ReadFile blocks or times out. In the future if you don't have an answer to the heart of the question please resist the urge to wax pedagogical. ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Palatable Windows IO using Ada 2006-04-07 19:21 ` Le @ 2006-04-07 20:49 ` Dmitry A. Kazakov 0 siblings, 0 replies; 15+ messages in thread From: Dmitry A. Kazakov @ 2006-04-07 20:49 UTC (permalink / raw) On 7 Apr 2006 12:21:11 -0700, Le wrote: >>You cannot read any device not knowing the size of input. > > You referred to blocks & lines previously indicating that you > understood what was being referred to. No, I didn't. Blocks were mentioned having block-oriented devices in mind. Lines were for record-oriented I/O existed, for example, in RSX-11. > I was speaking of byte sequences which vary in *length*. which length is determined by what? > ... ReadFile blocks or times out. This is plain wrong. Here is the citation from MSDN "Synchronization and Overlapped Input and Output" "The WriteFile, ReadFile, DeviceIoControl, WaitCommEvent, ConnectNamedPipe, and TransactNamedPipe functions can be performed either synchronously or asynchronously." and further: "... Functions called for overlapped operation can return immediately, even though the operation has not been completed. This enables a time-consuming I/O operation to be executed in the background while the calling thread is free to perform other tasks. For example, a single thread can perform simultaneous I/O operations on different handles, or even simultaneous read and write operations on the same handle." From MSDN "ReadFile" "... If hFile is opened with FILE_FLAG_OVERLAPPED and lpOverlapped is not NULL, the read operation starts at the offset that is specified in the OVERLAPPED structure, and ReadFile may return before the read operation is complete. In this scenario, ReadFile returns FALSE and the GetLastError function returns ERROR_IO_PENDING, which allows the calling process to continue while the read operation completes." -- Regards, Dmitry A. Kazakov http://www.dmitry-kazakov.de ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Palatable Windows IO using Ada 2006-04-07 7:29 ` Dmitry A. Kazakov 2006-04-07 16:30 ` Le @ 2006-04-08 2:07 ` Steve 2006-04-08 9:36 ` Dmitry A. Kazakov 1 sibling, 1 reply; 15+ messages in thread From: Steve @ 2006-04-08 2:07 UTC (permalink / raw) "Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> wrote in message news:15qxrbngb2kll.wkigwit0g1tn.dlg@40tude.net... > On 6 Apr 2006 17:21:44 -0700, Le wrote: > >> Wrappers we >> have seen for the Win32 API simply export the ReadFile and ReadFileEx >> which appear to be blocking by their very nature and demand that the >> reader have apriori knowledge of the length of input expected at any >> particular instance. > > which is true for all kinds of any I/O in any OS. You should know the size > of the byte, block, record, line before you start to read it. > I disagree. I have written a several high level drivers that communicate with serial devices. In my experience, knowing the size of the byte, block, record or line before you start to read is the exception rather than the rule. In my experience it is much more common to detect the end of input by the presence of special marker characters in the data or by the time between characters exceeding some threshold. In some cases an upper limit on the number of characters that will be available to read is known, but not always. In these cases multiple reads may be required. > Windows ReadFile isn't always blocking, it can be asynchronous, see > "overlapping" I/O in MSDN description of ReadFile. > I've been through the windows overlapped I/O for serial communications. In my opinion it's a royal pain in the ass. I think NT was the fourth OS I used to implement a bi-directional communications driver. I was very suprised to learn that you had to do something special to have one task reading a serial port at the same time a second task writes to the port in order to make things work. In the other systems I used, it just worked. >> Of course one can deal with the varying length by doing all reads one >> character at a time, but this is quite waseful in the face of Windows >> already daunting CPU requirements. > > Windows does buffering in background, so it isn't that wasteful. The > knowledge of how many items can be read at once depends solely on the > application. > You're kidding... right? Every transaction with a driver in NT involves a significant amount of overhead. If you read one character at a time you're incurring the overhead for each character. If you read a bunch of characters at once, there is just one transaction with the driver... less overhead. If you have a nice library that buffers things for you, you don't have to worry about it. But for serial I/O you'll have to build that on top of the call to ReadFile. Steve (The Duck) BTW: I sent the original poster a copy of a hight level driver for XP that should meet their needs. > As for serial I/O, there are further settings to keep in mind. See Windows > API SetCommTimeouts. > > -- > Regards, > Dmitry A. Kazakov > http://www.dmitry-kazakov.de ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Palatable Windows IO using Ada 2006-04-08 2:07 ` Steve @ 2006-04-08 9:36 ` Dmitry A. Kazakov 2006-04-09 13:50 ` Steve 0 siblings, 1 reply; 15+ messages in thread From: Dmitry A. Kazakov @ 2006-04-08 9:36 UTC (permalink / raw) On Fri, 7 Apr 2006 19:07:35 -0700, Steve wrote: > "Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> wrote in message > news:15qxrbngb2kll.wkigwit0g1tn.dlg@40tude.net... >> On 6 Apr 2006 17:21:44 -0700, Le wrote: >> >>> Wrappers we >>> have seen for the Win32 API simply export the ReadFile and ReadFileEx >>> which appear to be blocking by their very nature and demand that the >>> reader have apriori knowledge of the length of input expected at any >>> particular instance. >> >> which is true for all kinds of any I/O in any OS. You should know the size >> of the byte, block, record, line before you start to read it. >> > I have written a several high level drivers that communicate with serial > devices. In my experience, knowing the size of the byte, block, record or > line before you start to read is the exception rather than the rule. > > In my experience it is much more common to detect the end of input by the > presence of special marker characters in the data or by the time between > characters exceeding some threshold. This is a higher level protocol. The point is that when the OS does not natively support a device-specific application layer protocol, and it normally does not, then there is no other place to do it as in the application. So the communication item becomes one byte. Blocking for more than one byte at once is looking for troubles. Even if the protocol has things like bytes counts. Practically none of multiple serial devices we dealt with worked according to the specification. Using bytes stream full duplex communication model has saved us a lot of time, because we could easily adapt our software and overcome bugs in the hardware. >> Windows ReadFile isn't always blocking, it can be asynchronous, see >> "overlapping" I/O in MSDN description of ReadFile. >> > I've been through the windows overlapped I/O for serial communications. In > my opinion it's a royal pain in the ass. Yes. In my opinion it is overkill to use overlapped I/O for serial connections. Though if you have 1000 devices to handle simultaneously, there might be no other choice. In RSX and VAX, where the idea of asynchronous I/O seems was borrowed from, it worked well. I also think that overlapped I/O and asynchronous system trap could nicely fit to Ada's protected objects. The interface to overlapped I/O is so clumsy because API is in C, it should be in Ada! (:-)) > I think NT was the fourth OS I > used to implement a bi-directional communications driver. I was very > suprised to learn that you had to do something special to have one task > reading a serial port at the same time a second task writes to the port in > order to make things work. In the other systems I used, it just worked. ? >>> Of course one can deal with the varying length by doing all reads one >>> character at a time, but this is quite waseful in the face of Windows >>> already daunting CPU requirements. >> >> Windows does buffering in background, so it isn't that wasteful. The >> knowledge of how many items can be read at once depends solely on the >> application. > > You're kidding... right? Every transaction with a driver in NT involves a > significant amount of overhead. If you read one character at a time you're > incurring the overhead for each character. I didn't saw any significant difference when using synchronous vs. asynchronous read for serial devices. Taking into account serial connection speed I doubt one could notice it. A more interesting case is socket I/O. Our commercial software reads from sockets using 1 byte buffer and blocking read. Recently we did elaborated time measurements under Windows XP and Server 2003 for stream uni- and multicast sockets. It was all but easy, because of quite flawed design of Windows time services. But that's a different story. Anyway, our experiments have shown that the bottleneck wasn't communication. We tested publisher / subscriber services of our product and have noticed that the communication layer by margin outperforms other (local) parts. Roughly, global mutexes and context switches are much worse than I/O APIs, even if the former are used per data block and the later per byte. > If you have a nice library that buffers things for you, you don't have to > worry about it. But for serial I/O you'll have to build that on top of the > call to ReadFile. There is no other choice. The transport layer of serial connection is byte stream. It is a medical fact. (:-)) -- Regards, Dmitry A. Kazakov http://www.dmitry-kazakov.de ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Palatable Windows IO using Ada 2006-04-08 9:36 ` Dmitry A. Kazakov @ 2006-04-09 13:50 ` Steve 2006-04-10 7:43 ` Dmitry A. Kazakov 0 siblings, 1 reply; 15+ messages in thread From: Steve @ 2006-04-09 13:50 UTC (permalink / raw) "Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> wrote in message news:14sfx2j059hst$.1x6sfev1savpw.dlg@40tude.net... > On Fri, 7 Apr 2006 19:07:35 -0700, Steve wrote: > [snip] > > This is a higher level protocol. The point is that when the OS does not > natively support a device-specific application layer protocol, and it > normally does not, then there is no other place to do it as in the > application. So the communication item becomes one byte. Blocking for more > than one byte at once is looking for troubles. Even if the protocol has > things like bytes counts. Practically none of multiple serial devices we > dealt with worked according to the specification. Using bytes stream full > duplex communication model has saved us a lot of time, because we could > easily adapt our software and overcome bugs in the hardware. > It is interesting the opinions that are formed based on one's experiences. Mine is at least in part formed because in the past I have had to rewrite drivers that handled one character at a time to process as many as possible at a time due to performance issues. In one case, when communicating with an Allen Bradley PLC using DF1 protocol, I found that the driver that handled one character at a time used up a significant part of the available CPU time. Enough that the computer was observed to be very sluggish. The DF1 protocol uses markers (DLE) to mark key locations in the data stream. By using a "read until character" function instead of reading one character at a time, the CPU overhead was significantly reduced, and the system was no longer sluggish. Most of the applications I have worked on make heavy use of the CPU and are time critical (but not safety critical). Minimizing the processor overhead used by communication is often a goal. >>> Windows ReadFile isn't always blocking, it can be asynchronous, see >>> "overlapping" I/O in MSDN description of ReadFile. >>> >> I've been through the windows overlapped I/O for serial communications. >> In >> my opinion it's a royal pain in the ass. > > Yes. In my opinion it is overkill to use overlapped I/O for serial > connections. Though if you have 1000 devices to handle simultaneously, > there might be no other choice. In RSX and VAX, where the idea of > asynchronous I/O seems was borrowed from, it worked well. I also think > that > overlapped I/O and asynchronous system trap could nicely fit to Ada's > protected objects. The interface to overlapped I/O is so clumsy because > API > is in C, it should be in Ada! (:-)) On NT/XP for full duplex communication, you really have two choices: Use non-blocking I/O (poll) Use overlapped I/O The clumsy interface has no relation to the choice of language of the API. And it shouldn't. It is based on the design of the driver. In the good (or bad?) old days, the OS interfaces used to be described in a language neutral manner... as it should be. When I had to make duplex serial communications work right on NT, I found an article descrbing how great it was that NT could do overlapped I/O, and how it was something new and wonderful. I was left with the impression that the developer had only done the D part of R&D and thought this was a great new feature. I found it to be a clumsy feature that was working around a poor driver implementation. I had the same impression when Microsoft "invented" multi-threaded programming... after I had already been using multi-tasking systems for years. [snip] >>> Windows does buffering in background, so it isn't that wasteful. The >>> knowledge of how many items can be read at once depends solely on the >>> application. >> >> You're kidding... right? Every transaction with a driver in NT involves >> a >> significant amount of overhead. If you read one character at a time >> you're >> incurring the overhead for each character. > > I didn't saw any significant difference when using synchronous vs. > asynchronous read for serial devices. Taking into account serial > connection > speed I doubt one could notice it. A more interesting case is socket I/O. > Our commercial software reads from sockets using 1 byte buffer and > blocking > read. Recently we did elaborated time measurements under Windows XP and > Server 2003 for stream uni- and multicast sockets. It was all but easy, > because of quite flawed design of Windows time services. But that's a > different story. Anyway, our experiments have shown that the bottleneck > wasn't communication. We tested publisher / subscriber services of our > product and have noticed that the communication layer by margin > outperforms > other (local) parts. Roughly, global mutexes and context switches are much > worse than I/O APIs, even if the former are used per data block and the > later per byte. > Since you're using Windows XP and Server 2003, you're probably using recent hardware, which is fast enough that you may not see an overall time difference. If you use a high resolution performance counter to do you're timing you are much more likely to find a difference. I started working with networking on NT on Pentium 166 machines (or slower... it's been long enough that I can't remember). On those machines the time difference was more significant. Steve (The Duck) >> If you have a nice library that buffers things for you, you don't have to >> worry about it. But for serial I/O you'll have to build that on top of >> the >> call to ReadFile. > > There is no other choice. The transport layer of serial connection is byte > stream. It is a medical fact. (:-)) > > -- > Regards, > Dmitry A. Kazakov > http://www.dmitry-kazakov.de ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Palatable Windows IO using Ada 2006-04-09 13:50 ` Steve @ 2006-04-10 7:43 ` Dmitry A. Kazakov 2006-04-10 13:50 ` Steve 2006-04-10 19:28 ` Randy Brukardt 0 siblings, 2 replies; 15+ messages in thread From: Dmitry A. Kazakov @ 2006-04-10 7:43 UTC (permalink / raw) On Sun, 9 Apr 2006 06:50:23 -0700, Steve wrote: > It is interesting the opinions that are formed based on one's experiences. > Mine is at least in part formed because in the past I have had to rewrite > drivers that handled one character at a time to process as many as possible > at a time due to performance issues. > > In one case, when communicating with an Allen Bradley PLC using DF1 > protocol, I found that the driver that handled one character at a time used > up a significant part of the available CPU time. Enough that the computer > was observed to be very sluggish. > > The DF1 protocol uses markers (DLE) to mark key locations in the data > stream. By using a "read until character" function instead of reading one > character at a time, the CPU overhead was significantly reduced, and the > system was no longer sluggish. How would you tell that to the driver? Do you mean a true driver here or merely a program that uses an OS serial driver? I meant the later. Talking about drivers, you are absolutely right. Interrupts are very expensive on wintel platform. I had to redesign one driver (an ISA transputer link driver) for a similar reason. The card produced an interrupt per byte at up to 1Mbaud... > Most of the applications I have worked on make heavy use of the CPU and are > time critical (but not safety critical). Minimizing the processor overhead > used by communication is often a goal. > >>>> Windows ReadFile isn't always blocking, it can be asynchronous, see >>>> "overlapping" I/O in MSDN description of ReadFile. >>>> >>> I've been through the windows overlapped I/O for serial communications. >>> In >>> my opinion it's a royal pain in the ass. >> >> Yes. In my opinion it is overkill to use overlapped I/O for serial >> connections. Though if you have 1000 devices to handle simultaneously, >> there might be no other choice. In RSX and VAX, where the idea of >> asynchronous I/O seems was borrowed from, it worked well. I also think >> that >> overlapped I/O and asynchronous system trap could nicely fit to Ada's >> protected objects. The interface to overlapped I/O is so clumsy because >> API >> is in C, it should be in Ada! (:-)) > > On NT/XP for full duplex communication, you really have two choices: > Use non-blocking I/O (poll) > Use overlapped I/O I prefer two threads, one does blocking read (incoming bytes stream), another blocking write (outgoing packets), it works well. Overlapped I/O is too clumsy, then the read buffer cannot be accessed anyway when the operation is pending. > The clumsy interface has no relation to the choice of language of the API. > And it shouldn't. It is based on the design of the driver. In the good (or > bad?) old days, the OS interfaces used to be described in a language neutral > manner... as it should be. How can you do it language neutral? The problem is that asynchronous I/O inherently requires concurrency. If the language does not support concurrency, you simply cannot comprehensively describe what's going on. > When I had to make duplex serial communications work right on NT, I found an > article descrbing how great it was that NT could do overlapped I/O, and how > it was something new and wonderful. I was left with the impression that the > developer had only done the D part of R&D and thought this was a great new > feature. I found it to be a clumsy feature that was working around a poor > driver implementation. I had the same impression when Microsoft "invented" > multi-threaded programming... after I had already been using multi-tasking > systems for years. I started on RSX-11 and VAX-11, so to me it was a shock to see for the first time how many processes were running on an idle Unix machine. Slowly I adapted to the idea that each mouse button needs a separate process to deal with (:-)) Windows is somewhere between. It is unfair to blame alone Microsoft for all mess. It was Unix first, which destroyed the market of operating systems. We have what we have deserved. >> I didn't saw any significant difference when using synchronous vs. >> asynchronous read for serial devices. Taking into account serial >> connection >> speed I doubt one could notice it. A more interesting case is socket I/O. >> Our commercial software reads from sockets using 1 byte buffer and >> blocking >> read. Recently we did elaborated time measurements under Windows XP and >> Server 2003 for stream uni- and multicast sockets. It was all but easy, >> because of quite flawed design of Windows time services. But that's a >> different story. Anyway, our experiments have shown that the bottleneck >> wasn't communication. We tested publisher / subscriber services of our >> product and have noticed that the communication layer by margin >> outperforms >> other (local) parts. Roughly, global mutexes and context switches are much >> worse than I/O APIs, even if the former are used per data block and the >> later per byte. > > Since you're using Windows XP and Server 2003, you're probably using recent > hardware, which is fast enough that you may not see an overall time > difference. If you use a high resolution performance counter to do you're > timing you are much more likely to find a difference. > > I started working with networking on NT on Pentium 166 machines (or > slower... Once I managed to install Windows NT 4.0 on a 486DX 66MHz / 12Mb. It took about one day. The system booted in 10 minutes or so. I still remember VMS, which served 6 interactive users having 4Mb memory. It had DEC Ada too! (:-)) > it's been long enough that I can't remember). On those machines > the time difference was more significant. Actually it worked well on a Pentium 133 under Windows NT 4.0. COM port isn't fast enough to break it down. In fact, in one case we had to slow down the PC side, because the device crashed when PC had written too fast. I'm too lazy to look into Kernel32.dll, but I guess, that an ReadFile call isn't that expensive. I suppose that there is an OS-allocated buffer in the process space, so ReadFile does nothing extraordinary. Because the input is buffered, it makes little sense to do it again. -- Regards, Dmitry A. Kazakov http://www.dmitry-kazakov.de ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Palatable Windows IO using Ada 2006-04-10 7:43 ` Dmitry A. Kazakov @ 2006-04-10 13:50 ` Steve 2006-04-10 15:05 ` Dmitry A. Kazakov 2006-04-10 19:28 ` Randy Brukardt 1 sibling, 1 reply; 15+ messages in thread From: Steve @ 2006-04-10 13:50 UTC (permalink / raw) "Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> wrote in message news:o2i2brhcp49x$.1lmgnwrqgqzf7$.dlg@40tude.net... > On Sun, 9 Apr 2006 06:50:23 -0700, Steve wrote: [snip] >> >> In one case, when communicating with an Allen Bradley PLC using DF1 >> protocol, I found that the driver that handled one character at a time >> used >> up a significant part of the available CPU time. Enough that the >> computer >> was observed to be very sluggish. >> >> The DF1 protocol uses markers (DLE) to mark key locations in the data >> stream. By using a "read until character" function instead of reading >> one >> character at a time, the CPU overhead was significantly reduced, and the >> system was no longer sluggish. > > How would you tell that to the driver? Do you mean a true driver here or > merely a program that uses an OS serial driver? I meant the later... So did I. On the systems I have used there have been systems calls "read until character or timeout" or at least a mechnism of implementing that functionality. > Talking > about drivers, you are absolutely right. Interrupts are very expensive on > wintel platform. I had to redesign one driver (an ISA transputer link > driver) for a similar reason. The card produced an interrupt per byte at > up > to 1Mbaud... > [snip] >> On NT/XP for full duplex communication, you really have two choices: >> Use non-blocking I/O (poll) >> Use overlapped I/O > > I prefer two threads, one does blocking read (incoming bytes stream), > another blocking write (outgoing packets), it works well. Overlapped I/O > is > too clumsy, then the read buffer cannot be accessed anyway when the > operation is pending. > I prefer two threads as well. It is my understanding that the only way to do this on NT is using overlapped I/O. I have this working in my high level interface to NT's serial port. >> The clumsy interface has no relation to the choice of language of the >> API. >> And it shouldn't. It is based on the design of the driver. In the good >> (or >> bad?) old days, the OS interfaces used to be described in a language >> neutral >> manner... as it should be. > > How can you do it language neutral? The problem is that asynchronous I/O > inherently requires concurrency. If the language does not support > concurrency, you simply cannot comprehensively describe what's going on. > Sure you can. You just have to be explicit about how concurrency is handled, and can't take anything for granted. This leads to platform dependent code, which I usually handle by implementing a high level interface inside of a platform dependent package implementation. I believe that programming languages that handle concurrency are the exception rather than the rule. Which is one of the reasons I prefer Ada over those languages, but it certainly doesn't keep multi-tasking programs from being implemented in those languages. The majority of development I have done in the past has not been in Ada. When developing systems non-Ada systems I always face an OS learning curve and a programming language learning curve. The programming language learning curve is to learn the local dialect of the language, since the standard language doesn't include everything needed to do concurrent work. Ada covers enough bases I only have to learn a small part of working with the OS. This comes back to my claim that you can develop a system in almost any programming language, given enough time and money... but if you want to minimize either, use Ada. >> When I had to make duplex serial communications work right on NT, I found >> an >> article descrbing how great it was that NT could do overlapped I/O, and >> how >> it was something new and wonderful. I was left with the impression that >> the >> developer had only done the D part of R&D and thought this was a great >> new >> feature. I found it to be a clumsy feature that was working around a >> poor >> driver implementation. I had the same impression when Microsoft >> "invented" >> multi-threaded programming... after I had already been using >> multi-tasking >> systems for years. > > I started on RSX-11 and VAX-11, so to me it was a shock to see for the > first time how many processes were running on an idle Unix machine. Slowly > I adapted to the idea that each mouse button needs a separate process to > deal with (:-)) Windows is somewhere between. It is unfair to blame alone > Microsoft for all mess. It was Unix first, which destroyed the market of > operating systems. We have what we have deserved. > [snip] > > Once I managed to install Windows NT 4.0 on a 486DX 66MHz / 12Mb. It took > about one day. The system booted in 10 minutes or so. I still remember > VMS, > which served 6 interactive users having 4Mb memory. It had DEC Ada too! > (:-)) > >> it's been long enough that I can't remember). On those machines >> the time difference was more significant. > > Actually it worked well on a Pentium 133 under Windows NT 4.0. COM port > isn't fast enough to break it down. In fact, in one case we had to slow > down the PC side, because the device crashed when PC had written too fast. > > I'm too lazy to look into Kernel32.dll, but I guess, that an ReadFile call > isn't that expensive. I suppose that there is an OS-allocated buffer in > the > process space, so ReadFile does nothing extraordinary. Because the input > is > buffered, it makes little sense to do it again. I think this is the case for networked communication, but not for serial communication. Steve (The Duck) > > -- > Regards, > Dmitry A. Kazakov > http://www.dmitry-kazakov.de ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Palatable Windows IO using Ada 2006-04-10 13:50 ` Steve @ 2006-04-10 15:05 ` Dmitry A. Kazakov 0 siblings, 0 replies; 15+ messages in thread From: Dmitry A. Kazakov @ 2006-04-10 15:05 UTC (permalink / raw) On Mon, 10 Apr 2006 06:50:53 -0700, Steve wrote: > "Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> wrote in message > news:o2i2brhcp49x$.1lmgnwrqgqzf7$.dlg@40tude.net... >> How would you tell that to the driver? Do you mean a true driver here or >> merely a program that uses an OS serial driver? I meant the later... > > So did I. On the systems I have used there have been systems calls "read > until character or timeout" or at least a mechnism of implementing that > functionality. Then it should be "until delimiter or timeout or buffer is full." This is a too specific, IMO. For some devices you would need a pattern matching rather than simple delimiter. I don't think that it is worth to have something like that in OS. If protected objects were supported, one could just give a protected ring buffer to the driver. > I prefer two threads as well. It is my understanding that the only way to > do this on NT is using overlapped I/O. I have this working in my high level > interface to NT's serial port. We used both overlapped and synchronous I/O over Windows serial port, with no noticeable difference. In almost all cases the communication was without either hardware handshake or parity, which should be a quite hard test, if OS wouldn't buffer. In my experience, a correct use of SetCommTimeouts is more delicate issue. >>> The clumsy interface has no relation to the choice of language of the >>> API. >>> And it shouldn't. It is based on the design of the driver. In the good >>> (or >>> bad?) old days, the OS interfaces used to be described in a language >>> neutral >>> manner... as it should be. >> >> How can you do it language neutral? The problem is that asynchronous I/O >> inherently requires concurrency. If the language does not support >> concurrency, you simply cannot comprehensively describe what's going on. > > Sure you can. You just have to be explicit about how concurrency is > handled, and can't take anything for granted. But you just cannot be explicit. You have to document, to comment etc, rather than to code. It is like types, you don't need typed languages. Just carefully document each memory cell. The only problem is the abstraction level. [...] > Ada covers enough bases I only have to learn a small part of working with > the OS. This comes back to my claim that you can develop a system in almost > any programming language, given enough time and money... but if you want to > minimize either, use Ada. Actually there is a physical limit of complexity at which the amount of money becomes infinite. Though, long before that it will be bigger than the national product. With time it is even worse. I wouldn't even try to understand the Assembler code I wrote 20 years ago... (:-)) -- Regards, Dmitry A. Kazakov http://www.dmitry-kazakov.de ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Palatable Windows IO using Ada 2006-04-10 7:43 ` Dmitry A. Kazakov 2006-04-10 13:50 ` Steve @ 2006-04-10 19:28 ` Randy Brukardt 1 sibling, 0 replies; 15+ messages in thread From: Randy Brukardt @ 2006-04-10 19:28 UTC (permalink / raw) "Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> wrote in message news:o2i2brhcp49x$.1lmgnwrqgqzf7$.dlg@40tude.net... > Once I managed to install Windows NT 4.0 on a 486DX 66MHz / 12Mb. It took > about one day. The system booted in 10 minutes or so. Our main server is a 90 MHZ Pentium running Windows NT 4.0 Server. (It's fast enough, as it only has to manage a dozen user accounts and less than 100 e-mail messages per day.) It takes about 90 seconds to boot; it's faster than our Windows 95 test machine (which is a 250 MHZ machine). The "bad old days" were not really *that* bad. :-) Randy. ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Palatable Windows IO using Ada 2006-04-07 0:21 Palatable Windows IO using Ada Le 2006-04-07 7:29 ` Dmitry A. Kazakov @ 2006-04-07 10:55 ` Stephen Leake 2006-04-07 21:19 ` John 2 siblings, 0 replies; 15+ messages in thread From: Stephen Leake @ 2006-04-07 10:55 UTC (permalink / raw) "Le" <snelson1@llnl.gov> writes: > I are looking for Ada package(s) which make Windows serial IO more > palatable for varying length asynchronous inputs. http://www.toadmail.com/~ada_wizard/ada/win32_com_ports.zip provides a simple low-level binding to read and write serial ports, including asynchronous. I haven't tried it on Windows XP. -- -- Stephe ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Palatable Windows IO using Ada 2006-04-07 0:21 Palatable Windows IO using Ada Le 2006-04-07 7:29 ` Dmitry A. Kazakov 2006-04-07 10:55 ` Stephen Leake @ 2006-04-07 21:19 ` John 2 siblings, 0 replies; 15+ messages in thread From: John @ 2006-04-07 21:19 UTC (permalink / raw) Le wrote: > I are looking for Ada package(s) which make Windows serial IO more > palatable for varying length asynchronous inputs. I am aware of a body of work called "Ada Terminal Emulator" that is signed by Ross Higson. rosshigson@optusnet.com.au It's just part of my packrat's nest, so I can't easily say where I got it, but it might be that some portions of its implementation will be interesting. References in its documentation might lead to something as well. > My team is laying a large legacy Ada application on top of Windows XP. > There is a fair amount of serial IO which follows a VMS flavor - > pending reads while writes stimulate responses back to the reading > tasks & sets of terminator characters to delimit messages. Twenty or so years ago I worked for a year while between assignments on a product then called Silas. I wonder if you're dealing with one his grandsons. John Woodruff retired software engineer. ^ permalink raw reply [flat|nested] 15+ messages in thread
end of thread, other threads:[~2006-04-10 19:28 UTC | newest] Thread overview: 15+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2006-04-07 0:21 Palatable Windows IO using Ada Le 2006-04-07 7:29 ` Dmitry A. Kazakov 2006-04-07 16:30 ` Le 2006-04-07 18:51 ` Dmitry A. Kazakov 2006-04-07 19:21 ` Le 2006-04-07 20:49 ` Dmitry A. Kazakov 2006-04-08 2:07 ` Steve 2006-04-08 9:36 ` Dmitry A. Kazakov 2006-04-09 13:50 ` Steve 2006-04-10 7:43 ` Dmitry A. Kazakov 2006-04-10 13:50 ` Steve 2006-04-10 15:05 ` Dmitry A. Kazakov 2006-04-10 19:28 ` Randy Brukardt 2006-04-07 10:55 ` Stephen Leake 2006-04-07 21:19 ` John
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox