* 'Write attribute vs Write procedure. @ 2003-05-21 0:07 Jano 2003-05-21 0:42 ` tmoran 2003-05-24 14:11 ` 'Write attribute vs Write procedure Craig Carey 0 siblings, 2 replies; 26+ messages in thread From: Jano @ 2003-05-21 0:07 UTC (permalink / raw) Hi everybody. I'm getting abysmal differences in speed when writing large chunks (64k) of data to a stream. The stream is from a socket of Gnat.Sockets, I mention it because I don't know if it has something to do. See the two following snippets of code: ----common----- Buffer : Stream_element_array (1 .. Stream_element_offset (Upload_buffer_size)); ----common----- ----case A----- Write (This.Link.all, Buffer (1 .. Stream_element_offset (Chunk))); ----case A----- ----case B----- Stream_element_array'Write (This.Link, Buffer (1 .. Stream_element_offset (Chunk))); ----case B----- In the above example (without optimizations and -O2 tried, both the same), the Write procedure is taking no noticeable amount of time/cpu. In the contrary, the 'Write attribute takes like 3-4 seconds of 100% CPU usage, shared more or less equally by my program and the SYSTEM process, whatever it is, on WinXP. Compiler is Gnat 3.15p, computer Pentium III 450Mhz. I'm asking only out of curiosity, and to prevent any other surprise in the future. Could it be related to alignment checks/adjustments? P.s: The 'Read attribute is also fast, no noticeable delay. P.s2: I've tried also with strings, only the 'Write version of course, and results are the same. -- ------------------------- Jano 402450.at.cepsz.unizar.es ------------------------- ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: 'Write attribute vs Write procedure. 2003-05-21 0:07 'Write attribute vs Write procedure Jano @ 2003-05-21 0:42 ` tmoran 2003-05-21 7:48 ` Ole-Hjalmar Kristensen ` (2 more replies) 2003-05-24 14:11 ` 'Write attribute vs Write procedure Craig Carey 1 sibling, 3 replies; 26+ messages in thread From: tmoran @ 2003-05-21 0:42 UTC (permalink / raw) > Write (This.Link.all, > Buffer (1 .. Stream_element_offset (Chunk))); > Stream_element_array'Write (This.Link, > Buffer (1 .. Stream_element_offset (Chunk))); >... > the Write procedure is taking no noticeable amount of time/cpu. >In the contrary, the 'Write attribute takes like 3-4 seconds of 100% CPU Perhaps Gnat's 'Write is doing one 'Write(Stream_Element) call for each element of the Buffer, vs a single call for the whole thing in the procedure version? Since you've already got a Stream_Element_Array, perhaps you could call Ada.Streams.Write(Ada.Streams.Root_Stream_Type'class(This.Link), Buffer (1 .. Stream_element_offset (Chunk))); directly? Personally, I think using the default 'write is only reasonable in the very simplest cases. If something is big, or complicated, I'd write my own 'write for it. ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: 'Write attribute vs Write procedure. 2003-05-21 0:42 ` tmoran @ 2003-05-21 7:48 ` Ole-Hjalmar Kristensen 2003-05-21 9:23 ` Ole-Hjalmar Kristensen 2003-05-23 6:09 ` Craig Carey 2 siblings, 0 replies; 26+ messages in thread From: Ole-Hjalmar Kristensen @ 2003-05-21 7:48 UTC (permalink / raw) tmoran@acm.org writes: > > Write (This.Link.all, > > Buffer (1 .. Stream_element_offset (Chunk))); > > Stream_element_array'Write (This.Link, > > Buffer (1 .. Stream_element_offset (Chunk))); > >... > > the Write procedure is taking no noticeable amount of time/cpu. > >In the contrary, the 'Write attribute takes like 3-4 seconds of 100% CPU > Perhaps Gnat's 'Write is doing one 'Write(Stream_Element) call for > each element of the Buffer, vs a single call for the whole thing in > the procedure version? Since you've already got a Stream_Element_Array, > perhaps you could call You hit the nail on the head. > Ada.Streams.Write(Ada.Streams.Root_Stream_Type'class(This.Link), > Buffer (1 .. Stream_element_offset (Chunk))); > directly? > Personally, I think using the default 'write is only reasonable > in the very simplest cases. If something is big, or complicated, > I'd write my own 'write for it. -- Ole-Hj. Kristensen ****************************************************************************** * You cannot consistently believe this sentence. ****************************************************************************** ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: 'Write attribute vs Write procedure. 2003-05-21 0:42 ` tmoran 2003-05-21 7:48 ` Ole-Hjalmar Kristensen @ 2003-05-21 9:23 ` Ole-Hjalmar Kristensen 2003-05-21 16:22 ` tmoran ` (2 more replies) 2003-05-23 6:09 ` Craig Carey 2 siblings, 3 replies; 26+ messages in thread From: Ole-Hjalmar Kristensen @ 2003-05-21 9:23 UTC (permalink / raw) tmoran@acm.org writes: > > Write (This.Link.all, > > Buffer (1 .. Stream_element_offset (Chunk))); > > Stream_element_array'Write (This.Link, > > Buffer (1 .. Stream_element_offset (Chunk))); > >... > > the Write procedure is taking no noticeable amount of time/cpu. > >In the contrary, the 'Write attribute takes like 3-4 seconds of 100% CPU > Perhaps Gnat's 'Write is doing one 'Write(Stream_Element) call for > each element of the Buffer, vs a single call for the whole thing in > the procedure version? Since you've already got a Stream_Element_Array, > perhaps you could call > Ada.Streams.Write(Ada.Streams.Root_Stream_Type'class(This.Link), > Buffer (1 .. Stream_element_offset (Chunk))); > directly? > Personally, I think using the default 'write is only reasonable > in the very simplest cases. If something is big, or complicated, > I'd write my own 'write for it. Yes, but that removes much of the attraction of using a stream in the first place. It should only be necessary to do your own write if you wanted another representation of it in the external format. A prime example of rendering a very nice feature of the language pretty much unusable because of a poor implementation. -- Ole-Hj. Kristensen ****************************************************************************** * Forth: You yourself foot in shoot ****************************************************************************** ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: 'Write attribute vs Write procedure. 2003-05-21 9:23 ` Ole-Hjalmar Kristensen @ 2003-05-21 16:22 ` tmoran 2003-05-22 7:51 ` Ole-Hjalmar Kristensen 2003-05-21 16:24 ` tmoran 2003-05-21 21:17 ` Simon Wright 2 siblings, 1 reply; 26+ messages in thread From: tmoran @ 2003-05-21 16:22 UTC (permalink / raw) >> Personally, I think using the default 'write is only reasonable >> in the very simplest cases. If something is big, or complicated, >> I'd write my own 'write for it. > >Yes, but that removes much of the attraction of using a stream in the >first place. It should only be necessary to do your own write if you >wanted another representation of it in the external format. A prime >example of rendering a very nice feature of the language pretty much >unusable because of a poor implementation. I disagree. It is convenient when you use it that you needn't distinguish between a 'write that will write in a canonical, vs a special external format. In the OP's case, it was probably simpler to get the program up and running with the standard 'write, rather than having to make his own, and then worry about doing something special as a matter of performance tuning. You also still have use at a higher level, eg type Sound_Bytes is record Time : Ada.Calendar.Time; Audio : Wave; Title : String(1 .. 10); end record; If you have a special Wave'write, you can say Sound_Bytes'write and it will do the standard thing for Time and Name but call your special routine for Audio, so you probably don't need to make your own Sound_Bytes'write. At it's own abstraction level, Sound_Bytes is neither big nor complicated, so you probably don't need a special 'write. ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: 'Write attribute vs Write procedure. 2003-05-21 16:22 ` tmoran @ 2003-05-22 7:51 ` Ole-Hjalmar Kristensen 0 siblings, 0 replies; 26+ messages in thread From: Ole-Hjalmar Kristensen @ 2003-05-22 7:51 UTC (permalink / raw) tmoran@acm.org writes: > >> Personally, I think using the default 'write is only reasonable > >> in the very simplest cases. If something is big, or complicated, > >> I'd write my own 'write for it. > > > >Yes, but that removes much of the attraction of using a stream in the > >first place. It should only be necessary to do your own write if you > >wanted another representation of it in the external format. A prime > >example of rendering a very nice feature of the language pretty much > >unusable because of a poor implementation. > > I disagree. It is convenient when you use it that you needn't > distinguish between a 'write that will write in a canonical, vs a special > external format. In the OP's case, it was probably simpler to get the > program up and running with the standard 'write, rather than having to > make his own, and then worry about doing something special as a matter of > performance tuning. Yes, but why should performance tuning be necessary for the simple cases, like a record without tags or discriminants? The compiler has all the information necessary to make an *efficient* write. I figure the answer is that if you do it the simple-minded way by calling the write procedure of every component, you don't have to do any special compile time analysis, just note that this type has its own write already defined, and don't generate one for it. But the opposite strategy should also be simple : if on of the components has a special write, do it by calling write on every component, else just transfer it as a blob. > > You also still have use at a higher level, eg > type Sound_Bytes is record > Time : Ada.Calendar.Time; > Audio : Wave; > Title : String(1 .. 10); > end record; > If you have a special Wave'write, you can say > Sound_Bytes'write > and it will do the standard thing for Time and Name but call your > special routine for Audio, so you probably don't need to make your > own Sound_Bytes'write. At it's own abstraction level, Sound_Bytes is > neither big nor complicated, so you probably don't need a special 'write. -- Ole-Hj. Kristensen ****************************************************************************** * You cannot consistently believe this sentence. ****************************************************************************** ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: 'Write attribute vs Write procedure. 2003-05-21 9:23 ` Ole-Hjalmar Kristensen 2003-05-21 16:22 ` tmoran @ 2003-05-21 16:24 ` tmoran 2003-05-21 21:17 ` Simon Wright 2 siblings, 0 replies; 26+ messages in thread From: tmoran @ 2003-05-21 16:24 UTC (permalink / raw) >> Personally, I think using the default 'write is only reasonable >> in the very simplest cases. If something is big, or complicated, >> I'd write my own 'write for it. > >Yes, but that removes much of the attraction of using a stream in the >first place. It should only be necessary to do your own write if you >wanted another representation of it in the external format. A prime >example of rendering a very nice feature of the language pretty much >unusable because of a poor implementation. I disagree. It is convenient when you use it that you needn't distinguish between a 'write that will write in a canonical, vs a special external format. In the OP's case, it was probably simpler to get the program up and running with the standard 'write, rather than having to make his own, and then worry about doing something special as a matter of performance tuning. You also still have use at a higher level, eg type Sound_Bytes is record Time : Ada.Calendar.Time; Audio : Wave; Title : String(1 .. 10); end record; If you have a special Wave'write, you can say Sound_Bytes'write and it will do the standard thing for Time and Name but call your special routine for Audio, so you probably don't need to make your own Sound_Bytes'write. At it's own abstraction level, Sound_Bytes is neither big nor complicated, so you probably don't need a special 'write. ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: 'Write attribute vs Write procedure. 2003-05-21 9:23 ` Ole-Hjalmar Kristensen 2003-05-21 16:22 ` tmoran 2003-05-21 16:24 ` tmoran @ 2003-05-21 21:17 ` Simon Wright 2003-05-22 8:02 ` Ole-Hjalmar Kristensen 2003-05-22 13:49 ` Marc A. Criley 2 siblings, 2 replies; 26+ messages in thread From: Simon Wright @ 2003-05-21 21:17 UTC (permalink / raw) Ole-Hjalmar Kristensen <oleh@vlinux.voxelvision.no> writes: > tmoran@acm.org writes: > > > > Write (This.Link.all, > > > Buffer (1 .. Stream_element_offset (Chunk))); > > > Stream_element_array'Write (This.Link, > > > Buffer (1 .. Stream_element_offset (Chunk))); > > >... > > > the Write procedure is taking no noticeable amount of time/cpu. > > >In the contrary, the 'Write attribute takes like 3-4 seconds of 100% CPU > > Perhaps Gnat's 'Write is doing one 'Write(Stream_Element) call for > > each element of the Buffer, vs a single call for the whole thing in > > the procedure version? Since you've already got a Stream_Element_Array, > > perhaps you could call > > Ada.Streams.Write(Ada.Streams.Root_Stream_Type'class(This.Link), > > Buffer (1 .. Stream_element_offset (Chunk))); > > directly? > > Personally, I think using the default 'write is only reasonable > > in the very simplest cases. If something is big, or complicated, > > I'd write my own 'write for it. > > Yes, but that removes much of the attraction of using a stream in the > first place. It should only be necessary to do your own write if you > wanted another representation of it in the external format. A prime > example of rendering a very nice feature of the language pretty much > unusable because of a poor implementation. ARM 13.13.2(9) says [...] For composite types, the Write or Read attribute for each component is called in canonical order, which is last dimension varying fastest for an array, and positional aggregate order for a record. Bounds are not included in the stream if T is an array type. [...] I'm not sure whether an implementation has licence to produce code which works "as if" this happened. What about the exceptional cases? (where there isn't enough room in the stream for all the elements, for example). ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: 'Write attribute vs Write procedure. 2003-05-21 21:17 ` Simon Wright @ 2003-05-22 8:02 ` Ole-Hjalmar Kristensen 2003-05-22 13:49 ` Marc A. Criley 1 sibling, 0 replies; 26+ messages in thread From: Ole-Hjalmar Kristensen @ 2003-05-22 8:02 UTC (permalink / raw) Simon Wright <simon@pushface.org> writes: > Ole-Hjalmar Kristensen <oleh@vlinux.voxelvision.no> writes: > > > tmoran@acm.org writes: > > > > > > Write (This.Link.all, > > > > Buffer (1 .. Stream_element_offset (Chunk))); > > > > Stream_element_array'Write (This.Link, > > > > Buffer (1 .. Stream_element_offset (Chunk))); > > > >... > > > > the Write procedure is taking no noticeable amount of time/cpu. > > > >In the contrary, the 'Write attribute takes like 3-4 seconds of 100% CPU > > > Perhaps Gnat's 'Write is doing one 'Write(Stream_Element) call for > > > each element of the Buffer, vs a single call for the whole thing in > > > the procedure version? Since you've already got a Stream_Element_Array, > > > perhaps you could call > > > Ada.Streams.Write(Ada.Streams.Root_Stream_Type'class(This.Link), > > > Buffer (1 .. Stream_element_offset (Chunk))); > > > directly? > > > Personally, I think using the default 'write is only reasonable > > > in the very simplest cases. If something is big, or complicated, > > > I'd write my own 'write for it. > > > > Yes, but that removes much of the attraction of using a stream in the > > first place. It should only be necessary to do your own write if you > > wanted another representation of it in the external format. A prime > > example of rendering a very nice feature of the language pretty much > > unusable because of a poor implementation. > > ARM 13.13.2(9) says > > [...] For composite types, the Write or Read attribute for each > component is called in canonical order, which is last dimension > varying fastest for an array, and positional aggregate order for a > record. Bounds are not included in the stream if T is an array > type. [...] > > I'm not sure whether an implementation has licence to produce code > which works "as if" this happened. What about the exceptional cases? > (where there isn't enough room in the stream for all the elements, for > example). Too bad. But I cannot see what stops the compiler to produce code which works "as if" this happened in the case where every read/write attribute is compiler generated. After all, the standard does not specify how the compiler should implement read and write for elementary types. -- Ole-Hj. Kristensen ****************************************************************************** * C: You shoot yourself in the foot * Ada: If you are actually dumb enough to program in this language, the DOD will * kidnap you, stand you up in front of a firing squad, and tell the soldiers: * "Shoot at his feet" ****************************************************************************** ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: 'Write attribute vs Write procedure. 2003-05-21 21:17 ` Simon Wright 2003-05-22 8:02 ` Ole-Hjalmar Kristensen @ 2003-05-22 13:49 ` Marc A. Criley 2003-05-22 20:07 ` Simon Wright 1 sibling, 1 reply; 26+ messages in thread From: Marc A. Criley @ 2003-05-22 13:49 UTC (permalink / raw) Simon Wright <simon@pushface.org> wrote in message news:<x7v65o4j8y4.fsf@smaug.pushface.org>... > I'm not sure whether an implementation has licence to produce code > which works "as if" this happened. What about the exceptional cases? > (where there isn't enough room in the stream for all the elements, for > example). According to Robert Dewar, compilers do have such "as if" license: "Remember that a compiler is always allowed to treat a language definition 'as-if', which means that any generated code that makes a compiler run the code the same way as some other way is equivalent. " Posted 9/11/97 in comp.lang.ada. (If you want to see the whole context of the post, do a Google groups search in comp.lang.ada with keywords: Dewar "as if" "no semantic effect" The thread is titled "The Red Language". I didn't include the URL because it's quite long and unwieldy. Marc A. Criley ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: 'Write attribute vs Write procedure. 2003-05-22 13:49 ` Marc A. Criley @ 2003-05-22 20:07 ` Simon Wright 0 siblings, 0 replies; 26+ messages in thread From: Simon Wright @ 2003-05-22 20:07 UTC (permalink / raw) mcq95@earthlink.net (Marc A. Criley) writes: > Simon Wright <simon@pushface.org> wrote in message news:<x7v65o4j8y4.fsf@smaug.pushface.org>... > > > I'm not sure whether an implementation has licence to produce code > > which works "as if" this happened. What about the exceptional cases? > > (where there isn't enough room in the stream for all the elements, for > > example). > > According to Robert Dewar, compilers do have such "as if" license: > > "Remember that a compiler is always allowed to treat a language > definition 'as-if', which means that any generated code that makes a > compiler run the code the same way as some other way is equivalent. " > Posted 9/11/97 in comp.lang.ada. > > (If you want to see the whole context of the post, do a Google groups > search in comp.lang.ada with keywords: Dewar "as if" "no semantic > effect" > The thread is titled "The Red Language". I didn't include the URL > because it's quite long and unwieldy. I meant to say, it isn't clear that a failure due to overrun when writing as a blob will have the same external effect as writing individual elements; and I'm not sure whether the "as if" permission includes "unless an exceptional condition occurs, in which case you are allowed to behave differently". ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: 'Write attribute vs Write procedure. 2003-05-21 0:42 ` tmoran 2003-05-21 7:48 ` Ole-Hjalmar Kristensen 2003-05-21 9:23 ` Ole-Hjalmar Kristensen @ 2003-05-23 6:09 ` Craig Carey 2003-05-23 19:55 ` Simon Wright 2 siblings, 1 reply; 26+ messages in thread From: Craig Carey @ 2003-05-23 6:09 UTC (permalink / raw) On Wed, 21 May 2003 00:42:33 GMT, tmoran@acm.org wrote: >> Write (This.Link.all, >> Buffer (1 .. Stream_element_offset (Chunk))); >> Stream_element_array'Write (This.Link, >> Buffer (1 .. Stream_element_offset (Chunk))); >>... >> the Write procedure is taking no noticeable amount of time/cpu. >>In the contrary, the 'Write attribute takes like 3-4 seconds of 100% CPU > Perhaps Gnat's 'Write is doing one 'Write(Stream_Element) call for >each element of the Buffer, vs a single call for the whole thing in >the procedure version? Since you've already got a Stream_Element_Array, >perhaps you could call > Ada.Streams.Write(Ada.Streams.Root_Stream_Type'class(This.Link), > Buffer (1 .. Stream_element_offset (Chunk))); >directly? > Personally, I think using the default 'write is only reasonable >in the very simplest cases. [...] X'Write() would have to come out best in a comparison involving at least two choices. Some other principle, presumably something 'unreasonable', shows up when there is not much Ada 95 code. -- PS. GNAT is allocating tracking numbers for requests on GNAT.Sockets, including for requests for big rewrites. ACT might not be planning on rewriting the package (I don't know). I got this subject header back: "[C516-004 public] Programming style of exception raising in GNAT.Sockets, etc." So similarly, GNAT.Sockets could be altered so that the comments in it suggesting use of Ada 95's X'Read feature, are deleted or discouraged. X'Read() has no parameter for diagnostics on what the malfunction was. The language can be revised so that X'Read is deleted or else it returns error information out directly so that it is available to very next line in the source code. If Ada becomes a language where malfunctioning programs don't know why, and are really cryptic in describing the precise problem (e.g. since code in exception handlers tries to handle too many errors), then it could be all made reasonable (insufficiently reasonable) by the new style of new unneeded code that seems to do ntohing to a single byte but just lose the error info so that the next line can't get it (it has to be the line after if the code is in an exception handler). However ACT is saying that customers do not seek big changes in GNAT.Sockets. Craig Carey http://www.ijs.co.nz/ada_95.htm <- mailing lists ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: 'Write attribute vs Write procedure. 2003-05-23 6:09 ` Craig Carey @ 2003-05-23 19:55 ` Simon Wright 2003-05-23 20:48 ` AI-248 and counting the elements in an external file Larry Kilgallen 0 siblings, 1 reply; 26+ messages in thread From: Simon Wright @ 2003-05-23 19:55 UTC (permalink / raw) Craig Carey <research@ijs.co.nz> writes: > So similarly, GNAT.Sockets could be altered so that the comments in it > suggesting use of Ada 95's X'Read feature, are deleted or discouraged. > > X'Read() has no parameter for diagnostics on what the malfunction was. > The language can be revised so that X'Read is deleted or else it returns > error information out directly so that it is available to very next > line in the source code. If Ada becomes a language where malfunctioning > programs don't know why, and are really cryptic in describing the > precise problem (e.g. since code in exception handlers tries to handle too > many errors), then it could be all made reasonable (insufficiently > reasonable) by the new style of new unneeded code that seems to do > ntohing to a single byte but just lose the error info so that the next > line can't get it (it has to be the line after if the code is in an > exception handler). I don't understand this .. A "simple" solution to your problem (not to mine!) might be to have a package -- _not_ a revised GNAT.Sockets, please! -- which doesn't deal with Streams, but with something else. You still have the option to (for example) encapsulate the data you're actually interested in in a "datagram" preceded by length information, so you can read the "datagram" into a Stream_Element_Array and then internally stream from that (this is the approach we are adopting for UDP sockets). > However ACT is saying that customers do not seek big changes in > GNAT.Sockets. There is certainly a problem with reading classwide objects from UDP sockets. ^ permalink raw reply [flat|nested] 26+ messages in thread
* AI-248 and counting the elements in an external file 2003-05-23 19:55 ` Simon Wright @ 2003-05-23 20:48 ` Larry Kilgallen 2003-05-23 21:12 ` Stephen Leake 0 siblings, 1 reply; 26+ messages in thread From: Larry Kilgallen @ 2003-05-23 20:48 UTC (permalink / raw) I suppose my lack of experience with Ada95 streams ('read, etc.) may be the reason for my confusion, but in reading http://www.ada-auth.org/cgi-bin/cvsweb.cgi/AIs/AI-00248.TXT?rev=1.16 I see in several places the text: The size of an external file is the number of stream elements contained in the file. That is straightforward for a file of some record type, or a file of integers, but what if the file is one created with Ada95 stream semantics ? I thought that 'write and write() relied upon the reading program to know "how" to read the file, and that implementations would not have kept a tally of how many elements were in the file. I don't know enough about this to know if it is possible to do a linear search of a file to count the elements, but even if it were, that would be a very slow operation. Is it envisioned by the authors of AI-248 that implementations would preserve metadata containing the element count ? If so, what about stream files that were written under Ada95 ? Or can someone suggest what it is that I don't understand about Ada streams ? ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: AI-248 and counting the elements in an external file 2003-05-23 20:48 ` AI-248 and counting the elements in an external file Larry Kilgallen @ 2003-05-23 21:12 ` Stephen Leake 2003-05-23 21:45 ` Randy Brukardt 2003-05-24 1:42 ` Larry Kilgallen 0 siblings, 2 replies; 26+ messages in thread From: Stephen Leake @ 2003-05-23 21:12 UTC (permalink / raw) Kilgallen@SpamCop.net (Larry Kilgallen) writes: > I suppose my lack of experience with Ada95 streams ('read, etc.) > may be the reason for my confusion, but in reading > > http://www.ada-auth.org/cgi-bin/cvsweb.cgi/AIs/AI-00248.TXT?rev=1.16 > > I see in several places the text: > > The size of an external file is the number of > stream elements contained in the file. a "stream element" is just a "byte"; it's Ada-ese for "smallest writeable chunk". On all "normal" OS's, it's 8 bits. I think perhaps you are thinking of "stream element" as some higher-level object? > That is straightforward for a file of some record type, or a file of > integers, but what if the file is one created with Ada95 stream > semantics ? It makes no difference. All of these types must be converted to raw stream elements (bytes) before they can be written to the file. That's the nature of files! > I thought that 'write and write() relied upon the reading program to > know "how" to read the file, and that implementations would not have > kept a tally of how many elements were in the file. The OS certainly has to know how big the file is; that's all that matters here. > I don't know enough about this to know if it is possible to do a > linear search of a file to count the elements, but even if it were, > that would be a very slow operation. Yes, but the OS keeps track. > Is it envisioned by the authors of AI-248 that implementations would > preserve metadata containing the element count ? Only if the underlying OS doesn't; in which case this whole package makes no sense anyway. > If so, what about stream files that were written under Ada95 ? I don't see why you think stream files are different from other files, as far as this package is concerned. > Or can someone suggest what it is that I don't understand about Ada > streams ? -- -- Stephe ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: AI-248 and counting the elements in an external file 2003-05-23 21:12 ` Stephen Leake @ 2003-05-23 21:45 ` Randy Brukardt 2003-05-24 1:45 ` Larry Kilgallen 2003-05-24 1:42 ` Larry Kilgallen 1 sibling, 1 reply; 26+ messages in thread From: Randy Brukardt @ 2003-05-23 21:45 UTC (permalink / raw) Stephen Leake wrote in message ... >Kilgallen@SpamCop.net (Larry Kilgallen) writes: > >> I suppose my lack of experience with Ada95 streams ('read, etc.) >> may be the reason for my confusion, but in reading >> >> http://www.ada-auth.org/cgi-bin/cvsweb.cgi/AIs/AI-00248.TXT?rev=1.16 >> >> I see in several places the text: >> >> The size of an external file is the number of >> stream elements contained in the file. > >a "stream element" is just a "byte"; it's Ada-ese for "smallest >writeable chunk". On all "normal" OS's, it's 8 bits. > >I think perhaps you are thinking of "stream element" as some >higher-level object? > >> That is straightforward for a file of some record type, or a file of >> integers, but what if the file is one created with Ada95 stream >> semantics ? > >It makes no difference. All of these types must be converted to raw >stream elements (bytes) before they can be written to the file. That's >the nature of files! Right, the intent is that this value is whatever the OS returns for the file size. That's true for all of the results of this package (with a few exceptions). There is no attempt to tie the file size to any other Ada semantics (that would be a mess). Randy. ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: AI-248 and counting the elements in an external file 2003-05-23 21:45 ` Randy Brukardt @ 2003-05-24 1:45 ` Larry Kilgallen 2003-05-24 22:00 ` Robert I. Eachus 0 siblings, 1 reply; 26+ messages in thread From: Larry Kilgallen @ 2003-05-24 1:45 UTC (permalink / raw) In article <vct5imh4199m7f@corp.supernews.com>, "Randy Brukardt" <randy@rrsoftware.com> writes: > Stephen Leake wrote in message ... >>Kilgallen@SpamCop.net (Larry Kilgallen) writes: >>> That is straightforward for a file of some record type, or a file of >>> integers, but what if the file is one created with Ada95 stream >>> semantics ? >> >>It makes no difference. All of these types must be converted to raw >>stream elements (bytes) before they can be written to the file. That's >>the nature of files! > > > Right, the intent is that this value is whatever the OS returns for the > file size. That's true for all of the results of this package (with a > few exceptions). There is no attempt to tie the file size to any other > Ada semantics (that would be a mess). Meaning it is permissible to return the number of 512-byte disk blocks (or file-specific-size tape blocks) for this function on VMS ? Or is the answer supposed to be normalized to bytes even though the number of overhead bytes required (and thus the file size) will vary, for instance, when copying a file from disk to tape ? ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: AI-248 and counting the elements in an external file 2003-05-24 1:45 ` Larry Kilgallen @ 2003-05-24 22:00 ` Robert I. Eachus 2003-05-25 3:54 ` Larry Kilgallen 0 siblings, 1 reply; 26+ messages in thread From: Robert I. Eachus @ 2003-05-24 22:00 UTC (permalink / raw) Larry Kilgallen wrote: > Meaning it is permissible to return the number of 512-byte disk blocks > (or file-specific-size tape blocks) for this function on VMS ? Or is the > answer supposed to be normalized to bytes even though the number of > overhead bytes required (and thus the file size) will vary, for instance, > when copying a file from disk to tape ? Let me answer differently. VMS supports many file formats. Ada stream files would map nicely to VMS record oriented ANSI tape files. There are other VMS formats that I would not tend to support stream files for. You could do the blocking and unblocking in the file system, but why not use the OS to do what it does best, and only try to read and write file types that support stream semantics as stream files. What if the program needs one type of file, and you have the data in another? Write a program to read the file in and write it in as the necessary file type. Won't be the first time I had to do that. I remember one case where I had to distribute a data file to several projects. One file format covered all the Unix systems, but I needed three different formats for the three groups with VAXen. One of them actually overrode the default block size with 4K. (To save tape.) Argh! ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: AI-248 and counting the elements in an external file 2003-05-24 22:00 ` Robert I. Eachus @ 2003-05-25 3:54 ` Larry Kilgallen 2003-05-27 20:19 ` Randy Brukardt 0 siblings, 1 reply; 26+ messages in thread From: Larry Kilgallen @ 2003-05-25 3:54 UTC (permalink / raw) In article <3ECFEB59.4010401@attbi.com>, "Robert I. Eachus" <rieachus@attbi.com> writes: > Larry Kilgallen wrote: > >> Meaning it is permissible to return the number of 512-byte disk blocks >> (or file-specific-size tape blocks) for this function on VMS ? Or is the >> answer supposed to be normalized to bytes even though the number of >> overhead bytes required (and thus the file size) will vary, for instance, >> when copying a file from disk to tape ? > > Let me answer differently. VMS supports many file formats. Ada stream > files would map nicely to VMS record oriented ANSI tape files. There > are other VMS formats that I would not tend to support stream files for. But my understanding is that the Ada.Directory functions are to support all files that might be accessed from Ada, and that is not limited to stream files, nor to files written by Ada. ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: AI-248 and counting the elements in an external file 2003-05-25 3:54 ` Larry Kilgallen @ 2003-05-27 20:19 ` Randy Brukardt 2003-05-27 20:57 ` Larry Kilgallen 0 siblings, 1 reply; 26+ messages in thread From: Randy Brukardt @ 2003-05-27 20:19 UTC (permalink / raw) Larry Kilgallen wrote in message ... >In article <3ECFEB59.4010401@attbi.com>, "Robert I. Eachus" <rieachus@attbi.com> writes: >> Larry Kilgallen wrote: >> >>> Meaning it is permissible to return the number of 512-byte disk blocks >>> (or file-specific-size tape blocks) for this function on VMS ? Or is the >>> answer supposed to be normalized to bytes even though the number of >>> overhead bytes required (and thus the file size) will vary, for instance, >>> when copying a file from disk to tape ? >> >> Let me answer differently. VMS supports many file formats. Ada stream >> files would map nicely to VMS record oriented ANSI tape files. There >> are other VMS formats that I would not tend to support stream files for. > >But my understanding is that the Ada.Directory functions are to support >all files that might be accessed from Ada, and that is not limited to >stream files, nor to files written by Ada. The intent is that it be well-defined (which might include raising Use_Error if if cannot be figured out, as for a keyboard) for files that can be read or written by Ada. If Ada doesn't understand the files at all, then we're not trying to say anything. I would expect that this functionality would take more work on some OSes than it does on Windows or Unix. On the original CP/M, for instance, you had to figure out the file size by finding out the block size and multiplying it by the number of blocks. That's fine, and keep in mind that this result really is intended to report the physical size of the file, which may or may not be directly relatable to the 'logical' size returned by Direct_IO.Size and Stream_IO.Size. Randy. ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: AI-248 and counting the elements in an external file 2003-05-27 20:19 ` Randy Brukardt @ 2003-05-27 20:57 ` Larry Kilgallen 2003-05-28 18:26 ` Randy Brukardt 0 siblings, 1 reply; 26+ messages in thread From: Larry Kilgallen @ 2003-05-27 20:57 UTC (permalink / raw) In article <vd7hvtd5dkqr17@corp.supernews.com>, "Randy Brukardt" <randy@rrsoftware.com> writes: > Larry Kilgallen wrote in message ... >>In article <3ECFEB59.4010401@attbi.com>, "Robert I. Eachus" > <rieachus@attbi.com> writes: >>> Larry Kilgallen wrote: >>> >>>> Meaning it is permissible to return the number of 512-byte disk > blocks >>>> (or file-specific-size tape blocks) for this function on VMS ? Or > is the >>>> answer supposed to be normalized to bytes even though the number of >>>> overhead bytes required (and thus the file size) will vary, for > instance, >>>> when copying a file from disk to tape ? > The intent is that it be well-defined (which might include raising > Use_Error if if cannot be figured out, as for a keyboard) for files that > can be read or written by Ada. If Ada doesn't understand the files at > all, then we're not trying to say anything. > > I would expect that this functionality would take more work on some OSes > than it does on Windows or Unix. On the original CP/M, for instance, you > had to figure out the file size by finding out the block size and > multiplying it by the number of blocks. That's fine, and keep in mind > that this result really is intended to report the physical size of the > file, The idea of "physical size -- does not have to accurately represent size of data contained" was not at all clear to me reading AI-248. Is that stated in there, or are there plans for an AI- about AI-248 ? ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: AI-248 and counting the elements in an external file 2003-05-27 20:57 ` Larry Kilgallen @ 2003-05-28 18:26 ` Randy Brukardt 0 siblings, 0 replies; 26+ messages in thread From: Randy Brukardt @ 2003-05-28 18:26 UTC (permalink / raw) Larry Kilgallen wrote in message <$ghCR$mci5sd@eisner.encompasserve.org>... >In article <vd7hvtd5dkqr17@corp.supernews.com>, "Randy Brukardt" <randy@rrsoftware.com> writes: > >> The intent is that it be well-defined (which might include raising >> Use_Error if if cannot be figured out, as for a keyboard) for files that >> can be read or written by Ada. If Ada doesn't understand the files at >> all, then we're not trying to say anything. >> >> I would expect that this functionality would take more work on some OSes >> than it does on Windows or Unix. On the original CP/M, for instance, you >> had to figure out the file size by finding out the block size and >> multiplying it by the number of blocks. That's fine, and keep in mind >> that this result really is intended to report the physical size of the >> file, > >The idea of "physical size -- does not have to accurately represent >size of data contained" was not at all clear to me reading AI-248. >Is that stated in there, or are there plans for an AI- about AI-248 ? Why would you expect otherwise? The 'size in Storage_Elements' for a Direct_IO file certainly doesn't have to be the number of records (Count) * the size of a record, even though it is often implemented that way. There may be control information, or (like on the CP/M systems I mentioned previously), there may be some rounding up. Certainly the standard has nothing to say about this, and Ada.Directories is not intended to change anything in this area. It was carefully written with the intent of not adding any unintended requirements about file representation, because that would be incompatible, and for no good reason. Anyway, AI-248 appears to require that Ada.Directories.Size = Ada.Streams.Stream_IO.Size for a stream file, but that ought to not have any implementation burden. But for any file type that is not a stream file (that is, could not be Created by Stream_IO.Open) (or is a non-positionable stream file), the meaning of Ada.Directories.Size is essentially implementation defined, because the standard says nothing about the representation of Sequential_IO, Text_IO, and Direct_IO files, and thus the maping to stream elements is not defined by the standard. You have to remember that the wording part of an AI is written to be part of the standard. That means if it doesn't say something, that means that the interpretation is being left to the implementor. It's not practical to put every possible nuance into an AI; indeed, when I've done that in the past, I've generally been asked to remove such discussions as clutter. Too much clutter makes an AI hard to read and digest. If you have a specific suggestion for an improvement, I'd be happy to hear it. (Even better, submit it to Ada-Comment to put it into the official record.) Randy Brukardt ARG Editor ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: AI-248 and counting the elements in an external file 2003-05-23 21:12 ` Stephen Leake 2003-05-23 21:45 ` Randy Brukardt @ 2003-05-24 1:42 ` Larry Kilgallen 2003-05-28 21:17 ` Brian Gaffney 1 sibling, 1 reply; 26+ messages in thread From: Larry Kilgallen @ 2003-05-24 1:42 UTC (permalink / raw) In article <u4r3l5pvw.fsf@nasa.gov>, Stephen Leake <Stephe.Leake@nasa.gov> writes: > Kilgallen@SpamCop.net (Larry Kilgallen) writes: > >> I suppose my lack of experience with Ada95 streams ('read, etc.) >> may be the reason for my confusion, but in reading >> >> http://www.ada-auth.org/cgi-bin/cvsweb.cgi/AIs/AI-00248.TXT?rev=1.16 >> >> I see in several places the text: >> >> The size of an external file is the number of >> stream elements contained in the file. > > a "stream element" is just a "byte"; it's Ada-ese for "smallest > writeable chunk". On all "normal" OS's, it's 8 bits. > > I think perhaps you are thinking of "stream element" as some > higher-level object? Yes, that was my problem. >> I don't know enough about this to know if it is possible to do a >> linear search of a file to count the elements, but even if it were, >> that would be a very slow operation. > > Yes, but the OS keeps track. On VMS that would be true if one writes a fixed-length record file, or a stream-lf, stream-cr or stream-crlf file, but not if one is writing a (VMS default) variable length record file. > >> Is it envisioned by the authors of AI-248 that implementations would >> preserve metadata containing the element count ? > > Only if the underlying OS doesn't; in which case this whole package > makes no sense anyway. In DEC Ada83, there is no function to return "bytes in a file", and certainly an Ada program reading a normal VMS file would not know down to the byte level how large a file is if it had been written by a program from another language. File length statistics on VMS are kept in disk blocks, and the number of actual data bytes in standard (variable length record) files will vary according to the number of records, whether they have odd or even lengths, and whether they evenly fill an integral number of blocks. Does anyone reading this use GNAT Ada95 on VMS, and know whether they have dealt with any of this (I thought they had some GNAT-specific directory functions). ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: AI-248 and counting the elements in an external file 2003-05-24 1:42 ` Larry Kilgallen @ 2003-05-28 21:17 ` Brian Gaffney 2003-05-28 22:06 ` Larry Kilgallen 0 siblings, 1 reply; 26+ messages in thread From: Brian Gaffney @ 2003-05-28 21:17 UTC (permalink / raw) Kilgallen@SpamCop.net (Larry Kilgallen) wrote in message news:<X5uMu8X9J4P6@eisner.encompasserve.org>... > In article <u4r3l5pvw.fsf@nasa.gov>, Stephen Leake <Stephe.Leake@nasa.gov> writes: > > Kilgallen@SpamCop.net (Larry Kilgallen) writes: > > > >> I suppose my lack of experience with Ada95 streams ('read, etc.) > >> may be the reason for my confusion, but in reading > >> > >> http://www.ada-auth.org/cgi-bin/cvsweb.cgi/AIs/AI-00248.TXT?rev=1.16 > >> > >> I see in several places the text: > >> > >> The size of an external file is the number of > >> stream elements contained in the file. > > > > a "stream element" is just a "byte"; it's Ada-ese for "smallest > > writeable chunk". On all "normal" OS's, it's 8 bits. > > > > I think perhaps you are thinking of "stream element" as some > > higher-level object? > > Yes, that was my problem. > > >> I don't know enough about this to know if it is possible to do a > >> linear search of a file to count the elements, but even if it were, > >> that would be a very slow operation. > > > > Yes, but the OS keeps track. > > On VMS that would be true if one writes a fixed-length record file, > or a stream-lf, stream-cr or stream-crlf file, but not if one is > writing a (VMS default) variable length record file. > > > >> Is it envisioned by the authors of AI-248 that implementations would > >> preserve metadata containing the element count ? > > > > Only if the underlying OS doesn't; in which case this whole package > > makes no sense anyway. > > In DEC Ada83, there is no function to return "bytes in a file", > and certainly an Ada program reading a normal VMS file would not > know down to the byte level how large a file is if it had been > written by a program from another language. > > File length statistics on VMS are kept in disk blocks, and the number > of actual data bytes in standard (variable length record) files will > vary according to the number of records, whether they have odd or even > lengths, and whether they evenly fill an integral number of blocks. > My suggestion for VMS would be for File_Size to represent the size in blocks, since that is what the VMS user would be used to working with. This would imply File_Size would have an implementation-defined unit as well as range. (BTW why is stream elements used as the size definition, shouldn't it be whatever is natural to the implementation/OS enviroment?) (Or should a stream element on VMS be a block?:-) File_Size isn't the only issue VMS would have with this package. What would an implementation do about Logicals in returning a Full_Name? Depending on how the Logical is defined, sometimes they are translated to give the full path, and sometimes they are not (i.e. used like the drive letter in DOS). The user may never see the true "full paths", and the system administrator may have it set up that way intentionally. Also, Set_Directory raises an exception if the target doesn't exist. This is not usual in VMS (from a user's perspective). I can SET DEF to a nonexistant directory, and only get an error if I try to access it. (I have found this to be a feature, since I can still log in if the drive my home directory is on has crashed.) VMS is not the only enviroment where this package, as written, would cause problems. For example, the Discussion for the function Compose states that a null Name is not valid, however a file with no Name and an Extension (as defined in the AI), is allowed in DOS/Windows and is actually a 'feature' in Linux (and Unix?). --Brian ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: AI-248 and counting the elements in an external file 2003-05-28 21:17 ` Brian Gaffney @ 2003-05-28 22:06 ` Larry Kilgallen 0 siblings, 0 replies; 26+ messages in thread From: Larry Kilgallen @ 2003-05-28 22:06 UTC (permalink / raw) In article <5e9b8c34.0305281317.3a6eb6a4@posting.google.com>, Brian.Gaffney@myrealbox.com (Brian Gaffney) writes: > My suggestion for VMS would be for File_Size to represent the size in > blocks, since that is what the VMS user would be used to working with. But the last I read from Randy in this topic indicated to me the expectation was to normalize to bytes. > File_Size isn't the only issue VMS would have with this package. What > would an implementation do about Logicals in returning a Full_Name? > Depending on how the Logical is defined, sometimes they are translated > to give the full path, and sometimes they are not (i.e. used like the > drive letter in DOS). The user may never see the true "full paths", > and the system administrator may have it set up that way > intentionally. That part seemed clear to me in the VMS context, depending on the setting of the "concealed" flag on the logical names. > Also, Set_Directory raises an exception if the target doesn't exist. > This is not usual in VMS (from a user's perspective). I can SET DEF to > a nonexistant directory, and only get an error if I try to access it. But it certainly is easy to program in the Ada package to make it obey the specification. > (I have found this to be a feature, since I can still log in if the > drive my home directory is on has crashed.) If one were to rewrite LOGINOUT in Ada :-), it could likewise be written to ignore that exception. ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: 'Write attribute vs Write procedure. 2003-05-21 0:07 'Write attribute vs Write procedure Jano 2003-05-21 0:42 ` tmoran @ 2003-05-24 14:11 ` Craig Carey 1 sibling, 0 replies; 26+ messages in thread From: Craig Carey @ 2003-05-24 14:11 UTC (permalink / raw) On Wed, 21 May 2003 02:07:05 +0200, Jano <nono@celes.unizar.es> wrote: >Hi everybody. > >I'm getting abysmal differences in speed when writing large chunks (64k) >of data to a stream. The stream is from a socket of Gnat.Sockets, I >mention it because I don't know if it has something to do. > ... >In the contrary, the 'Write attribute takes like 3-4 seconds of 100% CPU >usage, shared more or less equally by my program and the SYSTEM process, ... >P.s2: I've tried also with strings, only the 'Write version of course, >and results are the same. I guess that the ACT vendor designed that part of their Ada compiler to be intolerably inefficient. It would not affect authors of programs written in a fairly good style. That speculation could be tested by: (1) finding out where ACT could have speeded up their X'Write() for TCP data, by adding a "pragma Suppress()" pragma statement. (2) Then that information would be reported in the form of a bug report. (3) Then the speculation is suggested to be incorrect if a tracking number comes back in the tailored response if any. If the text in the GNAT.Sockets package caused a loss of time in doing timing tests, then a bug report about the comments in the package could be sent in to ACT. If the topic is about TCP Ada streams then it probably about extremely simple programs that might be much modified later. Craig Carey Bindings to GNAT.Sockets: http://www.ijs.co.nz/code/ada95_http_hdrs.zip ^ permalink raw reply [flat|nested] 26+ messages in thread
end of thread, other threads:[~2003-05-28 22:06 UTC | newest] Thread overview: 26+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2003-05-21 0:07 'Write attribute vs Write procedure Jano 2003-05-21 0:42 ` tmoran 2003-05-21 7:48 ` Ole-Hjalmar Kristensen 2003-05-21 9:23 ` Ole-Hjalmar Kristensen 2003-05-21 16:22 ` tmoran 2003-05-22 7:51 ` Ole-Hjalmar Kristensen 2003-05-21 16:24 ` tmoran 2003-05-21 21:17 ` Simon Wright 2003-05-22 8:02 ` Ole-Hjalmar Kristensen 2003-05-22 13:49 ` Marc A. Criley 2003-05-22 20:07 ` Simon Wright 2003-05-23 6:09 ` Craig Carey 2003-05-23 19:55 ` Simon Wright 2003-05-23 20:48 ` AI-248 and counting the elements in an external file Larry Kilgallen 2003-05-23 21:12 ` Stephen Leake 2003-05-23 21:45 ` Randy Brukardt 2003-05-24 1:45 ` Larry Kilgallen 2003-05-24 22:00 ` Robert I. Eachus 2003-05-25 3:54 ` Larry Kilgallen 2003-05-27 20:19 ` Randy Brukardt 2003-05-27 20:57 ` Larry Kilgallen 2003-05-28 18:26 ` Randy Brukardt 2003-05-24 1:42 ` Larry Kilgallen 2003-05-28 21:17 ` Brian Gaffney 2003-05-28 22:06 ` Larry Kilgallen 2003-05-24 14:11 ` 'Write attribute vs Write procedure Craig Carey
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox