* Required Metrics @ 2000-04-29 0:00 Ken Garlington 2000-04-29 0:00 ` swhalen 2000-05-01 0:00 ` Required Metrics Ted Dennison 0 siblings, 2 replies; 68+ messages in thread From: Ken Garlington @ 2000-04-29 0:00 UTC (permalink / raw) In the Ada Reference Manual, paragraphs D.8:37-45 states that "The implementation shall document" a series of metrics related to the real-time clock, etc. paragraph D:2 states that "The metrics are documentation requirements; an implementation shall document the values of the language-defined metrics for at least one configuration of hardware or an underlying system supported by the implementation, and shall document the details of that configuration." However, although my compiler's reference manual claims that "The real-time systems annex is fully implemented", it goes on to say " Information on metrics is not yet available." Does a compiler fully implement the real-time systems annex if it does not meet the documentation requirements? ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Required Metrics 2000-04-29 0:00 Required Metrics Ken Garlington @ 2000-04-29 0:00 ` swhalen 2000-05-01 0:00 ` Required Metrics (GNAT et al) Ken Garlington 2000-05-01 0:00 ` Required Metrics Ted Dennison 1 sibling, 1 reply; 68+ messages in thread From: swhalen @ 2000-04-29 0:00 UTC (permalink / raw) Ken Garlington <Ken.Garlington@computer.org> wrote: : In the Ada Reference Manual, paragraphs D.8:37-45 states that "The : implementation shall document" a series of metrics related to the real-time : clock, etc. paragraph D:2 states that "The metrics are documentation : requirements; an implementation shall document the values of the : language-defined metrics for at least one configuration of hardware or an : underlying system supported by the implementation, and shall document the : details of that configuration." However, although my compiler's reference : manual claims that "The real-time systems annex is fully implemented", it : goes on to say " Information on metrics is not yet available." Does a : compiler fully implement the real-time systems annex if it does not meet the : documentation requirements? I'd say you don't know if it meets the documentation requirements unless you have the documentation for _ALL_ of that vendor's "implementations". I'd expect that for GNAT for example, that the hardware implementation they "documented" would be one besides the two I have documentation for (WinNT & Linux). I'd expect the metrics documentation to be for a more "real-time" hardware / OS combination. And I would expect to see the documentation in the distribution for that platform, not distributed with all platforms from that vendor. I'm also not sure I'd expect so see the metrics with the documentation on a non-validated compiler (I fool around with the public versions of GNAT mainly). But then I'm no language lawyer, so my expectations are probably meaningless <g>. Steve -- {===--------------------------------------------------------------===} Steve Whalen swhalen@netcom.com {===--------------------------------------------------------------===} ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Required Metrics (GNAT et al) 2000-04-29 0:00 ` swhalen @ 2000-05-01 0:00 ` Ken Garlington 2000-05-01 0:00 ` swhalen 0 siblings, 1 reply; 68+ messages in thread From: Ken Garlington @ 2000-05-01 0:00 UTC (permalink / raw) <swhalen@netcom.com> wrote in message news:8efg68$44u$1@slb1.atl.mindspring.net... > Ken Garlington <Ken.Garlington@computer.org> wrote: > : In the Ada Reference Manual, paragraphs D.8:37-45 states that "The > : implementation shall document" a series of metrics related to the real-time > : clock, etc. paragraph D:2 states that "The metrics are documentation > : requirements; an implementation shall document the values of the > : language-defined metrics for at least one configuration of hardware or an > : underlying system supported by the implementation, and shall document the > : details of that configuration." However, although my compiler's reference > : manual claims that "The real-time systems annex is fully implemented", it > : goes on to say " Information on metrics is not yet available." Does a > : compiler fully implement the real-time systems annex if it does not meet the > : documentation requirements? > > I'd say you don't know if it meets the documentation requirements > unless you have the documentation for _ALL_ of that vendor's > "implementations". Hmmm... that's not how I'd parse that sentence. It appears to me that all implementations have to have documentation, but that each implementation doesn't have to be documented. For example, the documentation for GNAT might not describe how it works on WinNT or Linux, but if there were metrics available for VxWorks, I'd expect to see those VxWorks metrics in the documentation for _all_ implementations to at least technically meet the requirements? > I'd expect that for GNAT for example, that the hardware implementation > they "documented" would be one besides the two I have documentation > for (WinNT & Linux). OK - Does anyone know if that's how it works, for GNAT or anyone else who claims conformance to the annex? > I'd expect the metrics documentation to be for a more "real-time" > hardware / OS combination. It seems to me that, if the reason the annex isn't fully implemented is because the particular platform isn't considered "real time", that no vendor should claim conformance on that platform. It not like these metrics are just being provided as a marketing tool; we're talking about requirements from the standard. > I'm also not sure I'd expect so see the metrics with the > documentation on a non-validated compiler (I fool > around with the public versions of GNAT mainly). OK - Does anyone know if the metrics are provided only to supported customers (GNAT or otherwise)? ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Required Metrics (GNAT et al) 2000-05-01 0:00 ` Required Metrics (GNAT et al) Ken Garlington @ 2000-05-01 0:00 ` swhalen 2000-05-01 0:00 ` Ken Garlington 0 siblings, 1 reply; 68+ messages in thread From: swhalen @ 2000-05-01 0:00 UTC (permalink / raw) I'm not sure it would be practical to include, say, the VxWorks metrics with the WinNT compiler, because there would be too many references to VxWorks that would be easy to explain in the context of the VxWorks documentation, but would make no sense in the WinNT docs. The more I think about this, the more I'm convinced it would be wrong for a vendor to ship metrics with anything other than a validated compiler. The metrics would have to be very hardware / OS / compiler dependent. Metrics should be reproducible, but they would be reproducible only on the platform that was validated. Steve -- {===--------------------------------------------------------------===} Steve Whalen swhalen@netcom.com {===--------------------------------------------------------------===} ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Required Metrics (GNAT et al) 2000-05-01 0:00 ` swhalen @ 2000-05-01 0:00 ` Ken Garlington 0 siblings, 0 replies; 68+ messages in thread From: Ken Garlington @ 2000-05-01 0:00 UTC (permalink / raw) <swhalen@netcom.com> wrote in message news:8eksr5$kf8$1@slb1.atl.mindspring.net... > I'm not sure it would be practical to include, say, the VxWorks > metrics with the WinNT compiler, because there would be too many > references to VxWorks that would be easy to explain in the context of > the VxWorks documentation, but would make no sense in the WinNT docs. > > Metrics should be reproducible, but they would be reproducible only on > the platform that was validated. > > The metrics would have to be very hardware / OS / compiler > dependent. Well, I don't think there's any requirement to provide the _entire_ documentation suite, just the _metrics_ (formula, whatever) from the platform of choice. Using GNAT as an example, the documentation already defines some features that really only make sense on specific platforms (e.g. certain DEC Ada "emulations"), so there's certainly some precedent here. The standard already appears to recognize that the metrics depend upon context, so that shouldn't be an issue. > The more I think about this, the more I'm convinced it would be wrong > for a vendor to ship metrics with anything other than a validated > compiler. OK - does ACT do this? I wasn't aware you got different documentation for the supported version versus the public version. ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Required Metrics 2000-04-29 0:00 Required Metrics Ken Garlington 2000-04-29 0:00 ` swhalen @ 2000-05-01 0:00 ` Ted Dennison 2000-05-01 0:00 ` Tucker Taft 2000-05-01 0:00 ` Ken Garlington 1 sibling, 2 replies; 68+ messages in thread From: Ted Dennison @ 2000-05-01 0:00 UTC (permalink / raw) In article <5DDO4.2237$wb7.194854@news.flash.net>, "Ken Garlington" <Ken.Garlington@computer.org> wrote: > In the Ada Reference Manual, paragraphs D.8:37-45 states that "The > implementation shall document" a series of metrics related to the ... > details of that configuration." However, although my compiler's > reference manual claims that "The real-time systems annex is fully > implemented", it goes on to say " Information on metrics is not yet > available." Does a compiler fully implement the real-time systems > annex if it does not meet the documentation requirements? I remember asking almost this exact question here about a year and a half ago. You can search deja for the thread. But the answer I remember getting was that since there is (and can be) no test in the Ada validation test suite for documentation requirements, implementors can pretty much get away with murder here. -- T.E.D. http://www.telepath.com/~dennison/Ted/TED.html Sent via Deja.com http://www.deja.com/ Before you buy. ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Required Metrics 2000-05-01 0:00 ` Required Metrics Ted Dennison @ 2000-05-01 0:00 ` Tucker Taft 2000-05-01 0:00 ` Ken Garlington 2000-05-01 0:00 ` Ken Garlington 1 sibling, 1 reply; 68+ messages in thread From: Tucker Taft @ 2000-05-01 0:00 UTC (permalink / raw) Ted Dennison wrote: > > In article <5DDO4.2237$wb7.194854@news.flash.net>, > "Ken Garlington" <Ken.Garlington@computer.org> wrote: > > In the Ada Reference Manual, paragraphs D.8:37-45 states that "The > > implementation shall document" a series of metrics related to the > ... > > details of that configuration." However, although my compiler's > > reference manual claims that "The real-time systems annex is fully > > implemented", it goes on to say " Information on metrics is not yet > > available." Does a compiler fully implement the real-time systems > > annex if it does not meet the documentation requirements? > > I remember asking almost this exact question here about a year and a > half ago. You can search deja for the thread. But the answer I remember > getting was that since there is (and can be) no test in the Ada > validation test suite for documentation requirements, implementors can > pretty much get away with murder here. The validation process (aka "conformance assessment" process) does not include checking conformance to the documentation requirements, largely for pragmatic reasons. It was felt that the cost of document conformance assessment would be excessive and difficult to standardize. In any case, the marketplace needs to play a role here. Either make good documentation a condition of sale, or give your existing vendor a hard time if they don't provide the documentation you need. > > -- > T.E.D. > > http://www.telepath.com/~dennison/Ted/TED.html -- -Tucker Taft stt@averstar.com http://www.averstar.com/~stt/ Technical Director, Commercial Division, AverStar (formerly Intermetrics) (http://www.averstar.com/services/IT_consulting.html) Burlington, MA USA ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Required Metrics 2000-05-01 0:00 ` Tucker Taft @ 2000-05-01 0:00 ` Ken Garlington 2000-05-02 0:00 ` Ken Garlington 2000-05-02 0:00 ` Ted Dennison 0 siblings, 2 replies; 68+ messages in thread From: Ken Garlington @ 2000-05-01 0:00 UTC (permalink / raw) "Tucker Taft" <stt@averstar.com> wrote in message news:390DC8AD.59B5EBEE@averstar.com... > Ted Dennison wrote: > > > > In article <5DDO4.2237$wb7.194854@news.flash.net>, > > "Ken Garlington" <Ken.Garlington@computer.org> wrote: > > > In the Ada Reference Manual, paragraphs D.8:37-45 states that "The > > > implementation shall document" a series of metrics related to the > > ... > > > details of that configuration." However, although my compiler's > > > reference manual claims that "The real-time systems annex is fully > > > implemented", it goes on to say " Information on metrics is not yet > > > available." Does a compiler fully implement the real-time systems > > > annex if it does not meet the documentation requirements? > > > > I remember asking almost this exact question here about a year and a > > half ago. You can search deja for the thread. But the answer I remember > > getting was that since there is (and can be) no test in the Ada > > validation test suite for documentation requirements, implementors can > > pretty much get away with murder here. > > The validation process (aka "conformance assessment" process) does > not include checking conformance to the documentation requirements, > largely for pragmatic reasons. It was felt that the cost of document > conformance assessment would be excessive and difficult to standardize. Well, yeah, sure, but I think my question was more "abstract" (to coin a keyword :) than this. Simply put: Does the implementation meet the standard? I understand that a validation suite might not detect all deviations with respect to _functionality_, never mind documentation, since no test can be exhaustive. However, I've seen people post behavior seen in a compiler, and ask if it meets the standard, and few people have responded with "Sorry - that specific behavior wasn't/wouldn't be caught in the validation suite, so you're on your own." > In any case, the marketplace needs to play a role > here. Either make good documentation a condition of sale, or give > your existing vendor a hard time if they don't provide the documentation > you need. Ditto for this. I can ask for useful "stuff" (documentation, optimizations, etc.) whether or not it's in the standard. Alternate phrasing of the question: If a requirement is in the standard, and no one makes an effort to follow it, what's the requirement doing in the standard? Why not have an interpretation that says, in essence: "Oops. Never mind"? Or am I just missing some fundamental point here? ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Required Metrics 2000-05-01 0:00 ` Ken Garlington @ 2000-05-02 0:00 ` Ken Garlington 2000-05-02 0:00 ` Ted Dennison 2000-05-03 0:00 ` Robert Dewar 2000-05-02 0:00 ` Ted Dennison 1 sibling, 2 replies; 68+ messages in thread From: Ken Garlington @ 2000-05-02 0:00 UTC (permalink / raw) Having gone back and read the thread on this subject from May 1999, the closest thing I found to an answer to my question was, essentially, "It's not practical for a vendor to document the required metrics if it relies on an underlying operating system." I'm not sure I buy this in all cases -- what if the OS vendor provides sufficient information on their thread support, for example? -- but let's assume it's a reasonable caveat. Given this, then: 1. Why wouldn't this be the response given in the documentation; i.e. "This implementation conforms to the Real-Time Annex, except for the documentation of metrics, which is not practical"? (Better yet, why didn't anyone put this in the ARM to begin with, or write an interpretation after the fact?) 2. What would this mean for an implementation that didn't use an operating system; e.g. GNORT? (By the way, if it wasn't obvious, this isn't a rant about GNAT. It's a question about the meaning -- or maybe, lack of meaning -- of the standard...) I'm starting to wonder about the responses to this question (and the lack thereof). Are there other requirements in the ARM that are optional in practice? Is it a general rule that if it's not tested in the validation suite, a vendor can decline to implement the requirement but still be considered conforming? If so, what does this mean with respect to the assumed benefits of standardization? "Hey, this compiler doesn't implement tasking, but it says it conforms to the ARM!" "Well, you can always call the pthreads package and implement it yourself. Call the vendor, they'll provide assistance. Alternately, you can ask for tasking support in a later release." ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Required Metrics 2000-05-02 0:00 ` Ken Garlington @ 2000-05-02 0:00 ` Ted Dennison 2000-05-03 0:00 ` Robert Dewar 1 sibling, 0 replies; 68+ messages in thread From: Ted Dennison @ 2000-05-02 0:00 UTC (permalink / raw) In article <GwzP4.3979$wb7.383449@news.flash.net>, "Ken Garlington" <Ken.Garlington@computer.org> wrote: > Having gone back and read the thread on this subject from May 1999, Interesting coincidence that this would come up two Mays in a row. I know that once a year is not exactly "frequently", but I think a coherent accouting of this issue would be worthy of inclusion in the FAQ. I'd put it in myself, but I can't access AdaPower right now for some bizzare reason. -- T.E.D. http://www.telepath.com/~dennison/Ted/TED.html Sent via Deja.com http://www.deja.com/ Before you buy. ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Required Metrics 2000-05-02 0:00 ` Ken Garlington 2000-05-02 0:00 ` Ted Dennison @ 2000-05-03 0:00 ` Robert Dewar 2000-05-03 0:00 ` Ken Garlington 1 sibling, 1 reply; 68+ messages in thread From: Robert Dewar @ 2000-05-03 0:00 UTC (permalink / raw) The trouble with documentation requirements is that the standard, by necessity, does not define exactly what the meaning of these requirements is, or even what documentation consists of. In the case of typical implementations on top of an operating system, about the best documentation of the conventional kind that you could give for tasking metrics would be to clearly document the sequence of operating systems calls that is made for any particular tasking language construct. Our view in the case of GNAT is that the sources of the run-time which are an integral part of our complete technical documentation contain precisely this information, and so far we have not had any instances of users requiring the information in any other form. I don't think this is so surprising. We could of course run benchmarks on particular machines under particular conditions and publish numbers that appear to meet the requirements of the RM. Why "appear"? Because in practice they won't be useful for users. If users need hard figures on timings of tasking operations, what we would advise is that they run tests corresponding to their actual use of tasking constructs in the exact conditions they will be using them. This is the same advice that applies to any use of any features of the language. I personally think the metrics requirements in annex D do not belong in a language standard. They are too vague from a formal point of view, and not helpful from a pragmatic point of view. Historically they were put there to satisfy people who actually wanted to try to quantify performance with specific RM requirements for performance, keyed to processor clock speed. That's of course an even WORSE idea for infusing the standard with inappropriate junk :-) Robert Dewar Ada Core Technologies Sent via Deja.com http://www.deja.com/ Before you buy. ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Required Metrics 2000-05-03 0:00 ` Robert Dewar @ 2000-05-03 0:00 ` Ken Garlington 2000-05-03 0:00 ` Robert A Duff ` (2 more replies) 0 siblings, 3 replies; 68+ messages in thread From: Ken Garlington @ 2000-05-03 0:00 UTC (permalink / raw) "Robert Dewar" <robert_dewar@my-deja.com> wrote in message news:8ep0k3$jlr$1@nnrp1.deja.com... > The trouble with documentation requirements is that the > standard, by necessity, does not define exactly what the > meaning of these requirements is, or even what documentation > consists of. > > In the case of typical implementations on top of an operating > system, about the best documentation of the conventional kind > that you could give for tasking metrics would be to clearly > document the sequence of operating systems calls that is made > for any particular tasking language construct. > > Our view in the case of GNAT is that the sources of the run-time > which are an integral part of our complete technical > documentation contain precisely this information, and so far > we have not had any instances of users requiring the information > in any other form. I don't have any problem with interpreting the term "formula" as a sequence of operating system calls, but I don't think the GNAT document set (again, as an example only) does this. Looking in the most logical place (the GNAT Reference Manual), it says "Information on metrics is not yet available." This implies, to me at least, that this information is *not* available elsewhere in the "documentation". If this statement were completely absent, and/or if there were a general statement somewhere that the source code was also considered part of the documentation, I think I'd have an easier time accepting that the requirement were met. Why wouldn't the reference manual, at a minimum, refer to the applicable run-time sources (making them clearly part of the "documentation")? This is done in other cases (e.g. for storage pools), and it certainly would have helped me when I needed this information! Considering all of the effort that's gone into good error messages and the like, this sort of easily-corrected "gap" in addressing a *language requirement* (not, I want to point out, "desirable documentation") is surprising. > I don't think this is so surprising. We could of course run > benchmarks on particular machines under particular conditions > and publish numbers that appear to meet the requirements of > the RM. Why "appear"? Because in practice they won't be useful > for users. It's perfectly OK to think that a requirement is not useful, and in fact I agree with you to some extent. However, that is beside the point. I get a queasy feeling when it seems that the Ada vendor community may not be addressing requirements they don't like. First, it doesn't seem to be a very "software engineering" oriented solution, given how Ada folks are supposedly more "software engineering" oriented than users of other languages. Second, how can anyone claim that an advantage of Ada is standardization, if vendors don't have to follow parts of the standard that are hard to implement, or that they just don't like? I'd hope we all agree that a compiler that passes the validation suite, but can't handle *any* other valid Ada program, is also in practice not useful for users, and should be universally denounced as not compliant to the standard. However, once we start down the path of ignoring the requirements we don't like, we lose any right to complain about such a case. It seems to me in the Ada83 days that AIs were used to develop and document consensus on clarifications, etc. to the standard. Is this no longer used? ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Required Metrics 2000-05-03 0:00 ` Ken Garlington @ 2000-05-03 0:00 ` Robert A Duff 2000-05-04 0:00 ` Ken Garlington 2000-05-04 0:00 ` Robert Dewar 2000-05-04 0:00 ` Robert Dewar 2 siblings, 1 reply; 68+ messages in thread From: Robert A Duff @ 2000-05-03 0:00 UTC (permalink / raw) I think the answer to your original question is, "Yes, the Metrics are real requirements, but it's not clear exactly what they mean." In other words, it's a matter of judgement whether or not a given piece of text constitutes proper documentation. I would have to agree that a statement like, "Documentation not yet available" is a clear violation -- I can say that without defining precisely what the rule means. I strongly agree with Robert Dewar's statement that Documentation Requirements should not be in the standard, and I said so many times during the Ada 9X design. Many people, including Tucker, did not agree. And I think the Metrics are the worst sort of Documentation Requirements. Putting Doc Requirements in a standard shows confusion as to what standards are for. Standards are not about forcing or encouraging people to make high quality implementations. Standards are about encouraging uniformity. IMHO, of course. ;-) Furthermore, Doc Requirements distract implementors from the real goal, which is good documentation. Look at any vendors Doc Requirements documentation. You will usually see a long list of poorly organized snippets of confusing, out-of-context, information. > I get a queasy feeling when it seems that the Ada vendor community may not > be addressing requirements they don't like. First, it doesn't seem to be a > very "software engineering" oriented solution, given how Ada folks are > supposedly more "software engineering" oriented than users of other > languages. Second, how can anyone claim that an advantage of Ada is > standardization, if vendors don't have to follow parts of the standard that > are hard to implement, or that they just don't like? You have a point. However, in this particular case, if vendors fail to provide the required documentation, that has absolutely zero effect on portability. The advantage of having a standard is portability. So it's hard to get too excited about a rule in the standard that has no effect on portability. > I'd hope we all agree that a compiler that passes the validation suite, but > can't handle *any* other valid Ada program, is also in practice not useful > for users, and should be universally denounced as not compliant to the > standard. Sure. >... However, once we start down the path of ignoring the requirements > we don't like, we lose any right to complain about such a case. That would be a stronger argument if we were talking about some language feature. Eg, we all know Robert Dewar hates asynchronous transfer of control, but it would be annoying indeed if GNAT didn't implement it. Likewise, I hate modular types -- but of course I don't refuse to implement them in the AverStar compilers, and I don't insist on changing the rules to be more to my liking. These would be true even if the ACATS didn't test for these features. > It seems to me in the Ada83 days that AIs were used to develop and document > consensus on clarifications, etc. to the standard. Is this no longer used? The ARG still exists, and still generates AI's. I suspect that if you sent in a question saying "Are the metrics really required?", the ARG would issue a ruling saying, "Yes, of course, it says so in plain English." (I'm not sure of that; some people might use it as an opportunity to get rid of these silly rules. Some people might take the attitude that you can't require something unless you can precisely define it -- and thus declare the metrics to be meaningless gibberish.) On the other hand, if you asked, "What, exactly, do the metrics require?", the ARG would refuse to waste time trying to answer the impossible. So, whatever compiler it was that said "Documentation not yet available", send them a bug report. I wouldn't be surprised if it's our compiler. ;-) - Bob ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Required Metrics 2000-05-03 0:00 ` Robert A Duff @ 2000-05-04 0:00 ` Ken Garlington 2000-05-04 0:00 ` Robert Dewar 0 siblings, 1 reply; 68+ messages in thread From: Ken Garlington @ 2000-05-04 0:00 UTC (permalink / raw) "Robert A Duff" <bobduff@world.std.com> wrote in message news:wcczoq7jyt9.fsf@world.std.com... > The advantage of having a standard is portability. So > it's hard to get too excited about a rule in the standard that has no > effect on portability.... Eg, we all know Robert Dewar hates asynchronous transfer of > control, but it would be annoying indeed if GNAT didn't implement it. > Likewise, I hate modular types -- but of course I don't refuse to > implement them in the AverStar compilers, and I don't insist on changing > the rules to be more to my liking. These would be true even if the > ACATS didn't test for these features. You *have* changed the rules to be more to your liking, if you can *unilaterally* decide which rules affect portability (or any other desirable aspect of standardization, such as the ability to reduce training costs). For example, if a vendor decides that there's some required aspect of ATC that no one is ever going to use, and it's not required to pass the validation suite, then it sounds like that vendor is permitted to not implement the feature. Certainly, no one could in good conscience complain that this violates the standard, if they accept your argument. In addition, if vendors alone are permitted to decide what really _isn't_ in the standard, why can't vendors alone decide what's really _in_ the standard? Why spend all this time fooling with ISO standardization procedures if vendors have, in essence, a veto? Is it just for the publicity value? > The ARG still exists, and still generates AI's. > > I suspect that if you sent in a question saying "Are the metrics really > required?", the ARG would issue a ruling saying, "Yes, of course, it > says so in plain English." (I'm not sure of that; some people might use > it as an opportunity to get rid of these silly rules. Some people might > take the attitude that you can't require something unless you can > precisely define it -- and thus declare the metrics to be meaningless > gibberish.) Per the standard, I just sent the mail message attached at the end. We'll see what happens... > On the other hand, if you asked, "What, exactly, do the metrics > require?", the ARG would refuse to waste time trying to answer the > impossible. > > So, whatever compiler it was that said "Documentation not yet > available", send them a bug report. I wouldn't be surprised if it's our > compiler. ;-) If it's impossible to answer whether or not an implementation meets the requirement, what's the bug? How would I know if it's fixed? In particular, if _every_ vendor is doing this (and no one has said otherwise), it sounds like something a bug report to a single vendor won't fix. ----- From: Ken Garlington [mailto:Ken.Garlington@computer.org] Sent: Wednesday, May 03, 2000 7:23 PM To: ada-comment@sw-eng.falls-church.va.us Subject: Minimum criteria for metrics documentation !topic Minimum criteria for metrics documentation !reference RM95-D(2) !from Author Name 00-05-03 !keywords metrics, documentation, real-time !discussion Is there any criteria that can be used to determine if a vendor has formally met the requirements to document metrics described in D(2-6)? For example, which (if any) of the following conditions would be considered acceptable? 1. The vendor states that the metrics are not currently available. 2. Same as #1, but vendor provides source code for its implementation of interfaces to an underlying operating system. 3. Same as #2, but vendor explicitly states that source code is provided to meet requirement. 4. Same as #3, but vendor provides a list of which source code files are applicable. ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Required Metrics 2000-05-04 0:00 ` Ken Garlington @ 2000-05-04 0:00 ` Robert Dewar 2000-05-04 0:00 ` Robert A Duff 2000-05-05 0:00 ` Ken Garlington 0 siblings, 2 replies; 68+ messages in thread From: Robert Dewar @ 2000-05-04 0:00 UTC (permalink / raw) In article <zw3Q4.5766$wb7.485167@news.flash.net>, "Ken Garlington" <Ken.Garlington@computer.org> wrote: > You *have* changed the rules to be more to your liking, if you can > *unilaterally* decide which rules affect portability (or any other desirable > aspect of standardization, such as the ability to reduce training costs). > For example, if a vendor decides that there's some required aspect of ATC > that no one is ever going to use, and it's not required to pass the > validation suite, then it sounds like that vendor is permitted to not > implement the feature. Certainly, no one could in good conscience complain > that this violates the standard, if they accept your argument. That's a strawman. Obviously no one can sign the DOC if they have deliberately failed to implement some aspect of the standard, whether or not it is tested in the ACAATS tests > In addition, if vendors alone are permitted to decide what really _isn't_ in > the standard, why can't vendors alone decide what's really _in_ the > standard? Why spend all this time fooling with ISO standardization > procedures if vendors have, in essence, a veto? Is it just for the publicity > value? That's an absurd over-reaction to the thread at hand, not based in any reality. Remember this thread is all about a requirement that is clearly semantically meaningless in formal terms (the RM does not even describe what the word "documentation" means, and that's a serious omission. Note that we could if we wanted perfectly well declare that our sources are part of the documentation and constitute the full compliance with the requirements of annex M. That would meet the RM standards from a legal point of view, but it would be useless to our customers. I really think the documentation requirements of the standard have almost no effect on compiler writers. Documentation is written for the benefit of users. It is presumptious in any case for the RM to think it can know what kinds of documentation the user will require. Furthermore, if it requires documentation without any indication of what the requirement means, then the requirement is plain useless. But Ken, extending this to clearly defined technical features, where the requirements are meaningful and semantically sound makes no sense at all. Bob Duff was quite clear in saying that in THIS area, vendors follow the RM closely even if they don't like what it says, and even if the tests do not test something. His examples were well chosen ones. I do indeed think that ATC was a huge mistake in the design of Ada, but Ada Core Technologies has invested significant resources in making this feature work completely and well, not just to pass the tests, but to meet the requirements of our users. Your concerns about vendors running amok would make more sense if you would give us some nice examples of what it is that you do not like :-) Robert Dewar Ada Core Technologies > > > The ARG still exists, and still generates AI's. > > > > I suspect that if you sent in a question saying "Are the metrics really > > required?", the ARG would issue a ruling saying, "Yes, of course, it > > says so in plain English." (I'm not sure of that; some people might use > > it as an opportunity to get rid of these silly rules. Some people might > > take the attitude that you can't require something unless you can > > precisely define it -- and thus declare the metrics to be meaningless > > gibberish.) > > Per the standard, I just sent the mail message attached at the end. We'll > see what happens... > > > On the other hand, if you asked, "What, exactly, do the metrics > > require?", the ARG would refuse to waste time trying to answer the > > impossible. > > > > So, whatever compiler it was that said "Documentation not yet > > available", send them a bug report. I wouldn't be surprised if it's our > > compiler. ;-) > > If it's impossible to answer whether or not an implementation meets the > requirement, what's the bug? How would I know if it's fixed? In particular, > if _every_ vendor is doing this (and no one has said otherwise), it sounds > like something a bug report to a single vendor won't fix. > > ----- > > From: Ken Garlington [mailto:Ken.Garlington@computer.org] > Sent: Wednesday, May 03, 2000 7:23 PM > To: ada-comment@sw-eng.falls-church.va.us > Subject: Minimum criteria for metrics documentation > > !topic Minimum criteria for metrics documentation > !reference RM95-D(2) > !from Author Name 00-05-03 > !keywords metrics, documentation, real-time > !discussion > > Is there any criteria that can be used to determine if a vendor has formally > met the requirements to document metrics described in D(2-6)? For example, > which (if any) of the following conditions would be considered acceptable? > > 1. The vendor states that the metrics are not currently available. > > 2. Same as #1, but vendor provides source code for its implementation of > interfaces to an underlying operating system. > > 3. Same as #2, but vendor explicitly states that source code is provided to > meet requirement. > > 4. Same as #3, but vendor provides a list of which source code files are > applicable. > > Sent via Deja.com http://www.deja.com/ Before you buy. ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Required Metrics 2000-05-04 0:00 ` Robert Dewar @ 2000-05-04 0:00 ` Robert A Duff 2000-05-04 0:00 ` Robert Dewar 2000-05-05 0:00 ` Ken Garlington 1 sibling, 1 reply; 68+ messages in thread From: Robert A Duff @ 2000-05-04 0:00 UTC (permalink / raw) Robert Dewar <dewar@gnat.com> writes: >...Note that we could if we wanted perfectly > well declare that our sources are part of the documentation > and constitute the full compliance with the requirements > of annex M. Why not say that the binary object code of the compiler and run-time system constitute the full documentation? The information's in there. ;-) ;-) - Bob ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Required Metrics 2000-05-04 0:00 ` Robert A Duff @ 2000-05-04 0:00 ` Robert Dewar 0 siblings, 0 replies; 68+ messages in thread From: Robert Dewar @ 2000-05-04 0:00 UTC (permalink / raw) In article <wccsnvy1ddr.fsf@world.std.com>, Robert A Duff <bobduff@world.std.com> wrote: > Why not say that the binary object code of the compiler and run-time > system constitute the full documentation? The information's in there. True -- of course your printed documentation must mention this or you will not be in full compliance with the standard :-) Sent via Deja.com http://www.deja.com/ Before you buy. ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Required Metrics 2000-05-04 0:00 ` Robert Dewar 2000-05-04 0:00 ` Robert A Duff @ 2000-05-05 0:00 ` Ken Garlington 1 sibling, 0 replies; 68+ messages in thread From: Ken Garlington @ 2000-05-05 0:00 UTC (permalink / raw) "Robert Dewar" <dewar@gnat.com> wrote in message news:8es5sk$5cr$1@nnrp1.deja.com... > That's a strawman. Obviously no one can sign the DOC if they > have deliberately failed to implement some aspect of the > standard, whether or not it is tested in the ACAATS tests Why not, if the vendor believes it's an unreasonable requirement? For example, if a vendor believes that "the documentation requirements of the standard have almost no effect on compiler writers," then he might be justified in signing off without ever examining his documentation for compliance. I also wonder about inadvertant failures. Let's take a specific example: Before ACT signs off, what items -- other than ensuring all ACAATS tests pass -- are on your checklist? > > In addition, if vendors alone are permitted to decide what > really _isn't_ in > > the standard, why can't vendors alone decide what's really > _in_ the > > standard? Why spend all this time fooling with ISO > standardization > > procedures if vendors have, in essence, a veto? Is it just for > the publicity > > value? > > That's an absurd over-reaction to the thread at hand, not based > in any reality. How can questions be "over-reactions?" Sounds like _someone_ doesn't have any answers handy... :) More to the point, I have to at least assume this is a possibility, given that there (a) is evidence that vendors can choose what they want to implement and (b) no obvious counter-evidence that there are any limits beyond the ACAATS tests as to what vendors might ignore. > Remember this thread is all about a requirement that is clearly > semantically meaningless in formal terms (the RM does not even > describe what the word "documentation" means, and that's a > serious omission. Note that we could if we wanted perfectly > well declare that our sources are part of the documentation > and constitute the full compliance with the requirements > of annex M. That would meet the RM standards from a legal > point of view, but it would be useless to our customers. But (and this is the key point, which I will keep repeating until someone bothers to address it), isn't there an intrinsic value in meeting the standards "from a legal point of view"? Or is it all just based on what a particular vendor's customers want? Secondly, if there is something in the standards that is unreasonable, isn't there an intrinsic value in going through the formal process to fix it? > I really think the documentation requirements of the standard > have almost no effect on compiler writers. Documentation is > written for the benefit of users. It is presumptious in any > case for the RM to think it can know what kinds of documentation > the user will require. Furthermore, if it requires documentation > without any indication of what the requirement means, then the > requirement is plain useless. > But Ken, extending this to clearly defined technical features, > where the requirements are meaningful and semantically sound > makes no sense at all. Bob Duff was quite clear in saying that > in THIS area, vendors follow the RM closely even if they don't > like what it says, and even if the tests do not test something. > His examples were well chosen ones. I do indeed think that ATC > was a huge mistake in the design of Ada, but Ada Core > Technologies has invested significant resources in making this > feature work completely and well, not just to pass the tests, > but to meet the requirements of our users. I was really feeling good until the last line ("meet the requirements of our users"). Again, I think you're missing the point. I assume you're investing in ATC because users are asking you for this feature. I assume Sun invested in a good implementation of java.net without benefit of an ISO standard, as well. Without the benefit of knowing what your customers are asking from you, let me take a stab in the dark at another example, that maybe will get the point across. Let's say that a vendor claims to fully implement the Safety and Secuity annex. Pragma Reviewable is part of that annex. It has a series of requirements identified as "implementation requirements", not documentation requirements. Are these "clearly defined technical features?" Are they "semantically sound"? I know there's compilers (including Ada83 compilers) that meet these requirements, so I assume that at least some vendors believe they are implementable. I also know that it's really expensive to implement these (because I kept having to pay big bucks to get them implemented, because they weren't part of the standard). Before this conversation, I would have said "If a vendor claims conformance to Annex H, these requirements have to be implemented, or the vendor has to explicitly say that they aren't". However, let's say this vendor doesn't even identify pragma Reviewable in their documentation, much less implement the required behavior. If I complained about this, would I have a leg to stand on? Unless I could get some other users to agree, sounds like I'm out of luck. It sounds like we may have gone to all the trouble to identify requirements that IHMO are really important for safety-critical systems, and managed to get it into the standard so that we could share the cost among a larger group of users, just to end up in the same fix we were in back in 1985. I can hear the conversation now: "You claim conformance to the standard, but you don't implement these features." "Well, they don't affect program execution, and they're not tested as part of validation, and no other users have asked for them, so we never did them." "Maybe that's because we're the first safety-critical application to use your product?" "Well, if you don't like it, pick another vendor!" "Unfortunately, you're the only vendor for our platform!" "OK - pay us and we'll do it." That conversation is easy to reproduce, because I heard it a lot over the years. The main difference? Before, we said "yes". Now, we say "C". :) (The next time anyone complains about why companies are abandoning Ada, keep this conversation in mind...) > Your concerns about vendors running amok would make more sense > if you would give us some nice examples of what it is that you > do not like :-) Are you saying that something really bad has to happen before anyone considers it a risk? Remember what I was saying earlier about good software engineering practices? Here's an example from the Ada83 days. We had built an application on a particular vendor's product, and it worked fine. We delivered the product. Then, we ported it to a different platform and a different vendor. It wouldn't compile. It turned out we had a subprogram, declared in a package body, which used another subprogram in the same package body before it was declared. The old (validated) compiler, we discovered, had a few problems in enforcing the visibility rules. This probably isn't a common programming error, so no other users had complained. (Actually, I think we wrote about 75% of all the problem reports this vendor _ever_ received.) Obviously, it wasn't caught by the then-ACVC. It wasn't that hard to change the package body (although I was now stuck with two variants to maintain, at least until the "new" package body could be ported back to the old system.) So, it sounds like I really didn't have any reason to complain about the compiler's behavior, based on the criteria presented above? ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Required Metrics 2000-05-03 0:00 ` Ken Garlington 2000-05-03 0:00 ` Robert A Duff @ 2000-05-04 0:00 ` Robert Dewar 2000-05-04 0:00 ` Ken Garlington 2000-05-04 0:00 ` Robert Dewar 2 siblings, 1 reply; 68+ messages in thread From: Robert Dewar @ 2000-05-04 0:00 UTC (permalink / raw) In article <yLUP4.4522$wb7.450636@news.flash.net>, "Ken Garlington" <Ken.Garlington@computer.org> wrote: > Why wouldn't the reference manual, at a minimum, refer to the > applicabl run-time sources (making them clearly part of the > "documentation")? There is a simple answer to this. I wrote this paragraph over five years ago, and no one ever bothered to notice or comment on it since, which may give some indication as to the real importance of this issue :-) The sentence in question has been edited appropriately. Robert Dewar Ada Core Technologies Sent via Deja.com http://www.deja.com/ Before you buy. ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Required Metrics 2000-05-04 0:00 ` Robert Dewar @ 2000-05-04 0:00 ` Ken Garlington 2000-05-05 0:00 ` Robert Dewar 0 siblings, 1 reply; 68+ messages in thread From: Ken Garlington @ 2000-05-04 0:00 UTC (permalink / raw) "Robert Dewar" <robert_dewar@my-deja.com> wrote in message news:8es58n$4lo$1@nnrp1.deja.com... > In article <yLUP4.4522$wb7.450636@news.flash.net>, > "Ken Garlington" <Ken.Garlington@computer.org> wrote: > > Why wouldn't the reference manual, at a minimum, refer to the > > applicabl run-time sources (making them clearly part of the > > "documentation")? > > There is a simple answer to this. I wrote this paragraph over > five years ago, and no one ever bothered to notice or comment > on it since, which may give some indication as to the real > importance of this issue :-) The sentence in question has been > edited appropriately. Actually, there's a thread from May of last year that touches on this issue. > > Robert Dewar > Ada Core Technologies > > > Sent via Deja.com http://www.deja.com/ > Before you buy. ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Required Metrics 2000-05-04 0:00 ` Ken Garlington @ 2000-05-05 0:00 ` Robert Dewar 0 siblings, 0 replies; 68+ messages in thread From: Robert Dewar @ 2000-05-05 0:00 UTC (permalink / raw) In article <6FnQ4.7883$wb7.551417@news.flash.net>, "Ken Garlington" <Ken.Garlington@computer.org> wrote: > Actually, there's a thread from May of last year that touches > on this issue. When I said no one had raised this issue, I meant none of our customers. I doubt any vendor is greatly influenced by wants expressed by contributors to comp.lang.ada. Certainly Ada Core Technologies essentially ignores this forum in terms of language suggestions etc (surely you would be surprised if it were otherwise). We look to what our customers and potential customers ask us for -- we get plenty of good input that way, and a much clearer indication of true priorities! My particular point was that no one ever commented on the obsolete statement in our reference manual, and I find that interesting :-) Sent via Deja.com http://www.deja.com/ Before you buy. ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Required Metrics 2000-05-03 0:00 ` Ken Garlington 2000-05-03 0:00 ` Robert A Duff 2000-05-04 0:00 ` Robert Dewar @ 2000-05-04 0:00 ` Robert Dewar 2000-05-04 0:00 ` Wes Groleau 2000-05-04 0:00 ` Ken Garlington 2 siblings, 2 replies; 68+ messages in thread From: Robert Dewar @ 2000-05-04 0:00 UTC (permalink / raw) In article <yLUP4.4522$wb7.450636@news.flash.net>, "Ken Garlington" <Ken.Garlington@computer.org> wrote: > I get a queasy feeling when it seems that the Ada vendor > community may not be addressing requirements they don't like. I think that's the wrong take entirely. In the case of metrics (see Bob Duff's post), there is a broad feeling that these are completely bogus requirements for modern compilers. If they ever had any meaning it would only be for bare-board compilers for very specific architectures. It is not that the vendor "does not like" this particular requirement in our case, it is simply that a) it is meaningless, but we do meet the requirement anyway, perhaps not in the most useful way, but since it is pretty much useless, there is not much point in trying to do useless things in a useful manner! b) our customers have zero interest in us doing any more than we do now. As we often find, CLA readers and contributors seem to have rather different priorities from actual Ada users when we get to discussing various theoretical issues surrounding the RM :-) Robert Dewar Ada Core Technologies Sent via Deja.com http://www.deja.com/ Before you buy. ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Required Metrics 2000-05-04 0:00 ` Robert Dewar @ 2000-05-04 0:00 ` Wes Groleau 2000-05-04 0:00 ` Ken Garlington 1 sibling, 0 replies; 68+ messages in thread From: Wes Groleau @ 2000-05-04 0:00 UTC (permalink / raw) > a) it is meaningless, but we do meet the requirement anyway, > perhaps not in the most useful way, but since it is pretty > much useless, there is not much point in trying to do useless > things in a useful manner! Although the GNAT RM may have gaps in "documentation requirements," I feel that in general, it is clear and well-organized. There are vendors for which the required documentation either does not exist or is impossible to find. -- Wes Groleau http://freepages.genealogy.rootsweb.com/~wgroleau ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Required Metrics 2000-05-04 0:00 ` Robert Dewar 2000-05-04 0:00 ` Wes Groleau @ 2000-05-04 0:00 ` Ken Garlington 2000-05-05 0:00 ` Robert Dewar 1 sibling, 1 reply; 68+ messages in thread From: Ken Garlington @ 2000-05-04 0:00 UTC (permalink / raw) "Robert Dewar" <robert_dewar@my-deja.com> wrote in message news:8es5fv$4ov$1@nnrp1.deja.com... > In article <yLUP4.4522$wb7.450636@news.flash.net>, > "Ken Garlington" <Ken.Garlington@computer.org> wrote: > > > I get a queasy feeling when it seems that the Ada vendor > > community may not be addressing requirements they don't like. > > I think that's the wrong take entirely. In the case of metrics > (see Bob Duff's post), there is a broad feeling that these are > completely bogus requirements for modern compilers. Here's the first problem I have with this statement. Either (a) it's not true, and so this is a real issue, or (b) it is true, and has been true for some time, in which case how did it manage to get into the standard? or (c) something changed between 1994-ish and now (really well before now, since apparently no one has ever met this part of the standard), and if so, what changed? Here's the second (and larger) problem, as I've mentioned before. Why should "feelings" be related to requirements? Why go through the motions to formally establish the requirement and get a consensus documented and approved, if there's not an equal intent to do something formal (an AI, at least) to remove such a "bogus" requirement? It appears like the classic case of people writing down a requirements document just because the customer says you have to do one, but promptly ignoring it through the rest of the life cycle. Regardless of the facts of the matter, should the Ada community give such an appearance? > If they > ever had any meaning it would only be for bare-board compilers > for very specific architectures. However, I'd claim that there are still a significant number of real-time systems that do, in fact, get implemented on bare-board systems. I don't know which are the "specific architectures," but certainly real-time systems use a variety of them. Since we're talking about the real-time annex, then surely the environments in which real-time systems operate should be a strong consideration for the appropriateness of such a requirement. For example, is there a GNORT target for which the metrics are meaningful? > It is not that the vendor "does not like" this particular > requirement in our case, it is simply that > > a) it is meaningless, but we do meet the requirement anyway, > perhaps not in the most useful way, but since it is pretty > much useless, there is not much point in trying to do useless > things in a useful manner! Again, the issue of whether it's an appropriate requirement is interesting, but it's not the real issue. I'm still not convinced that you meet the requirement in the current release, for that matter. More importantly, since this is not about a specific compiler, I'm not convinced _anyone_ has bothered to meet the requirement. I don't have proof that this is the case, but it concerns me. Most importantly, it seems this argument can be used to justify all sorts of mischief. > b) our customers have zero interest in us doing any more > than we do now. > > As we often find, CLA readers and contributors seem to have > rather different priorities from actual Ada users when we get > to discussing various theoretical issues surrounding the RM :-) Again, I find the argument of user priorities a very slippery slope, which can be used to invisibly drop any number of requirements. It's useful for discussing the options for implementing a requirement (e.g. a fancy calculator for computing the metric vs. "here's the sources, have at it"), but good software engineering would say that failing to meet a requirement due to perceived user desire, particularly when the discrepancy is not explicitly documented, is definitely not a Good Thing. ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Required Metrics 2000-05-04 0:00 ` Ken Garlington @ 2000-05-05 0:00 ` Robert Dewar 2000-05-06 0:00 ` Ken Garlington 0 siblings, 1 reply; 68+ messages in thread From: Robert Dewar @ 2000-05-05 0:00 UTC (permalink / raw) In article <_HnQ4.7884$wb7.550012@news.flash.net>, "Ken Garlington" <Ken.Garlington@computer.org> wrote: > > I think that's the wrong take entirely. In the case of metrics > > (see Bob Duff's post), there is a broad feeling that these are > > completely bogus requirements for modern compilers. > > Here's the first problem I have with this statement. Either > > (a) it's not true, and so this is a real issue It's true, but always remember the issue here is not whether the documentation might or might not be useful, but whether it belongs in a language standard. > (b) it is true, and has been true for some time, in which case > how did it manage to get into the standard? It has always been true that documentation requirements like this do not belong in the standard. As to why they got in, I think your posts have vividly illustrated the thinking behind those who argued for putting them in. To me a classical case of confusing desirable implementation features with formal semantic requirements. > (c) something changed between 1994-ish and now (really well > before now, No, nothing has changed > since apparently no one has ever met this part of the > standard), You keep saying this, which means you still do not understand the issue here, which is that we are talking about formal semantic requirements. On the contrary, everyone has met it, since from a formal point of view, documentation is not defined, and therefore can be defined anyway you want. As Bob Duff points out, you can define the compiler itself as an operational definition of its own behavior, and therefore it constitutes documentation in a formal sense. Providing the sources is a more realistic form of this approach to documentation, but still may not meet your *non-formal* implementation requirements. > and if so, > what changed? I think you could argue that things have indeed change from the point of view of informal documentation requirements here. And that probably accounts for the overwhelming lack of interest in getting more accessible forms of this documentation, since it was (like much of the RT annex) framed with bare board single processor implementations in mind, whereas modern reality is more likely implementation over an OS or 3rd party RTOS. But that does not affect the discussion about whether such requirements should be formal semantic requirements in the language. The answer to that is, and has always been, no. The fact that such requirements got in was just a compromise necessitated by a standards process involving many people, not all of whom understand what formal language requirements are about, and who don't mind having undefined wooly requirements in what should be a precise specification. > Here's the second (and larger) problem, as I've mentioned > before. Why should > "feelings" be related to requirements? They should not be. The problem with these requirements was precisely that the "feeling" expressed by you forcefully that Ada implementations should be "required" to provide useful documentation got translated into requirements. > Why go through the motions to > formally establish the requirement and get a consensus documented and > approved, if there's not an equal intent to do something formal (an AI, at > least) to remove such a "bogus" requirement? Lots of us knew these requirements were bogus, including key people on the design team (e.g. Bob Duff). You fight many battles in a consensus process. This one was not worth fighting at the time, since the effect of these requirements is negligible in practice. We knew they would end up being useless of course, but we also knew they would not be harmful. > It appears like the classic > case of people writing down a requirements document just > because the > customer says you have to do one, Well, you paint with too broad a brush. This particular phenomenon (documentation requirements) was indeed a matter of responding to customers who did not really understand the issue. But the great majority of the Ada RM is not vaguely related to this process. > but promptly ignoring it through the rest > of the life cycle. Regardless of the facts of the matter, > should the Ada > community give such an appearance? Of course not, but as you have made very clear, there are significant players in the Ada community who do not understand that requirements like this do not belong in the RM, so even now it might be a difficult battle to win, and frankly it is not worth it, since it is unimportant. > > If they > > ever had any meaning it would only be for bare-board compilers > > for very specific architectures. Note that I am talking about the value of these items as pragmatic implementation features, not formal requirements when I say this. > However, I'd claim that there are still a significant number > of real-time systems that do, in fact, get implemented on > bare-board systems. It's certainly a dieing breed. We see almost no demand for this, not zero, but barely on the radar screen. The trend is clearly to 3rd party RTOS based systems (VxWorks, and also see Greenhills new Integrity push). > I don't > know which are the "specific architectures," but certainly real-time systems > use a variety of them. Since we're talking about the real-time annex, then > surely the environments in which real-time systems operate should be a > strong consideration for the appropriateness of such a requirement. I think if you are in such an environment, you should require your implementor to provide whatever information you need. The RM will be no help in this task, since the issue is not formal requirements, but rather providing information in the manner YOU require. > For example, is there a GNORT target for which the metrics are > meaningful? Seeing as GNORT excludes tasking and the real time annex in its entirety, no :-) > > It is not that the vendor "does not like" this particular > > requirement in our case, it is simply that > > > > a) it is meaningless, but we do meet the requirement anyway, > > perhaps not in the most useful way, but since it is pretty > > much useless, there is not much point in trying to do > > useless > > things in a useful manner! > Again, the issue of whether it's an appropriate requirement is > interesting, but it's not the real issue. I'm still not > convinced that you meet the > requirement in the current release, for that matter. Well try to prove this formally. First you will need a formal definition of "documentation" *derived from the RM*. That will be hard. For example, are you sure that this documentation must be in English? What if I provide it in Navaho or Klingon, is that sufficient, or, more to the point here, what if I provide it in Ada 95. Furthermore, if your idea is that you want to see metrics in milliseconds, then this is clearly infeasible in general working over third party systems, so the conditions of RM 1.1.3(6) would apply: 6 Contain no variations except those explicitly permitted by this International Standard, or those that are impossible or impractical to avoid given the implementation's execution environment; > More importantly, since > this is not about a specific compiler, I'm not convinced > _anyone_ has > bothered to meet the requirement. You still think of this as a requirement that can be met or not met. And it is this invalid thinking that lead to having these kind of requirements in the RM. Since the requirement is semantically vacuous, I think it is clear that all compilers meet it. > I don't have proof that this is the case, > but it concerns me. Most importantly, it seems this argument > can be used to > justify all sorts of mischief. That's FUD unless you can back it up. I see no possible spill over from requirements that are meaningless to handling of requirements that are well stated and semantically meaningful. > > b) our customers have zero interest in us doing any more > > than we do now. > > > > As we often find, CLA readers and contributors seem to have > > rather different priorities from actual Ada users when we get > > to discussing various theoretical issues surrounding the RM :-) > Again, I find the argument of user priorities a very slippery > slope, which can be used to invisibly drop any number of > requirements. No, that's completely wrong, and I begin to think that it is a hopeless task to convince you. No requirements have been dropped by anyone here. The only issue is that you have in your mind some interpretation of these requirements that is not in the RM, and you note that compilers do not follow Ken Garlington's interpretation of these requirements. Very possibly so, but that has nothing to do with whether the requirements are followed. Now take the visibility rules of Ada in chapter 8. Here we don't care what Ken thinks about them, because a) they are well defined b) it is well defined whether a compiler implements them correctly. A totally different situation. It is really critical in a formal language standard that all requirements be objective and well defined. Once again requirements for "good" documentation are as silly as requirements for good performance. As I mentioned before, there were actually people seriously asking for quantitative requirements on performance of tasking constructs in terms of processor clock cycles. I know, that's hard to believe that anyone could be that misguided but it's true. The undefined metrics requirements were a compromise to deal with this unreasonable extreme. Since they are undefined and harmless, it was a way of getting things done without running into a road block. > It's useful for > discussing the options for implementing a requirement (e.g. a fancy > calculator for computing the metric vs. "here's the sources, have at it"), > but good software engineering would say that failing to meet a > requirement > due to perceived user desire The trouble with these undefined requirements is that the only POSSIBLE meaning is in terms of "perceivced user desires". In other words if we try to ask if a compiler meets certain documentation requirements, we cannot look to any formal definitions and we cannot look to the RM, the ONLY way of determining whether such requirements have been met is to interrogate "perceived user desire". Yes, that's a horrible state of affairs and that's what this thread is all about, but it is really stil a tempest in a teapot in terms of overall significance. By the way, the ACVC review board discussed this briefly. They immediately agreed that these requirements were too ill-defined to even consider including them in conformance assessment. There was no controversy in this decision, it was clear to this committee that these "requirements" have to be treated differently, namely ignored completely when it comes to formal conformance testing. Sent via Deja.com http://www.deja.com/ Before you buy. ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Required Metrics 2000-05-05 0:00 ` Robert Dewar @ 2000-05-06 0:00 ` Ken Garlington 2000-05-06 0:00 ` Ken Garlington 2000-05-06 0:00 ` Robert Dewar 0 siblings, 2 replies; 68+ messages in thread From: Ken Garlington @ 2000-05-06 0:00 UTC (permalink / raw) "Robert Dewar" <robert_dewar@my-deja.com> wrote in message news:8eukm0$ssm$1@nnrp1.deja.com... > It has always been true that documentation requirements like > this do not belong in the standard. As to why they got in, I > think your posts have vividly illustrated the thinking behind > those who argued for putting them in. To me a classical > case of confusing desirable implementation features with > formal semantic requirements. In a word: Bullshit. This gets repeated (somewhat condescendingly) throughout your response, but it's a total strawman. Could you cite ONE case in this thread where I argued in favor of including ANY requirement, documentation or otherwise, in the standard? It's getting depressing, but I'll make one last run at my position: ONCE A REQUIREMENT ENTERS THE STANDARD, IT'S A REQUIREMENT UNTIL IT'S REMOVED. If a requirement shouldn't be in the standard, don't add it. If a requirement is in there that shouldn't be, then remove it (or indicate that it's only advice). How hard could this be? At best, it's weakly coupled to the issue of which requirements _should_ be in the standard. If you want to discuss whether a requirement should or should not be in the standard, fine. Start another thread. If you'd go back and read what I said earlier, you'd see that I'd probably agree that documentation requirements shouldn't be in there. So what? > You keep saying this, which means you still do not understand > the issue here, which is that we are talking about formal > semantic requirements. > > On the contrary, everyone has met it, since from a formal > point of view, documentation is not defined, and therefore > can be defined anyway you want. OK, so everyone always meets the requirements of the standard that aren't formal semantic requirements? Good software engineering practice would say that they aren't requirements then **and should be identified as such**. If I understand ARM 1.1.2 (24-39), then the only "real" requirements are: 25 Syntax: Syntax rules (indented). 28 Static Semantics: A definition of the compile-time effect of each construct. 30 Dynamic Semantics: A definition of the run-time effect of each construct. Is this correct? I'd be happy to send in a comment to this effect to note this in the standard, if we agree on this list. Actually, since there appears to be complete agreement among the vendor community (and the ACVC review board) as to what's in and what's not in, I guess it doesn't matter. See the comment at the end. > And that probably accounts for the overwhelming lack of > interest in getting more accessible forms of this > documentation, since it was (like much of the RT annex) > framed with bare board single processor implementations > in mind, whereas modern reality is more likely implementation > over an OS or 3rd party RTOS. Which is really depressing for those of us who still build bare-board systems, but the use of DSPs and other "non-mainstream" processors has pretty much taken Ada out of the running for us, anyway, so that's probably why you're not seeing any demand. However, that's a different issue. > > Why go through the motions to > > formally establish the requirement and get a consensus > documented and > > approved, if there's not an equal intent to do something > formal (an AI, at > > least) to remove such a "bogus" requirement? > > Lots of us knew these requirements were bogus, including > key people on the design team (e.g. Bob Duff). You fight > many battles in a consensus process. This one was not > worth fighting at the time, since the effect of these > requirements is negligible in practice. We knew they > would end up being useless of course, but we also > knew they would not be harmful. Except, of course, for those of us who read the word "requirement" as it's used in software engineering. However, once we get the list of non-requirement requirements accepted, this is easy to fix. You know, I lose those requirements battles from time to time myself. It never occured to me to just ignore the ones that I considered infeasible to meet. > > It appears like the classic > > case of people writing down a requirements document just > > because the > > customer says you have to do one, > > Well, you paint with too broad a brush. This particular > phenomenon (documentation requirements) was indeed a > matter of responding to customers who did not really > understand the issue. But the great majority of the > Ada RM is not vaguely related to this process. Whenever someone makes a post about how Ada is used by software engineers, and other languages are used by hackers, I'm going to think about this paragraph. > > Again, the issue of whether it's an appropriate requirement is > > interesting, but it's not the real issue. I'm still not > > convinced that you meet the > > requirement in the current release, for that matter. > > Well try to prove this formally. First you will need a > formal definition of "documentation" *derived from the > RM*. Why can't I use the definition of "documentation" *provided by the vendor*? > That will be hard. For example, are you sure that > this documentation must be in English? What if I provide > it in Navaho or Klingon, is that sufficient, or, more to > the point here, what if I provide it in Ada 95. What if you do, so long as you say "This is what I'm providing to meet the requirement"? Again, you're confusing "what a customer wants" with "what the standard requires". In software engineering, validation addresses the former, verification the latter. >> > For a validated compiler -- difficult! In particular, how >> > can the DOC be signed if you have ignored a requirement. >> >> I suspect with a pen in the hand, and a song in the heart! > Ken, do you think it is fair to accuse vendors of dishonesty > and duplicity with no evidence at all to back up your > statements. Do you think it's fair to accuse me of such an accusation, when the quoted statement doesn't actually say that? > I will say right away that I personally sign > all DOC's for Ada Core Technologies, and in all cases I > do so with integrity knowing that what I am signing is > absolutely correct. I would assume that all other vendors > would make the same statement. Like I said, "with a pen in the hand, and a song in the heart." Of course, your signature is based on your determination as to which requirements are "real" requirements, believing that all other requirements are "met" no matter what. You may believe that the ACVC review board, and all other vendors, agree with you. You may be right, for that matter. If you're wrong, there's no way to find out, since apparently the list of non-requirements has never been formally established, and nobody (not even the vendor) checks that any requirement is met beyond passing the conformance test. You might want to re-read your original statement: "How can the DOC be signed if you have ignored a requirement?" Note that the word "intentionally" does not appear. >> I suspect, after you read the discussion of pragma Reviewable, >> we're going to be expanding this to "documentation and >> some implementation requirements" > Nope, the issues with pragma Reviewable are also all > documentation based, let's look: Yes, let's. Specifically, let's look at the words which appear just before paragraph 5: ***** Implementation Requirements ***** "5 The implementation shall provide the following information ..." Now, I suspect you're using the term "documentation" informally, not in the sense of ARM 1.1.2. However, since all Implementation Requirements, like all Documentation Requirements, are not really requirements, perhaps we _should_ use the same name for both. When we were discussing these safety and security requirements during that industry-implementors meeting lo these many years ago, I was frankly surprised that the vendor representatives (including yourself) didn't put up more of a fuss. Now I see why. What a total waste of our time. ----- From: Ken Garlington [mailto:Ken.Garlington@computer.org] Sent: Friday, May 05, 2000 7:55 PM To: ada-comment@sw-eng.falls-church.va.us Subject: Non-Requirement Requirements !topic Non-Requirement Requirements !reference RM95-1.1.2(24-39) !from Ken Garlington 00-05-05 !keywords requirements, advise !discussion The referenced paragraphs define several different categories of the standard. Some of these categories use the term "requirements," although they are not considered requirements by the Ada vendor community. There should be an explicit list of which categories can be considered requirements (i.e., it is possible for a vendor to "fail" to meet it). All other categories should be described in the terms currently used for "Implementation Advice." ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Required Metrics 2000-05-06 0:00 ` Ken Garlington @ 2000-05-06 0:00 ` Ken Garlington 2000-05-06 0:00 ` Robert Dewar 2000-05-06 0:00 ` Robert Dewar 2000-05-06 0:00 ` Robert Dewar 1 sibling, 2 replies; 68+ messages in thread From: Ken Garlington @ 2000-05-06 0:00 UTC (permalink / raw) By the way, while writing up a bug report, I found this in the gnatinfo.txt file... "The Documentation for GNAT consists of two manuals, the GNAT User's Guide and the GNAT Reference Manual. These are available in a number of formats (ASCII, HTML, PostScript and info) and are bundled as a separate documentation distribution and can be found at the same places as the GNAT binary and source distributions." ...and I just had to laugh! ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Required Metrics 2000-05-06 0:00 ` Ken Garlington @ 2000-05-06 0:00 ` Robert Dewar 2000-05-06 0:00 ` Robert Dewar 1 sibling, 0 replies; 68+ messages in thread From: Robert Dewar @ 2000-05-06 0:00 UTC (permalink / raw) In article <24VQ4.8453$wb7.646902@news.flash.net>, "Ken Garlington" <Ken.Garlington@computer.org> wrote: > By the way, while writing up a bug report, I found this in the gnatinfo.txt > file... > > "The Documentation for GNAT consists of two manuals, the GNAT User's Guide > and the GNAT Reference Manual. These are available in a number of formats > (ASCII, HTML, PostScript and info) and are bundled as a separate > documentation > distribution and can be found at the same places as the GNAT binary and > source distributions." > > ...and I just had to laugh! Ken actually submitted this as a bug report. I commented to him that this was probably the most insignificant bug report we had ever received :-) We "fixed" it by adding the word primary in front of documentation. Perhaps that will suddenly make Ken think we conform to the standard where we did not before :-) Sent via Deja.com http://www.deja.com/ Before you buy. ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Required Metrics 2000-05-06 0:00 ` Ken Garlington 2000-05-06 0:00 ` Robert Dewar @ 2000-05-06 0:00 ` Robert Dewar 2000-05-07 0:00 ` Ken Garlington 2000-05-08 0:00 ` Ole-Hjalmar Kristensen 1 sibling, 2 replies; 68+ messages in thread From: Robert Dewar @ 2000-05-06 0:00 UTC (permalink / raw) In article <24VQ4.8453$wb7.646902@news.flash.net>, "Ken Garlington" <Ken.Garlington@computer.org> wrote: > By the way, while writing up a bug report, I found this in the gnatinfo.txt > file... > > "The Documentation for GNAT consists of two manuals, the GNAT User's Guide > and the GNAT Reference Manual. These are available in a number of formats > (ASCII, HTML, PostScript and info) and are bundled as a separate > documentation > distribution and can be found at the same places as the GNAT binary and > source distributions." > > ...and I just had to laugh! By the way, the documentation for GNAT very definitely includes the Ada Reference Manual -- but perhaps that's a surprise to Ken too ... hard to tell! This thread is quite instructive. I think the entire problem is that in the realm of software engineering, you define a set of "requirements", and then you can tell whether you have done a good job of implementing the software by seeing if it meets these requirements. It is, I guess, quite understandable, if quite wrong, for people to make the mistake of thinking that a language definition is like such a requirements document, and that you will be able to tell if a vendor has done a good job of writing an Ada compiler by seeing if it has met these requirements. That's quite wrong of course, the defining language document is not a set of requirements in this sense at all. Well more properly it is a small part of the requirements. Left out entirely are issues of performance, reliability, efficiency, usability, maintainability etc etc. By the way Ken, you questioned me saying that you were one of the people arguing for inclusion of these ill-defined requirements in the language. I may remember wrong, but I distinctly remember you arguing for this approach in the meetings we had on safety-critical requirements, and your previous post seems to confirm that memory (the one where you say that it was a waste of time attending that meeting). I sure hope that SOME people reading this thread come away with a little bit better understanding of what language definitions are all about (and also an understanding of why validation cannot guarantee usability or quality). Sent via Deja.com http://www.deja.com/ Before you buy. ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Required Metrics 2000-05-06 0:00 ` Robert Dewar @ 2000-05-07 0:00 ` Ken Garlington 2000-05-07 0:00 ` Robert Dewar 2000-05-08 0:00 ` Ole-Hjalmar Kristensen 1 sibling, 1 reply; 68+ messages in thread From: Ken Garlington @ 2000-05-07 0:00 UTC (permalink / raw) "Robert Dewar" <robert_dewar@my-deja.com> wrote in message news:8f279n$me2$1@nnrp1.deja.com... > I think the entire problem is that in the realm of software > engineering, you define a set of "requirements", and then > you can tell whether you have done a good job of implementing > the software by seeing if it meets these requirements. > > It is, I guess, quite understandable, if quite wrong, for people > to make the mistake of thinking that a language definition is > like such a requirements document, and that you will be able > to tell if a vendor has done a good job of writing an Ada > compiler by seeing if it has met these requirements. > > That's quite wrong of course, the defining language document > is not a set of requirements in this sense at all. Well more > properly it is a small part of the requirements. Left out > entirely are issues of performance, reliability, efficiency, > usability, maintainability etc etc. > > By the way Ken, you questioned me saying that you were one of > the people arguing for inclusion of these ill-defined > requirements in the language. I may remember wrong, but I > distinctly remember you arguing for this approach in the > meetings we had on safety-critical requirements, and your > previous post seems to confirm that memory (the one where > you say that it was a waste of time attending that meeting). What I said (with emphasis pointing out the important part): "Could you cite ONE case *** in this thread *** where I argued in favor of including ANY requirement, documentation or otherwise, in the standard?" Certainly, I have argued for requirements *** in other conversations, in other places, at other times ***. In those discussions, I've expected vendors to say "Wait a minute -- I don't think I understand what that means" or "I don't know of any reasonable way to do what you're asking". Now I know why they didn't object to them (at least in the meetings I attended) -- it was preferable to just go along to make the users happy, and then not consider themselves constrained by these "requirements" when implementing the standard. However, that's not related AT ALL to the original question I've raised, which I think you've answered completely. Thank you for your time. ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Required Metrics 2000-05-07 0:00 ` Ken Garlington @ 2000-05-07 0:00 ` Robert Dewar 0 siblings, 0 replies; 68+ messages in thread From: Robert Dewar @ 2000-05-07 0:00 UTC (permalink / raw) In article <pr7R4.8717$wb7.694952@news.flash.net>, "Ken Garlington" <Ken.Garlington@computer.org> wrote: > "Could you cite ONE case *** in this thread *** where I argued in favor of > including ANY requirement, documentation or otherwise, in the standard?" No, and no one said you had. What I said, cued by YOUR post was that you were one of the people who had suggested such requirements during the design process. > Certainly, I have argued for requirements *** in other > conversations, in > other places, at other times ***. And that is what I was referring to. > In those discussions, I've expected > vendors to say "Wait a minute -- I don't think I understand > what that means" Why? Vendors were not present at these meetings, and this is not a vendor issue, it is a language design issue. The meeting I am thinking of was a listening session to understand what people wanted in the Ada 95 standard. When you are listening to what people want, you do not quarrel with them, you listen! That was the ground rule (a good one) set by Chris Anderson. > or "I don't know of any reasonable way to do what you're > asking". Now I know > why they didn't object to them No you don't, your are guessing (wrong!). The interesting thing is that this battle for including documentation requirements was fought much later on, and as you know people (like me and Bob Duff) DID protest that it would be more useful to have these as implementation advice. We lost the argument. > It was preferable to just go along to make the users happy, > and then not consider themselves constrained by these > "requirements" when implementing the standard. Ken, the only thing I can figure, since you keep repeating the incorrect statement above is that you just don't understand what language definitions and semantic requirements are about. Of course one is constrained by requirements when implementing. But the trouble is that if a requirement has no formal content, then in the context of a language standard, it can be met in almost anyway. The RM essentially says You must provide kerbloggle, and then does not define what kerbloggle means. This means that everyone makes up their own mind what kerbloggle means. The standard is no longer very useful, one has instead to have informal market oriented acceptance discussions of whether YOUR idea of kerbloggle is close enough to the implementors view of kerbloggle. Remember, the standard is NOT written in english. It is written in formal english (this was a phrase that Jean Ichbian coined to try to explain this crucial difference). The fact that Ken Garlington has some informal idea of what documentation means has nothing whatsoever to do with this word as used in the standard. The standard has to define all terms that it uses, just as a mathematician has to in writing a proof. Could one come up with a definition of documentation that was not over-restrictive? I very much doubt it. > However, that's not related AT ALL to the original question > I've raised, which I think you've answered completely. Thank > you for your time. Sorry, at this stage, I really don't know what question you are raising :-) Sent via Deja.com http://www.deja.com/ Before you buy. ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Required Metrics 2000-05-06 0:00 ` Robert Dewar 2000-05-07 0:00 ` Ken Garlington @ 2000-05-08 0:00 ` Ole-Hjalmar Kristensen 1 sibling, 0 replies; 68+ messages in thread From: Ole-Hjalmar Kristensen @ 2000-05-08 0:00 UTC (permalink / raw) Robert Dewar <robert_dewar@my-deja.com> writes: > In article <24VQ4.8453$wb7.646902@news.flash.net>, > "Ken Garlington" <Ken.Garlington@computer.org> wrote: > > By the way, while writing up a bug report, I found this in the > gnatinfo.txt > > file... > > > > "The Documentation for GNAT consists of two manuals, the GNAT > User's Guide > > and the GNAT Reference Manual. These are available in a number > of formats > > (ASCII, HTML, PostScript and info) and are bundled as a > separate > > documentation > > distribution and can be found at the same places as the GNAT > binary and > > source distributions." > > > > ...and I just had to laugh! > > > By the way, the documentation for GNAT very definitely > includes the Ada Reference Manual -- but perhaps that's a > surprise to Ken too ... hard to tell! > > This thread is quite instructive. > > I think the entire problem is that in the realm of software > engineering, you define a set of "requirements", and then > you can tell whether you have done a good job of implementing > the software by seeing if it meets these requirements. > > It is, I guess, quite understandable, if quite wrong, for people > to make the mistake of thinking that a language definition is > like such a requirements document, and that you will be able > to tell if a vendor has done a good job of writing an Ada > compiler by seeing if it has met these requirements. > > That's quite wrong of course, the defining language document > is not a set of requirements in this sense at all. Well more > properly it is a small part of the requirements. Left out > entirely are issues of performance, reliability, efficiency, > usability, maintainability etc etc. But it *is* the definition of the language. In that case, it is the *minimum* requirement if you want a complete implementation. > > By the way Ken, you questioned me saying that you were one of > the people arguing for inclusion of these ill-defined > requirements in the language. I may remember wrong, but I > distinctly remember you arguing for this approach in the > meetings we had on safety-critical requirements, and your > previous post seems to confirm that memory (the one where > you say that it was a waste of time attending that meeting). > > I sure hope that SOME people reading this thread come away with > a little bit better understanding of what language definitions > are all about (and also an understanding of why validation > cannot guarantee usability or quality). > > > > Sent via Deja.com http://www.deja.com/ > Before you buy. -- E pluribus Unix ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Required Metrics 2000-05-06 0:00 ` Ken Garlington 2000-05-06 0:00 ` Ken Garlington @ 2000-05-06 0:00 ` Robert Dewar 2000-05-07 0:00 ` Ken Garlington 1 sibling, 1 reply; 68+ messages in thread From: Robert Dewar @ 2000-05-06 0:00 UTC (permalink / raw) In article <kdKQ4.8331$wb7.618601@news.flash.net>, "Ken Garlington" <Ken.Garlington@computer.org> wrote: > Except, of course, for those of us who read the word > "requirement" as it's > used in software engineering. And there perhaps lies the crux of the problem, a formal language definition has almost NOTHING to do with an informal "software engineering" requirements document. It is that confusion in the understanding of the word requirement that lies at the heart of the confusion. Indeed, most software projects are specified with a requirements document where the requirements are often very much subjective, and not anywhere *near* being a mathematical formal specification. > You know, I lose those requirements battles from time to time > myself. It never occured to me to just ignore the ones that I > considered infeasible to meet. You keep saying this, but that does not make the argument here any more convincing. The documenatation requirements are trivial to meet. Everyone meets them, no one ignores them. That of course is from a formal language specification point of view, but the standard is a formal language specification, nothing more and nothing less. > Again, you're confusing "what a customer wants" with "what the > standard requires". In software engineering, validation > addresses the former, > verification the latter. Please don't bring your software engineering background to the table, you will just confuse yourself, it is irrelevant. Yes, a lot of people don't understand this, including some people who participated in the language design effort. A background in formal language semantics is much more relevant here, and I definitely think you will have trouble writing semantic equations for the documentation requirements, no matter WHAT formalism you choose. Yes, we chose to use formal english (which is not that great mathematically) to describe the semantics of Ada 95, so that it would be more accessible, but it has to be equivalent to a formal specification in some sense, or it is useless. It's a bit like the controversy over translating the bible into english. When this was first done, priests worried that people would look at the result and try to interpret in an informal context which would lead to misunderstandings. I leave it to others to judge whether there concern was misplaced, but for sure there is a danger in making standards accessible if people start reading them like software engineering specifications. Now the "implementation advice" sections of the RM *can* be read this way, and including these sections was a brilliant innovation on the design team's point of view, because it allowed us to state informal pragmatic implementation "requirements" where requirements here is used in Ken's sense of software engineering, without contaminating the formal definition with junk. > Do you think it's fair to accuse me of such an accusation, > when the quoted statement doesn't actually say that? Yes, I think it's fair, reread what you said. > Like I said, "with a pen in the hand, and a song in the > heart." Nope! No "song in the heart", which I take to mean with an intent other than a formal declaration. Perhaps you have some other meaning, if so, it is obscure to me. > Of course, > your signature is based on your determination as to which requirements are > "real" requirements, believing that all other requirements are "met" no > matter what. All requirements are requirements. I am not saying that the documentation requirements are not requirements, just that they are trivially met. > You may believe that the ACVC review board, and all other > vendors, agree with you. You mean I agree that there can be no formal tests for whether you meet the documentation rquirements. I cannot imagine anyone with any reasonable knowledge thinking otherwise. > You may be right, for that matter. If you're wrong, > there's no way to find out, since apparently the list of non-requirements > has never been formally established, and nobody (not even the vendor) checks > that any requirement is met beyond passing the conformance test. There is no list of non-requirements, because there are no non-requirements. THe DOC requires that you have not made any intentional deviations from the FULL SET of requirements, including the documentation requirements. > You might want to re-read your original statement: "How can the DOC be > signed if you have ignored a requirement?" Note that the word > "intentionally" does not appear. intentionally is of course critical. Otherwise the DOC would be a declaration that your compiler was bug free. This is not intended, and would be impractical. > > Nope, the issues with pragma Reviewable are also all > > documentation based, let's look: > > Yes, let's. Specifically, let's look at the words which appear just before > paragraph 5: > > ***** Implementation Requirements ***** > > "5 The implementation shall provide the following information ..." > > Now, I suspect you're using the term "documentation" informally, not in the > sense of ARM 1.1.2. However, since all Implementation Requirements, like all > Documentation Requirements, are not really requirements, perhaps we _should_ > use the same name for both. Of course implementation requirements are requirements, what gave you the lunatic idea that this is not the case. I say lunatic here, because it simply makes no sense. There are many critical implentation requirements. Where ON EARTH did you get this idea, certainly not from anything I wrote. Here is an example (there are hundreds maybe thousands of others) Implementation Requirements 15 An implementation shall support lines of at least 200 characters in length, not counting any characters used to signify the end of a line. An implementation shall support lexical elements of at least 200 characters in length. The maximum supported line length and lexical element length are implementation defined. This is formally definable, and of course tested by the ACVC tests. Again, I ask where did you get the idea that implementation requirements are not requirements??????? > When we were discussing these safety and security requirements during that > industry-implementors meeting lo these many years ago, I was frankly > surprised that the vendor representatives (including yourself) didn't put up > more of a fuss. Now I see why. What a total waste of our time. Actually many useful things came out of those meetings, but if you thought that the meetings were about requiring vendors to provide useful usable implementations, you were indeed deluding yourself, since language standards can never guarantee this. > > !topic Non-Requirement Requirements > !reference RM95-1.1.2(24-39) > !from Ken Garlington 00-05-05 > !keywords requirements, advise > !discussion > > The referenced paragraphs define several different categories of the > standard. Some of these categories use the term "requirements," although > they are not considered requirements by the Ada vendor community. There > should be an explicit list of which categories can be considered > requirements (i.e., it is possible for a vendor to "fail" to meet it). All > other categories should be described in the terms currently used for > "Implementation Advice." This is plain silly, I can't imagine the ARG giving it much attention, since it amounts to nothing more than an unfounded polemic. There is no part of the Ada community that considers any requirements in the RM not to be requirements, and certainly the vendor community does not ignore any requirements. Again, I think the entire problem comes from the fact that you don't really understand the purpose of a formal language definition, and that you read the word "requirement" in (to use your terms) "software engineering" sense, which is totally misleading in this context. This reminds me of people who use to operate under the curious illusion that validation had something to say about guaranteeing that a compiler was usable (of course failing validation does say that a compiler may NOT be usable, but passing validation does not for a MOMENT constitute a guarantee that a compiler is usable). Perhaps we should have used VDM for the Ada standard after all -- it would certainly have helped eliminate uncertainties and for sure would have avoided this kind of user confusion :-) Sent via Deja.com http://www.deja.com/ Before you buy. ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Required Metrics 2000-05-06 0:00 ` Robert Dewar @ 2000-05-07 0:00 ` Ken Garlington 2000-05-07 0:00 ` Robert Dewar 0 siblings, 1 reply; 68+ messages in thread From: Ken Garlington @ 2000-05-07 0:00 UTC (permalink / raw) "Robert Dewar" <robert_dewar@my-deja.com> wrote in message news:8f26s6$lrj$1@nnrp1.deja.com... > > Do you think it's fair to accuse me of such an accusation, > > when the quoted statement doesn't actually say that? > > Yes, I think it's fair, reread what you said. > > > Like I said, "with a pen in the hand, and a song in the > > heart." > > Nope! No "song in the heart", which I take to mean with an > intent other than a formal declaration. Perhaps you have > some other meaning, if so, it is obscure to me. I do have some other meaning, and apparently it is obscure to you (even though you quote its meaning in the following): > > Of course, > > your signature is based on your determination as to which > requirements are > > "real" requirements, believing that all other requirements are > "met" no > > matter what. > > All requirements are requirements. I am not saying that the > documentation requirements are not requirements, just that > they are trivially met. or vacuously met, to quote previous posts? as in: "all other requirements are 'met' no matter what"? Perhaps you have some meaning for vacuous or trivial that is obscure to me! > > You may believe that the ACVC review board, and all other > > vendors, agree with you. > > You mean I agree that there can be no formal tests for whether > you meet the documentation rquirements. I cannot imagine anyone > with any reasonable knowledge thinking otherwise. No, I mean that you may believe that the ACVC review board agrees with you, based on your statement in a previous post: "By the way, the ACVC review board discussed this briefly. They immediately agreed that these requirements were too ill-defined to even consider including them in conformance assessment." Isn't it at least possible that I simply mean what I say? > There is no list of non-requirements, because there are no > non-requirements. THe DOC requires that you have not made > any intentional deviations from the FULL SET of requirements, > including the documentation requirements. > > > You might want to re-read your original statement: "How can > the DOC be > > signed if you have ignored a requirement?" Note that the word > > "intentionally" does not appear. > > intentionally is of course critical. Otherwise the DOC would > be a declaration that your compiler was bug free. This is not > intended, and would be impractical. That's right. You can sign the DOC "with a pen in the hand and a song in the heart", and still make an unintentional mistake. Perhaps, even, a certain kind of unintentional mistake relevant to the topic at hand. And yet, the meaning of my statement above still eludes you. Unfortunate, but there's no point in banging my head against that wall any further. > > > Nope, the issues with pragma Reviewable are also all > > > documentation based, let's look: > > > > Yes, let's. Specifically, let's look at the words which appear > just before > > paragraph 5: > > > > ***** Implementation Requirements ***** > > > > "5 The implementation shall provide the following information > ..." > > > > Now, I suspect you're using the term "documentation" > informally, not in the > > sense of ARM 1.1.2. However, since all Implementation > Requirements, like all > > Documentation Requirements, are not really requirements, > perhaps we _should_ > > use the same name for both. > > Of course implementation requirements are requirements, what > gave you the lunatic idea that this is not the case. I say > lunatic here, because it simply makes no sense. There are many > critical implentation requirements. Where ON EARTH did you > get this idea, certainly not from anything I wrote. Except for the bit you quoted from your earlier post: "Nope, the issues with pragma Reviewable are also all documentation based, let's look:" So, which is it? (a) the ***implementation*** requirements for pragram Reviewable should be treated like documentation requirements (vacuously met), or (b) they shouldn't (i.e. it is possible to fail to meet them). If (a), then how does someone determine which other ***implementation*** requirements are vacuously met, and which are critical? > This is formally definable, and of course tested by the ACVC > tests. Again, I ask where did you get the idea that > implementation requirements are not requirements??????? From you. You should really read the implementation requirements for pragma Reviewable, they're quite extensive. None are tested by the ACVC, of course... > > > When we were discussing these safety and security requirements > during that > > industry-implementors meeting lo these many years ago, I was > frankly > > surprised that the vendor representatives (including yourself) > didn't put up > > more of a fuss. Now I see why. What a total waste of our time. > > Actually many useful things came out of those meetings, but if > you thought that the meetings were about requiring vendors to > provide useful usable implementations, you were indeed deluding > yourself, since language standards can never guarantee this. Actually, I thought we were working on a useful usable standard. That is, an agreement from the vendors as to the text of the implementation requirements included the concept that the vendors understood what they meant! Apparently, I was mistaken. > > !topic Non-Requirement Requirements > > !reference RM95-1.1.2(24-39) > > !from Ken Garlington 00-05-05 > > !keywords requirements, advise > > !discussion > > > > The referenced paragraphs define several different categories > of the > > standard. Some of these categories use the term > "requirements," although > > they are not considered requirements by the Ada vendor > community. There > > should be an explicit list of which categories can be > considered > > requirements (i.e., it is possible for a vendor to "fail" to > meet it). All > > other categories should be described in the terms currently > used for > > "Implementation Advice." > > > This is plain silly, I can't imagine the ARG giving it much > attention, since it amounts to nothing more than an unfounded > polemic. There is no part of the Ada community that considers > any requirements in the RM not to be requirements, and > certainly the vendor community does not ignore any requirements. Note my definition of a requirement: "i.e., it is possible for a vendor to 'fail' to meet it." So, which is it? (a) There are areas of the standard that have the word "requirement" attached to them which are vacuously met by all vendors. (b) There aren't. You may consider it "plain silly" to take items in (a) and label them implementation advice, and I'm sure the ARG will give it no attention (I mean, how much weight should my opinion hold?). At least, when a user comes to me and says, "if this is crap, why is in the standard?" I can say, "beats me, I tried to get it fixed, but no one cared." > Again, I think the entire problem comes from the fact that you > don't really understand the purpose of a formal language > definition, and that you read the word "requirement" in > (to use your terms) "software engineering" sense, which is > totally misleading in this context. Absolutely! An equally elegant solution to the problem to this would be to put in the front of the standard: "WARNING: Terms used in this standard, such as 'requirement,' do not have their usual English meaning." > (by the way, I suddenly realize that you Ken > converts my general use of semantic into the headings in the > manual, don't make that mistake, semantics includes a lot more > :-) Excellent! Light begins to dawn. Now, the slippery part: How much more? ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Required Metrics 2000-05-07 0:00 ` Ken Garlington @ 2000-05-07 0:00 ` Robert Dewar 2000-05-07 0:00 ` Ken Garlington 0 siblings, 1 reply; 68+ messages in thread From: Robert Dewar @ 2000-05-07 0:00 UTC (permalink / raw) In article <Gt7R4.8719$wb7.695096@news.flash.net>, "Ken Garlington" <Ken.Garlington@computer.org> wrote: > > Nope! No "song in the heart", which I take to mean with an > > intent other than a formal declaration. Perhaps you have > > some other meaning, if so, it is obscure to me. > I do have some other meaning, and apparently it is obscure to > you (even though you quote its meaning in the following): OK, you still don't say what that "other meaning" is. > or vacuously met, to quote previous posts? as in: "all other > requirements are 'met' no matter what"? > > Perhaps you have some meaning for vacuous or trivial that is > obscure to me! Right, as I said earlier, I think the problem you have in understanding just comes from unfamiliarity with the notion of formal language definition, and the history and function of language standards documents. You are expecting them to do something they can NOT by nature do, a common misunderstanding indeed. As I say, closely related to the peculiar notion, so very common in Ada 83 days, that validation guaranteed quality and usability. > No, I mean that you may believe that the ACVC review board > agrees with you, based on your statement in a previous post: > > "By the way, the ACVC review board discussed this briefly. They immediately > agreed that these requirements were too ill-defined to even consider > including them in conformance assessment." Right, this was a completely uncontroversial position. I cannot imagine anyone who understood the notion of language definitions, conformance and validation testing disagreeing. What would you have in mind? That we have a section of the ACVC testing that includes Ken Garlington sitting down, reading the documentation and announcing that it meets his (nowhere defined) criteria for acceptable documentation? Ken is free (more than free, encouraged) to do this as part of his acceptance testing, but only for his own purposes. Actually Ken, you said something VERY revealing earlier on. You talked about tightening up the standard to make compiler vendors do stuff that they had not done in Ada 83 compilers in this area so that the costs could be shared. Let's examine this thought a bit further. First. It's really an inappropriate distortion of the market to attempt to include requirements that YOU want, but the market place does not in general want, in an attempt to get other people to pay for things that you need that they don't want (if they wanted them generally, they would be there, competition works that way). Second, Even ignoring the dubious justification for such an attempt, it is bound to fail. Why? Because language standards are not about providing good documentation, useful information, or even about producing usable compilers. They are nothing more than a set of mathematical formulae, informally expressed in this case, that dictate certain input-output properties that a compiler must have. The language standard by its nature can only ever be a small part of the requirements. You might hope that "beefing up" the standard would help solve YOUR problem and relieve you of YOUR responsibility for devising the set of requirements for your tools and making sure you acquire tools that meet these requirements, but this attempt is bound to fail. Language standards are NOT software engineering requirements documents, and no amount of yelling or complaining can change this fundamental fact. > > Isn't it at least possible that I simply mean what I say? > > > There is no list of non-requirements, because there are no > > non-requirements. THe DOC requires that you have not made > > any intentional deviations from the FULL SET of requirements, > > including the documentation requirements. > > > > > You might want to re-read your original statement: "How can > > the DOC be > > > signed if you have ignored a requirement?" Note that the word > > > "intentionally" does not appear. > > > > intentionally is of course critical. Otherwise the DOC would > > be a declaration that your compiler was bug free. This is not > > intended, and would be impractical. > > That's right. You can sign the DOC "with a pen in the hand and a song in the > heart", and still make an unintentional mistake. Perhaps, even, a certain > kind of unintentional mistake relevant to the topic at hand. And yet, the > meaning of my statement above still eludes you. Unfortunate, but there's no > point in banging my head against that wall any further. > > > > > Nope, the issues with pragma Reviewable are also all > > > > documentation based, let's look: > > > > > > Yes, let's. Specifically, let's look at the words which appear > > just before > > > paragraph 5: > > > > > > ***** Implementation Requirements ***** > > > > > > "5 The implementation shall provide the following information > > ..." > > > > > > Now, I suspect you're using the term "documentation" > > informally, not in the > > > sense of ARM 1.1.2. However, since all Implementation > > Requirements, like all > > > Documentation Requirements, are not really requirements, > > perhaps we _should_ > > > use the same name for both. > > > > Of course implementation requirements are requirements, what > > gave you the lunatic idea that this is not the case. I say > > lunatic here, because it simply makes no sense. There are many > > critical implentation requirements. Where ON EARTH did you > > get this idea, certainly not from anything I wrote. > > Except for the bit you quoted from your earlier post: > > "Nope, the issues with pragma Reviewable are also all documentation based, > let's look:" > > So, which is it? > > (a) the ***implementation*** requirements for pragram Reviewable should be > treated like documentation requirements (vacuously met), or > (b) they shouldn't (i.e. it is possible to fail to meet them). > > If (a), then how does someone determine which other ***implementation*** > requirements are vacuously met, and which are critical? > > > This is formally definable, and of course tested by the ACVC > > tests. Again, I ask where did you get the idea that > > implementation requirements are not requirements??????? > > From you. You should really read the implementation requirements for pragma > Reviewable, they're quite extensive. None are tested by the ACVC, of > course... > > > > > > When we were discussing these safety and security requirements > > during that > > > industry-implementors meeting lo these many years ago, I was > > frankly > > > surprised that the vendor representatives (including yourself) > > didn't put up > > > more of a fuss. Now I see why. What a total waste of our time. > > > > Actually many useful things came out of those meetings, but if > > you thought that the meetings were about requiring vendors to > > provide useful usable implementations, you were indeed deluding > > yourself, since language standards can never guarantee this. > > Actually, I thought we were working on a useful usable standard. That is, an > agreement from the vendors as to the text of the implementation requirements > included the concept that the vendors understood what they meant! > Apparently, I was mistaken. > > > > !topic Non-Requirement Requirements > > > !reference RM95-1.1.2(24-39) > > > !from Ken Garlington 00-05-05 > > > !keywords requirements, advise > > > !discussion > > > > > > The referenced paragraphs define several different categories > > of the > > > standard. Some of these categories use the term > > "requirements," although > > > they are not considered requirements by the Ada vendor > > community. There > > > should be an explicit list of which categories can be > > considered > > > requirements (i.e., it is possible for a vendor to "fail" to > > meet it). All > > > other categories should be described in the terms currently > > used for > > > "Implementation Advice." > > > > > > This is plain silly, I can't imagine the ARG giving it much > > attention, since it amounts to nothing more than an unfounded > > polemic. There is no part of the Ada community that considers > > any requirements in the RM not to be requirements, and > > certainly the vendor community does not ignore any requirements. > > Note my definition of a requirement: "i.e., it is possible for a vendor to > 'fail' to meet it." So, which is it? > > (a) There are areas of the standard that have the word "requirement" > attached to them which are vacuously met by all vendors. > (b) There aren't. > > You may consider it "plain silly" to take items in (a) and label them > implementation advice, and I'm sure the ARG will give it no attention (I > mean, how much weight should my opinion hold?). At least, when a user comes > to me and says, "if this is crap, why is in the standard?" I can say, "beats > me, I tried to get it fixed, but no one cared." > > > Again, I think the entire problem comes from the fact that you > > don't really understand the purpose of a formal language > > definition, and that you read the word "requirement" in > > (to use your terms) "software engineering" sense, which is > > totally misleading in this context. > > Absolutely! An equally elegant solution to the problem to this would be to > put in the front of the standard: > > "WARNING: Terms used in this standard, such as 'requirement,' do not have > their usual English meaning." > > > (by the way, I suddenly realize that you Ken > > converts my general use of semantic into the headings in the > > manual, don't make that mistake, semantics includes a lot more > > :-) > > Excellent! Light begins to dawn. Now, the slippery part: How much more? > > Sent via Deja.com http://www.deja.com/ Before you buy. ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Required Metrics 2000-05-07 0:00 ` Robert Dewar @ 2000-05-07 0:00 ` Ken Garlington 2000-05-07 0:00 ` Robert Dewar 0 siblings, 1 reply; 68+ messages in thread From: Ken Garlington @ 2000-05-07 0:00 UTC (permalink / raw) "Robert Dewar" <robert_dewar@my-deja.com> wrote in message news:8f3s87$bsc$1@nnrp1.deja.com... > In article <Gt7R4.8719$wb7.695096@news.flash.net>, > "Ken Garlington" <Ken.Garlington@computer.org> wrote: > > > (by the way, I suddenly realize that you Ken > > > converts my general use of semantic into the headings in the > > > manual, don't make that mistake, semantics includes a lot > more > > > :-) > > > > Excellent! Light begins to dawn. Now, the slippery part: How > > much more? This, of course, is a key question, which went unanswered in the last response. I thought I'd put it up here, so it wouldn't be missed. > But the trouble is that if a requirement has no formal content, > then in the context of a language standard, it can be met in > almost anyway. The word "almost" is the one that interests me. All we have to do now is decide if it really belongs in that sentence, and we're home free. Earlier, you appeared to be saying that certain items in the standard, labelled as requirements, did not constrain implementor's behavior -- they were vacuously met. Now, you appear to be saying, through the use of the word "almost", that it is possible for an implementor to fail to meet a requirement with "no formal content." Again, to review, there are three questions I care about: 1. Under what conditions could a vendor fail to meet the metrics requirements, the documentation requirements, and other requirements of this type? 2. If there are no such conditions -- they do not constrain the implementation in any way -- is it appropriate to call them requirements? 3. Is there a consensus as to which "requirements" are always met by definition? Note that discussions of what the language standard _should_ contain, or how it should be tested, or whether users should express their needs in language revision meetings (?!?) are interesting, but off-topic. For example: --- OFF-TOPIC CONVERSATIONS FOLLOW --- > > No, I mean that you may believe that the ACVC review board > > agrees with you, based on your statement in a previous post: > > > > "By the way, the ACVC review board discussed this briefly. > They immediately > > agreed that these requirements were too ill-defined to even > consider > > including them in conformance assessment." > > Right, this was a completely uncontroversial position. I cannot > imagine anyone who understood the notion of language > definitions, conformance and validation testing disagreeing. > > What would you have in mind? That we have a section of the ACVC > testing that includes Ken Garlington sitting down, reading the > documentation and announcing that it meets his (nowhere defined) > criteria for acceptable documentation? Ken is free (more than > free, encouraged) to do this as part of his acceptance testing, > but only for his own purposes. Nope, I have in mind what I've said all along. If there are requirements in the standard, then they should in some way constrain the behavior of those implementing them. How they constrain the behavior is a different subject. I'm perfectly comfortable with the statement that some requirements are vague, can't be qualified by testing, etc. Have been from the beginning of the post. For example, I noticed you failed to respond to my answer to your question, "what happens if the vendor decides to write his documentation in Klingon?". That same answer applies here. I know you feel more comfortable bringing up this ACVC strawman, or this "user preference" strawman, as you have done it several times. It has no bearing on my point, which I made in an earlier part of the thread, and will repeat now: IF IT'S A REQUIREMENT, IT SHOULD BE TREATED AS SUCH. THAT IS, IT SHOULD CAUSE THE IMPLEMENTOR TO DO SOMETHING. IF IT'S NOT A REQUIREMENT, IT SHOULD NOT BE IDENTIFIED AS SUCH. USERS (AND VENDORS) SHOULD BE CONFIDENT THEY KNOW WHICH PARTS OF THE STANDARD ARE IDENTIFIED AS REQUIREMENTS, BUT DO NOT CONSTRAIN BEHAVIOR. Clearly, what you don't understand is that a requirement that is always met by definition, is not a requirement. It doesn't constrain behavior, so it's not a requirement. I know you're reading these really fast, and trying very hard to pull out some underlying meaning having to do with testing, etc., but you need to stop. You're just confusing yourself. For a moment, just a moment, assume there's no underlying agenda. Just read these words, slowly: REQUIREMENTS SHOULD CONSTRAIN BEHAVIOR. IF THEY DON'T, IT'S WRONG TO CALL THEM REQUIREMENTS. If you disagree, let me know. From what you've posted elsewhere, it appears that you agree. If that's the case, then all you have to do is identify which "requirements" fall into this category. > Actually Ken, you said something VERY revealing earlier on. You > talked about tightening up the standard to make compiler vendors > do stuff that they had not done in Ada 83 compilers in this > area so that the costs could be shared. > > Let's examine this thought a bit further. > > First. It's really an inappropriate distortion of the market > to attempt to include requirements that YOU want, but the > market place does not in general want, in an attempt to get > other people to pay for things that you need that they don't > want (if they wanted them generally, they would be there, > competition works that way). Isn't this why you have a standards process? Users bring ideas to the table; they're discussed by other users, implementors, and other interested parties, and if THERE'S A CONSENSUS they get implemented? I brought a requirement to an SEI meeting with Goodenough and co. during the language revision process for "static pointers." We were having to do a lot of non-portable things like Unchecked_Conversion in Ada83 to achieve this. The 95 standard has new features that address this need. Did I "distort the market" by doing this? If so, why did you guys ask me for inputs? Seems to me the argument "competition works that way" could imply that there's no real reason to have an ISO standard. It seems to be working reasonably well with Java; I haven't had any problems porting between MS Visual J++ and Sun Java, for example. With respect to whether others in the market place want the requirements of pragma Reviewable; at least for the ones I wanted, Aonix seems to think there's a market, and I didn't even have to pay them! Too bad we couldn't get the vendors together and say, "if you're going to implement this sort of feature, we'd like to have some level of standardization as to what it means. Is this possible?" > Second, Even ignoring the dubious justification for such an > attempt, it is bound to fail. Why? Because language standards > are not about providing good documentation, useful information, > or even about producing usable compilers. They are nothing more > than a set of mathematical formulae, informally expressed in > this case, that dictate certain input-output properties that > a compiler must have. The language standard by its nature can > only ever be a small part of the requirements. Fine. So which parts of the standard are this "formal English," and which aren't? For example, the following implementation requirement seems to dictate certain input-output properties that a compiler must have: "The implementation shall provide control- and data-flow information, both within each compilation unit and across the compilation units of the partition." Input -> source code defining compilation units of the partition. Output -> Control-flow and data-flow information. I know it's not testable through an ACVC-style test, and that people can differ on the meaning of the definition, and that this requirement "distorts" the standard. I just don't see how it answers the question. Please don't spend too much time explaining all of this to me again. It's a simple "yes" or "no" question. > You might hope that "beefing up" the standard would help solve > YOUR problem and relieve you of YOUR responsibility for devising > the set of requirements for your tools and making sure you > acquire tools that meet these requirements, but this attempt is > bound to fail. You've made it fully clear that a language standard does not exist to address user needs. ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Required Metrics 2000-05-07 0:00 ` Ken Garlington @ 2000-05-07 0:00 ` Robert Dewar 2000-05-07 0:00 ` Ken Garlington 0 siblings, 1 reply; 68+ messages in thread From: Robert Dewar @ 2000-05-07 0:00 UTC (permalink / raw) In article <mUfR4.8835$wb7.709941@news.flash.net>, "Ken Garlington" <Ken.Garlington@computer.org> wrote: > IF IT'S A REQUIREMENT, IT SHOULD BE TREATED AS SUCH. THAT IS, > IT SHOULD CAUSE THE IMPLEMENTOR TO DO SOMETHING. Sounds reasonable to me! No one ever argued any differently. In fact most people would go further and say that if something is not testable, then it's not a requirement. > IF IT'S NOT A REQUIREMENT, IT SHOULD NOT BE IDENTIFIED AS > SUCH. I think you mean "if it's not a requirement in the above sense". Obviously requirements are precisely things that are identified as requirements, so the above sentence stated as you gave it is tautologically true. > USERS (AND VENDORS) SHOULD BE CONFIDENT THEY KNOW WHICH PARTS > OF THE STANDARD ARE IDENTIFIED AS REQUIREMENTS, BUT DO NOT > CONSTRAIN BEHAVIOR. Well I'm pretty confident, since it's pretty clear, sorry that it continues to be unclear for you! > Clearly, what you don't understand is that a requirement that > is always met by definition, is not a requirement. So Ken Garlington pronounces, but can Ken Garlington prove this statement from the RM. I think not. Again, the "is not" here is confusing, I think you mean "should not be". It is quite clear in the RM what is and what is not a requirement, from the headings. The fact that the definition of requirement does not meet your definition is pretty much irrelevant from the point of view of analyzing the existing RM. You could get lots of people to agree with a statement that all requirements in the reference manual should be testable, at least conceptually. > REQUIREMENTS SHOULD CONSTRAIN BEHAVIOR. IF THEY DON'T, IT'S > WRONG TO CALL THEM REQUIREMENTS. Ah now that of course is a much better form, because now you are giving Ken Garlington's opinion. Do I agree? Well I don't understand "behavior" in a mathematical sense, it sounds more like a software engineering sense. If the above statement is equivalent to: REQUIREMENTS SHOULD BE TESTABLE. IF THEY ARE NOT, IT'S WRONG TO CALL THEM REQUIREMENTS. Fits better into a language standard, and that of course I agree with. Remember that Bob Duff and I both strongly objected to all these ill-defined "requirements" in the reference manual. Informally I agree with your statement, it is just not stated in a well defined manner, because it uses terms "constrain" and "behavior" that cannot be given formal semantic meaning. Remember we are ALWAYS in the realm of the formal semantic definition, so pronouncements about what this formal semantic definition should or should not contain should try to avoid making the mistake of themselves not being formally stated. The standard is NOT about constraining implementors, in fact the standard has nothing to do with implementors at all. It is simply a definition of what it means to conform to the ISO standard. If someone constrains an implementor to conform to the standard, then the language of the standard will be relevant. But it is the someone, not the requirements, that are doing the constraining. That's an important difference, because for example, the ACVC test suite IS all about constraining implementors and has quite a different viewpoint than the standard itself. > If that's the case, then all you have to do is identify > which "requirements" fall into this category. Nope, I don't *have* to do anything of the kind, it does not seem important to me to do so at this stage for several reasons: 1) I don't think any real users are affected by this one way or another. Theoretical conversations on CLA don't have very much to do with real users. 2) We went through the excercise during the design process, and surprisingly lots of people wanted to keep in these ill-defined "requirements". There is no reason to think that anything has changed. The fact that Ken Garlington may or many not understand things differently is irrelevant. Certainly Robert Dewar's understanding has not been changed one iota by this exchange. He still thinks that documentation requirements etc should never have been included in the standard as normative, and nothing that has been said here has changed his mind, but he never makes the mistake of rearguing old points unless there is new data! 3) The ARG has many more important things to do than sit around in angels-on-a-pin-head discussions that have no actual effect either on compiler writers or compiler users. > Isn't this why you have a standards process? Users bring ideas > to the table; they're discussed by other users, implementors, > and other interested parties, and if THERE'S A CONSENSUS they > get implemented? Only for well defined semantic features. This discussion is all about things not meeting that requirement. Go look up your previous post where you mention sharing costs, you were not talking about static pointers, but about metrics! > "if you're going to implement this sort of > feature, we'd like to have some level of standardization as to > what it means. Is this possible?" Just as I thought you were beginning to understand :-( Now, not only are you defending requirements like the documentation requirements for pragma Reviewable, you are suggesting that this information be output in a standardized manner -- worse and worse if you ask me! Do you want to specify the font used :-) > Fine. So which parts of the standard are this "formal > English," and which > aren't? The entire normative part of the standard should be read as formal english. I am surprised anyone would ask this question. It just goes to show the danger of making standards accessible :-) Don't get me wrong, I think it is great for the standard to be accessible, and I am one person who liked the less formal style of the Ada 83 standard better than the more formal language of the Ada 95 standard, but it does have this risk of people reading things and interpreting them informally. > For example, the following implementation requirement seems to > dictate > certain input-output properties that a compiler must have: > "The implementation shall provide control- and data-flow > information, both within each compilation unit and across the > compilation units of the > partition." It might, except the phrase "control-flow information" is undefined, and even as an expert compiler writer, I can only guess what it might mean. Actually I don't think it dictates any input-output properties, since the RM makes clear that documentation on implementation dependent features (most certainly this information is in that category) can be provided by simply pointing to HOW to find out the information. I assume this means that "control flow information" can be determined by following a debugger, now that's REAL useful. Perhaps the writers of this paragraph had something specific in mind, but they sure did not express it. I suspect if they HAD tried to be more specific, no one would have put up with it even in annex H (which is worse than other sections of the RM in many respects when it comes to being formal and precise). > I know it's not testable through an ACVC-style test, and that people can > differ on the meaning of the definition, and that this requirement > "distorts" the standard. I just don't see how it answers the question. > Please don't spend too much time explaining all of this to me again. It's a > simple "yes" or "no" question. No it isn't, I can't find any simple yes or no question in what you wrote. > You've made it fully clear that a language standard does not > exist to address user needs. And that's a bit of a pathetic, but VERY revealing statement. The language standard exists for one purpose and one purpose only, to define exactly what it means for a compiler to conform to the ISO standard. Ken Garlington may not find that this narrow objective addresses his "user needs", but I can *assure* you that the great majority of our users find the existence of the standard and the associated conformance testing vital, and it most certainly addresses their needs. Now of course they do NOT make Ken's mistake of thinking that the standard can do any more than that, and that my some magic rewording of the standard, he will no longer have to write special requirements for his compilers, and will be able to avoid spending unreasonable amounts of money getting the tools they need. It's been useful to have someone like Ken put into very clear perspective the frustration that he and others feel that the standard cannot guarantee them high level compilers, and you can see from his posts how reluctant he is to abandon this goal. Moreover, as he begins to see that this guarantee might not be possible, he rushes to the conclusion that the standard does "exist to meet user's needs". This viewpoint of the standard as a complete requirements document is an amazingly common one. But it was never and will never be a useful one. We probably even still have people around who hold the closely related fantasy that validation testing of compilers will assure quality. This is a closely related issue, and the natural consequence of Ken's view of the standard is presumably that ACVC testing does not "exist to meet user's needs", since it shows partial conformance to the standard, nothing less and nothing more. Sent via Deja.com http://www.deja.com/ Before you buy. ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Required Metrics 2000-05-07 0:00 ` Robert Dewar @ 2000-05-07 0:00 ` Ken Garlington 2000-05-07 0:00 ` Robert Dewar 0 siblings, 1 reply; 68+ messages in thread From: Ken Garlington @ 2000-05-07 0:00 UTC (permalink / raw) "Robert Dewar" <robert_dewar@my-deja.com> wrote in message news:8f49je$pti$1@nnrp1.deja.com... > In article <mUfR4.8835$wb7.709941@news.flash.net>, > "Ken Garlington" <Ken.Garlington@computer.org> wrote: > > > IF IT'S A REQUIREMENT, IT SHOULD BE TREATED AS SUCH. THAT IS, > > IT SHOULD CAUSE THE IMPLEMENTOR TO DO SOMETHING. > > Sounds reasonable to me! No one ever argued any differently. Excellent! So, when you were implementing the requirement described in H.3.1 that says, "The implementation shall provide control- and data-flow information, both within each compilation unit and across the compilation units of the partition," what was the something you did in the implementation to meet that requirement? > > Clearly, what you don't understand is that a requirement that > > is always met by definition, is not a requirement. > > So Ken Garlington pronounces, but can Ken Garlington prove this > statement from the RM. I think not. No, but I can "prove" this by referring to the nearest dictionary: "requirement. something essential to the existence or occurrence of something else." If there are statements in the RM that are not essential to the existence of anything in the implementation, why would they be considered requirements? Perhaps the language specification does not use terms in either the software engineering or English language sense? I can, of course, also indirectly infer the meaning of requirement from 1.1.2:37. > > IF IT'S NOT A REQUIREMENT, IT SHOULD NOT BE IDENTIFIED AS > > SUCH. > > I think you mean "if it's not a requirement in the above > sense". Obviously requirements are precisely things that are > identified as requirements, so the above sentence stated as > you gave it is tautologically true. No more than "if an apple is identified as a banana, it's a banana ." Apparently, however, this does not translate to language specifications. If I understand you correctly, I could replace the word "requirement" with "fodasugfdu" with no ill effects, or (more importantly) replace "advice" with "requirement". > > USERS (AND VENDORS) SHOULD BE CONFIDENT THEY KNOW WHICH PARTS > > OF THE STANDARD ARE IDENTIFIED AS REQUIREMENTS, BUT DO NOT > > CONSTRAIN BEHAVIOR. > > Well I'm pretty confident, since it's pretty clear, sorry that > it continues to be unclear for you! Can you then translate that confidence into words more precise than "formal semantics" (which, as we've learned, is not related to the terminology used in the ARM), or "a specification of input-output relationships" (for which counter-examples exist; e.g. pragma Reviewable), so that we know which parts of the RM are essential to the existence or occurence of something else? > Nope, I don't *have* to do anything of the kind, it does not > seem important to me to do so OK, then why are you responding to this thread? Simply say, "Sorry -- your problem is just not interesting enough to work" and be done with it? It seems important to me to do so, because: > I think it's definitely a problem, if for no other reason than it leads you and others to be confused. Given that, it is strange to conclude that: > It is quite clear in the RM what is and what is not a > requirement, from the headings. The fact that the definition > of requirement does not meet your definition is pretty much > irrelevant from the point of view of analyzing the existing > RM. Just as it is quite clear from the headings what is and is not formal semantics, I presume? Just as you continue to refer to implementation requirements as documentation requirements, as in: > Now, not only are you defending requirements like the > documentation requirements for pragma Reviewable, And yet, somehow the phrase "well defined" continues to appear, as in > > Isn't this why you have a standards process? Users bring ideas > > to the table; they're discussed by other users, implementors, > > and other interested parties, and if THERE'S A CONSENSUS they > > get implemented? > > Only for well defined semantic features. Where each vendor not only gets to define those features, but in fact may not be able to articulate which specific parts of the standard are in that category? That doesn't seem very "well defined". > Go look up your > previous post where you mention sharing costs, you were not > talking about static pointers, but about metrics! Actually, I was talking about the "implementation requirements" of pragma Reviewable -- the idea being that if users in a "special needs" area (like, say Safety and Security) commonly needed some output from the compiler (e.g., those commonly required by DO-178B), that it made sense to identify that output in a standard, to the level of detail that the implementors could stand (which, of course, would not include the report content or font, or even if it were generated as a standard "report" at all). I don't think I provided one word of input regarding metrics during the standarization process. However, now that I've learned that: > The standard is NOT about constraining implementors, in fact > the standard has nothing to do with implementors at all. then I guess it doesn't make any sense to talk about the standard with implementors! [the usual strawmen of testing, the extent to which the standard addresses all the users needs, etc. ignored] ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Required Metrics 2000-05-07 0:00 ` Ken Garlington @ 2000-05-07 0:00 ` Robert Dewar 0 siblings, 0 replies; 68+ messages in thread From: Robert Dewar @ 2000-05-07 0:00 UTC (permalink / raw) I'm signing off from this thread now. I think I have said everything I want to say (more than once), and it is simply beginning to repeat itself :-) Sent via Deja.com http://www.deja.com/ Before you buy. ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Required Metrics 2000-05-01 0:00 ` Ken Garlington 2000-05-02 0:00 ` Ken Garlington @ 2000-05-02 0:00 ` Ted Dennison 2000-05-04 0:00 ` Robert Dewar 1 sibling, 1 reply; 68+ messages in thread From: Ted Dennison @ 2000-05-02 0:00 UTC (permalink / raw) In article <OKnP4.3694$wb7.343653@news.flash.net>, "Ken Garlington" <Ken.Garlington@computer.org> wrote: > Alternate phrasing of the question: If a requirement is in the > standard, and no one makes an effort to follow it, what's the > requirement doing in the standard? Why not have an interpretation that > says, in essence: "Oops. Never mind"? Or am I just missing some > fundamental point here? :-) I'm glad to see I'm not the only one who had this reaction. If a requirement falls in the forest... -- T.E.D. http://www.telepath.com/~dennison/Ted/TED.html Sent via Deja.com http://www.deja.com/ Before you buy. ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Required Metrics 2000-05-02 0:00 ` Ted Dennison @ 2000-05-04 0:00 ` Robert Dewar 2000-05-04 0:00 ` Ted Dennison 2000-05-05 0:00 ` Ken Garlington 0 siblings, 2 replies; 68+ messages in thread From: Robert Dewar @ 2000-05-04 0:00 UTC (permalink / raw) In article <8emkv7$uf6$1@nnrp1.deja.com>, Ted Dennison <dennison@telepath.com> wrote: > > Alternate phrasing of the question: If a requirement is in the > > standard, and no one makes an effort to follow it, what's the > > requirement doing in the standard? Why not have an interpretation that > > says, in essence: "Oops. Never mind"? Or am I just missing some > > fundamental point here? Well of course, this should be the point of view. And indeed properly following that point of view would have avoided all the meaningless documentation requirements in the RM. The ARG does in fact sometimes decide that things that look like requirements in the RM are meaningess. For instance we recently discovered that the Suppress (.., On=> ...) form of this pragma is completely meaningless, and the language in the standard is bogus -- compilers do different things, and what appears like a requirement is in fact in practice a statement that this feature is completely implementation dependent. Sent via Deja.com http://www.deja.com/ Before you buy. ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Required Metrics 2000-05-04 0:00 ` Robert Dewar @ 2000-05-04 0:00 ` Ted Dennison 2000-05-05 0:00 ` Ken Garlington 1 sibling, 0 replies; 68+ messages in thread From: Ted Dennison @ 2000-05-04 0:00 UTC (permalink / raw) In article <8es60q$5fm$1@nnrp1.deja.com>, Robert Dewar <robert_dewar@my-deja.com> wrote: > In article <8emkv7$uf6$1@nnrp1.deja.com>, > Ted Dennison <dennison@telepath.com> wrote: > > > > Alternate phrasing of the question: If a requirement is in Just a nit here, but none of the quoted text is mine. I eventually resigned myself to not trusting the "required" documentation last year when this issue came up. Its tough to regenerate any outrage about it now. :-) -- T.E.D. http://www.telepath.com/~dennison/Ted/TED.html Sent via Deja.com http://www.deja.com/ Before you buy. ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Required Metrics 2000-05-04 0:00 ` Robert Dewar 2000-05-04 0:00 ` Ted Dennison @ 2000-05-05 0:00 ` Ken Garlington 2000-05-05 0:00 ` Robert Dewar 1 sibling, 1 reply; 68+ messages in thread From: Ken Garlington @ 2000-05-05 0:00 UTC (permalink / raw) "Robert Dewar" <robert_dewar@my-deja.com> wrote in message news:8es60q$5fm$1@nnrp1.deja.com... > In article <8emkv7$uf6$1@nnrp1.deja.com>, > Ted Dennison <dennison@telepath.com> wrote: > > > > Alternate phrasing of the question: If a requirement is in > the > > > standard, and no one makes an effort to follow it, what's > the > > > requirement doing in the standard? Why not have an > interpretation that > > > says, in essence: "Oops. Never mind"? Or am I just missing > some > > > fundamental point here? > > > Well of course, this should be the point of view. And indeed > properly following that point of view would have avoided all > the meaningless documentation requirements in the RM. > > The ARG does in fact sometimes decide that things that look > like requirements in the RM are meaningess. For instance we > recently discovered that the Suppress (.., On=> ...) form > of this pragma is completely meaningless, and the language > in the standard is bogus -- compilers do different things, > and what appears like a requirement is in fact in practice > a statement that this feature is completely implementation > dependent. So, why are some meaningless requirements addressed by the ARG, and others not? (By the way, I think I was the one to make the comment quoted at the top, not Ted Dennison, although by the end I may wish it had been someone else :) ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Required Metrics 2000-05-05 0:00 ` Ken Garlington @ 2000-05-05 0:00 ` Robert Dewar 0 siblings, 0 replies; 68+ messages in thread From: Robert Dewar @ 2000-05-05 0:00 UTC (permalink / raw) In article <CJoQ4.7913$wb7.555817@news.flash.net>, "Ken Garlington" <Ken.Garlington@computer.org> wrote: > So, why are some meaningless requirements addressed by the > ARG, and others not? Because the ARG regards it as a waste of time to spend any time on the issue of obviously subjective requirements like the documentation requirements. All it takes is one member of the ARG to press to raise this issue, but no one has and no one is likely to. Even those who supported the inclusion of these requirements in the RM have no inclination to revisit the matter. I personally agree it would be a waste of time. But if there is a requirement which on its face *appears* to be well defined, but on careful technical analysis is not, then that's a good point for the ARG to resolve. In the particular case (Suppress ON) that I mentioned, this came up during the attempt to semi-standardize the GNAT pragma Unsuppress, and the question was what does UNsuppress/ON mean. THe facile answer: the same as Suppress/ON. But Unsuppress has far more potent semantics since it guarantees that a check will not be performed, unlike the case with Suppress/ON which guarantees nothing at all. So we really needed to be more precise, and we discovered that we were building on shifting sands here. Sent via Deja.com http://www.deja.com/ Before you buy. ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Required Metrics 2000-05-01 0:00 ` Required Metrics Ted Dennison 2000-05-01 0:00 ` Tucker Taft @ 2000-05-01 0:00 ` Ken Garlington 2000-05-04 0:00 ` Robert Dewar 2000-05-04 0:00 ` Roger Barnett 1 sibling, 2 replies; 68+ messages in thread From: Ken Garlington @ 2000-05-01 0:00 UTC (permalink / raw) "Ted Dennison" <dennison@telepath.com> wrote in message news:8ek4ea$5ta$1@nnrp1.deja.com... > In article <5DDO4.2237$wb7.194854@news.flash.net>, > "Ken Garlington" <Ken.Garlington@computer.org> wrote: > > In the Ada Reference Manual, paragraphs D.8:37-45 states that "The > > implementation shall document" a series of metrics related to the > ... > > details of that configuration." However, although my compiler's > > reference manual claims that "The real-time systems annex is fully > > implemented", it goes on to say " Information on metrics is not yet > > available." Does a compiler fully implement the real-time systems > > annex if it does not meet the documentation requirements? > > I remember asking almost this exact question here about a year and a > half ago. You can search deja for the thread. But the answer I remember > getting was that since there is (and can be) no test in the Ada > validation test suite for documentation requirements, implementors can > pretty much get away with murder here. I'm not surprised, but I don't think it really answers my question. The issue isn't, in my mind, "How easy is it for vendors to ignore certain requirements?". It's "Is there really a requirement?" (See my response to Tucker's post for more...) ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Required Metrics 2000-05-01 0:00 ` Ken Garlington @ 2000-05-04 0:00 ` Robert Dewar 2000-05-05 0:00 ` Ken Garlington 2000-05-04 0:00 ` Roger Barnett 1 sibling, 1 reply; 68+ messages in thread From: Robert Dewar @ 2000-05-04 0:00 UTC (permalink / raw) In article <BQnP4.3698$wb7.344147@news.flash.net>, "Ken Garlington" <Ken.Garlington@computer.org> wrote: > I'm not surprised, but I don't think it really answers my question. The > issue isn't, in my mind, "How easy is it for vendors to ignore certain > requirements?". It's "Is there really a requirement?" (See my response to > Tucker's post for more...) For a validated compiler -- difficult! In particular, how can the DOC be signed if you have ignored a requirement. Once again, the documentation requirements, the primary subject of this thread, are in a rather special category, since these are untestable, undefined requirements. Normally one would not expect to find untestable, undefined stuff in a language standard, and for the most part, one does not in the Ada RM, but I am afraid there are exceptions :-) Sent via Deja.com http://www.deja.com/ Before you buy. ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Required Metrics 2000-05-04 0:00 ` Robert Dewar @ 2000-05-05 0:00 ` Ken Garlington 2000-05-05 0:00 ` Robert Dewar 0 siblings, 1 reply; 68+ messages in thread From: Ken Garlington @ 2000-05-05 0:00 UTC (permalink / raw) "Robert Dewar" <robert_dewar@my-deja.com> wrote in message news:8es65n$5hn$1@nnrp1.deja.com... > In article <BQnP4.3698$wb7.344147@news.flash.net>, > "Ken Garlington" <Ken.Garlington@computer.org> wrote: > > > I'm not surprised, but I don't think it really answers my > question. The > > issue isn't, in my mind, "How easy is it for vendors to ignore > certain > > requirements?". It's "Is there really a requirement?" (See my > response to > > Tucker's post for more...) > > > For a validated compiler -- difficult! In particular, how can > the DOC be signed if you have ignored a requirement. I suspect with a pen in the hand, and a song in the heart! I think you read the statement backwards, by the way. To repeat: The issue isn't, in my mind, "How easy is it for vendors to ignore certain requirements?". It's "Is there really a requirement?" > Once again, the documentation requirements, the primary subject > of this thread, are in a rather special category, since these > are untestable, undefined requirements. Normally one would not > expect to find untestable, undefined stuff in a language > standard, and for the most part, one does not in the Ada RM, > but I am afraid there are exceptions :-) I suspect, after you read the discussion of pragma Reviewable, we're going to be expanding this to "documentation and some implementation requirements" ;) ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Required Metrics 2000-05-05 0:00 ` Ken Garlington @ 2000-05-05 0:00 ` Robert Dewar 2000-05-05 0:00 ` Ted Dennison 2000-05-07 0:00 ` Robert I. Eachus 0 siblings, 2 replies; 68+ messages in thread From: Robert Dewar @ 2000-05-05 0:00 UTC (permalink / raw) In article <%MoQ4.7915$wb7.556168@news.flash.net>, "Ken Garlington" <Ken.Garlington@computer.org> wrote: > > For a validated compiler -- difficult! In particular, how > > can the DOC be signed if you have ignored a requirement. > > I suspect with a pen in the hand, and a song in the heart! Ken, do you think it is fair to accuse vendors of dishonesty and duplicity with no evidence at all to back up your statements. I will say right away that I personally sign all DOC's for Ada Core Technologies, and in all cases I do so with integrity knowing that what I am signing is absolutely correct. I would assume that all other vendors would make the same statement. > I think you read the statement backwards, by the way. To > repeat: The issue > isn't, in my mind, "How easy is it for vendors to ignore > certain > requirements?". It's "Is there really a requirement?" For the case of documentation requirements, the answer is basically no, it is not a requirement at all in the usual formal semantic sense. > > Once again, the documentation requirements, the primary subject > > of this thread, are in a rather special category, since these > > are untestable, undefined requirements. Normally one would not > > expect to find untestable, undefined stuff in a language > > standard, and for the most part, one does not in the Ada RM, > > but I am afraid there are exceptions :-) > I suspect, after you read the discussion of pragma Reviewable, we're going > to be expanding this to "documentation and some implementation requirements" Nope, the issues with pragma Reviewable are also all documentation based, let's look: The definition of pragma reviewable is in two sections BOTH of them have the language: THe implementation shall provide the following information .... Well, "shall" is an impressive word, it sure *sounds* like a requirement. Unfortunately, what on earth does it mean to "provide the following information". Ooops, totally undefined. And since the information for ALL sections is equivalent to information provided in the object file, you can argue from a formal point of view that the object file satisfies the requirement. Now of course from a pragmatic point of view, we want the information in a much more usable form, and GNAT provides a lot of *non-required* capabilities in this area. For example, let's take: 6 Where compiler-generated run-time checks remain; From a formal point of view, we could just say "look in the object module for all calls to __gnat_raise_constraint_error or whatever, but in practice it is useful (but not required) to have this information in a clearer source-related fashion. So GNAT provides the generated source listing, where for example if you write x := a(b,c); in the source, then the generated source listing (-gnatG) might say: x := a(b,{c}); where, as documented in the GNAT documentation, the curly brackets mean that the expression inside is range checked at run time. That's a nice feature, very useful for users, and definitely meeting the spirit of the requirement quoted above. But in a formal language definition, we are not in the business of meeting the spirit of requirements! It would have been SO much better to make all these documentation "requirements" into implementation advice for several reasons: 1. In IA sections, since they are not part of the formal standard, we are much freer to make informal requirements of the "you-will-know-it-when-you-see-it" form. 2. IA sections get interpreted in a pragmatic manner, which is just right for this purpose. 3. Implementors have to document which IA they follow and which they do not, leading to a useful account of how each IA section is addressed (see the section in the GNAT RM that does this for example). Sent via Deja.com http://www.deja.com/ Before you buy. ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Required Metrics 2000-05-05 0:00 ` Robert Dewar @ 2000-05-05 0:00 ` Ted Dennison 2000-05-06 0:00 ` Robert Dewar 2000-05-07 0:00 ` Robert I. Eachus 1 sibling, 1 reply; 68+ messages in thread From: Ted Dennison @ 2000-05-05 0:00 UTC (permalink / raw) In article <8eulom$u8m$1@nnrp1.deja.com>, Robert Dewar <robert_dewar@my-deja.com> wrote: > It would have been SO much better to make all these > documentation "requirements" into implementation advice > for several reasons: > > 3. Implementors have to document which IA they follow and > which they do not, leading to a useful account of how > each IA section is addressed (see the section in the > GNAT RM that does this for example). But that is a documentation requirement too. If documentation requirements became implementation advice, implementors would no loger *have* to document which IA they follow, right? Or is that your point? -- T.E.D. http://www.telepath.com/~dennison/Ted/TED.html Sent via Deja.com http://www.deja.com/ Before you buy. ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Required Metrics 2000-05-05 0:00 ` Ted Dennison @ 2000-05-06 0:00 ` Robert Dewar 0 siblings, 0 replies; 68+ messages in thread From: Robert Dewar @ 2000-05-06 0:00 UTC (permalink / raw) In article <8eutl8$7gh$1@nnrp1.deja.com>, Ted Dennison <dennison@telepath.com> wrote: > In article <8eulom$u8m$1@nnrp1.deja.com>, > Robert Dewar <robert_dewar@my-deja.com> wrote: > > > It would have been SO much better to make all these > > documentation "requirements" into implementation advice > > for several reasons: > > > > 3. Implementors have to document which IA they follow and > > which they do not, leading to a useful account of how > > each IA section is addressed (see the section in the > > GNAT RM that does this for example). > > But that is a documentation requirement too. If documentation > requirements became implementation advice, implementors would no loger > *have* to document which IA they follow, right? Or is that > your point? No, you are missing my point. The IA is stuff that we interpret informally (in Ken's terms, from a "software engineering" point of view). If someone fails to document which IA they follow in a convenient manner, then they are not formally non-conformant, but in practice this will not be an issue. It would surprise me if any vendor does not provide this information, and if they don't you should definitely complain. Right now, there is no suggestion AT ALL in the RM, formal or informal, that vendors should describe how they meet the documentation "requirements". Why not? Because they are treated like any other requirements. I don't have to document that I meet the requirement of 3.2.1: 5 A given type shall not have a subcomponent whose type is the given type itself. or document *how* I meet it, since it is a well defined semantic requirement (by the way, I suddenly realize that you Ken converts my general use of semantic into the headings in the manual, don't make that mistake, semantics includes a lot more :-) Similarly the "requirements" associated with pragma Reviewable do not have any associated documentation requirements on how they are met, since they are treated by the standard as equivalent to the 3.2.1(5) statement above, even though they are not. That's too bad. When we did the documentation for GNAT, we went all through annex M making sure we provided all this information, but we do not include documentation on how we provide the pragma Reviewable information, because it's not in Annex M. Now I definitely agree that this is useful information to provide, and I made a note to add this to our docs for some future release, but if the documentation requirements had been in IA sections, you would have got this automatically. In practice, making the documentation requirements IA would INCREASE their utility, not decrease it. Sent via Deja.com http://www.deja.com/ Before you buy. ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Required Metrics 2000-05-05 0:00 ` Robert Dewar 2000-05-05 0:00 ` Ted Dennison @ 2000-05-07 0:00 ` Robert I. Eachus 2000-05-07 0:00 ` Ken Garlington ` (2 more replies) 1 sibling, 3 replies; 68+ messages in thread From: Robert I. Eachus @ 2000-05-07 0:00 UTC (permalink / raw) Robert Dewar wrote: > Well, "shall" is an impressive word, it sure *sounds* like > a requirement. Unfortunately, what on earth does it mean to > "provide the following information". Ooops, totally undefined. > And since the information for ALL sections is equivalent to > information provided in the object file, you can argue from > a formal point of view that the object file satisfies the > requirement. Let me see if I can spread a little oil on the water here. Ken sees the shalls, and is used to saying "Aha, a requirement. I must find all such requirements--explicit, implicit, and derived--during requirements analysis, allocate them, and plan to test them." I've spent time in this environment as well, and so had a lot of those developing and approving the Ada standard. It somehow seems wrong to use shall in a non-testable 'requirement.' The requirement should either be made testable, or not treated as a requirement. On the other hand, everyone involved in the process was aware of what had happened in Ada 83 with most of Chapter 13. Some vendors ignored the parts that weren't tested, and other vendors since the requirements there were not tested put implementation of those features at low priority. (If a customer came alone with a need and sufficient cash, they move up the list. But the users were not particularly happy with that either.) So why were there no meaningful Chapter 13 tests? Not because there were not real requirements there, but because the real requirements were not testable, and the parts that were formally testable were such that tests were counterproductive. For example, how do you design a portable test for the bit patterns used to represent enumeration types? It is much more difficult than it seems, since S'Pos returns the position number, not the representation. Of course, a new attribute could be required, that would make the enumeration representation clause testable, but that is adding a significant burden. (Note that RM 95 says in 13.4(11) that "Unchecked_Conversion may be used to query the internal codes used for an enumeration type." But this is a note, and intentionally does not say how to do it or to place any requirements on the results of such conversions, so this is not even a derived requirement. You can, from real requirements derive some tests that are useful, but not necessarily as useful as you would expect due to byte-sex and bit ordering issues. see 13.5.3.) For Ada 95, there was a strong desire to make Chapter 13 more useful. But since the nature of Chapter 13 is to be very implementation dependent, and difficult to test, users, especially embedded system users, wanted a stronger "moral" commitment to features that they felt were necessary, even if they were untestable. Such moral commitments became Implementation Advice and Documentation Requirements. But note that in most Documentation Requirements which require a measurement, the reason for adding the requirement was not to ensure that these measurements appear in some manual that may bear little or no relation to the intended hardware configuration. The idea was that for those particular quantities, it was important that it be possible to determine what they are. A computer program that performed the measurements is more than adequate documentation, as long as the underlying moral commitment is met. Since for most of the numeric documentation requirements, those programs exist, the requirement is trivially met IF THE QUANTITY IS MEASURABLE. So in most (all?) cases, the existance of an ACES report meets the documentation requirements as long as the compiler does not engage in bizarre behavior. (For example, if redezvous take an extra three milliseconds on alternate Tuesdays, that needs to be in the documentation.) One last word on testable requirements. Is a requirement that an error will be raised if the sun rises in the West testable? Not really. The requirement is clear, but it is not possible to create the environment necessary to perform the test. What about a requirement to document the number of planets in the solar system? Somewhat testable, but what do you do if the documentation says 10? Is the documentation wrong, or is there a planet you don't know about? This is why many of the documentation requirements are untestable. For example, the intent of D.8(10) is clear: "An upper bound on the execution time, in processor clock cycles, of a delay_relative_statement whose requested value of the delay expression is less than or equal to zero." If the documentation says the upper limit is one million clocks is it useful? Probably not. Testable? Not likely. It is possible that a test program would require more than a million clock cycles for some input, and by luck your testing finds that case. But more more likely is that you have a meaningless test that can only fail to falsify the documentation. If the documentation specified 50 clocks, the test is much more interesting, but from a formal point of view nothing has changed. > Now of course from a pragmatic point of view, we want the > information in a much more usable form, and GNAT provides > a lot of *non-required* capabilities in this area. For > example... And this is where Robert Dewar the implementor, can and will give much more acceptable answers than Robert Dewar the langauge lawyer. Real users don't want useless volumes of meaningless measurements, they want answers to the real questions that lay behind the Documentation Requirements. There shouldn't be--and there is not--a requirement that the documentation must be in 10 point Sans-Serif font on 8 1/2" by 11" paper with one inch margins. The real requirement is that the information should be available to the user when needed and in a form that applies to the target system. So the real checkoff for any of the documentation requirements is not, where is the piece of paper? It is the ability of the user to obtain this information when needed. If ACT chooses to provide this for their validated compilers in the form of good customer support, great. If some other vendor chooses to provide a program which when compiled and run prints out the documentation, that's good too. And maybe some vendor chooses to provide a set of formulas which can be used to get the numbers. That is also acceptable, even if it is not very convenient. So there is no conspiracy here by compiler vendors or language lawyers to dilute or subvert the meaning of the Reference Manual. The ARG has enough work to do dealing with other issues. If the issue can be resolved by writing a page of code, then believe me, that is much easier than creating an AI. The compromise, if it can be called that, is found in RM 1.1.3(19): "The implementation may choose to document implementation-defined behavior either by documenting what happens in general, or by providing some mechanism for the user to determine what happens in a particular case." ANY documentation requirement can theoretically be satisfied by providing a program which does the measurement. Of course, not all documentation requirements can actually be satisfied that way. It is left up to the implementor to choose which Annex M issues have to be dealt with in the documentation and which are better satisfied using a tool or program. It is possible that for any particular implementation requirement some compilers will not be able to provide meaningful documentation, while some others cannot provide a test program. And that is another reason why this is a can of worms from a testing standpoint. The determination of which Annex M issues can be tested is something that cannot be done without looking at a particular compiler in depth, and the answers will only be appropriate to that compiler. Again, from a user's point of view, all this nit-picking is irrelevant. A user who needs, say, a compiler which never takes more than 1 millisecond for a clock interrupt is not going to be satisfied with some random document. He or she is going to write those requirements in the purchasing contract. Any such requirements in the contract will not specify that this or that feature be documented, but the actual required performance. ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Required Metrics 2000-05-07 0:00 ` Robert I. Eachus @ 2000-05-07 0:00 ` Ken Garlington 2000-05-07 0:00 ` Robert Dewar 2000-05-07 0:00 ` Robert Dewar 2000-05-08 0:00 ` Ole-Hjalmar Kristensen 2 siblings, 1 reply; 68+ messages in thread From: Ken Garlington @ 2000-05-07 0:00 UTC (permalink / raw) "Robert I. Eachus" <rieachus@earthlink.net> wrote in message news:3914F1DC.A5EE1751@earthlink.net... > Let me see if I can spread a little oil on the water here. Ken sees > the shalls, and is used to saying "Aha, a requirement. I must find all > such requirements--explicit, implicit, and derived--during requirements > analysis, allocate them, and plan to test them." Actually, it has nothing to do with the word "shall." It has to do with the word "requirement". It has nothing to do with allocation or testing. It has to do with whether requirements for which vendors believe any conceivable implementation will pass -- that is, it does not constrain their actions in any way whatsoever -- are requirements. If a vendor says, "I implemented this requirement a certain way" -- that is, the requirement was acknowledged in some form in the implementation -- it's a requirement. You can argue about whether it was the best implementation, or whether it's testable, etc. but that's a different argument. When a vendor says, "This requirement does not constrain me in any way," that's trouble. > But since the nature of Chapter 13 is to be very implementation > dependent, and difficult to test, users, especially embedded system > users, wanted a stronger "moral" commitment to features that they felt > were necessary, even if they were untestable. Such moral commitments > became Implementation Advice and Documentation Requirements. "Committment" implies that vendors take some specific action in response to the requirement. As we've seen, that isn't necessary to sign the DOC with a clear conscience. So, do you still believe this "moral" committment is being met? > Again, from a user's point of view, all this nit-picking is > irrelevant. A user who needs, say, a compiler which never takes more > than 1 millisecond for a clock interrupt is not going to be satisfied > with some random document. He or she is going to write those > requirements in the purchasing contract. Any such requirements in the > contract will not specify that this or that feature be documented, but > the actual required performance. Well, I would hope less often that they used to -- I know my group has sworn off these custom contracts wherever possible. After having spent well into seven digits on compilers in the last 15 years, I don't think we'll be back anytime soon. ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Required Metrics 2000-05-07 0:00 ` Ken Garlington @ 2000-05-07 0:00 ` Robert Dewar 2000-05-07 0:00 ` Ken Garlington 0 siblings, 1 reply; 68+ messages in thread From: Robert Dewar @ 2000-05-07 0:00 UTC (permalink / raw) In article <Bj7R4.8714$wb7.694404@news.flash.net>, "Ken Garlington" <Ken.Garlington@computer.org> wrote: > When a vendor says, "This requirement does not constrain me in > any way", that's trouble. A vendor can only say this if it is correct, and if it is correct that the requirement is not constraining, then it is not trouble as far as the requirement goes. > "Committment" implies that vendors take some specific action > in response to the requirement. Right! Absolutely! YOu have to make sure you meet all the requirements. Including documentation requirements in the sense of the RM. > As we've seen, that isn't necessary to sign the DOC with a > clear conscience. We have seen nothing of the kind. When I sign the DOC I am very careful to ensure that the statement signed is formally true with respect to ALL requirements, including documentation requirements. Indeed I have often held up signing a DOC because there is some small issue on which I am not satisfied that we do not have a deviation from the standard. > So, do you still believe this "moral" committment is being > met? Absolutely it is being met. The requirements are being met. THe trouble is that, unjustified by the RM, Ken Garlington has some idea in his mind that meeting them involves something other than meeting the requirements of the RM. Fine, but not relevant to the validation process. > Well, I would hope less often that they used to -- I know my > group has sworn off these custom contracts wherever possible. > After having spent well into seven digits on compilers in the > last 15 years, I don't think we'll be back > anytime soon. If you have spent millions of dollars on Ada compilers, and not got what you wanted, then I would say the blame must fall on you as well as your suppliers. Again, this statement shows the fantasy you have that the RM can ensure that off-the-shelf compilers meet your needs. Now of course it might be the case that OTS compilers can meet your needs, but if this is true, it is a reflection of many other factors than the contents of the standard. Perhaps the real trouble is the DoD notion of custom contracts. It tends to be far too formal, and thus far too expensive. What's needed in general is a more flexible, much less expensive much less formal, but far more effective procedure, where, as in buying a car or almost anything else, there is informal discussion between the vendor and consumer. In the GNAT context, there are very few users for whom we have not provided enhancements to the compiler from time to time in response to their needs. You have all seen the big list of new features and capabilities that come with each new version, and most of these entries come from informal discussions and requests. In some cases, significant features are specially costed of course. But again, if you have been spending millions of dollars on Ada compilers and have not got what you want, something is quite wrong on BOTH sides. Sent via Deja.com http://www.deja.com/ Before you buy. ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Required Metrics 2000-05-07 0:00 ` Robert Dewar @ 2000-05-07 0:00 ` Ken Garlington 2000-05-07 0:00 ` Robert Dewar 0 siblings, 1 reply; 68+ messages in thread From: Ken Garlington @ 2000-05-07 0:00 UTC (permalink / raw) "Robert Dewar" <robert_dewar@my-deja.com> wrote in message news:8f3u36$dnk$1@nnrp1.deja.com... > In article <Bj7R4.8714$wb7.694404@news.flash.net>, > "Ken Garlington" <Ken.Garlington@computer.org> wrote: > > > When a vendor says, "This requirement does not constrain me in > > any way", that's trouble. > > A vendor can only say this if it is correct, and if it is > correct that the requirement is not constraining, then it is > not trouble as far as the requirement goes. Apparently, we're going to have to agree to disagree as to whether it's a problem if the vendor decides a requirement doesn't constrain his behavior (and doesn't even have to tell anyone which requirements he decided fell into this category). > > "Committment" implies that vendors take some specific action > > in response to the requirement. > > Right! Absolutely! YOu have to make sure you meet all the > requirements. Including documentation requirements in the > sense of the RM. Why does someone have to "make sure" they have to meet a requirement that they've decided does not constrain them in any way? Do people who don't drive cars spend much time worrying if they're driving over the speed limit? > > As we've seen, that isn't necessary to sign the DOC with a > > clear conscience. > > We have seen nothing of the kind. When I sign the DOC I am > very careful to ensure that the statement signed is formally > true with respect to ALL requirements, including documentation > requirements. Indeed I have often held up signing a DOC because > there is some small issue on which I am not satisfied that we > do not have a deviation from the standard. Now we're getting to the question that wen't unanswered before. How do you do this? Is there a checklist? What criteria do you use to determine if you've met all of the requirements that aren't part of the "formal semantics"? If there are requirements that the vendor decides are vacuously true, is there any "moral" responsibility to either (a) verify this is the case, and/or (b) document the decision? --- AND THEN, THE OFF-TOPIC STUFF --- > If you have spent millions of dollars on Ada compilers, and not > got what you wanted, then I would say the blame must fall on > you as well as your suppliers. Oh, we always got what we wanted -- just before the vendor went out of business. We're just tired of having to go through the cycle over, and over, and over again. You're quite correct that we have to shoulder our part of the blame. We thought that Ada would naturally win out in the marketplace, because it had all of these advantages over other languages, and so we were an early adopter. Now, we know better. Rather than trying to drive the marketplace, we adapt our requirements to what's out there, and to what vendors are willing to implement on their own dime. Custom requirements are a sucker's game. ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Required Metrics 2000-05-07 0:00 ` Ken Garlington @ 2000-05-07 0:00 ` Robert Dewar 0 siblings, 0 replies; 68+ messages in thread From: Robert Dewar @ 2000-05-07 0:00 UTC (permalink / raw) In article <uegR4.8838$wb7.710909@news.flash.net>, "Ken Garlington" <Ken.Garlington@computer.org> wrote: > Apparently, we're going to have to agree to disagree as to > whether it's a problem if the vendor decides a requirement > doesn't constrain his behavior > (and doesn't even have to tell anyone which requirements he > decided fell > into this category). Ken, always remember that I was one of the ones who think it was a huge mistake to include ill-defined requirements in the RM. I think it's definitely a problem, if for no other reason than it leads you and others to be confused. That's why I think it would have been FAR better to have all such "requirements" be implementation advice instead for the following reasons: 1. We can say far more information things under the IA than in the requirements section. The non-normative sections of the RM, including all IA, are not in formal english, and can be read informally, which is far more useful in this case. 2. We would word them with this in mind, so that they would be clearer when interpreted this way. 3. Implementors would indeed have to say whether they meet the IA in each case or not (OK, to T.E.D, yes this is not a formal requirement, but in practice it is well understood, and customarily followed, at least I assume all decent Ada 95 compilers provide this useful information). If it is important to you to have implementors tell you how they deal with each such "requirement", you can only get that if they are IA. For real requirements, you either meet it or you don't, and there is not a hint of an obligation to say *how* you met the requirement. For example, I don't have to tell you how I compute the sqrt function. There is no such requirement for information provision, even in an informal woolly sense. One would have thought that the people who demanded that documentation be included in the RM would have also pushed for requiring documentation documenting that documentation :-) But they did not, or very properly, such a suggestion was rejected in the context of formal requirements. Ken, is this really all new to you? It is really a very familiar issue to those involved in the language design. I am a bit surprised by all this sturm and drang over an issue that was laid to rest 5 years ago, after extensive discussion. > Why does someone have to "make sure" they have to meet a > requirement that they've decided does not constrain them in > any way? Sorry, I don't understand. Each requirement is addressed from the point of view of whether the compiler meets it, not whether it "constrains" you, whatever that means. > Do people who don't > drive cars spend much time worrying if they're driving over > the speed limit? This is not relevant at all, nice rhetoric, but completely irrelevant. A compiler vendor must look through the documentation requirements, and make sure that they are satisfied according to the vendors best reading of the reference manual. If the vendor is relying on 1.1.3(19), they need to think about whether this applies. It may take little thought, but little is not zero. > Now we're getting to the question that wen't unanswered > before. How do you do this? Is there a checklist? What > criteria do you use to determine if you've met all of the > requirements that aren't part of the "formal > semantics"? This is simply not a hard job to do for people who understand what the standard is about, and who know and understand the standard well. Remember the DOC does NOT say you meet all requirements (such a DOC would be unsignable, since as I mentioned before, it would be a declaration that you had no bugs). The DOC declares that you have no intentional deviations from the standard. So the process of signing the DOC is precisely to make sure that you have no intentional deviations. > If there are requirements that the vendor decides are vacuously true, is > there any "moral" responsibility to either (a) verify this is the case, > and/or (b) document the decision? For requirements that are ill-specified and untestable, it is clearly impossible to formally verify that the requirement has been met. You simply do the best you can in such cases to make sure that the requirement is met. There is no hard and fast division into requirements that are or are not vacuously true. You have seized on this phrase, but it is not useful to do so. I have no idea what morality has to do with a formal language definition. Most certainly there is no requirement at all to document decisions, because no decisions have been made. You paint a picture in which the vendor lines up all the requirements, and then "decides" which are vacuously true and can be ignored. I know of know vendor who goes through such a process. Rather you go through all the requirements, and as best you can, make sure you have met them. That's all, nothing more, nothing less, and you do NOT have to document the fact that you conform to each individual decision, or how you conform to it. > Oh, we always got what we wanted -- just before the vendor > went out of business. Well I guess you were not very good at picking vendors then. Too bad, but it's not something the standard can help you with. > We're just tired of having to go through the cycle over, and > over, and over again. Lots of Ada users do NOT have the experience of going over a cycle of vendors going out of business over and over again, so this is definitely avoidable. Perhaps you need to look to your procurement procedures, they sound shaky, at least if judged by results. In any case the standard is not going to be able to prevent this I am afraid. > You're quite correct that we have to shoulder our part of the > blame. We thought that Ada would naturally win out in the > marketplace, because it had all of these advantages over other > languages, and so we were an early > adopter. Well many early adopters of Ada have had very good experience. Maybe they know something you don't, or they have better procurement procedures. I don't think you can blame all your difficulties on early adoption. > Now, we know better. Rather than trying to drive the > marketplace, we adapt our requirements to what's out there, > and to what vendors are > willing to implement on their own dime. Well more accurately, what they are willing to implement to meet general requirements of the Ada community. That's a little different. Certainly all of ACT's development is paid for by our paying users. We have no outside investment, so we have no "our own dime". All the dimes come from customers, so we try to spend them wisely in a way that corresponds to the general needs that they have. > Custom requirements are a sucker's game. They certainly can be, and it sounds like you have been sucked in, but that's largely your fault. Your new approach of looking to see what's generally available, rather than demanding the implementation of custom tools sounds like a FAR better way to go. Of course it's not absolute. As I said before, we do many small enhancements for our customers as part of our regular support. We also implement special features on a contract basis when there are special needs, but we will only do this if we think the features are generally useful. I definitely agree that the Ada 83 world was too full of custom requirements. Isn't this a general problem in the past history of the DoD? Aren't million dollar Ada compiler tool sets pretty closely related to $500 toilet seats? :-) Robert Dewar Ada Core Technologies Sent via Deja.com http://www.deja.com/ Before you buy. ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Required Metrics 2000-05-07 0:00 ` Robert I. Eachus 2000-05-07 0:00 ` Ken Garlington @ 2000-05-07 0:00 ` Robert Dewar 2000-05-18 0:00 ` Robert I. Eachus 2000-05-08 0:00 ` Ole-Hjalmar Kristensen 2 siblings, 1 reply; 68+ messages in thread From: Robert Dewar @ 2000-05-07 0:00 UTC (permalink / raw) In article <3914F1DC.A5EE1751@earthlink.net>, "Robert I. Eachus" <rieachus@earthlink.net> wrote: > On the other hand, everyone involved in the process was aware of what > had happened in Ada 83 with most of Chapter 13. Some vendors ignored > the parts that weren't tested, and other vendors since the requirements > there were not tested put implementation of those features at low > priority. (If a customer came alone with a need and sufficient cash, > they move up the list. But the users were not particularly happy with > that either.) As one very much involved in this issue (I campaigned for the chapter 13 tests to be included in the ACVC suite, and consulted extensively with the AVO making that happen - quite a bit of the code in those tests is mine :-) let me comment. It's not so simple. These were not clear requirements that were being ignored. The precise trouble was the infamous "simply" clause. Here it is: An implementation may limit its acceptance of representation clauses to those that can be handled simply by the underlying hardware. And the word "simply" is nowhere defined. Well this is the proverbial hole big enough to drive a truck through. Robert Dewar (and others) argued that simply must be interpreted in solely technical terms, looking at sequences of machine instructions. Even this leaves some subjective margin (is 3 instructions simple? 10? 20?). But others considered that simple means simple to implement and used this as an escape hatch to pick and choose what to implement. Not till the ARG ruled clearly on this issue, and the AVO backed it up with the tests (which as I mention above, I helped write), in ACVC version 1.10, did a proper set of tests get written. Now in Ada 95, we did two things 1. Required (in the systems programming annex) that rep clauses be implemented. 2. Specified an exact set of rep clauses that must be implemented. Was this an improvement? Yes, and No. Yes, if you agreed that the old Ada 83 requirement was so ill-defined as to say nothing, since now we clearly DO have a set of required rep clauses. Of course we achieved this in Ada 83 too by ARG rulings and the introduction of ACVC 1.10. No, if you take Robert Dewar's fierce interpretation of the original Ada 83 rule. Right now, there are lots of rep clauses that can be implemented simply in the hardware, which are in my opinion required by the Ada 83 standard, but which FOR SURE are not required by the Ada 95 standard. Overall are we ahead? Well in practice I find the subset of rep clauses required by the Ada 95 RM pathetically inadequate. In particular, an Ada 95 compiler that implements ONLY this minimal subset will have a really painful time with legacy Ada 83 compilers, which in general implemented a FAR larger set of representation clauses. Certainly in GNAT we go way beyond the Ada 95 requirements, and we attempt to completely match the capabilities of the Ada 83 compilers and in some cases, we go significantly beyond those requirements as well. Why? Because the standard is not the driving force when it comes to rep clauses, yes, of course we meet the limited requirements of the RM, but the driving force here is customers and Ada 83 legacy code. > For example, how do you design a portable > test for the bit patterns used to represent enumeration types? <<discussion of why this is hard snipped>> Yes, it's tricky, but can be done. This is one particular problem that I provided the solution for, and my solution (involving carefully chosen use of unchecked conversion is in the ACVC suite since 1.10). I think you can find my name on that test still :-) > This is why many of the documentation requirements are > untestable. For example, the intent of D.8(10) is clear: > If the documentation says the upper limit is one million > clocks is it useful? Probably not. Ah thanks Robert, I had forgotten that very useful point. Yes indeed, another completely accurate way of meeting the metrics requirements is to provide silly bogus figures like this. In a software engineering document, you can avoid this problem, most likely by specifying actual absolute performance criterion for the particular hardware combination you are using, but an attempt to plug this hole in the RM would lead to even more murky situations. > The compromise, if it can be called that, > is found in RM 1.1.3(19): > > "The implementation may choose to document > implementation-defined behavior either by documenting what > happens in general, or by providin some mechanism for the > user to determine what happens in a particular case." Interesting -- I had COMPLETELY forgotten this statement in the standard. It is of course an exact codification of some of the things that Bob Duff and I have argued. In particular, it provides an absolutely CLEAR basis for Bob Duff's observation that the behavior of the delivered binary compiler code, and the behavior of code generated by this compiler constitute documentation in the sense of the RM. So the RM *did* try to define what documentation meant, and came up with something about as far from what Ken has in mind as one can imagine. All the more reason to accept my point that we would have been far ahead putting documentation requirements in IA sections, where the word documentation can appear without being defined in a more useful subjective form. Sent via Deja.com http://www.deja.com/ Before you buy. ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Required Metrics 2000-05-07 0:00 ` Robert Dewar @ 2000-05-18 0:00 ` Robert I. Eachus 2000-05-18 0:00 ` Robert A Duff 0 siblings, 1 reply; 68+ messages in thread From: Robert I. Eachus @ 2000-05-18 0:00 UTC (permalink / raw) Robert Dewar wrote: > Not till the ARG ruled clearly on this issue, and the AVO > backed it up with the tests (which as I mention above, I > helped write), in ACVC version 1.10, did a proper set of > tests get written. I have no disagreement with what Robert Dwer wrote here (hmmm enthusiastically agree? applaud wildly?). However, the Chapter 13 tests which RBKD wrote had less effect on Ada 95 than he and others including myself hoped, for several reasons. First, by the time some of these were required by the ACVC vendors were all working on Ada 9X compilers and were unwilling to make fundamental changes not required by Ada 9X. Second, the conformance rules were changed to allow compilers not to comply with up to ten tests, as long as the intent was to correct these in the next validated version. In practice, since most new tests in 1.10 and after were chapter 13 tests, it allowed vendors to be much more agressive in stating that "this cannot be 'simply' implemented in this compiler." Of course, in many cases this was exactly right--it was the tests that would have required major compiler changes that didn't get made. Finally, since many of the Chapter 13 tests were, in effect, pushed back by challenges to later ACVC versions, some of these tests were never effectively required. Since the design of Ada 9X started well before ACVC 1.10 affected production compilers, the experience of users during the early phases of the Ada 9X development process was with compilers that in some cases did not take representation clauses, etc. seriously. So as Robert Dewar points out, while the requirements in Ada 95 are much more clearly spelled out, they are definitely a subset of what was finally required in Ada 83. (Fortunately, the marketplace has tended to require this 'additional' support in any case. I hope 'de facto' becomes 'de jure' with the next Ada standard... > Interesting -- I had COMPLETELY forgotten this > statement in the standard. It is of course an exact codification > of some of the things that Bob Duff and I have argued. In > particular, it provides an absolutely CLEAR basis for Bob Duff's > observation that the behavior of the delivered binary compiler > code, and the behavior of code generated by this compiler > constitute documentation in the sense of the RM. > > So the RM *did* try to define what documentation meant, and > came up with something about as far from what Ken has in mind > as one can imagine. Yes, the intent was to prevent documentation requirements from limiting actual implementations. In Ada 83, the ACVC tests effectively required every Ada compiler to implement a "usable" maximum input line length. And the blasted ACVC tests checked that the compiler DID NOT accept longer lines, whle requiring that you accept an identifier that was the same length as a line. So you had to set an arbitrary maximum line length and enforce it even if in the absence of the ACVC test the limit would only be imposed by memory size or file characteristics. In Ada 95 the requirement concerning line length is in RM 2.2(15): "...The maximum supported line length and lexical element length are implementation defined." (The minimum for each is 200 characters.) ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Required Metrics 2000-05-18 0:00 ` Robert I. Eachus @ 2000-05-18 0:00 ` Robert A Duff 2000-05-19 0:00 ` Robert I. Eachus 2000-05-21 0:00 ` Robert Dewar 0 siblings, 2 replies; 68+ messages in thread From: Robert A Duff @ 2000-05-18 0:00 UTC (permalink / raw) "Robert I. Eachus" <rieachus@earthlink.net> writes: >...And the blasted ACVC tests checked that the compiler DID NOT > accept longer lines, whle requiring that you accept an identifier that > was the same length as a line. That test is clearly wrong. I'm surprised nobody has challenged it. There is no requirement that a compiler impose a maximum line-length limit, and there never has been. I guess it's easier to make your compiler pass a test than it is to go through the challenge process. Sigh. - Bob ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Required Metrics 2000-05-18 0:00 ` Robert A Duff @ 2000-05-19 0:00 ` Robert I. Eachus 2000-05-21 0:00 ` Robert Dewar 1 sibling, 0 replies; 68+ messages in thread From: Robert I. Eachus @ 2000-05-19 0:00 UTC (permalink / raw) Robert A Duff wrote: > That test is clearly wrong. I'm surprised nobody has challenged it. > There is no requirement that a compiler impose a maximum line-length > limit, and there never has been. > I guess it's easier to make your compiler pass a test than it is to go > through the challenge process. Sigh. If the test is still in the suite, it definitely should be challenged. The Ada 95 rules explicitly allow the maximum identifier length to be different from the maximum line length. However, in the early days of Ada 83 validation--around ACVC 1.4--you were faced with an impossible Catch-22. You could argue that the test had to be modified for your implementation, but you still had to demonstrate somehow, that an identifier of maximum length was accepted, and an identifier of maximum length + 1 was either rejected by the compiler, or impossible to create in a file. And yes, we took the easy way out on the compiler for the Honeywell DPS6. The "real" maximum line length for {normal} text files was 255 characters. It was MUCH easier to set the maximum Ada source file line length to 253 and pass the ACVC test as written, than to try to demonstrate that we couldn't create a non-standard file containing an identifier longer than the maximum line length. (The choice of 253 was for consistancy with Ada Text_IO files, where we needed a couple extra characters for new page and end of file makers.) ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Required Metrics 2000-05-18 0:00 ` Robert A Duff 2000-05-19 0:00 ` Robert I. Eachus @ 2000-05-21 0:00 ` Robert Dewar 2000-06-03 0:00 ` Robert I. Eachus 1 sibling, 1 reply; 68+ messages in thread From: Robert Dewar @ 2000-05-21 0:00 UTC (permalink / raw) In article <wccbt23uaut.fsf@world.std.com>, Robert A Duff <bobduff@world.std.com> wrote: > "Robert I. Eachus" <rieachus@earthlink.net> writes: > > >...And the blasted ACVC tests checked that the compiler DID NOT > > accept longer lines, whle requiring that you accept an identifier that > > was the same length as a line. > > That test is clearly wrong. I'm surprised nobody has challenged it. > There is no requirement that a compiler impose a maximum line-length > limit, and there never has been. > > I guess it's easier to make your compiler pass a test than it is to go > through the challenge process. Sigh. You missed the point. There was a requirement that if you support lines of length N, you must also support identifiers of length N, including externally visible names of length N. It is impractical in most environments to allow arbitrary length for identifier names. A compiler that limited names to 256 characters, but had no limit on line length would NOT have been valid in Ada 83 (I think we fixed this in Ada 95? but on the other hand we recommend a line limit of 250 in Ada 95!) Sent via Deja.com http://www.deja.com/ Before you buy. ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Required Metrics 2000-05-21 0:00 ` Robert Dewar @ 2000-06-03 0:00 ` Robert I. Eachus 0 siblings, 0 replies; 68+ messages in thread From: Robert I. Eachus @ 2000-06-03 0:00 UTC (permalink / raw) Robert Dewar wrote: > You missed the point. There was a requirement that if you > support lines of length N, you must also support identifiers > of length N, including externally visible names of length N. > It is impractical in most environments to allow arbitrary > length for identifier names. No, I didn't miss the point, since at Honeywell we did have a compiler that ran into exactly this problem. The compiler used a hash table with the actual identifier strings stored in the heap. Originally, the only limit on identifier length was imposed by the type of text file used for the source file. This limit was always shorter than the limit imposed by the compiler (none) or by the size of memory on the machine used to compile the program. We could pass the ACVC test on the largest DPS-6 machines with the maximum line length set to 2**16-4, which was the longest line allowed if you used a special file type. However, on the low end machines, that ACVC test would fail to compile when it couldn't allocate one of the long identifiers. In this case, we could have broken the test up into several parts, but it was much easier to require a smaller maximum line length--I'd have to look but I think we used 8K--so that all validations were identical. (Later, I think DDC changed their front-end to store the identifier strings in a file and just keep the hash table in memory. But the Line length limit stayed.) So I do personally know that in at least one case, this ACVC test did force a compiler limitation. And we did fix this in Ada 95--you are allowed to have different limits on identifier size and line length. (And I do mean we. I remember giving an example where requiring 255 instead of 254 would cause problems, and you (Robert Dewar) suggested 250, since there was no particular reason not to choose a decimal, not binary, round number.) Hmmm. Just remembering a different effect of this rule in Ada 83. There were some ACVC floating point tests that had long numeric literals. The length of these literals depended on the number of digits allowed in a floating point type declaration. I remember at least one compiler choosing not to support extended floating point since their compiler couldn't support numeric literals as long as required by those tests. (Does anyone remember? Was that the ALS? It certainly was a VAX targeted compiler.) In any case choosing a "short enough" maximum line length negated the problem. ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Required Metrics 2000-05-07 0:00 ` Robert I. Eachus 2000-05-07 0:00 ` Ken Garlington 2000-05-07 0:00 ` Robert Dewar @ 2000-05-08 0:00 ` Ole-Hjalmar Kristensen 2000-05-08 0:00 ` Robert Dewar ` (2 more replies) 2 siblings, 3 replies; 68+ messages in thread From: Ole-Hjalmar Kristensen @ 2000-05-08 0:00 UTC (permalink / raw) "Robert I. Eachus" <rieachus@earthlink.net> writes: <stuff deleted> > > Again, from a user's point of view, all this nit-picking is > irrelevant. A user who needs, say, a compiler which never takes more > than 1 millisecond for a clock interrupt is not going to be satisfied > with some random document. He or she is going to write those > requirements in the purchasing contract. Any such requirements in the > contract will not specify that this or that feature be documented, but > the actual required performance. Or is simply going to test it, and throw away those compilers that do not meet his requirement. Or implement the feature himself. Or use another language, since the need for implementing it yourself really removes much of the reason for using Ada in the first place. -- E pluribus Unix ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Required Metrics 2000-05-08 0:00 ` Ole-Hjalmar Kristensen @ 2000-05-08 0:00 ` Robert Dewar 2000-05-08 0:00 ` Robert Dewar 2000-05-18 0:00 ` Robert I. Eachus 2 siblings, 0 replies; 68+ messages in thread From: Robert Dewar @ 2000-05-08 0:00 UTC (permalink / raw) In article <umqzoq1h3u6.fsf@gong2.clustra.com>, Ole-Hjalmar Kristensen <ohk@clustra.com> wrote: > Or use another language, since the need for implementing it > yourself really removes much of the reason for using Ada in > the first place. No, that's MUCH too strong. It is still the case that many uses of Ada simply avoid tasking completely. One case is safety critical programming which wants to avoid the complexities of tasking semantics. Another case is where the Ada programmer *prefers* to use the underlying thread capabilities of the system directly. There is nothing wrong with this. Furthermore, there primary advantages of Ada to most users do not stem from performance of rendez-vous, but rather many other features. Of course people might switch to other languages for many reasons but the "much" in your sentence above does not hold water. In any case, I think we agree that the RM is not the issue here :-) Sent via Deja.com http://www.deja.com/ Before you buy. ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Required Metrics 2000-05-08 0:00 ` Ole-Hjalmar Kristensen 2000-05-08 0:00 ` Robert Dewar @ 2000-05-08 0:00 ` Robert Dewar 2000-05-18 0:00 ` Robert I. Eachus 2 siblings, 0 replies; 68+ messages in thread From: Robert Dewar @ 2000-05-08 0:00 UTC (permalink / raw) In article <umqzoq1h3u6.fsf@gong2.clustra.com>, Ole-Hjalmar Kristensen <ohk@clustra.com> wrote: > Or use another language, since the need for implementing it > yourself really removes much of the reason for using Ada in > the first place. No, that's MUCH too strong. It is still the case that many uses of Ada simply avoid tasking completely. One case is safety critical programming which wants to avoid the complexities of tasking semantics. Another case is where the Ada programmer *prefers* to use the underlying thread capabilities of the system directly. There is nothing wrong with this. Furthermore, there primary advantages of Ada to most users do not stem from performance of rendez-vous, but rather many other features. Of course people might switch to other languages for many reasons but the "much" in your sentence above does not hold water. In any case, I think we agree that the RM is not the issue here :-) Sent via Deja.com http://www.deja.com/ Before you buy. ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Required Metrics 2000-05-08 0:00 ` Ole-Hjalmar Kristensen 2000-05-08 0:00 ` Robert Dewar 2000-05-08 0:00 ` Robert Dewar @ 2000-05-18 0:00 ` Robert I. Eachus 2000-05-18 0:00 ` Ken Garlington 2 siblings, 1 reply; 68+ messages in thread From: Robert I. Eachus @ 2000-05-18 0:00 UTC (permalink / raw) (I said:) > > Again, from a user's point of view, all this nit-picking is > > irrelevant. A user who needs, say, a compiler which never takes more > > than 1 millisecond for a clock interrupt is not going to be satisfied > > with some random document. He or she is going to write those > > requirements in the purchasing contract. Any such requirements in the > > contract will not specify that this or that feature be documented, but > > the actual required performance. I was not saying that you should have a compiler built to your special order, although I have been in the position of writing such contracts (for hardware vendors). I was saying that when you are setting the requirements for a compiler for YOUR particular project, you look at what YOUR specific requirements are, and then write a requirements document, possibly an RFP, and a purchase contract. If a compiler is available off the shelf that does meet your requirements for a large, long-duration project, there is a major advantage to writing those key requirements into the purchase order. It informs the vendor that those particular features are key requirements for your project, and that you will need long term support for a compiler that meets those requirements. The compiler vendor, when planning enhancements of his product, will look at his support requirements and pay attention to them. Of course, if you write a detailed contract which documents every possible detail of the expected product, the vendor will ignore those requirements when planning significant revisions. He will expect you to continue to use the current compiler version, perhaps with some patches. He can afford to halt progress to support you and still be a viable competitor. (Actually, it sounds like Ken did get into this situation--imposing enough requirements on the vendor to prevent him from staying competitive.) ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Required Metrics 2000-05-18 0:00 ` Robert I. Eachus @ 2000-05-18 0:00 ` Ken Garlington 0 siblings, 0 replies; 68+ messages in thread From: Ken Garlington @ 2000-05-18 0:00 UTC (permalink / raw) > (Actually, it sounds like Ken did get into this situation--imposing > enough requirements on the vendor to prevent him from staying > competitive.) Not really - more that the vendor made poor business decisions (one couldn't sell their product overseas, for example), and so lost out in the inevitable Ada market contraction. Many of the "requirements" we imposed are now described in the Ada standard. The real moral of the story, IMHO, is to never plan on a long-term relationship with a single vendor, toolset, platform or language. ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Required Metrics 2000-05-01 0:00 ` Ken Garlington 2000-05-04 0:00 ` Robert Dewar @ 2000-05-04 0:00 ` Roger Barnett 2000-05-05 0:00 ` Robert Dewar 1 sibling, 1 reply; 68+ messages in thread From: Roger Barnett @ 2000-05-04 0:00 UTC (permalink / raw) As an aside, what is the status of the validation suite nowadays, i.e. is there an Ada 95 revision administered by the DoD and their appointed agencies outside of the US ? [ I was last involved with this stuff in the 1980s, so I suspect I'm a little out of date ] -- Roger Barnett ^ permalink raw reply [flat|nested] 68+ messages in thread
* Re: Required Metrics 2000-05-04 0:00 ` Roger Barnett @ 2000-05-05 0:00 ` Robert Dewar 0 siblings, 0 replies; 68+ messages in thread From: Robert Dewar @ 2000-05-05 0:00 UTC (permalink / raw) In article <842359703wnr@natron.demon.co.uk>, Roger@natron.demon.co.uk wrote: > > As an aside, what is the status of the validation suite nowadays, > i.e. is there an Ada 95 revision administered by the DoD and their > appointed agencies outside of the US ? In brief, validation has become internationalized and is now under control of ISO WG9, it is no longer a process administered by the authorities in one member nation, and thus completely outside the ISO process. The ISO WG9/ARG now adjudicates validation issues, and there is an ISO standard for validation of Ada compilers (not quite sure if it is formally approved yet, but if not it is essentially at that stage). Any organization following the requirements of this ISO standard can carry out ISO style validations. In the US, the Ada Resource Association funds an organization that carries out validations. New tests are being written as appropriate, under the aegis of the ARG. SO in fact AJPO getting out of the validation business has resulted in a considerable upgrade of the entire validation process. Validations under the new procedures and test suite have already been successfully performed, and now have formal international recognition. Perhaps Randy Burkhardt would like to add something to this thread at this point if he is reading it :-) Sent via Deja.com http://www.deja.com/ Before you buy. ^ permalink raw reply [flat|nested] 68+ messages in thread
end of thread, other threads:[~2000-06-03 0:00 UTC | newest] Thread overview: 68+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2000-04-29 0:00 Required Metrics Ken Garlington 2000-04-29 0:00 ` swhalen 2000-05-01 0:00 ` Required Metrics (GNAT et al) Ken Garlington 2000-05-01 0:00 ` swhalen 2000-05-01 0:00 ` Ken Garlington 2000-05-01 0:00 ` Required Metrics Ted Dennison 2000-05-01 0:00 ` Tucker Taft 2000-05-01 0:00 ` Ken Garlington 2000-05-02 0:00 ` Ken Garlington 2000-05-02 0:00 ` Ted Dennison 2000-05-03 0:00 ` Robert Dewar 2000-05-03 0:00 ` Ken Garlington 2000-05-03 0:00 ` Robert A Duff 2000-05-04 0:00 ` Ken Garlington 2000-05-04 0:00 ` Robert Dewar 2000-05-04 0:00 ` Robert A Duff 2000-05-04 0:00 ` Robert Dewar 2000-05-05 0:00 ` Ken Garlington 2000-05-04 0:00 ` Robert Dewar 2000-05-04 0:00 ` Ken Garlington 2000-05-05 0:00 ` Robert Dewar 2000-05-04 0:00 ` Robert Dewar 2000-05-04 0:00 ` Wes Groleau 2000-05-04 0:00 ` Ken Garlington 2000-05-05 0:00 ` Robert Dewar 2000-05-06 0:00 ` Ken Garlington 2000-05-06 0:00 ` Ken Garlington 2000-05-06 0:00 ` Robert Dewar 2000-05-06 0:00 ` Robert Dewar 2000-05-07 0:00 ` Ken Garlington 2000-05-07 0:00 ` Robert Dewar 2000-05-08 0:00 ` Ole-Hjalmar Kristensen 2000-05-06 0:00 ` Robert Dewar 2000-05-07 0:00 ` Ken Garlington 2000-05-07 0:00 ` Robert Dewar 2000-05-07 0:00 ` Ken Garlington 2000-05-07 0:00 ` Robert Dewar 2000-05-07 0:00 ` Ken Garlington 2000-05-07 0:00 ` Robert Dewar 2000-05-02 0:00 ` Ted Dennison 2000-05-04 0:00 ` Robert Dewar 2000-05-04 0:00 ` Ted Dennison 2000-05-05 0:00 ` Ken Garlington 2000-05-05 0:00 ` Robert Dewar 2000-05-01 0:00 ` Ken Garlington 2000-05-04 0:00 ` Robert Dewar 2000-05-05 0:00 ` Ken Garlington 2000-05-05 0:00 ` Robert Dewar 2000-05-05 0:00 ` Ted Dennison 2000-05-06 0:00 ` Robert Dewar 2000-05-07 0:00 ` Robert I. Eachus 2000-05-07 0:00 ` Ken Garlington 2000-05-07 0:00 ` Robert Dewar 2000-05-07 0:00 ` Ken Garlington 2000-05-07 0:00 ` Robert Dewar 2000-05-07 0:00 ` Robert Dewar 2000-05-18 0:00 ` Robert I. Eachus 2000-05-18 0:00 ` Robert A Duff 2000-05-19 0:00 ` Robert I. Eachus 2000-05-21 0:00 ` Robert Dewar 2000-06-03 0:00 ` Robert I. Eachus 2000-05-08 0:00 ` Ole-Hjalmar Kristensen 2000-05-08 0:00 ` Robert Dewar 2000-05-08 0:00 ` Robert Dewar 2000-05-18 0:00 ` Robert I. Eachus 2000-05-18 0:00 ` Ken Garlington 2000-05-04 0:00 ` Roger Barnett 2000-05-05 0:00 ` Robert Dewar
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox