Required Metrics

Required Metrics

[Ken Garlington]

In the Ada Reference Manual, paragraphs D.8:37-45 states that "The
implementation shall document" a series of metrics related to the real-time
clock, etc. paragraph D:2 states that "The metrics are documentation
requirements; an implementation shall document the values of the
language-defined metrics for at least one configuration of hardware or an
underlying system supported by the implementation, and shall document the
details of that configuration." However, although my compiler's reference
manual claims that "The real-time systems annex is fully implemented", it
goes on to say " Information on metrics is not yet available." Does a
compiler fully implement the real-time systems annex if it does not meet the
documentation requirements?

Re: Required Metrics

[swhalen]

Ken Garlington <Ken.Garlington@computer.org> wrote:
: In the Ada Reference Manual, paragraphs D.8:37-45 states that "The
: implementation shall document" a series of metrics related to the real-time
: clock, etc. paragraph D:2 states that "The metrics are documentation
: requirements; an implementation shall document the values of the
: language-defined metrics for at least one configuration of hardware or an
: underlying system supported by the implementation, and shall document the
: details of that configuration." However, although my compiler's reference
: manual claims that "The real-time systems annex is fully implemented", it
: goes on to say " Information on metrics is not yet available." Does a
: compiler fully implement the real-time systems annex if it does not meet the
: documentation requirements?

I'd say you don't know if it meets the documentation requirements
unless you have the documentation for _ALL_ of that vendor's
"implementations". I'd expect that for GNAT for example, that the
hardware implementation they "documented" would be one besides the two
I have documentation for (WinNT & Linux).

I'd expect the metrics documentation to be for a more "real-time"
hardware / OS combination. And I would expect to see the documentation
in the distribution for that platform, not distributed with all
platforms from that vendor. I'm also not sure I'd expect so see the
metrics with the documentation on a non-validated compiler (I fool
around with the public versions of GNAT mainly).

But then I'm no language lawyer, so my expectations are probably
meaningless <g>.

Steve

-- 
{===--------------------------------------------------------------===}
                Steve Whalen     swhalen@netcom.com
{===--------------------------------------------------------------===}

Re: Required Metrics

[Ted Dennison]

In article <5DDO4.2237$wb7.194854@news.flash.net>,
  "Ken Garlington" <Ken.Garlington@computer.org> wrote:
> In the Ada Reference Manual, paragraphs D.8:37-45 states that "The
> implementation shall document" a series of metrics related to the
...
> details of that configuration." However, although my compiler's
> reference manual claims that "The real-time systems annex is fully
> implemented", it goes on to say " Information on metrics is not yet
> available." Does a compiler fully implement the real-time systems
> annex if it does not meet the documentation requirements?

I remember asking almost this exact question here about a year and a
half ago. You can search deja for the thread. But the answer I remember
getting was that since there is (and can be) no test in the Ada
validation test suite for documentation requirements, implementors can
pretty much get away with murder here.

--
T.E.D.

http://www.telepath.com/~dennison/Ted/TED.html


Sent via Deja.com http://www.deja.com/
Before you buy.

Re: Required Metrics (GNAT et al)

[swhalen]

I'm not sure it would be practical to include, say, the VxWorks
metrics with the WinNT compiler, because there would be too many
references to VxWorks that would be easy to explain in the context of
the VxWorks documentation, but would make no sense in the WinNT docs.

The more I think about this, the more I'm convinced it would be wrong
for a vendor to ship metrics with anything other than a validated
compiler.  The metrics would have to be very hardware / OS / compiler
dependent.

Metrics should be reproducible, but they would be reproducible only on
the platform that was validated.

Steve

-- 
{===--------------------------------------------------------------===}
                Steve Whalen     swhalen@netcom.com
{===--------------------------------------------------------------===}

Re: Required Metrics

[Ken Garlington]

"Tucker Taft" <stt@averstar.com> wrote in message
news:390DC8AD.59B5EBEE@averstar.com...
> Ted Dennison wrote:
> >
> > In article <5DDO4.2237$wb7.194854@news.flash.net>,
> >   "Ken Garlington" <Ken.Garlington@computer.org> wrote:
> > > In the Ada Reference Manual, paragraphs D.8:37-45 states that "The
> > > implementation shall document" a series of metrics related to the
> > ...
> > > details of that configuration." However, although my compiler's
> > > reference manual claims that "The real-time systems annex is fully
> > > implemented", it goes on to say " Information on metrics is not yet
> > > available." Does a compiler fully implement the real-time systems
> > > annex if it does not meet the documentation requirements?
> >
> > I remember asking almost this exact question here about a year and a
> > half ago. You can search deja for the thread. But the answer I remember
> > getting was that since there is (and can be) no test in the Ada
> > validation test suite for documentation requirements, implementors can
> > pretty much get away with murder here.
>
> The validation process (aka "conformance assessment" process) does
> not include checking conformance to the documentation requirements,
> largely for pragmatic reasons. It was felt that the cost of document
> conformance assessment would be excessive and difficult to standardize.

Well, yeah, sure, but I think my question was more "abstract" (to coin a
keyword :) than this. Simply put: Does the implementation meet the standard?
I understand that a validation suite might not detect all deviations with
respect to _functionality_, never mind documentation, since no test can be
exhaustive. However, I've seen people post behavior seen in a compiler, and
ask if it meets the standard, and few people have responded with "Sorry -
that specific behavior wasn't/wouldn't be caught in the validation suite, so
you're on your own."

> In any case, the marketplace needs to play a role
> here.  Either make good documentation a condition of sale, or give
> your existing vendor a hard time if they don't provide the documentation
> you need.

Ditto for this. I can ask for useful "stuff" (documentation, optimizations,
etc.) whether or not it's in the standard.

Alternate phrasing of the question: If a requirement is in the standard, and
no one makes an effort to follow it, what's the requirement doing in the
standard? Why not have an interpretation that says, in essence: "Oops. Never
mind"? Or am I just missing some fundamental point here?

Re: Required Metrics (GNAT et al)

[Ken Garlington]

<swhalen@netcom.com> wrote in message
news:8eksr5$kf8$1@slb1.atl.mindspring.net...
> I'm not sure it would be practical to include, say, the VxWorks
> metrics with the WinNT compiler, because there would be too many
> references to VxWorks that would be easy to explain in the context of
> the VxWorks documentation, but would make no sense in the WinNT docs.
>
> Metrics should be reproducible, but they would be reproducible only on
> the platform that was validated.
>
> The metrics would have to be very hardware / OS / compiler
> dependent.

Well, I don't think there's any requirement to provide the _entire_
documentation suite, just the _metrics_ (formula, whatever) from the
platform of choice. Using GNAT as an example, the documentation already
defines some features that really only make sense on specific platforms
(e.g. certain DEC Ada "emulations"), so there's certainly some precedent
here. The standard already appears to recognize that the metrics depend upon
context, so that shouldn't be an issue.

> The more I think about this, the more I'm convinced it would be wrong
> for a vendor to ship metrics with anything other than a validated
> compiler.

OK - does ACT do this? I wasn't aware you got different documentation for
the supported version versus the public version.

Re: Required Metrics

[Ken Garlington]

"Ted Dennison" <dennison@telepath.com> wrote in message
news:8ek4ea$5ta$1@nnrp1.deja.com...
> In article <5DDO4.2237$wb7.194854@news.flash.net>,
>   "Ken Garlington" <Ken.Garlington@computer.org> wrote:
> > In the Ada Reference Manual, paragraphs D.8:37-45 states that "The
> > implementation shall document" a series of metrics related to the
> ...
> > details of that configuration." However, although my compiler's
> > reference manual claims that "The real-time systems annex is fully
> > implemented", it goes on to say " Information on metrics is not yet
> > available." Does a compiler fully implement the real-time systems
> > annex if it does not meet the documentation requirements?
>
> I remember asking almost this exact question here about a year and a
> half ago. You can search deja for the thread. But the answer I remember
> getting was that since there is (and can be) no test in the Ada
> validation test suite for documentation requirements, implementors can
> pretty much get away with murder here.

I'm not surprised, but I don't think it really answers my question. The
issue isn't, in my mind, "How easy is it for vendors to ignore certain
requirements?".  It's "Is there really a requirement?" (See my response to
Tucker's post for more...)

Re: Required Metrics (GNAT et al)

[Ken Garlington]

<swhalen@netcom.com> wrote in message
news:8efg68$44u$1@slb1.atl.mindspring.net...
> Ken Garlington <Ken.Garlington@computer.org> wrote:
> : In the Ada Reference Manual, paragraphs D.8:37-45 states that "The
> : implementation shall document" a series of metrics related to the
real-time
> : clock, etc. paragraph D:2 states that "The metrics are documentation
> : requirements; an implementation shall document the values of the
> : language-defined metrics for at least one configuration of hardware or
an
> : underlying system supported by the implementation, and shall document
the
> : details of that configuration." However, although my compiler's
reference
> : manual claims that "The real-time systems annex is fully implemented",
it
> : goes on to say " Information on metrics is not yet available." Does a
> : compiler fully implement the real-time systems annex if it does not meet
the
> : documentation requirements?
>
> I'd say you don't know if it meets the documentation requirements
> unless you have the documentation for _ALL_ of that vendor's
> "implementations".

Hmmm... that's not how I'd parse that sentence. It appears to me that all
implementations have to have documentation, but that each implementation
doesn't have to be documented. For example, the documentation for GNAT might
not describe how it works on WinNT or Linux, but if there were metrics
available for VxWorks, I'd expect to see those VxWorks metrics in the
documentation for _all_ implementations to at least technically meet the
requirements?

> I'd expect that for GNAT for example, that the hardware implementation
> they "documented" would be one besides the two I have documentation
> for (WinNT & Linux).

OK - Does anyone know if that's how it works, for GNAT or anyone else who
claims conformance to the annex?

> I'd expect the metrics documentation to be for a more "real-time"
> hardware / OS combination.

It seems to me that, if the reason the annex isn't fully implemented is
because the particular platform isn't considered "real time", that no vendor
should claim conformance on that platform. It not like these metrics are
just being provided as a marketing tool; we're talking about requirements
from the standard.

> I'm also not sure I'd expect so see the metrics with the
> documentation on a non-validated compiler (I fool
> around with the public versions of GNAT mainly).

OK - Does anyone know if the metrics are provided only to supported
customers (GNAT or otherwise)?

Re: Required Metrics

[Tucker Taft]

Ted Dennison wrote:
> 
> In article <5DDO4.2237$wb7.194854@news.flash.net>,
>   "Ken Garlington" <Ken.Garlington@computer.org> wrote:
> > In the Ada Reference Manual, paragraphs D.8:37-45 states that "The
> > implementation shall document" a series of metrics related to the
> ...
> > details of that configuration." However, although my compiler's
> > reference manual claims that "The real-time systems annex is fully
> > implemented", it goes on to say " Information on metrics is not yet
> > available." Does a compiler fully implement the real-time systems
> > annex if it does not meet the documentation requirements?
> 
> I remember asking almost this exact question here about a year and a
> half ago. You can search deja for the thread. But the answer I remember
> getting was that since there is (and can be) no test in the Ada
> validation test suite for documentation requirements, implementors can
> pretty much get away with murder here.

The validation process (aka "conformance assessment" process) does
not include checking conformance to the documentation requirements,
largely for pragmatic reasons. It was felt that the cost of document
conformance assessment would be excessive and difficult to standardize. 

In any case, the marketplace needs to play a role
here.  Either make good documentation a condition of sale, or give
your existing vendor a hard time if they don't provide the documentation
you need.


> 
> --
> T.E.D.
> 
> http://www.telepath.com/~dennison/Ted/TED.html

-- 
-Tucker Taft   stt@averstar.com   http://www.averstar.com/~stt/
Technical Director, Commercial Division, AverStar (formerly Intermetrics)
(http://www.averstar.com/services/IT_consulting.html)  Burlington, MA  USA

Re: Required Metrics

[Ken Garlington]

Having gone back and read the thread on this subject from May 1999, the
closest thing I found to an answer to my question was, essentially, "It's
not practical for a vendor to document the required metrics if it relies on
an underlying operating system." I'm not sure I buy this in all cases --
what if the OS vendor provides sufficient information on their thread
support, for example? -- but let's assume it's a reasonable caveat. Given
this, then:

1. Why wouldn't this be the response given in the documentation; i.e. "This
implementation conforms to the Real-Time Annex, except for the documentation
of metrics, which is not practical"? (Better yet, why didn't anyone put this
in the ARM to begin with, or write an interpretation after the fact?)

2. What would this mean for an implementation that didn't use an operating
system; e.g. GNORT? (By the way, if it wasn't obvious, this isn't a rant
about GNAT. It's a question about the meaning -- or maybe, lack of
meaning -- of the standard...)

I'm starting to wonder about the responses to this question (and the lack
thereof). Are there other requirements in the ARM that are optional in
practice? Is it a general rule that if it's not tested in the validation
suite, a vendor can decline to implement the requirement but still be
considered conforming? If so, what does this mean with respect to the
assumed benefits of standardization?

"Hey, this compiler doesn't implement tasking, but it says it conforms to
the ARM!"
"Well, you can always call the pthreads package and implement it yourself.
Call the vendor, they'll provide assistance. Alternately, you can ask for
tasking support in a later release."

Re: Required Metrics

[Ted Dennison]

In article <OKnP4.3694$wb7.343653@news.flash.net>,
  "Ken Garlington" <Ken.Garlington@computer.org> wrote:

> Alternate phrasing of the question: If a requirement is in the
> standard, and no one makes an effort to follow it, what's the
> requirement doing in the standard? Why not have an interpretation that
> says, in essence: "Oops. Never mind"? Or am I just missing some
> fundamental point here?

:-)
I'm glad to see I'm not the only one who had this reaction.

If a requirement falls in the forest...
--
T.E.D.

http://www.telepath.com/~dennison/Ted/TED.html


Sent via Deja.com http://www.deja.com/
Before you buy.

Re: Required Metrics

[Ted Dennison]

In article <GwzP4.3979$wb7.383449@news.flash.net>,
  "Ken Garlington" <Ken.Garlington@computer.org> wrote:
> Having gone back and read the thread on this subject from May 1999,

Interesting coincidence that this would come up two Mays in a row.

I know that once a year is not exactly "frequently", but I think a
coherent accouting of this issue would be worthy of inclusion in the
FAQ. I'd put it in myself, but I can't access AdaPower right now for
some bizzare reason.

--
T.E.D.

http://www.telepath.com/~dennison/Ted/TED.html


Sent via Deja.com http://www.deja.com/
Before you buy.

Re: Required Metrics

[Robert Dewar]

The trouble with documentation requirements is that the
standard, by necessity, does not define exactly what the
meaning of these requirements is, or even what documentation
consists of.

In the case of typical implementations on top of an operating
system, about the best documentation of the conventional kind
that you could give for tasking metrics would be to clearly
document the sequence of operating systems calls that is made
for any particular tasking language construct.

Our view in the case of GNAT is that the sources of the run-time
which are an integral part of our complete technical
documentation contain precisely this information, and so far
we have not had any instances of users requiring the information
in any other form.

I don't think this is so surprising. We could of course run
benchmarks on particular machines under particular conditions
and publish numbers that appear to meet the requirements of
the RM. Why "appear"? Because in practice they won't be useful
for users. If users need hard figures on timings of tasking
operations, what we would advise is that they run tests
corresponding to their actual use of tasking constructs in
the exact conditions they will be using them. This is the
same advice that applies to any use of any features of the
language.

I personally think the metrics requirements in annex D do not
belong in a language standard. They are too vague from a formal
point of view, and not helpful from a pragmatic point of view.

Historically they were put there to satisfy people who actually
wanted to try to quantify performance with specific RM
requirements for performance, keyed to processor clock speed.
That's of course an even WORSE idea for infusing the standard
with inappropriate junk :-)

Robert Dewar
Ada Core Technologies


Sent via Deja.com http://www.deja.com/
Before you buy.

Re: Required Metrics

[Ken Garlington]

"Robert Dewar" <robert_dewar@my-deja.com> wrote in message
news:8ep0k3$jlr$1@nnrp1.deja.com...
> The trouble with documentation requirements is that the
> standard, by necessity, does not define exactly what the
> meaning of these requirements is, or even what documentation
> consists of.
>
> In the case of typical implementations on top of an operating
> system, about the best documentation of the conventional kind
> that you could give for tasking metrics would be to clearly
> document the sequence of operating systems calls that is made
> for any particular tasking language construct.
>
> Our view in the case of GNAT is that the sources of the run-time
> which are an integral part of our complete technical
> documentation contain precisely this information, and so far
> we have not had any instances of users requiring the information
> in any other form.

I don't have any problem with interpreting the term "formula" as a sequence
of operating system calls, but I don't think the GNAT document set (again,
as an example only) does this. Looking in the most logical place (the GNAT
Reference Manual), it says "Information on metrics is not yet available."
This implies, to me at least, that this information is *not* available
elsewhere in the "documentation". If this statement were completely absent,
and/or if there were a general statement somewhere that the source code was
also considered part of the documentation, I think I'd have an easier time
accepting that the requirement were met.

Why wouldn't the reference manual, at a minimum, refer to the applicable
run-time sources (making them clearly part of the "documentation")? This is
done in other cases (e.g. for storage pools), and it certainly would have
helped me when I needed this information! Considering all of the effort
that's gone into good error messages and the like, this sort of
easily-corrected "gap" in addressing a *language requirement* (not, I want
to point out, "desirable documentation") is surprising.

> I don't think this is so surprising. We could of course run
> benchmarks on particular machines under particular conditions
> and publish numbers that appear to meet the requirements of
> the RM. Why "appear"? Because in practice they won't be useful
> for users.

It's perfectly OK to think that a requirement is not useful, and in fact I
agree with you to some extent. However, that is beside the point.

I get a queasy feeling when it seems that the Ada vendor community may not
be addressing requirements they don't like. First, it doesn't seem to be a
very "software engineering" oriented solution, given how Ada folks are
supposedly more "software engineering" oriented than users of other
languages. Second, how can anyone claim that an advantage of Ada is
standardization, if vendors don't have to follow parts of the standard that
are hard to implement, or that they just don't like?

I'd hope we all agree that a compiler that passes the validation suite, but
can't handle *any* other valid Ada program, is also in practice not useful
for users, and should be universally denounced as not compliant to the
standard. However, once we start down the path of ignoring the requirements
we don't like, we lose any right to complain about such a case.

It seems to me in the Ada83 days that AIs were used to develop and document
consensus on clarifications, etc. to the standard. Is this no longer used?

Re: Required Metrics

[Robert A Duff]

I think the answer to your original question is, "Yes, the Metrics are
real requirements, but it's not clear exactly what they mean."  In other
words, it's a matter of judgement whether or not a given piece of text
constitutes proper documentation.  I would have to agree that a
statement like, "Documentation not yet available" is a clear violation
-- I can say that without defining precisely what the rule means.

I strongly agree with Robert Dewar's statement that Documentation
Requirements should not be in the standard, and I said so many times
during the Ada 9X design.  Many people, including Tucker, did not agree.

And I think the Metrics are the worst sort of Documentation
Requirements.

Putting Doc Requirements in a standard shows confusion as to what
standards are for.  Standards are not about forcing or encouraging
people to make high quality implementations.  Standards are about
encouraging uniformity.  IMHO, of course.  ;-)

Furthermore, Doc Requirements distract implementors from the real goal,
which is good documentation.  Look at any vendors Doc Requirements
documentation.  You will usually see a long list of poorly organized
snippets of confusing, out-of-context, information.

> I get a queasy feeling when it seems that the Ada vendor community may not
> be addressing requirements they don't like. First, it doesn't seem to be a
> very "software engineering" oriented solution, given how Ada folks are
> supposedly more "software engineering" oriented than users of other
> languages. Second, how can anyone claim that an advantage of Ada is
> standardization, if vendors don't have to follow parts of the standard that
> are hard to implement, or that they just don't like?

You have a point.  However, in this particular case, if vendors fail to
provide the required documentation, that has absolutely zero effect on
portability.  The advantage of having a standard is portability.  So
it's hard to get too excited about a rule in the standard that has no
effect on portability.

> I'd hope we all agree that a compiler that passes the validation suite, but
> can't handle *any* other valid Ada program, is also in practice not useful
> for users, and should be universally denounced as not compliant to the
> standard.

Sure.

>... However, once we start down the path of ignoring the requirements
> we don't like, we lose any right to complain about such a case.

That would be a stronger argument if we were talking about some language
feature.  Eg, we all know Robert Dewar hates asynchronous transfer of
control, but it would be annoying indeed if GNAT didn't implement it.
Likewise, I hate modular types -- but of course I don't refuse to
implement them in the AverStar compilers, and I don't insist on changing
the rules to be more to my liking.  These would be true even if the
ACATS didn't test for these features.

> It seems to me in the Ada83 days that AIs were used to develop and document
> consensus on clarifications, etc. to the standard. Is this no longer used?

The ARG still exists, and still generates AI's.

I suspect that if you sent in a question saying "Are the metrics really
required?", the ARG would issue a ruling saying, "Yes, of course, it
says so in plain English."  (I'm not sure of that; some people might use
it as an opportunity to get rid of these silly rules.  Some people might
take the attitude that you can't require something unless you can
precisely define it -- and thus declare the metrics to be meaningless
gibberish.)

On the other hand, if you asked, "What, exactly, do the metrics
require?", the ARG would refuse to waste time trying to answer the
impossible.

So, whatever compiler it was that said "Documentation not yet
available", send them a bug report.  I wouldn't be surprised if it's our
compiler.  ;-)

- Bob

Re: Required Metrics

[Wes Groleau]

> a) it is meaningless, but we do meet the requirement anyway,
> perhaps not in the most useful way, but since it is pretty
> much useless, there is not much point in trying to do useless
> things in a useful manner!

Although the GNAT RM may have gaps in "documentation requirements,"
I feel that in general, it is clear and well-organized.
There are vendors for which the required documentation either does not
exist or is impossible to find.

-- 
Wes Groleau
http://freepages.genealogy.rootsweb.com/~wgroleau

Re: Required Metrics

[Robert A Duff]

Robert Dewar <dewar@gnat.com> writes:

>...Note that we could if we wanted perfectly
> well declare that our sources are part of the documentation
> and constitute the full compliance with the requirements
> of annex M.

Why not say that the binary object code of the compiler and run-time
system constitute the full documentation?  The information's in there.
;-) ;-)

- Bob

Re: Required Metrics

[Robert Dewar]

In article <wccsnvy1ddr.fsf@world.std.com>,
  Robert A Duff <bobduff@world.std.com> wrote:

> Why not say that the binary object code of the compiler and
run-time
> system constitute the full documentation?  The information's
in there.


True -- of course your printed documentation must mention this
or you will not be in full compliance with the standard :-)


Sent via Deja.com http://www.deja.com/
Before you buy.

Re: Required Metrics

[Ken Garlington]

"Robert Dewar" <robert_dewar@my-deja.com> wrote in message
news:8es58n$4lo$1@nnrp1.deja.com...
> In article <yLUP4.4522$wb7.450636@news.flash.net>,
>   "Ken Garlington" <Ken.Garlington@computer.org> wrote:
> > Why wouldn't the reference manual, at a minimum, refer to the
> > applicabl run-time sources (making them clearly part of the
> > "documentation")?
>
> There is a simple answer to this. I wrote this paragraph over
> five years ago, and no one ever bothered to notice or comment
> on it since, which may give some indication as to the real
> importance of this issue :-) The sentence in question has been
> edited appropriately.

Actually, there's a thread from May of last year that touches on this issue.

>
> Robert Dewar
> Ada Core Technologies
>
>
> Sent via Deja.com http://www.deja.com/
> Before you buy.

Re: Required Metrics

[Ken Garlington]

"Robert Dewar" <robert_dewar@my-deja.com> wrote in message
news:8es5fv$4ov$1@nnrp1.deja.com...
> In article <yLUP4.4522$wb7.450636@news.flash.net>,
>   "Ken Garlington" <Ken.Garlington@computer.org> wrote:
>
> > I get a queasy feeling when it seems that the Ada vendor
> > community may not be addressing requirements they don't like.
>
> I think that's the wrong take entirely. In the case of metrics
> (see Bob Duff's post), there is a broad feeling that these are
> completely bogus requirements for modern compilers.

Here's the first problem I have with this statement. Either

(a) it's not true, and so this is a real issue, or
(b) it is true, and has been true for some time, in which case how did it
manage to get into the standard? or
(c) something changed between 1994-ish and now (really well before now,
since apparently no one has ever met this part of the standard), and if so,
what changed?

Here's the second (and larger) problem, as I've mentioned before. Why should
"feelings" be related to requirements? Why go through the motions to
formally establish the requirement and get a consensus documented and
approved, if there's not an equal intent to do something formal (an AI, at
least) to remove such a "bogus" requirement? It appears like the classic
case of people writing down a requirements document just because the
customer says you have to do one, but promptly ignoring it through the rest
of the life cycle. Regardless of the facts of the matter, should the Ada
community give such an appearance?

> If they
> ever had any meaning it would only be for bare-board compilers
> for very specific architectures.

However, I'd claim that there are still a significant number of real-time
systems that do, in fact, get implemented on bare-board systems. I don't
know which are the "specific architectures," but certainly real-time systems
use a variety of them. Since we're talking about the real-time annex, then
surely the environments in which real-time systems operate should be a
strong consideration for the appropriateness of such a requirement.

For example, is there a GNORT target for which the metrics are meaningful?

> It is not that the vendor "does not like" this particular
> requirement in our case, it is simply that
>
> a) it is meaningless, but we do meet the requirement anyway,
> perhaps not in the most useful way, but since it is pretty
> much useless, there is not much point in trying to do useless
> things in a useful manner!

Again, the issue of whether it's an appropriate requirement is interesting,
but it's not the real issue. I'm still not convinced that you meet the
requirement in the current release, for that matter. More importantly, since
this is not about a specific compiler, I'm not convinced _anyone_ has
bothered to meet the requirement. I don't have proof that this is the case,
but it concerns me. Most importantly, it seems this argument can be used to
justify all sorts of mischief.

> b) our customers have zero interest in us doing any more
> than we do now.
>
> As we often find, CLA readers and contributors seem to have
> rather different priorities from actual Ada users when we get
> to discussing various theoretical issues surrounding the RM :-)

Again, I find the argument of user priorities a very slippery slope, which
can be used to invisibly drop any number of requirements. It's useful for
discussing the options for implementing a requirement (e.g. a fancy
calculator for computing the metric vs. "here's the sources, have at it"),
but good software engineering would say that failing to meet a requirement
due to perceived user desire, particularly when the discrepancy is not
explicitly documented, is definitely not a Good Thing.

Re: Required Metrics

[Roger Barnett]

As an aside, what is the status of the validation suite nowadays,
i.e. is there an Ada 95 revision administered by the DoD and their 
appointed agencies outside of the US ?

[ I was last involved with this stuff in the 1980s, so I suspect
  I'm a little out of date ]

-- 
Roger Barnett

Re: Required Metrics

[Ken Garlington]

"Robert A Duff" <bobduff@world.std.com> wrote in message
news:wcczoq7jyt9.fsf@world.std.com...

> The advantage of having a standard is portability.  So
> it's hard to get too excited about a rule in the standard that has no
> effect on portability.... Eg, we all know Robert Dewar hates asynchronous
transfer of
> control, but it would be annoying indeed if GNAT didn't implement it.
> Likewise, I hate modular types -- but of course I don't refuse to
> implement them in the AverStar compilers, and I don't insist on changing
> the rules to be more to my liking.  These would be true even if the
> ACATS didn't test for these features.

You *have* changed the rules to be more to your liking, if you can
*unilaterally* decide which rules affect portability (or any other desirable
aspect of standardization, such as the ability to reduce training costs).
For example, if a vendor decides that there's some required aspect of ATC
that no one is ever going to use, and it's not required to pass the
validation suite, then it sounds like that vendor is permitted to not
implement the feature. Certainly, no one could in good conscience complain
that this violates the standard, if they accept your argument.

In addition, if vendors alone are permitted to decide what really _isn't_ in
the standard, why can't vendors alone decide what's really _in_ the
standard? Why spend all this time fooling with ISO standardization
procedures if vendors have, in essence, a veto? Is it just for the publicity
value?

> The ARG still exists, and still generates AI's.
>
> I suspect that if you sent in a question saying "Are the metrics really
> required?", the ARG would issue a ruling saying, "Yes, of course, it
> says so in plain English."  (I'm not sure of that; some people might use
> it as an opportunity to get rid of these silly rules.  Some people might
> take the attitude that you can't require something unless you can
> precisely define it -- and thus declare the metrics to be meaningless
> gibberish.)

Per the standard, I just sent the mail message attached at the end. We'll
see what happens...

> On the other hand, if you asked, "What, exactly, do the metrics
> require?", the ARG would refuse to waste time trying to answer the
> impossible.
>
> So, whatever compiler it was that said "Documentation not yet
> available", send them a bug report.  I wouldn't be surprised if it's our
> compiler.  ;-)

If it's impossible to answer whether or not an implementation meets the
requirement, what's the bug? How would I know if it's fixed? In particular,
if _every_ vendor is doing this (and no one has said otherwise), it sounds
like something a bug report to a single vendor won't fix.

-----

From: Ken Garlington [mailto:Ken.Garlington@computer.org]
Sent: Wednesday, May 03, 2000 7:23 PM
To: ada-comment@sw-eng.falls-church.va.us
Subject: Minimum criteria for metrics documentation

!topic Minimum criteria for metrics documentation
!reference RM95-D(2)
!from Author Name 00-05-03
!keywords metrics, documentation, real-time
!discussion

Is there any criteria that can be used to determine if a vendor has formally
met the requirements to document metrics described in D(2-6)? For example,
which (if any) of the following conditions would be considered acceptable?

1. The vendor states that the metrics are not currently available.

2. Same as #1, but vendor provides source code for its implementation of
interfaces to an underlying operating system.

3. Same as #2, but vendor explicitly states that source code is provided to
meet requirement.

4. Same as #3, but vendor provides a list of which source code files are
applicable.

Re: Required Metrics

[Robert Dewar]

In article <yLUP4.4522$wb7.450636@news.flash.net>,
  "Ken Garlington" <Ken.Garlington@computer.org> wrote:
> Why wouldn't the reference manual, at a minimum, refer to the
> applicabl run-time sources (making them clearly part of the
> "documentation")?

There is a simple answer to this. I wrote this paragraph over
five years ago, and no one ever bothered to notice or comment
on it since, which may give some indication as to the real
importance of this issue :-) The sentence in question has been
edited appropriately.

Robert Dewar
Ada Core Technologies


Sent via Deja.com http://www.deja.com/
Before you buy.

Re: Required Metrics

[Robert Dewar]

In article <yLUP4.4522$wb7.450636@news.flash.net>,
  "Ken Garlington" <Ken.Garlington@computer.org> wrote:

> I get a queasy feeling when it seems that the Ada vendor
> community may not be addressing requirements they don't like.

I think that's the wrong take entirely. In the case of metrics
(see Bob Duff's post), there is a broad feeling that these are
completely bogus requirements for modern compilers. If they
ever had any meaning it would only be for bare-board compilers
for very specific architectures.

It is not that the vendor "does not like" this particular
requirement in our case, it is simply that

a) it is meaningless, but we do meet the requirement anyway,
perhaps not in the most useful way, but since it is pretty
much useless, there is not much point in trying to do useless
things in a useful manner!

b) our customers have zero interest in us doing any more
than we do now.

As we often find, CLA readers and contributors seem to have
rather different priorities from actual Ada users when we get
to discussing various theoretical issues surrounding the RM :-)

Robert Dewar
Ada Core Technologies


Sent via Deja.com http://www.deja.com/
Before you buy.

Re: Required Metrics

[Robert Dewar]

In article <zw3Q4.5766$wb7.485167@news.flash.net>,
  "Ken Garlington" <Ken.Garlington@computer.org> wrote:
> You *have* changed the rules to be more to your liking, if you
can
> *unilaterally* decide which rules affect portability (or any
other desirable
> aspect of standardization, such as the ability to reduce
training costs).
> For example, if a vendor decides that there's some required
aspect of ATC
> that no one is ever going to use, and it's not required to
pass the
> validation suite, then it sounds like that vendor is permitted
to not
> implement the feature. Certainly, no one could in good
conscience complain
> that this violates the standard, if they accept your argument.

That's a strawman. Obviously no one can sign the DOC if they
have deliberately failed to implement some aspect of the
standard, whether or not it is tested in the ACAATS tests

> In addition, if vendors alone are permitted to decide what
really _isn't_ in
> the standard, why can't vendors alone decide what's really
_in_ the
> standard? Why spend all this time fooling with ISO
standardization
> procedures if vendors have, in essence, a veto? Is it just for
the publicity
> value?

That's an absurd over-reaction to the thread at hand, not based
in any reality.

Remember this thread is all about a requirement that is clearly
semantically meaningless in formal terms (the RM does not even
describe what the word "documentation" means, and that's a
serious omission. Note that we could if we wanted perfectly
well declare that our sources are part of the documentation
and constitute the full compliance with the requirements
of annex M. That would meet the RM standards from a legal
point of view, but it would be useless to our customers.

I really think the documentation requirements of the standard
have almost no effect on compiler writers. Documentation is
written for the benefit of users. It is presumptious in any
case for the RM to think it can know what kinds of documentation
the user will require. Furthermore, if it requires documentation
without any indication of what the requirement means, then the
requirement is plain useless.

But Ken, extending this to clearly defined technical features,
where the requirements are meaningful and semantically sound
makes no sense at all. Bob Duff was quite clear in saying that
in THIS area, vendors follow the RM closely even if they don't
like what it says, and even if the tests do not test something.
His examples were well chosen ones. I do indeed think that ATC
was a huge mistake in the design of Ada, but Ada Core
Technologies has invested significant resources in making this
feature work completely and well, not just to pass the tests,
but to meet the requirements of our users.

Your concerns about vendors running amok would make more sense
if you would give us some nice examples of what it is that you
do not like :-)

Robert Dewar
Ada Core Technologies



>
> > The ARG still exists, and still generates AI's.
> >
> > I suspect that if you sent in a question saying "Are the
metrics really
> > required?", the ARG would issue a ruling saying, "Yes, of
course, it
> > says so in plain English."  (I'm not sure of that; some
people might use
> > it as an opportunity to get rid of these silly rules.  Some
people might
> > take the attitude that you can't require something unless
you can
> > precisely define it -- and thus declare the metrics to be
meaningless
> > gibberish.)
>
> Per the standard, I just sent the mail message attached at the
end. We'll
> see what happens...
>
> > On the other hand, if you asked, "What, exactly, do the
metrics
> > require?", the ARG would refuse to waste time trying to
answer the
> > impossible.
> >
> > So, whatever compiler it was that said "Documentation not
yet
> > available", send them a bug report.  I wouldn't be surprised
if it's our
> > compiler.  ;-)
>
> If it's impossible to answer whether or not an implementation
meets the
> requirement, what's the bug? How would I know if it's fixed?
In particular,
> if _every_ vendor is doing this (and no one has said
otherwise), it sounds
> like something a bug report to a single vendor won't fix.
>
> -----
>
> From: Ken Garlington [mailto:Ken.Garlington@computer.org]
> Sent: Wednesday, May 03, 2000 7:23 PM
> To: ada-comment@sw-eng.falls-church.va.us
> Subject: Minimum criteria for metrics documentation
>
> !topic Minimum criteria for metrics documentation
> !reference RM95-D(2)
> !from Author Name 00-05-03
> !keywords metrics, documentation, real-time
> !discussion
>
> Is there any criteria that can be used to determine if a
vendor has formally
> met the requirements to document metrics described in D(2-6)?
For example,
> which (if any) of the following conditions would be considered
acceptable?
>
> 1. The vendor states that the metrics are not currently
available.
>
> 2. Same as #1, but vendor provides source code for its
implementation of
> interfaces to an underlying operating system.
>
> 3. Same as #2, but vendor explicitly states that source code
is provided to
> meet requirement.
>
> 4. Same as #3, but vendor provides a list of which source code
files are
> applicable.
>
>


Sent via Deja.com http://www.deja.com/
Before you buy.

Re: Required Metrics

[Robert Dewar]

In article <8emkv7$uf6$1@nnrp1.deja.com>,
  Ted Dennison <dennison@telepath.com> wrote:

> > Alternate phrasing of the question: If a requirement is in
the
> > standard, and no one makes an effort to follow it, what's
the
> > requirement doing in the standard? Why not have an
interpretation that
> > says, in essence: "Oops. Never mind"? Or am I just missing
some
> > fundamental point here?


Well of course, this should be the point of view. And indeed
properly following that point of view would have avoided all
the meaningless documentation requirements in the RM.

The ARG does in fact sometimes decide that things that look
like requirements in the RM are meaningess. For instance we
recently discovered that the Suppress (.., On=> ...) form
of this pragma is completely meaningless, and the language
in the standard is bogus -- compilers do different things,
and what appears like a requirement is in fact in practice
a statement that this feature is completely implementation
dependent.



Sent via Deja.com http://www.deja.com/
Before you buy.

Re: Required Metrics

[Robert Dewar]

In article <BQnP4.3698$wb7.344147@news.flash.net>,
  "Ken Garlington" <Ken.Garlington@computer.org> wrote:

> I'm not surprised, but I don't think it really answers my
question. The
> issue isn't, in my mind, "How easy is it for vendors to ignore
certain
> requirements?".  It's "Is there really a requirement?" (See my
response to
> Tucker's post for more...)


For a validated compiler -- difficult! In particular, how can
the DOC be signed if you have ignored a requirement.

Once again, the documentation requirements, the primary subject
of this thread, are in a rather special category, since these
are untestable, undefined requirements. Normally one would not
expect to find untestable, undefined stuff in a language
standard, and for the most part, one does not in the Ada RM,
but I am afraid there are exceptions :-)




Sent via Deja.com http://www.deja.com/
Before you buy.

Re: Required Metrics

[Ted Dennison]

In article <8es60q$5fm$1@nnrp1.deja.com>,
  Robert Dewar <robert_dewar@my-deja.com> wrote:
> In article <8emkv7$uf6$1@nnrp1.deja.com>,
>   Ted Dennison <dennison@telepath.com> wrote:
>
> > > Alternate phrasing of the question: If a requirement is in

Just a nit here, but none of the quoted text is mine. I eventually
resigned myself to not trusting the "required" documentation last year
when this issue came up. Its tough to regenerate any outrage about it
now. :-)


--
T.E.D.

http://www.telepath.com/~dennison/Ted/TED.html


Sent via Deja.com http://www.deja.com/
Before you buy.

Re: Required Metrics

[Robert Dewar]

In article <_HnQ4.7884$wb7.550012@news.flash.net>,
  "Ken Garlington" <Ken.Garlington@computer.org> wrote:

> > I think that's the wrong take entirely. In the case of
metrics
> > (see Bob Duff's post), there is a broad feeling that these
are
> > completely bogus requirements for modern compilers.
>
> Here's the first problem I have with this statement. Either
>
> (a) it's not true, and so this is a real issue

  It's true, but always remember the issue here is not whether
  the documentation might or might not be useful, but whether
  it belongs in a language standard.

> (b) it is true, and has been true for some time, in which case
>     how did it manage to get into the standard?

  It has always been true that documentation requirements like
  this do not belong in the standard. As to why they got in, I
  think your posts have vividly illustrated the thinking behind
  those who argued for putting them in. To me a classical
  case of confusing desirable implementation features with
  formal semantic requirements.

> (c) something changed between 1994-ish and now (really well
> before now,

  No, nothing has changed

> since apparently no one has ever met this part of the
> standard),

  You keep saying this, which means you still do not understand
  the issue here, which is that we are talking about formal
  semantic requirements.

  On the contrary, everyone has met it, since from a formal
  point of view, documentation is not defined, and therefore
  can be defined anyway you want. As Bob Duff points out,
  you can define the compiler itself as an operational
  definition of its own behavior, and therefore it
  constitutes documentation in a formal sense. Providing
  the sources is a more realistic form of this approach
  to documentation, but still may not meet your *non-formal*
  implementation requirements.

> and if so,
> what changed?

  I think you could argue that things have indeed change from
  the point of view of informal documentation requirements here.
  And that probably accounts for the overwhelming lack of
  interest in getting more accessible forms of this
  documentation, since it was (like much of the RT annex)
  framed with bare board single processor implementations
  in mind, whereas modern reality is more likely implementation
  over an OS or 3rd party RTOS.

  But that does not affect the discussion about whether such
  requirements should be formal semantic requirements in the
  language. The answer to that is, and has always been, no.
  The fact that such requirements got in was just a compromise
  necessitated by a standards process involving many people,
  not all of whom understand what formal language requirements
  are about, and who don't mind having undefined wooly
  requirements in what should be a precise specification.

> Here's the second (and larger) problem, as I've mentioned
> before. Why should
> "feelings" be related to requirements?

  They should not be. The problem with these requirements
  was precisely that the "feeling" expressed by you forcefully
  that Ada implementations should be "required" to provide
  useful documentation got translated into requirements.

> Why go through the motions to
> formally establish the requirement and get a consensus
documented and
> approved, if there's not an equal intent to do something
formal (an AI, at
> least) to remove such a "bogus" requirement?

  Lots of us knew these requirements were bogus, including
  key people on the design team (e.g. Bob Duff). You fight
  many battles in a consensus process. This one was not
  worth fighting at the time, since the effect of these
  requirements is negligible in practice. We knew they
  would end up being useless of course, but we also
  knew they would not be harmful.

> It appears like the classic
> case of people writing down a requirements document just
> because the
> customer says you have to do one,

  Well, you paint with too broad a brush. This particular
  phenomenon (documentation requirements) was indeed a
  matter of responding to customers who did not really
  understand the issue. But the great majority of the
  Ada RM is not vaguely related to this process.

> but promptly ignoring it through the rest
> of the life cycle. Regardless of the facts of the matter,
> should the Ada
> community give such an appearance?

  Of course not, but as you have made very clear, there are
  significant players in the Ada community who do not understand
  that requirements like this do not belong in the RM, so even
  now it might be a difficult battle to win, and frankly it is
  not worth it, since it is unimportant.

> > If they
> > ever had any meaning it would only be for bare-board
compilers
> > for very specific architectures.

  Note that I am talking about the value of these items as
  pragmatic implementation features, not formal requirements
  when I say this.

> However, I'd claim that there are still a significant number
> of real-time systems that do, in fact, get implemented on
> bare-board systems.

  It's certainly a dieing breed. We see almost no demand for
  this, not zero, but barely on the radar screen. The trend
  is clearly to 3rd party RTOS based systems (VxWorks, and
  also see Greenhills new Integrity push).

> I don't
> know which are the "specific architectures," but certainly
real-time systems
> use a variety of them. Since we're talking about the real-time
annex, then
> surely the environments in which real-time systems operate
should be a
> strong consideration for the appropriateness of such a
requirement.

  I think if you are in such an environment, you should require
  your implementor to provide whatever information you need.
  The RM will be no help in this task, since the issue is not
  formal requirements, but rather providing information in the
  manner YOU require.

> For example, is there a GNORT target for which the metrics are
> meaningful?

  Seeing as GNORT excludes tasking and the real time annex in
  its entirety, no :-)

> > It is not that the vendor "does not like" this particular
> > requirement in our case, it is simply that
> >
> > a) it is meaningless, but we do meet the requirement anyway,
> > perhaps not in the most useful way, but since it is pretty
> > much useless, there is not much point in trying to do
> > useless
> > things in a useful manner!


> Again, the issue of whether it's an appropriate requirement is
> interesting, but it's not the real issue. I'm still not
> convinced that you meet the
> requirement in the current release, for that matter.

  Well try to prove this formally. First you will need a
  formal definition of "documentation" *derived from the
  RM*. That will be hard. For example, are you sure that
  this documentation must be in English? What if I provide
  it in Navaho or Klingon, is that sufficient, or, more to
  the point here, what if I provide it in Ada 95.
  Furthermore, if your idea is that you want to see
  metrics in milliseconds, then this is clearly infeasible
  in general working over third party systems, so the
  conditions of RM 1.1.3(6) would apply:

    6  Contain no variations except those explicitly permitted
       by this
       International Standard, or those that are impossible or
       impractical to avoid given the implementation's execution
       environment;

> More importantly, since
> this is not about a specific compiler, I'm not convinced
> _anyone_ has
> bothered to meet the requirement.

  You still think of this as a requirement that can be met or
  not met. And it is this invalid thinking that lead to having
  these kind of requirements in the RM. Since the requirement
  is semantically vacuous, I think it is clear that all
  compilers meet it.

> I don't have proof that this is the case,
> but it concerns me. Most importantly, it seems this argument
> can be used to
> justify all sorts of mischief.

  That's FUD unless you can back it up. I see no possible
  spill over from requirements that are meaningless to
  handling of requirements that are well stated and
  semantically meaningful.

> > b) our customers have zero interest in us doing any more
> > than we do now.
> >
> > As we often find, CLA readers and contributors seem to have
> > rather different priorities from actual Ada users when we
get
> > to discussing various theoretical issues surrounding the RM
:-)

> Again, I find the argument of user priorities a very slippery
> slope, which can be used to invisibly drop any number of
> requirements.

  No, that's completely wrong, and I begin to think that it
  is a hopeless task to convince you. No requirements have
  been dropped by anyone here.

  The only issue is that you have in your mind some
  interpretation of these requirements that is not in the
  RM, and you note that compilers do not follow Ken
  Garlington's interpretation of these requirements.

  Very possibly so, but that has nothing to do with whether
  the requirements are followed.

  Now take the visibility rules of Ada in chapter 8. Here
  we don't care what Ken thinks about them, because

    a) they are well defined

    b) it is well defined whether a compiler implements
       them correctly.

  A totally different situation.

  It is really critical in a formal language standard that
  all requirements be objective and well defined. Once again
  requirements for "good" documentation are as silly as
  requirements for good performance.

  As I mentioned before, there were actually people seriously
  asking for quantitative requirements on performance of tasking
  constructs in terms of processor clock cycles. I know, that's
  hard to believe that anyone could be that misguided but it's
  true. The undefined metrics requirements were a compromise to
  deal with this unreasonable extreme. Since they are undefined
  and harmless, it was a way of getting things done without
  running into a road block.

> It's useful for
> discussing the options for implementing a requirement (e.g. a
fancy
> calculator for computing the metric vs. "here's the sources,
have at it"),
> but good software engineering would say that failing to meet a
> requirement
> due to perceived user desire

  The trouble with these undefined requirements is that the
  only POSSIBLE meaning is in terms of "perceivced user
  desires". In other words if we try to ask if a compiler
  meets certain documentation requirements, we cannot look
  to any formal definitions and we cannot look to the RM,
  the ONLY way of determining whether such requirements have
  been met is to interrogate "perceived user desire".

  Yes, that's a horrible state of affairs and that's what
  this thread is all about, but it is really stil a tempest
  in a teapot in terms of overall significance.

  By the way, the ACVC review board discussed this briefly.
  They immediately agreed that these requirements were too
  ill-defined to even consider including them in conformance
  assessment. There was no controversy in this decision, it
  was clear to this committee that these "requirements" have
  to be treated differently, namely ignored completely when
  it comes to formal conformance testing.


Sent via Deja.com http://www.deja.com/
Before you buy.

Re: Required Metrics

[Robert Dewar]

In article <6FnQ4.7883$wb7.551417@news.flash.net>,
  "Ken Garlington" <Ken.Garlington@computer.org> wrote:
> Actually, there's a thread from May of last year that touches
> on this issue.

When I said no one had raised this issue, I meant none of our
customers. I doubt any vendor is greatly influenced by wants
expressed by contributors to comp.lang.ada. Certainly Ada
Core Technologies essentially ignores this forum in terms
of language suggestions etc (surely you would be surprised
if it were otherwise). We look to what our customers and
potential customers ask us for -- we get plenty of good
input that way, and a much clearer indication of true
priorities!

My particular point was that no one ever commented on the
obsolete statement in our reference manual, and I find that
interesting :-)


Sent via Deja.com http://www.deja.com/
Before you buy.

Re: Required Metrics

[Robert Dewar]

In article <CJoQ4.7913$wb7.555817@news.flash.net>,
  "Ken Garlington" <Ken.Garlington@computer.org> wrote:
> So, why are some meaningless requirements addressed by the
> ARG, and others not?

Because the ARG regards it as a waste of time to spend any
time on the issue of obviously subjective requirements like
the documentation requirements. All it takes is one member
of the ARG to press to raise this issue, but no one has and
no one is likely to. Even those who supported the inclusion
of these requirements in the RM have no inclination to
revisit the matter. I personally agree it would be a waste
of time.

But if there is a requirement which on its face *appears* to
be well defined, but on careful technical analysis is not, then
that's a good point for the ARG to resolve.

In the particular case (Suppress ON) that I mentioned, this
came up during the attempt to semi-standardize the GNAT pragma
Unsuppress, and the question was what does UNsuppress/ON mean.
THe facile answer: the same as Suppress/ON. But Unsuppress has
far more potent semantics since it guarantees that a check will
not be performed, unlike the case with Suppress/ON which
guarantees nothing at all. So we really needed to be more
precise, and we discovered that we were building on shifting
sands here.



Sent via Deja.com http://www.deja.com/
Before you buy.

Re: Required Metrics

[Robert Dewar]

In article <%MoQ4.7915$wb7.556168@news.flash.net>,
  "Ken Garlington" <Ken.Garlington@computer.org> wrote:

> > For a validated compiler -- difficult! In particular, how
> > can the DOC be signed if you have ignored a requirement.
>
> I suspect with a pen in the hand, and a song in the heart!

Ken, do you think it is fair to accuse vendors of dishonesty
and duplicity with no evidence at all to back up your
statements. I will say right away that I personally sign
all DOC's for Ada Core Technologies, and in all cases I
do so with integrity knowing that what I am signing is
absolutely correct. I would assume that all other vendors
would make the same statement.

> I think you read the statement backwards, by the way. To
> repeat: The issue
> isn't, in my mind, "How easy is it for vendors to ignore
> certain
> requirements?".  It's "Is there really a requirement?"

For the case of documentation requirements, the answer is
basically no, it is not a requirement at all in the usual
formal semantic sense.

> > Once again, the documentation requirements, the primary
subject
> > of this thread, are in a rather special category, since
these
> > are untestable, undefined requirements. Normally one would
not
> > expect to find untestable, undefined stuff in a language
> > standard, and for the most part, one does not in the Ada RM,
> > but I am afraid there are exceptions :-)

> I suspect, after you read the discussion of pragma Reviewable,
we're going
> to be expanding this to "documentation and some implementation
requirements"

  Nope, the issues with pragma Reviewable are also all
  documentation based, let's look:

  The definition of pragma reviewable is in two sections
  BOTH of them have the language:

    THe implementation shall provide the following
    information ....

Well, "shall" is an impressive word, it sure *sounds* like
a requirement. Unfortunately, what on earth does it mean to
"provide the following information". Ooops, totally undefined.
And since the information for ALL sections is equivalent to
information provided in the object file, you can argue from
a formal point of view that the object file satisfies the
requirement.

Now of course from a pragmatic point of view, we want the
information in a much more usable form, and GNAT provides
a lot of *non-required* capabilities in this area. For
example, let's take:

    6  Where compiler-generated run-time checks remain;

From a formal point of view, we could just say "look in the
object module for all calls to __gnat_raise_constraint_error
or whatever, but in practice it is useful (but not required)
to have this information in a clearer source-related fashion.
So GNAT provides the generated source listing, where for
example if you write

    x := a(b,c);

in the source, then the generated source listing (-gnatG) might
say:

    x := a(b,{c});

where, as documented in the GNAT documentation, the curly
brackets mean that the expression inside is range checked
at run time.

That's a nice feature, very useful for users, and definitely
meeting the spirit of the requirement quoted above.

But in a formal language definition, we are not in the business
of meeting the spirit of requirements!

It would have been SO much better to make all these
documentation "requirements" into implementation advice
for several reasons:

 1. In IA sections, since they are not part of the formal
    standard, we are much freer to make informal requirements
    of the "you-will-know-it-when-you-see-it" form.

 2. IA sections get interpreted in a pragmatic manner, which
    is just right for this purpose.

 3. Implementors have to document which IA they follow and
    which they do not, leading to a useful account of how
    each IA section is addressed (see the section in the
    GNAT RM that does this for example).








Sent via Deja.com http://www.deja.com/
Before you buy.

Re: Required Metrics

[Robert Dewar]

In article <842359703wnr@natron.demon.co.uk>,
  Roger@natron.demon.co.uk wrote:
>
> As an aside, what is the status of the validation suite
nowadays,
> i.e. is there an Ada 95 revision administered by the DoD and
their
> appointed agencies outside of the US ?


In brief, validation has become internationalized and is now
under control of ISO WG9, it is no longer a process administered
by the authorities in one member nation, and thus completely
outside the ISO process. The ISO WG9/ARG now adjudicates
validation issues, and there is an ISO standard for validation
of Ada compilers (not quite sure if it is formally approved yet,
but if not it is essentially at that stage).

Any organization following the requirements of this ISO
standard can carry out ISO style validations. In the
US, the Ada Resource Association funds an organization
that carries out validations.

New tests are being written as appropriate, under the aegis
of the ARG.

SO in fact AJPO getting out of the validation business has
resulted in a considerable upgrade of the entire validation
process.

Validations under the new procedures and test suite have already
been successfully performed, and now have formal international
recognition.

Perhaps Randy Burkhardt would like to add something to this
thread at this point if he is reading it :-)


Sent via Deja.com http://www.deja.com/
Before you buy.

Re: Required Metrics

[Ted Dennison]

In article <8eulom$u8m$1@nnrp1.deja.com>,
  Robert Dewar <robert_dewar@my-deja.com> wrote:

> It would have been SO much better to make all these
> documentation "requirements" into implementation advice
> for several reasons:
>
>  3. Implementors have to document which IA they follow and
>     which they do not, leading to a useful account of how
>     each IA section is addressed (see the section in the
>     GNAT RM that does this for example).

But that is a documentation requirement too. If documentation
requirements became implementation advice, implementors would no loger
*have* to document which IA they follow, right? Or is that your point?

--
T.E.D.

http://www.telepath.com/~dennison/Ted/TED.html


Sent via Deja.com http://www.deja.com/
Before you buy.

Re: Required Metrics

[Ken Garlington]

"Robert Dewar" <dewar@gnat.com> wrote in message
news:8es5sk$5cr$1@nnrp1.deja.com...
> That's a strawman. Obviously no one can sign the DOC if they
> have deliberately failed to implement some aspect of the
> standard, whether or not it is tested in the ACAATS tests

Why not, if the vendor believes it's an unreasonable requirement? For
example, if a vendor believes that "the documentation requirements of the
standard have almost no effect on compiler writers," then he might be
justified in signing off without ever examining his documentation for
compliance.

I also wonder about inadvertant failures. Let's take a specific example:
Before ACT signs off, what items -- other than ensuring all ACAATS tests
pass -- are on your checklist?

> > In addition, if vendors alone are permitted to decide what
> really _isn't_ in
> > the standard, why can't vendors alone decide what's really
> _in_ the
> > standard? Why spend all this time fooling with ISO
> standardization
> > procedures if vendors have, in essence, a veto? Is it just for
> the publicity
> > value?
>
> That's an absurd over-reaction to the thread at hand, not based
> in any reality.

How can questions be "over-reactions?" Sounds like _someone_ doesn't have
any answers handy... :)

More to the point, I have to at least assume this is a possibility, given
that there (a) is evidence that vendors can choose what they want to
implement and (b) no obvious counter-evidence that there are any limits
beyond the ACAATS tests as to what vendors might ignore.

> Remember this thread is all about a requirement that is clearly
> semantically meaningless in formal terms (the RM does not even
> describe what the word "documentation" means, and that's a
> serious omission. Note that we could if we wanted perfectly
> well declare that our sources are part of the documentation
> and constitute the full compliance with the requirements
> of annex M. That would meet the RM standards from a legal
> point of view, but it would be useless to our customers.

But (and this is the key point, which I will keep repeating until someone
bothers to address it), isn't there an intrinsic value in meeting the
standards "from a legal point of view"? Or is it all just based on what a
particular vendor's customers want?

Secondly, if there is something in the standards that is unreasonable, isn't
there an intrinsic value in going through the formal process to fix it?

> I really think the documentation requirements of the standard
> have almost no effect on compiler writers. Documentation is
> written for the benefit of users. It is presumptious in any
> case for the RM to think it can know what kinds of documentation
> the user will require. Furthermore, if it requires documentation
> without any indication of what the requirement means, then the
> requirement is plain useless.

> But Ken, extending this to clearly defined technical features,
> where the requirements are meaningful and semantically sound
> makes no sense at all. Bob Duff was quite clear in saying that
> in THIS area, vendors follow the RM closely even if they don't
> like what it says, and even if the tests do not test something.
> His examples were well chosen ones. I do indeed think that ATC
> was a huge mistake in the design of Ada, but Ada Core
> Technologies has invested significant resources in making this
> feature work completely and well, not just to pass the tests,
> but to meet the requirements of our users.

I was really feeling good until the last line ("meet the requirements of our
users"). Again, I think you're missing the point. I assume you're investing
in ATC because users are asking you for this feature. I assume Sun invested
in a good implementation of java.net without benefit of an ISO standard, as
well.

Without the benefit of knowing what your customers are asking from you, let
me take a stab in the dark at another example, that maybe will get the point
across. Let's say that a vendor claims to fully implement the Safety and
Secuity annex. Pragma Reviewable is part of that annex. It has a series of
requirements identified as "implementation requirements", not documentation
requirements. Are these "clearly defined technical features?" Are they
"semantically sound"? I know there's compilers (including Ada83 compilers)
that meet these requirements, so I assume that at least some vendors believe
they are implementable. I also know that it's really expensive to implement
these (because I kept having to pay big bucks to get them implemented,
because they weren't part of the standard).

Before this conversation, I would have said "If a vendor claims conformance
to Annex H, these requirements have to be implemented, or the vendor has to
explicitly say that they aren't". However, let's say this vendor doesn't
even identify pragma Reviewable in their documentation, much less implement
the required behavior. If I complained about this, would I have a leg to
stand on? Unless I could get some other users to agree, sounds like I'm out
of luck.

It sounds like we may have gone to all the trouble to identify requirements
that IHMO are really important for safety-critical systems, and managed to
get it into the standard so that we could share the cost among a larger
group of users, just to end up in the same fix we were in back in 1985. I
can hear the conversation now:

"You claim conformance to the standard, but you don't implement these
features."
"Well, they don't affect program execution, and they're not tested as part
of validation, and no other users have asked for them, so we never did
them."
"Maybe that's because we're the first safety-critical application to use
your product?"
"Well, if you don't like it, pick another vendor!"
"Unfortunately, you're the only vendor for our platform!"
"OK - pay us and we'll do it."

That conversation is easy to reproduce, because I heard it a lot over the
years. The main difference? Before, we said "yes". Now, we say "C". :)

(The next time anyone complains about why companies are abandoning Ada, keep
this conversation in mind...)

> Your concerns about vendors running amok would make more sense
> if you would give us some nice examples of what it is that you
> do not like :-)

Are you saying that something really bad has to happen before anyone
considers it a risk? Remember what I was saying earlier about good software
engineering practices?

Here's an example from the Ada83 days. We had built an application on a
particular vendor's product, and it worked fine. We delivered the product.
Then, we ported it to a different platform and a different vendor. It
wouldn't compile. It turned out we had a subprogram, declared in a package
body, which used another subprogram in the same package body before it was
declared. The old (validated) compiler, we discovered, had a few problems in
enforcing the visibility rules. This probably isn't a common programming
error, so no other users had complained. (Actually, I think we wrote about
75% of all the problem reports this vendor _ever_ received.) Obviously, it
wasn't caught by the then-ACVC. It wasn't that hard to change the package
body (although I was now stuck with two variants to maintain, at least until
the "new" package body could be ported back to the old system.) So, it
sounds like I really didn't have any reason to complain about the compiler's
behavior, based on the criteria presented above?

Re: Required Metrics

[Ken Garlington]

"Robert Dewar" <robert_dewar@my-deja.com> wrote in message
news:8es60q$5fm$1@nnrp1.deja.com...
> In article <8emkv7$uf6$1@nnrp1.deja.com>,
>   Ted Dennison <dennison@telepath.com> wrote:
>
> > > Alternate phrasing of the question: If a requirement is in
> the
> > > standard, and no one makes an effort to follow it, what's
> the
> > > requirement doing in the standard? Why not have an
> interpretation that
> > > says, in essence: "Oops. Never mind"? Or am I just missing
> some
> > > fundamental point here?
>
>
> Well of course, this should be the point of view. And indeed
> properly following that point of view would have avoided all
> the meaningless documentation requirements in the RM.
>
> The ARG does in fact sometimes decide that things that look
> like requirements in the RM are meaningess. For instance we
> recently discovered that the Suppress (.., On=> ...) form
> of this pragma is completely meaningless, and the language
> in the standard is bogus -- compilers do different things,
> and what appears like a requirement is in fact in practice
> a statement that this feature is completely implementation
> dependent.

So, why are some meaningless requirements addressed by the ARG, and others
not?

(By the way, I think I was the one to make the comment quoted at the top,
not Ted Dennison, although by the end I may wish it had been someone else :)

Re: Required Metrics

[Ken Garlington]

"Robert Dewar" <robert_dewar@my-deja.com> wrote in message
news:8es65n$5hn$1@nnrp1.deja.com...
> In article <BQnP4.3698$wb7.344147@news.flash.net>,
>   "Ken Garlington" <Ken.Garlington@computer.org> wrote:
>
> > I'm not surprised, but I don't think it really answers my
> question. The
> > issue isn't, in my mind, "How easy is it for vendors to ignore
> certain
> > requirements?".  It's "Is there really a requirement?" (See my
> response to
> > Tucker's post for more...)
>
>
> For a validated compiler -- difficult! In particular, how can
> the DOC be signed if you have ignored a requirement.

I suspect with a pen in the hand, and a song in the heart!

I think you read the statement backwards, by the way. To repeat: The issue
isn't, in my mind, "How easy is it for vendors to ignore certain
requirements?".  It's "Is there really a requirement?"

> Once again, the documentation requirements, the primary subject
> of this thread, are in a rather special category, since these
> are untestable, undefined requirements. Normally one would not
> expect to find untestable, undefined stuff in a language
> standard, and for the most part, one does not in the Ada RM,
> but I am afraid there are exceptions :-)

I suspect, after you read the discussion of pragma Reviewable, we're going
to be expanding this to "documentation and some implementation requirements"
;)

Re: Required Metrics

[Ken Garlington]

"Robert Dewar" <robert_dewar@my-deja.com> wrote in message
news:8eukm0$ssm$1@nnrp1.deja.com...

>   It has always been true that documentation requirements like
>   this do not belong in the standard. As to why they got in, I
>   think your posts have vividly illustrated the thinking behind
>   those who argued for putting them in. To me a classical
>   case of confusing desirable implementation features with
>   formal semantic requirements.

In a word: Bullshit. This gets repeated (somewhat condescendingly)
throughout your response, but it's a total strawman.

Could you cite ONE case in this thread where I argued in favor of including
ANY requirement, documentation or otherwise, in the standard?

It's getting depressing, but I'll make one last run at my position:

ONCE A REQUIREMENT ENTERS THE STANDARD,  IT'S A REQUIREMENT UNTIL IT'S
REMOVED.

If a requirement shouldn't be in the standard, don't add it.
If a requirement is in there that shouldn't be, then remove it (or indicate
that it's only advice).

How hard could this be? At best, it's weakly coupled to the issue of which
requirements _should_ be in the standard.

If you want to discuss whether a requirement should or should not be in the
standard, fine. Start another thread. If you'd go back and read what I said
earlier, you'd see that I'd probably agree that documentation requirements
shouldn't be in there. So what?

>   You keep saying this, which means you still do not understand
>   the issue here, which is that we are talking about formal
>   semantic requirements.
>
>   On the contrary, everyone has met it, since from a formal
>   point of view, documentation is not defined, and therefore
>   can be defined anyway you want.

OK, so everyone always meets the requirements of the standard that aren't
formal semantic requirements? Good software engineering practice would say
that they aren't requirements then **and should be identified as such**. If
I understand ARM 1.1.2 (24-39), then the only "real" requirements are:

25 Syntax: Syntax rules (indented).
28 Static Semantics: A definition of the compile-time effect of each
construct.
30 Dynamic Semantics: A definition of the run-time effect of each construct.

Is this correct? I'd be happy to send in a comment to this effect to note
this in the standard, if we agree on this list. Actually, since there
appears to be complete agreement among the vendor community (and the ACVC
review board) as to what's in and what's not in, I guess it doesn't matter.
See the comment at the end.

>   And that probably accounts for the overwhelming lack of
>   interest in getting more accessible forms of this
>   documentation, since it was (like much of the RT annex)
>   framed with bare board single processor implementations
>   in mind, whereas modern reality is more likely implementation
>   over an OS or 3rd party RTOS.

Which is really depressing for those of us who still build bare-board
systems, but the use of DSPs and other "non-mainstream" processors has
pretty much taken Ada out of the running for us, anyway, so that's probably
why you're not seeing any demand. However, that's a different issue.

> > Why go through the motions to
> > formally establish the requirement and get a consensus
> documented and
> > approved, if there's not an equal intent to do something
> formal (an AI, at
> > least) to remove such a "bogus" requirement?
>
>   Lots of us knew these requirements were bogus, including
>   key people on the design team (e.g. Bob Duff). You fight
>   many battles in a consensus process. This one was not
>   worth fighting at the time, since the effect of these
>   requirements is negligible in practice. We knew they
>   would end up being useless of course, but we also
>   knew they would not be harmful.

Except, of course, for those of us who read the word "requirement" as it's
used in software engineering. However, once we get the list of
non-requirement requirements accepted, this is easy to fix.

You know, I lose those requirements battles from time to time myself. It
never occured to me to just ignore the ones that I considered infeasible to
meet.

> > It appears like the classic
> > case of people writing down a requirements document just
> > because the
> > customer says you have to do one,
>
>   Well, you paint with too broad a brush. This particular
>   phenomenon (documentation requirements) was indeed a
>   matter of responding to customers who did not really
>   understand the issue. But the great majority of the
>   Ada RM is not vaguely related to this process.

Whenever someone makes a post about how Ada is used by software engineers,
and other languages are used by hackers, I'm going to think about this
paragraph.

> > Again, the issue of whether it's an appropriate requirement is
> > interesting, but it's not the real issue. I'm still not
> > convinced that you meet the
> > requirement in the current release, for that matter.
>
>   Well try to prove this formally. First you will need a
>   formal definition of "documentation" *derived from the
>   RM*.

Why can't I use the definition of "documentation" *provided by the vendor*?

> That will be hard. For example, are you sure that
>   this documentation must be in English? What if I provide
>   it in Navaho or Klingon, is that sufficient, or, more to
>   the point here, what if I provide it in Ada 95.

What if you do, so long as you say "This is what I'm providing to meet the
requirement"?

Again, you're confusing "what a customer wants" with "what the standard
requires". In software engineering, validation addresses the former,
verification the latter.

>> > For a validated compiler -- difficult! In particular, how
>> > can the DOC be signed if you have ignored a requirement.
>>
>> I suspect with a pen in the hand, and a song in the heart!

> Ken, do you think it is fair to accuse vendors of dishonesty
> and duplicity with no evidence at all to back up your
> statements.

Do you think it's fair to accuse me of such an accusation, when the quoted
statement doesn't actually say that?

> I will say right away that I personally sign
> all DOC's for Ada Core Technologies, and in all cases I
> do so with integrity knowing that what I am signing is
> absolutely correct. I would assume that all other vendors
> would make the same statement.

Like I said, "with a pen in the hand, and a song in the heart." Of course,
your signature is based on your determination as to which requirements are
"real" requirements, believing that all other requirements are "met" no
matter what. You may believe that the ACVC review board, and all other
vendors, agree with you. You may be right, for that matter. If you're wrong,
there's no way to find out, since apparently the list of non-requirements
has never been formally established, and nobody (not even the vendor) checks
that any requirement is met beyond passing the conformance test.

You might want to re-read your original statement: "How can the DOC be
signed if you have ignored a requirement?" Note that the word
"intentionally" does not appear.

>> I suspect, after you read the discussion of pragma Reviewable,
>> we're going to be expanding this to "documentation and
>> some implementation requirements"

> Nope, the issues with pragma Reviewable are also all
 > documentation based, let's look:

Yes, let's. Specifically, let's look at the words which appear just before
paragraph 5:

***** Implementation Requirements *****

"5 The implementation shall provide the following information ..."

Now, I suspect you're using the term "documentation" informally, not in the
sense of ARM 1.1.2. However, since all Implementation Requirements, like all
Documentation Requirements, are not really requirements, perhaps we _should_
use the same name for both.

When we were discussing these safety and security requirements during that
industry-implementors meeting lo these many years ago, I was frankly
surprised that the vendor representatives (including yourself) didn't put up
more of a fuss. Now I see why. What a total waste of our time.

-----

From: Ken Garlington [mailto:Ken.Garlington@computer.org]
Sent: Friday, May 05, 2000 7:55 PM
To: ada-comment@sw-eng.falls-church.va.us
Subject: Non-Requirement Requirements

!topic Non-Requirement Requirements
!reference RM95-1.1.2(24-39)
!from Ken Garlington 00-05-05
!keywords requirements, advise
!discussion

The referenced paragraphs define several different categories of the
standard. Some of these categories use the term "requirements," although
they are not considered requirements by the Ada vendor community. There
should be an explicit list of which categories can be considered
requirements (i.e., it is possible for a vendor to "fail" to meet it). All
other categories should be described in the terms currently used for
"Implementation Advice."

Re: Required Metrics

[Ken Garlington]

By the way, while writing up a bug report, I found this in the gnatinfo.txt
file...

"The Documentation for GNAT consists of two manuals, the GNAT User's Guide
and the GNAT Reference Manual. These are available in a number of formats
(ASCII, HTML, PostScript and info) and are bundled as a separate
documentation
distribution and can be found at the same places as the GNAT binary and
source distributions."

...and I just had to laugh!

Re: Required Metrics

[Robert Dewar]

In article <kdKQ4.8331$wb7.618601@news.flash.net>,
  "Ken Garlington" <Ken.Garlington@computer.org> wrote:
> Except, of course, for those of us who read the word
> "requirement" as it's
> used in software engineering.

And there perhaps lies the crux of the problem, a formal
language definition has almost NOTHING to do with an informal
"software engineering" requirements document. It is that
confusion in the understanding of the word requirement that
lies at the heart of the confusion. Indeed, most software
projects are specified with a requirements document where
the requirements are often very much subjective, and not
anywhere *near* being a mathematical formal specification.

> You know, I lose those requirements battles from time to time
> myself. It never occured to me to just ignore the ones that I
> considered infeasible to meet.

You keep saying this, but that does not make the argument
here any more convincing. The documenatation requirements
are trivial to meet. Everyone meets them, no one ignores them.
That of course is from a formal language specification point
of view, but the standard is a formal language specification,
nothing more and nothing less.


> Again, you're confusing "what a customer wants" with "what the
> standard requires". In software engineering, validation
> addresses the former,
> verification the latter.

Please don't bring your software engineering background to
the table, you will just confuse yourself, it is irrelevant.
Yes, a lot of people don't understand this, including some
people who participated in the language design effort. A
background in formal language semantics is much more relevant
here, and I definitely think you will have trouble writing
semantic equations for the documentation requirements, no
matter WHAT formalism you choose. Yes, we chose to use formal
english (which is not that great mathematically) to describe
the semantics of Ada 95, so that it would be more accessible,
but it has to be equivalent to a formal specification in
some sense, or it is useless.

It's a bit like the controversy over translating the bible into
english. When this was first done, priests worried that people
would look at the result and try to interpret in an informal
context which would lead to misunderstandings. I leave it to
others to judge whether there concern was misplaced, but for
sure there is a danger in making standards accessible if people
start reading them like software engineering specifications.

Now the "implementation advice" sections of the RM *can* be
read this way, and including these sections was a brilliant
innovation on the design team's point of view, because it
allowed us to state informal pragmatic implementation
"requirements" where requirements here is used in Ken's sense
of software engineering, without contaminating the formal
definition with junk.

> Do you think it's fair to accuse me of such an accusation,
> when the quoted statement doesn't actually say that?

Yes, I think it's fair, reread what you said.

> Like I said, "with a pen in the hand, and a song in the
> heart."

Nope! No "song in the heart", which I take to mean with an
intent other than a formal declaration. Perhaps you have
some other meaning, if so, it is obscure to me.

> Of course,
> your signature is based on your determination as to which
requirements are
> "real" requirements, believing that all other requirements are
"met" no
> matter what.

All requirements are requirements. I am not saying that the
documentation requirements are not requirements, just that
they are trivially met.

> You may believe that the ACVC review board, and all other
> vendors, agree with you.

You mean I agree that there can be no formal tests for whether
you meet the documentation rquirements. I cannot imagine anyone
with any reasonable knowledge thinking otherwise.

> You may be right, for that matter. If you're wrong,
> there's no way to find out, since apparently the list of
non-requirements
> has never been formally established, and nobody (not even the
vendor) checks
> that any requirement is met beyond passing the conformance
test.

There is no list of non-requirements, because there are no
non-requirements. THe DOC requires that you have not made
any intentional deviations from the FULL SET of requirements,
including the documentation requirements.

> You might want to re-read your original statement: "How can
the DOC be
> signed if you have ignored a requirement?" Note that the word
> "intentionally" does not appear.

intentionally is of course critical. Otherwise the DOC would
be a declaration that your compiler was bug free. This is not
intended, and would be impractical.


> > Nope, the issues with pragma Reviewable are also all
>  > documentation based, let's look:
>
> Yes, let's. Specifically, let's look at the words which appear
just before
> paragraph 5:
>
> ***** Implementation Requirements *****
>
> "5 The implementation shall provide the following information
..."
>
> Now, I suspect you're using the term "documentation"
informally, not in the
> sense of ARM 1.1.2. However, since all Implementation
Requirements, like all
> Documentation Requirements, are not really requirements,
perhaps we _should_
> use the same name for both.

Of course implementation requirements are requirements, what
gave you the lunatic idea that this is not the case. I say
lunatic here, because it simply makes no sense. There are many
critical implentation requirements. Where ON EARTH did you
get this idea, certainly not from anything I wrote. Here is
an example (there are hundreds maybe thousands of others)

                         Implementation Requirements
15   An implementation shall support lines of at least 200
characters in length, not counting any characters used to
signify the end of a line.  An implementation shall support
lexical elements of at least 200 characters in length.  The
maximum supported line length and lexical element length are
implementation defined.

This is formally definable, and of course tested by the ACVC
tests. Again, I ask where did you get the idea that
implementation requirements are not requirements???????

> When we were discussing these safety and security requirements
during that
> industry-implementors meeting lo these many years ago, I was
frankly
> surprised that the vendor representatives (including yourself)
didn't put up
> more of a fuss. Now I see why. What a total waste of our time.

Actually many useful things came out of those meetings, but if
you thought that the meetings were about requiring vendors to
provide useful usable implementations, you were indeed deluding
yourself, since language standards can never guarantee this.

>
> !topic Non-Requirement Requirements
> !reference RM95-1.1.2(24-39)
> !from Ken Garlington 00-05-05
> !keywords requirements, advise
> !discussion
>
> The referenced paragraphs define several different categories
of the
> standard. Some of these categories use the term
"requirements," although
> they are not considered requirements by the Ada vendor
community. There
> should be an explicit list of which categories can be
considered
> requirements (i.e., it is possible for a vendor to "fail" to
meet it). All
> other categories should be described in the terms currently
used for
> "Implementation Advice."


This is plain silly, I can't imagine the ARG giving it much
attention, since it amounts to nothing more than an unfounded
polemic. There is no part of the Ada community that considers
any requirements in the RM not to be requirements, and
certainly the vendor community does not ignore any requirements.

Again, I think the entire problem comes from the fact that you
don't really understand the purpose of a formal language
definition, and that you read the word "requirement" in
(to use your terms) "software engineering" sense, which is
totally misleading in this context.

This reminds me of people who use to operate under the curious
illusion that validation had something to say about guaranteeing
that a compiler was usable (of course failing validation does
say that a compiler may NOT be usable, but passing validation
does not for a MOMENT constitute a guarantee that a compiler
is usable).

Perhaps we should have used VDM for the Ada standard after
all -- it would certainly have helped eliminate uncertainties
and for sure would have avoided this kind of user confusion :-)





Sent via Deja.com http://www.deja.com/
Before you buy.

Re: Required Metrics

[Robert Dewar]

In article <24VQ4.8453$wb7.646902@news.flash.net>,
  "Ken Garlington" <Ken.Garlington@computer.org> wrote:
> By the way, while writing up a bug report, I found this in the
gnatinfo.txt
> file...
>
> "The Documentation for GNAT consists of two manuals, the GNAT
User's Guide
> and the GNAT Reference Manual. These are available in a number
of formats
> (ASCII, HTML, PostScript and info) and are bundled as a
separate
> documentation
> distribution and can be found at the same places as the GNAT
binary and
> source distributions."
>
> ...and I just had to laugh!


Ken actually submitted this as a bug report. I commented to
him that this was probably the most insignificant bug report
we had ever received :-)

We "fixed" it by adding the word primary in front of
documentation. Perhaps that will suddenly make Ken think
we conform to the standard where we did not before :-)


Sent via Deja.com http://www.deja.com/
Before you buy.

Re: Required Metrics

[Robert Dewar]

In article <24VQ4.8453$wb7.646902@news.flash.net>,
  "Ken Garlington" <Ken.Garlington@computer.org> wrote:
> By the way, while writing up a bug report, I found this in the
gnatinfo.txt
> file...
>
> "The Documentation for GNAT consists of two manuals, the GNAT
User's Guide
> and the GNAT Reference Manual. These are available in a number
of formats
> (ASCII, HTML, PostScript and info) and are bundled as a
separate
> documentation
> distribution and can be found at the same places as the GNAT
binary and
> source distributions."
>
> ...and I just had to laugh!


By the way, the documentation for GNAT very definitely
includes the Ada Reference Manual -- but perhaps that's a
surprise to Ken too ... hard to tell!

This thread is quite instructive.

I think the entire problem is that in the realm of software
engineering, you define a set of "requirements", and then
you can tell whether you have done a good job of implementing
the software by seeing if it meets these requirements.

It is, I guess, quite understandable, if quite wrong, for people
to make the mistake of thinking that a language definition is
like such a requirements document, and that you will be able
to tell if a vendor has done a good job of writing an Ada
compiler by seeing if it has met these requirements.

That's quite wrong of course, the defining language document
is not a set of requirements in this sense at all. Well more
properly it is a small part of the requirements. Left out
entirely are issues of performance, reliability, efficiency,
usability, maintainability etc etc.

By the way Ken, you questioned me saying that you were one of
the people arguing for inclusion of these ill-defined
requirements in the language. I may remember wrong, but I
distinctly remember you arguing for this approach in the
meetings we had on safety-critical requirements, and your
previous post seems to confirm that memory (the one where
you say that it was a waste of time attending that meeting).

I sure hope that SOME people reading this thread come away with
a little bit better understanding of what language definitions
are all about (and also an understanding of why validation
cannot guarantee usability or quality).



Sent via Deja.com http://www.deja.com/
Before you buy.

Re: Required Metrics

[Robert Dewar]

In article <8eutl8$7gh$1@nnrp1.deja.com>,
  Ted Dennison <dennison@telepath.com> wrote:
> In article <8eulom$u8m$1@nnrp1.deja.com>,
>   Robert Dewar <robert_dewar@my-deja.com> wrote:
>
> > It would have been SO much better to make all these
> > documentation "requirements" into implementation advice
> > for several reasons:
> >
> >  3. Implementors have to document which IA they follow and
> >     which they do not, leading to a useful account of how
> >     each IA section is addressed (see the section in the
> >     GNAT RM that does this for example).
>
> But that is a documentation requirement too. If documentation
> requirements became implementation advice, implementors would
no loger
> *have* to document which IA they follow, right? Or is that
> your point?

No, you are missing my point. The IA is stuff that we interpret
informally (in Ken's terms, from a "software engineering" point
of view).

If someone fails to document which IA they follow in a
convenient manner, then they are not formally non-conformant,
but in practice this will not be an issue. It would surprise
me if any vendor does not provide this information, and if
they don't you should definitely complain.

Right now, there is no suggestion AT ALL in the RM, formal or
informal, that vendors should describe how they meet the
documentation "requirements".

Why not? Because they are treated like any other requirements.
I don't have to document that I meet the requirement of
3.2.1:

5   A given type shall not have a subcomponent whose type is the
    given type itself.

or document *how* I meet it, since it is a well defined semantic
requirement (by the way, I suddenly realize that you Ken
converts my general use of semantic into the headings in the
manual, don't make that mistake, semantics includes a lot more
:-)

Similarly the "requirements" associated with pragma Reviewable
do not have any associated documentation requirements on how
they are met, since they are treated by the standard as
equivalent to the 3.2.1(5) statement above, even though they
are not.

That's too bad. When we did the documentation for GNAT, we went
all through annex M making sure we provided all this
information, but we do not include documentation on how we
provide the pragma Reviewable information, because it's not
in Annex M.

Now I definitely agree that this is useful information to
provide, and I made a note to add this to our docs for some
future release, but if the documentation requirements had
been in IA sections, you would have got this automatically.

In practice, making the documentation requirements IA would
INCREASE their utility, not decrease it.







Sent via Deja.com http://www.deja.com/
Before you buy.

Re: Required Metrics

[Robert I. Eachus]

Robert Dewar wrote:
  
> Well, "shall" is an impressive word, it sure *sounds* like
> a requirement. Unfortunately, what on earth does it mean to
> "provide the following information". Ooops, totally undefined.
> And since the information for ALL sections is equivalent to
> information provided in the object file, you can argue from
> a formal point of view that the object file satisfies the
> requirement.

   Let me see if I can spread a little oil on the water here.  Ken sees
the shalls, and is used to saying "Aha, a requirement.  I must find all
such requirements--explicit, implicit, and derived--during requirements
analysis, allocate them, and plan to test them."  I've spent time in
this environment as well, and so had a lot of those developing and
approving the Ada standard.  It somehow seems wrong to use shall in a
non-testable 'requirement.'  The requirement should either be made
testable, or not treated as a requirement.

   On the other hand, everyone involved in the process was aware of what
had happened in Ada 83 with most of Chapter 13.  Some vendors ignored
the parts that weren't tested, and other vendors since the requirements
there were not tested put implementation of those features at low
priority.  (If a customer came alone with a need and sufficient cash,
they move up the list.  But the users were not particularly happy with
that either.)

   So why were there no meaningful Chapter 13 tests?  Not because there
were not real requirements there, but because the real requirements were
not testable, and the parts that were formally testable were such that
tests were counterproductive.  For example, how do you design a portable
test for the bit patterns used to represent enumeration types?  It is
much more difficult than it seems, since S'Pos returns the position
number, not the representation.  Of course, a new attribute could be
required, that would make the enumeration representation clause
testable, but that is adding a significant burden.  (Note that RM 95
says in 13.4(11) that "Unchecked_Conversion may be used to query the
internal codes used for an enumeration type."  But this is a note, and
intentionally does not say how to do it or to place any requirements on
the results of such conversions,  so this is not even a derived
requirement.  You can, from real requirements derive some tests that are
useful, but not necessarily as useful as you would expect due to
byte-sex and bit ordering issues. see 13.5.3.)

    For Ada 95, there was a strong desire to make Chapter 13 more
useful.  But since the nature of Chapter 13 is to be very implementation
dependent, and difficult to test, users, especially embedded system
users, wanted a stronger "moral" commitment to features that they felt
were necessary, even if they were untestable.  Such moral commitments
became Implementation Advice and Documentation Requirements.  But note
that in most Documentation Requirements which require a measurement, the
reason for adding the requirement was not to ensure that these
measurements appear in some manual that may bear little or no relation
to the intended hardware configuration.
The idea was that for those particular quantities, it was important that
it be possible to determine what they are.  A computer program that
performed the measurements is more than adequate documentation, as long
as the underlying moral commitment is met.  Since for most of the
numeric documentation requirements, those programs exist, the
requirement is trivially met IF THE QUANTITY IS MEASURABLE.  So in most
(all?) cases, the existance of an ACES report meets the documentation
requirements as long as the compiler does not engage in bizarre
behavior.  (For example, if redezvous take an extra three milliseconds
on alternate Tuesdays, that needs to be in the documentation.)

    One last word on testable requirements.  Is a requirement that an
error will be raised if the sun rises in the West testable?  Not
really.  The requirement is clear, but it is not possible to create the
environment necessary to perform the test.  What about a requirement to
document the number of planets in the solar system?  Somewhat testable,
but what do you do if the documentation says 10?  Is the documentation
wrong, or is there a planet you don't know about?

    This is why many of the documentation requirements are untestable. 
For example, the intent of D.8(10) is clear:

    "An upper bound on the execution time, in processor clock cycles, of
a delay_relative_statement whose requested value of the delay expression
is less than or equal to zero."

    If the documentation says the upper limit is one million clocks is
it useful?  Probably not.  Testable?  Not likely.  It is possible that a
test program would require more than a million clock cycles for some
input, and by luck your testing finds that case.  But more more likely
is that you have a meaningless test that can only fail to falsify the
documentation.  If the documentation specified 50 clocks, the test is
much more interesting, but from a formal point of view nothing has
changed.

> Now of course from a pragmatic point of view, we want the
> information in a much more usable form, and GNAT provides
> a lot of *non-required* capabilities in this area. For
> example...

     And this is where Robert Dewar the implementor, can and will give
much more acceptable answers than Robert Dewar the langauge lawyer. 
Real users don't want useless volumes of meaningless measurements, they
want answers to the real questions that lay behind the Documentation
Requirements.  There shouldn't be--and there is not--a requirement that
the documentation must be in 10 point Sans-Serif font on 8 1/2" by 11"
paper with one inch margins.  The real requirement is that the
information should be available to the user when needed and in a form
that applies to the target system.  So the real checkoff for any of the
documentation requirements is not, where is the piece of paper?  It is
the ability of the user to obtain this information when needed.  If ACT
chooses to provide this for their validated compilers in the form of
good customer support, great.  If some other vendor chooses to provide a
program which when compiled and run prints out the documentation, that's
good too.  And maybe some vendor chooses to provide a set of formulas
which can be used to get the numbers.
That is also acceptable, even if it is not very convenient.

    So there is no conspiracy here by compiler vendors or language
lawyers to dilute or subvert the meaning of the Reference Manual.  The
ARG has enough work to do dealing with other issues.  If the issue can
be resolved by writing a page of code, then believe me, that is much
easier than creating an AI.  The compromise, if it can be called that,
is found in RM 1.1.3(19):

    "The implementation may choose to document implementation-defined
behavior either by documenting what happens in general, or by providing
some mechanism for the user to determine what happens in a particular
case."

      ANY documentation requirement can theoretically be satisfied by
providing a program which does the measurement.  Of course, not all
documentation requirements can actually be satisfied that way.  It is
left up to the implementor to choose which Annex M issues have to be
dealt with in the documentation and which are better satisfied using a
tool or program.   It is possible that for any particular implementation
requirement some compilers will not be able to provide meaningful
documentation, while some others cannot provide a test program.  And
that is another reason why this is a can of worms from a testing
standpoint.  The determination of which Annex M issues can be tested is
something that cannot be done without looking at a particular compiler
in depth, and the answers will only be appropriate to that compiler.

     Again, from a user's point of view, all this nit-picking is
irrelevant.  A user who needs, say, a compiler which never takes more
than 1 millisecond for a clock interrupt is not going to be satisfied
with some random document.  He or she is going to write those
requirements in the purchasing contract.  Any such requirements in the
contract will not specify that this or that feature be documented, but
the actual required performance.

Re: Required Metrics

[Ken Garlington]

"Robert I. Eachus" <rieachus@earthlink.net> wrote in message
news:3914F1DC.A5EE1751@earthlink.net...
>    Let me see if I can spread a little oil on the water here.  Ken sees
> the shalls, and is used to saying "Aha, a requirement.  I must find all
> such requirements--explicit, implicit, and derived--during requirements
> analysis, allocate them, and plan to test them."

Actually, it has nothing to do with the word "shall." It has to do with the
word "requirement". It has nothing to do with allocation or testing. It has
to do with whether requirements for which vendors believe any conceivable
implementation will pass -- that is, it does not constrain their actions in
any way whatsoever -- are requirements. If a vendor says, "I implemented
this requirement a certain way" -- that is, the requirement was acknowledged
in some form in the implementation -- it's a requirement. You can argue
about whether it was the best implementation, or whether it's testable, etc.
but that's a different argument.

When a vendor says, "This requirement does not constrain me in any way,"
that's trouble.

> But since the nature of Chapter 13 is to be very implementation
> dependent, and difficult to test, users, especially embedded system
> users, wanted a stronger "moral" commitment to features that they felt
> were necessary, even if they were untestable.  Such moral commitments
> became Implementation Advice and Documentation Requirements.

"Committment" implies that vendors take some specific action in response to
the requirement. As we've seen, that isn't necessary to sign the DOC with a
clear conscience. So, do you still believe this "moral" committment is being
met?

>      Again, from a user's point of view, all this nit-picking is
> irrelevant.  A user who needs, say, a compiler which never takes more
> than 1 millisecond for a clock interrupt is not going to be satisfied
> with some random document.  He or she is going to write those
> requirements in the purchasing contract.  Any such requirements in the
> contract will not specify that this or that feature be documented, but
> the actual required performance.

Well, I would hope less often that they used to -- I know my group has sworn
off these custom contracts wherever possible. After having spent well into
seven digits on compilers in the last 15 years, I don't think we'll be back
anytime soon.

Re: Required Metrics

[Ken Garlington]

"Robert Dewar" <robert_dewar@my-deja.com> wrote in message
news:8f279n$me2$1@nnrp1.deja.com...

> I think the entire problem is that in the realm of software
> engineering, you define a set of "requirements", and then
> you can tell whether you have done a good job of implementing
> the software by seeing if it meets these requirements.
>
> It is, I guess, quite understandable, if quite wrong, for people
> to make the mistake of thinking that a language definition is
> like such a requirements document, and that you will be able
> to tell if a vendor has done a good job of writing an Ada
> compiler by seeing if it has met these requirements.
>
> That's quite wrong of course, the defining language document
> is not a set of requirements in this sense at all. Well more
> properly it is a small part of the requirements. Left out
> entirely are issues of performance, reliability, efficiency,
> usability, maintainability etc etc.
>
> By the way Ken, you questioned me saying that you were one of
> the people arguing for inclusion of these ill-defined
> requirements in the language. I may remember wrong, but I
> distinctly remember you arguing for this approach in the
> meetings we had on safety-critical requirements, and your
> previous post seems to confirm that memory (the one where
> you say that it was a waste of time attending that meeting).

What I said (with emphasis pointing out the important part):

"Could you cite ONE case *** in this thread *** where I argued in favor of
including ANY requirement, documentation or otherwise, in the standard?"

Certainly, I have argued for requirements *** in other conversations, in
other places, at other times ***. In those discussions, I've expected
vendors to say "Wait a minute -- I don't think I understand what that means"
or "I don't know of any reasonable way to do what you're asking". Now I know
why they didn't object to them (at least in the meetings I attended) -- it
was preferable to just go along to make the users happy, and then not
consider themselves constrained by these "requirements" when implementing
the standard.

However, that's not related AT ALL to the original question I've raised,
which I think you've answered completely. Thank you for your time.

Re: Required Metrics

[Ken Garlington]

"Robert Dewar" <robert_dewar@my-deja.com> wrote in message
news:8f26s6$lrj$1@nnrp1.deja.com...
> > Do you think it's fair to accuse me of such an accusation,
> > when the quoted statement doesn't actually say that?
>
> Yes, I think it's fair, reread what you said.
>
> > Like I said, "with a pen in the hand, and a song in the
> > heart."
>
> Nope! No "song in the heart", which I take to mean with an
> intent other than a formal declaration. Perhaps you have
> some other meaning, if so, it is obscure to me.

I do have some other meaning, and apparently it is obscure to you (even
though you quote its meaning in the following):

> > Of course,
> > your signature is based on your determination as to which
> requirements are
> > "real" requirements, believing that all other requirements are
> "met" no
> > matter what.
>
> All requirements are requirements. I am not saying that the
> documentation requirements are not requirements, just that
> they are trivially met.

or vacuously met, to quote previous posts? as in: "all other requirements
are 'met' no matter what"?

Perhaps you have some meaning for vacuous or trivial that is obscure to me!

> > You may believe that the ACVC review board, and all other
> > vendors, agree with you.
>
> You mean I agree that there can be no formal tests for whether
> you meet the documentation rquirements. I cannot imagine anyone
> with any reasonable knowledge thinking otherwise.

No, I mean that you may believe that the ACVC review board agrees with you,
based on your statement in a previous post:

"By the way, the ACVC review board discussed this briefly. They immediately
agreed that these requirements were too ill-defined to even consider
including them in conformance assessment."

Isn't it at least possible that I simply mean what I say?

> There is no list of non-requirements, because there are no
> non-requirements. THe DOC requires that you have not made
> any intentional deviations from the FULL SET of requirements,
> including the documentation requirements.
>
> > You might want to re-read your original statement: "How can
> the DOC be
> > signed if you have ignored a requirement?" Note that the word
> > "intentionally" does not appear.
>
> intentionally is of course critical. Otherwise the DOC would
> be a declaration that your compiler was bug free. This is not
> intended, and would be impractical.

That's right. You can sign the DOC "with a pen in the hand and a song in the
heart", and still make an unintentional mistake. Perhaps, even, a certain
kind of unintentional mistake relevant to the topic at hand. And yet, the
meaning of my statement above still eludes you. Unfortunate, but there's no
point in banging my head against that wall any further.

> > > Nope, the issues with pragma Reviewable are also all
> >  > documentation based, let's look:
> >
> > Yes, let's. Specifically, let's look at the words which appear
> just before
> > paragraph 5:
> >
> > ***** Implementation Requirements *****
> >
> > "5 The implementation shall provide the following information
> ..."
> >
> > Now, I suspect you're using the term "documentation"
> informally, not in the
> > sense of ARM 1.1.2. However, since all Implementation
> Requirements, like all
> > Documentation Requirements, are not really requirements,
> perhaps we _should_
> > use the same name for both.
>
> Of course implementation requirements are requirements, what
> gave you the lunatic idea that this is not the case. I say
> lunatic here, because it simply makes no sense. There are many
> critical implentation requirements. Where ON EARTH did you
> get this idea, certainly not from anything I wrote.

Except for the bit you quoted from your earlier post:

"Nope, the issues with pragma Reviewable are also all documentation based,
let's look:"

So, which is it?

(a) the ***implementation*** requirements for pragram Reviewable should be
treated like documentation requirements (vacuously met), or
(b) they shouldn't (i.e. it is possible to fail to meet them).

If (a), then how does someone determine which other ***implementation***
requirements are vacuously met, and which are critical?

> This is formally definable, and of course tested by the ACVC
> tests. Again, I ask where did you get the idea that
> implementation requirements are not requirements???????

From you. You should really read the implementation requirements for pragma
Reviewable, they're quite extensive. None are tested by the ACVC, of
course...

>
> > When we were discussing these safety and security requirements
> during that
> > industry-implementors meeting lo these many years ago, I was
> frankly
> > surprised that the vendor representatives (including yourself)
> didn't put up
> > more of a fuss. Now I see why. What a total waste of our time.
>
> Actually many useful things came out of those meetings, but if
> you thought that the meetings were about requiring vendors to
> provide useful usable implementations, you were indeed deluding
> yourself, since language standards can never guarantee this.

Actually, I thought we were working on a useful usable standard. That is, an
agreement from the vendors as to the text of the implementation requirements
included the concept that the vendors understood what they meant!
Apparently, I was mistaken.

> > !topic Non-Requirement Requirements
> > !reference RM95-1.1.2(24-39)
> > !from Ken Garlington 00-05-05
> > !keywords requirements, advise
> > !discussion
> >
> > The referenced paragraphs define several different categories
> of the
> > standard. Some of these categories use the term
> "requirements," although
> > they are not considered requirements by the Ada vendor
> community. There
> > should be an explicit list of which categories can be
> considered
> > requirements (i.e., it is possible for a vendor to "fail" to
> meet it). All
> > other categories should be described in the terms currently
> used for
> > "Implementation Advice."
>
>
> This is plain silly, I can't imagine the ARG giving it much
> attention, since it amounts to nothing more than an unfounded
> polemic. There is no part of the Ada community that considers
> any requirements in the RM not to be requirements, and
> certainly the vendor community does not ignore any requirements.

Note my definition of a requirement: "i.e., it is possible for a vendor to
'fail' to meet it." So, which is it?

(a) There are areas of the standard that have the word "requirement"
attached to them which are vacuously met by all vendors.
(b) There aren't.

You may consider it "plain silly" to take items in (a) and label them
implementation advice, and I'm sure the ARG will give it no attention (I
mean, how much weight should my opinion hold?). At least, when a user comes
to me and says, "if this is crap, why is in the standard?" I can say, "beats
me, I tried to get it fixed, but no one cared."

> Again, I think the entire problem comes from the fact that you
> don't really understand the purpose of a formal language
> definition, and that you read the word "requirement" in
> (to use your terms) "software engineering" sense, which is
> totally misleading in this context.

Absolutely! An equally elegant solution to the problem to this would be to
put in the front of the standard:

"WARNING: Terms used in this standard, such as 'requirement,' do not have
their usual English meaning."

> (by the way, I suddenly realize that you Ken
> converts my general use of semantic into the headings in the
> manual, don't make that mistake, semantics includes a lot more
> :-)

Excellent! Light begins to dawn. Now, the slippery part: How much more?

Re: Required Metrics

[Robert Dewar]

In article <pr7R4.8717$wb7.694952@news.flash.net>,
  "Ken Garlington" <Ken.Garlington@computer.org> wrote:

> "Could you cite ONE case *** in this thread *** where I argued
in favor of
> including ANY requirement, documentation or otherwise, in the
standard?"

No, and no one said you had. What I said, cued by YOUR post was
that you were one of the people who had suggested such
requirements during the design process.


> Certainly, I have argued for requirements *** in other
> conversations, in
> other places, at other times ***.

And that is what I was referring to.

> In those discussions, I've expected
> vendors to say "Wait a minute -- I don't think I understand
> what that means"

Why? Vendors were not present at these meetings, and this
is not a vendor issue, it is a language design issue. The
meeting I am thinking of was a listening session to understand
what people wanted in the Ada 95 standard. When you are
listening to what people want, you do not quarrel with them,
you listen! That was the ground rule (a good one) set by
Chris Anderson.

> or "I don't know of any reasonable way to do what you're
> asking". Now I know
> why they didn't object to them

No you don't, your are guessing (wrong!). The interesting thing
is that this battle for including documentation requirements was
fought much later on, and as you know people (like me and
Bob Duff) DID protest that it would be more useful to have
these as implementation advice. We lost the argument.

> It was preferable to just go along to make the users happy,
> and then not consider themselves constrained by these
> "requirements" when implementing the standard.

Ken, the only thing I can figure, since you keep repeating the
incorrect statement above is that you just don't understand
what language definitions and semantic requirements are about.
Of course one is constrained by requirements when implementing.
But the trouble is that if a requirement has no formal content,
then in the context of a language standard, it can be met in
almost anyway. The RM essentially says

You must provide kerbloggle, and then does not define what
kerbloggle means. This means that everyone makes up their
own mind what kerbloggle means. The standard is no longer
very useful, one has instead to have informal market oriented
acceptance discussions of whether YOUR idea of kerbloggle
is close enough to the implementors view of kerbloggle.

Remember, the standard is NOT written in english. It is written
in formal english (this was a phrase that Jean Ichbian coined
to try to explain this crucial difference). The fact that Ken
Garlington has some informal idea of what documentation means
has nothing whatsoever to do with this word as used in the
standard. The standard has to define all terms that it uses,
just as a mathematician has to in writing a proof.

Could one come up with a definition of documentation that was
not over-restrictive? I very much doubt it.

> However, that's not related AT ALL to the original question
> I've raised, which I think you've answered completely. Thank
> you for your time.

Sorry, at this stage, I really don't know what question you
are raising :-)


Sent via Deja.com http://www.deja.com/
Before you buy.

Re: Required Metrics

[Robert Dewar]

In article <Gt7R4.8719$wb7.695096@news.flash.net>,
  "Ken Garlington" <Ken.Garlington@computer.org> wrote:

> > Nope! No "song in the heart", which I take to mean with an
> > intent other than a formal declaration. Perhaps you have
> > some other meaning, if so, it is obscure to me.

> I do have some other meaning, and apparently it is obscure to
> you (even though you quote its meaning in the following):

OK, you still don't say what that "other meaning" is.

> or vacuously met, to quote previous posts? as in: "all other
> requirements are 'met' no matter what"?
>
> Perhaps you have some meaning for vacuous or trivial that is
> obscure to me!

Right, as I said earlier, I think the problem you have in
understanding just comes from unfamiliarity with the notion
of formal language definition, and the history and function
of language standards documents. You are expecting them to
do something they can NOT by nature do, a common
misunderstanding indeed. As I say, closely related to the
peculiar notion, so very common in Ada 83 days, that validation
guaranteed quality and usability.

> No, I mean that you may believe that the ACVC review board
> agrees with you, based on your statement in a previous post:
>
> "By the way, the ACVC review board discussed this briefly.
They immediately
> agreed that these requirements were too ill-defined to even
consider
> including them in conformance assessment."

Right, this was a completely uncontroversial position. I cannot
imagine anyone who understood the notion of language
definitions, conformance and validation testing disagreeing.

What would you have in mind? That we have a section of the ACVC
testing that includes Ken Garlington sitting down, reading the
documentation and announcing that it meets his (nowhere defined)
criteria for acceptable documentation? Ken is free (more than
free, encouraged) to do this as part of his acceptance testing,
but only for his own purposes.

Actually Ken, you said something VERY revealing earlier on. You
talked about tightening up the standard to make compiler vendors
do stuff that they had not done in Ada 83 compilers in this
area so that the costs could be shared.

Let's examine this thought a bit further.

First. It's really an inappropriate distortion of the market
to attempt to include requirements that YOU want, but the
market place does not in general want, in an attempt to get
other people to pay for things that you need that they don't
want (if they wanted them generally, they would be there,
competition works that way).

Second, Even ignoring the dubious justification for such an
attempt, it is bound to fail. Why? Because language standards
are not about providing good documentation, useful information,
or even about producing usable compilers. They are nothing more
than a set of mathematical formulae, informally expressed in
this case, that dictate certain input-output properties that
a compiler must have. The language standard by its nature can
only ever be a small part of the requirements.

You might hope that "beefing up" the standard would help solve
YOUR problem and relieve you of YOUR responsibility for devising
the set of requirements for your tools and making sure you
acquire tools that meet these requirements, but this attempt is
bound to fail. Language standards are NOT software engineering
requirements documents, and no amount of yelling or complaining
can change this fundamental fact.




>
> Isn't it at least possible that I simply mean what I say?
>
> > There is no list of non-requirements, because there are no
> > non-requirements. THe DOC requires that you have not made
> > any intentional deviations from the FULL SET of
requirements,
> > including the documentation requirements.
> >
> > > You might want to re-read your original statement: "How
can
> > the DOC be
> > > signed if you have ignored a requirement?" Note that the
word
> > > "intentionally" does not appear.
> >
> > intentionally is of course critical. Otherwise the DOC would
> > be a declaration that your compiler was bug free. This is
not
> > intended, and would be impractical.
>
> That's right. You can sign the DOC "with a pen in the hand and
a song in the
> heart", and still make an unintentional mistake. Perhaps,
even, a certain
> kind of unintentional mistake relevant to the topic at hand.
And yet, the
> meaning of my statement above still eludes you. Unfortunate,
but there's no
> point in banging my head against that wall any further.
>
> > > > Nope, the issues with pragma Reviewable are also all
> > >  > documentation based, let's look:
> > >
> > > Yes, let's. Specifically, let's look at the words which
appear
> > just before
> > > paragraph 5:
> > >
> > > ***** Implementation Requirements *****
> > >
> > > "5 The implementation shall provide the following
information
> > ..."
> > >
> > > Now, I suspect you're using the term "documentation"
> > informally, not in the
> > > sense of ARM 1.1.2. However, since all Implementation
> > Requirements, like all
> > > Documentation Requirements, are not really requirements,
> > perhaps we _should_
> > > use the same name for both.
> >
> > Of course implementation requirements are requirements, what
> > gave you the lunatic idea that this is not the case. I say
> > lunatic here, because it simply makes no sense. There are
many
> > critical implentation requirements. Where ON EARTH did you
> > get this idea, certainly not from anything I wrote.
>
> Except for the bit you quoted from your earlier post:
>
> "Nope, the issues with pragma Reviewable are also all
documentation based,
> let's look:"
>
> So, which is it?
>
> (a) the ***implementation*** requirements for pragram
Reviewable should be
> treated like documentation requirements (vacuously met), or
> (b) they shouldn't (i.e. it is possible to fail to meet them).
>
> If (a), then how does someone determine which other
***implementation***
> requirements are vacuously met, and which are critical?
>
> > This is formally definable, and of course tested by the ACVC
> > tests. Again, I ask where did you get the idea that
> > implementation requirements are not requirements???????
>
> From you. You should really read the implementation
requirements for pragma
> Reviewable, they're quite extensive. None are tested by the
ACVC, of
> course...
>
> >
> > > When we were discussing these safety and security
requirements
> > during that
> > > industry-implementors meeting lo these many years ago, I
was
> > frankly
> > > surprised that the vendor representatives (including
yourself)
> > didn't put up
> > > more of a fuss. Now I see why. What a total waste of our
time.
> >
> > Actually many useful things came out of those meetings, but
if
> > you thought that the meetings were about requiring vendors
to
> > provide useful usable implementations, you were indeed
deluding
> > yourself, since language standards can never guarantee this.
>
> Actually, I thought we were working on a useful usable
standard. That is, an
> agreement from the vendors as to the text of the
implementation requirements
> included the concept that the vendors understood what they
meant!
> Apparently, I was mistaken.
>
> > > !topic Non-Requirement Requirements
> > > !reference RM95-1.1.2(24-39)
> > > !from Ken Garlington 00-05-05
> > > !keywords requirements, advise
> > > !discussion
> > >
> > > The referenced paragraphs define several different
categories
> > of the
> > > standard. Some of these categories use the term
> > "requirements," although
> > > they are not considered requirements by the Ada vendor
> > community. There
> > > should be an explicit list of which categories can be
> > considered
> > > requirements (i.e., it is possible for a vendor to "fail"
to
> > meet it). All
> > > other categories should be described in the terms
currently
> > used for
> > > "Implementation Advice."
> >
> >
> > This is plain silly, I can't imagine the ARG giving it much
> > attention, since it amounts to nothing more than an
unfounded
> > polemic. There is no part of the Ada community that
considers
> > any requirements in the RM not to be requirements, and
> > certainly the vendor community does not ignore any
requirements.
>
> Note my definition of a requirement: "i.e., it is possible for
a vendor to
> 'fail' to meet it." So, which is it?
>
> (a) There are areas of the standard that have the word
"requirement"
> attached to them which are vacuously met by all vendors.
> (b) There aren't.
>
> You may consider it "plain silly" to take items in (a) and
label them
> implementation advice, and I'm sure the ARG will give it no
attention (I
> mean, how much weight should my opinion hold?). At least, when
a user comes
> to me and says, "if this is crap, why is in the standard?" I
can say, "beats
> me, I tried to get it fixed, but no one cared."
>
> > Again, I think the entire problem comes from the fact that
you
> > don't really understand the purpose of a formal language
> > definition, and that you read the word "requirement" in
> > (to use your terms) "software engineering" sense, which is
> > totally misleading in this context.
>
> Absolutely! An equally elegant solution to the problem to this
would be to
> put in the front of the standard:
>
> "WARNING: Terms used in this standard, such as 'requirement,'
do not have
> their usual English meaning."
>
> > (by the way, I suddenly realize that you Ken
> > converts my general use of semantic into the headings in the
> > manual, don't make that mistake, semantics includes a lot
more
> > :-)
>
> Excellent! Light begins to dawn. Now, the slippery part: How
much more?
>
>


Sent via Deja.com http://www.deja.com/
Before you buy.

Re: Required Metrics

[Robert Dewar]

In article <3914F1DC.A5EE1751@earthlink.net>,
  "Robert I. Eachus" <rieachus@earthlink.net> wrote:

>    On the other hand, everyone involved in the process was
aware of what
> had happened in Ada 83 with most of Chapter 13.  Some vendors
ignored
> the parts that weren't tested, and other vendors since the
requirements
> there were not tested put implementation of those features at
low
> priority.  (If a customer came alone with a need and
sufficient cash,
> they move up the list.  But the users were not particularly
happy with
> that either.)

As one very much involved in this issue (I campaigned for
the chapter 13 tests to be included in the ACVC suite, and
consulted extensively with the AVO making that happen - quite
a bit of the code in those tests is mine :-) let me comment.

It's not so simple. These were not clear requirements that
were being ignored. The precise trouble was the infamous
"simply" clause. Here it is:

  An implementation may limit its acceptance of representation
  clauses to those that can be handled simply by the underlying
  hardware.

And the word "simply" is nowhere defined. Well this is the
proverbial hole big enough to drive a truck through.

Robert Dewar (and others) argued that simply must be interpreted
in solely technical terms, looking at sequences of machine
instructions. Even this leaves some subjective margin (is 3
instructions simple? 10? 20?).

But others considered that simple means simple to implement
and used this as an escape hatch to pick and choose what to
implement.

Not till the ARG ruled clearly on this issue, and the AVO
backed it up with the tests (which as I mention above, I
helped write), in ACVC version 1.10, did a proper set of
tests get written.

Now in Ada 95, we did two things

  1. Required (in the systems programming annex) that rep
  clauses be implemented.

  2. Specified an exact set of rep clauses that must be
  implemented.

Was this an improvement?

Yes, and No. Yes, if you agreed that the old Ada 83 requirement
was so ill-defined as to say nothing, since now we clearly DO
have a set of required rep clauses. Of course we achieved this
in Ada 83 too by ARG rulings and the introduction of ACVC 1.10.

No, if you take Robert Dewar's fierce interpretation of the
original Ada 83 rule. Right now, there are lots of rep clauses
that can be implemented simply in the hardware, which are in
my opinion required by the Ada 83 standard, but which FOR SURE
are not required by the Ada 95 standard.

Overall are we ahead? Well in practice I find the subset of
rep clauses required by the Ada 95 RM pathetically inadequate.
In particular, an Ada 95 compiler that implements ONLY this
minimal subset will have a really painful time with legacy Ada
83 compilers, which in general implemented a FAR larger set of
representation clauses.

Certainly in GNAT we go way beyond the Ada 95 requirements,
and we attempt to completely match the capabilities of the
Ada 83 compilers and in some cases, we go significantly beyond
those requirements as well.

Why? Because the standard is not the driving force when it
comes to rep clauses, yes, of course we meet the limited
requirements of the RM, but the driving force here is customers
and Ada 83 legacy code.



> For example, how do you design a portable
> test for the bit patterns used to represent enumeration types?

<<discussion of why this is hard snipped>>

Yes, it's tricky, but can be done. This is one particular
problem that I provided the solution for, and my solution
(involving carefully chosen use of unchecked conversion is
in the ACVC suite since 1.10). I think you can find my name
on that test still :-)

> This is why many of the documentation requirements are
> untestable.  For example, the intent of D.8(10) is clear:

> If the documentation says the upper limit is one million
> clocks is it useful?  Probably not.

Ah thanks Robert, I had forgotten that very useful point. Yes
indeed, another completely accurate way of meeting the metrics
requirements is to provide silly bogus figures like this. In a
software engineering document, you can avoid this problem, most
likely by specifying actual absolute performance criterion for
the particular hardware combination you are using, but an
attempt to plug this hole in the RM would lead to even more
murky situations.

> The compromise, if it can be called that,
> is found in RM 1.1.3(19):
>
>   "The implementation may choose to document
>    implementation-defined behavior either by documenting what
>    happens in general, or by providin some mechanism for the
>    user to determine what happens in a particular case."

Interesting -- I had COMPLETELY forgotten this
statement in the standard. It is of course an exact codification
of some of the things that Bob Duff and I have argued. In
particular, it provides an absolutely CLEAR basis for Bob Duff's
observation that the behavior of the delivered binary compiler
code, and the behavior of code generated by this compiler
constitute documentation in the sense of the RM.

So the RM *did* try to define what documentation meant, and
came up with something about as far from what Ken has in mind
as one can imagine.

All the more reason to accept my point that we would have been
far ahead putting documentation requirements in IA sections,
where the word documentation can appear without being defined
in a more useful subjective form.



Sent via Deja.com http://www.deja.com/
Before you buy.

Re: Required Metrics

[Robert Dewar]

In article <Bj7R4.8714$wb7.694404@news.flash.net>,
  "Ken Garlington" <Ken.Garlington@computer.org> wrote:

> When a vendor says, "This requirement does not constrain me in
> any way", that's trouble.

A vendor can only say this if it is correct, and if it is
correct that the requirement is not constraining, then it is
not trouble as far as the requirement goes.

> "Committment" implies that vendors take some specific action
> in response to the requirement.

Right! Absolutely! YOu have to make sure you meet all the
requirements. Including documentation requirements in the
sense of the RM.

> As we've seen, that isn't necessary to sign the DOC with a
> clear conscience.

We have seen nothing of the kind. When I sign the DOC I am
very careful to ensure that the statement signed is formally
true with respect to ALL requirements, including documentation
requirements. Indeed I have often held up signing a DOC because
there is some small issue on which I am not satisfied that we
do not have a deviation from the standard.

> So, do you still believe this "moral" committment is being
> met?

Absolutely it is being met. The requirements are being met. THe
trouble is that, unjustified by the RM, Ken Garlington has some
idea in his mind that meeting them involves something other
than meeting the requirements of the RM. Fine, but not relevant
to the validation process.


> Well, I would hope less often that they used to -- I know my
> group has sworn off these custom contracts wherever possible.
> After having spent well into seven digits on compilers in the
> last 15 years, I don't think we'll be back
> anytime soon.

If you have spent millions of dollars on Ada compilers, and not
got what you wanted, then I would say the blame must fall on
you as well as your suppliers.

Again, this statement shows the fantasy you have that the
RM can ensure that off-the-shelf compilers meet your needs.
Now of course it might be the case that OTS compilers can
meet your needs, but if this is true, it is a reflection of
many other factors than the contents of the standard.

Perhaps the real trouble is the DoD notion of custom contracts.
It tends to be far too formal, and thus far too expensive.
What's needed in general is a more flexible, much less expensive
much less formal, but far more effective procedure, where, as
in buying a car or almost anything else, there is informal
discussion between the vendor and consumer.

In the GNAT context, there are very few users for whom we have
not provided enhancements to the compiler from time to time
in response to their needs. You have all seen the big list of
new features and capabilities that come with each new version,
and most of these entries come from informal discussions and
requests. In some cases, significant features are specially
costed of course.

But again, if you have been spending millions of dollars on
Ada compilers and have not got what you want, something is
quite wrong on BOTH sides.


Sent via Deja.com http://www.deja.com/
Before you buy.

Re: Required Metrics

[Ken Garlington]

"Robert Dewar" <robert_dewar@my-deja.com> wrote in message
news:8f3s87$bsc$1@nnrp1.deja.com...
> In article <Gt7R4.8719$wb7.695096@news.flash.net>,
>   "Ken Garlington" <Ken.Garlington@computer.org> wrote:
> > > (by the way, I suddenly realize that you Ken
> > > converts my general use of semantic into the headings in the
> > > manual, don't make that mistake, semantics includes a lot
> more
> > > :-)
> >
> > Excellent! Light begins to dawn. Now, the slippery part: How
> > much more?

This, of course, is a key question, which went unanswered in the last
response. I thought I'd put it up here, so it wouldn't be missed.

> But the trouble is that if a requirement has no formal content,
> then in the context of a language standard, it can be met in
> almost anyway.

The word "almost" is the one that interests me. All we have to do now is
decide if it really belongs in that sentence, and we're home free. Earlier,
you appeared to be saying that certain items in the standard, labelled as
requirements, did not constrain implementor's behavior -- they were
vacuously met. Now, you appear to be saying, through the use of the word
"almost", that it is possible for an implementor to fail to meet a
requirement with "no formal content."

Again, to review, there are three questions I care about:

1. Under what conditions could a vendor fail to meet the metrics
requirements, the documentation requirements, and other requirements of this
type?
2. If there are no such conditions -- they do not constrain the
implementation in any way -- is it appropriate to call them requirements?
3. Is there a consensus as to which "requirements" are always met by
definition?

Note that discussions of what the language standard _should_ contain, or how
it should be tested, or whether users should express their needs in language
revision meetings (?!?) are interesting, but off-topic. For example:

--- OFF-TOPIC CONVERSATIONS FOLLOW ---

> > No, I mean that you may believe that the ACVC review board
> > agrees with you, based on your statement in a previous post:
> >
> > "By the way, the ACVC review board discussed this briefly.
> They immediately
> > agreed that these requirements were too ill-defined to even
> consider
> > including them in conformance assessment."
>
> Right, this was a completely uncontroversial position. I cannot
> imagine anyone who understood the notion of language
> definitions, conformance and validation testing disagreeing.
>
> What would you have in mind? That we have a section of the ACVC
> testing that includes Ken Garlington sitting down, reading the
> documentation and announcing that it meets his (nowhere defined)
> criteria for acceptable documentation? Ken is free (more than
> free, encouraged) to do this as part of his acceptance testing,
> but only for his own purposes.

Nope, I have in mind what I've said all along. If there are requirements in
the standard, then they should in some way constrain the behavior of those
implementing them. How they constrain the behavior is a different subject.
I'm perfectly comfortable with the statement that some requirements are
vague, can't be qualified by testing, etc. Have been from the beginning of
the post. For example, I noticed you failed to respond to my answer to your
question, "what happens if the vendor decides to write his documentation in
Klingon?". That same answer applies here.

I know you feel more comfortable bringing up this ACVC strawman, or this
"user preference" strawman, as you have done it several times. It has no
bearing on my point, which I made in an earlier part of the thread, and will
repeat now:

IF IT'S A REQUIREMENT, IT SHOULD BE TREATED AS SUCH. THAT IS, IT SHOULD
CAUSE THE IMPLEMENTOR TO DO SOMETHING.

IF IT'S NOT A REQUIREMENT, IT SHOULD NOT BE IDENTIFIED AS SUCH.

USERS (AND VENDORS) SHOULD BE CONFIDENT THEY KNOW WHICH PARTS OF THE
STANDARD ARE IDENTIFIED AS REQUIREMENTS, BUT DO NOT CONSTRAIN BEHAVIOR.

Clearly, what you don't understand is that a requirement that is always met
by definition, is not a requirement. It doesn't constrain behavior, so it's
not a requirement. I know you're reading these really fast, and trying very
hard to pull out some underlying meaning having to do with testing, etc.,
but you need to stop. You're just confusing yourself. For a moment, just a
moment, assume there's no underlying agenda. Just read these words, slowly:

REQUIREMENTS SHOULD CONSTRAIN BEHAVIOR. IF THEY DON'T, IT'S WRONG TO CALL
THEM REQUIREMENTS.

If you disagree, let me know. From what you've posted elsewhere, it appears
that you agree. If that's the case, then all you have to do is identify
which "requirements" fall into this category.

> Actually Ken, you said something VERY revealing earlier on. You
> talked about tightening up the standard to make compiler vendors
> do stuff that they had not done in Ada 83 compilers in this
> area so that the costs could be shared.
>
> Let's examine this thought a bit further.
>
> First. It's really an inappropriate distortion of the market
> to attempt to include requirements that YOU want, but the
> market place does not in general want, in an attempt to get
> other people to pay for things that you need that they don't
> want (if they wanted them generally, they would be there,
> competition works that way).

Isn't this why you have a standards process? Users bring ideas to the table;
they're discussed by other users, implementors, and other interested
parties, and if THERE'S A CONSENSUS they get implemented?

I brought a requirement to an SEI meeting with Goodenough and co. during the
language revision process for "static pointers." We were having to do a lot
of non-portable things like Unchecked_Conversion in Ada83 to achieve this.
The 95 standard has new features that address this need. Did I "distort the
market" by doing this? If so, why did you guys ask me for inputs?

Seems to me the argument "competition works that way" could imply that
there's no real reason to have an ISO standard. It seems to be working
reasonably well with Java; I haven't had any problems porting between MS
Visual J++ and Sun Java, for example.

With respect to whether others in the market place want the requirements of
pragma Reviewable; at least for the ones I wanted, Aonix seems to think
there's a market, and I didn't even have to pay them! Too bad we couldn't
get the vendors together and say, "if you're going to implement this sort of
feature, we'd like to have some level of standardization as to what it
means. Is this possible?"

> Second, Even ignoring the dubious justification for such an
> attempt, it is bound to fail. Why? Because language standards
> are not about providing good documentation, useful information,
> or even about producing usable compilers. They are nothing more
> than a set of mathematical formulae, informally expressed in
> this case, that dictate certain input-output properties that
> a compiler must have. The language standard by its nature can
> only ever be a small part of the requirements.

Fine. So which parts of the standard are this "formal English," and which
aren't?

For example, the following implementation requirement seems to dictate
certain input-output properties that a compiler must have:

"The implementation shall provide control- and data-flow information, both
within each compilation unit and across the compilation units of the
partition."

Input -> source code defining compilation units of the partition.
Output -> Control-flow and data-flow information.

I know it's not testable through an ACVC-style test, and that people can
differ on the meaning of the definition, and that this requirement
"distorts" the standard. I just don't see how it answers the question.
Please don't spend too much time explaining all of this to me again. It's a
simple "yes" or "no" question.

> You might hope that "beefing up" the standard would help solve
> YOUR problem and relieve you of YOUR responsibility for devising
> the set of requirements for your tools and making sure you
> acquire tools that meet these requirements, but this attempt is
> bound to fail.

You've made it fully clear that a language standard does not exist to
address user needs.

Re: Required Metrics

[Ken Garlington]

"Robert Dewar" <robert_dewar@my-deja.com> wrote in message
news:8f3u36$dnk$1@nnrp1.deja.com...
> In article <Bj7R4.8714$wb7.694404@news.flash.net>,
>   "Ken Garlington" <Ken.Garlington@computer.org> wrote:
>
> > When a vendor says, "This requirement does not constrain me in
> > any way", that's trouble.
>
> A vendor can only say this if it is correct, and if it is
> correct that the requirement is not constraining, then it is
> not trouble as far as the requirement goes.

Apparently, we're going to have to agree to disagree as to whether it's a
problem if the vendor decides a requirement doesn't constrain his behavior
(and doesn't even have to tell anyone which requirements he decided fell
into this category).

> > "Committment" implies that vendors take some specific action
> > in response to the requirement.
>
> Right! Absolutely! YOu have to make sure you meet all the
> requirements. Including documentation requirements in the
> sense of the RM.

Why does someone have to "make sure" they have to meet a requirement that
they've decided does not constrain them in any way? Do people who don't
drive cars spend much time worrying if they're driving over the speed limit?

> > As we've seen, that isn't necessary to sign the DOC with a
> > clear conscience.
>
> We have seen nothing of the kind. When I sign the DOC I am
> very careful to ensure that the statement signed is formally
> true with respect to ALL requirements, including documentation
> requirements. Indeed I have often held up signing a DOC because
> there is some small issue on which I am not satisfied that we
> do not have a deviation from the standard.

Now we're getting to the question that wen't unanswered before. How do you
do this? Is there a checklist? What criteria do you use to determine if
you've met all of the requirements that aren't part of the "formal
semantics"?

If there are requirements that the vendor decides are vacuously true, is
there any "moral" responsibility to either (a) verify this is the case,
and/or (b) document the decision?

--- AND THEN, THE OFF-TOPIC STUFF ---

> If you have spent millions of dollars on Ada compilers, and not
> got what you wanted, then I would say the blame must fall on
> you as well as your suppliers.

Oh, we always got what we wanted -- just before the vendor went out of
business. We're just tired of having to go through the cycle over, and over,
and over again.

You're quite correct that we have to shoulder our part of the blame. We
thought that Ada would naturally win out in the marketplace, because it had
all of these advantages over other languages, and so we were an early
adopter. Now, we know better. Rather than trying to drive the marketplace,
we adapt our requirements to what's out there, and to what vendors are
willing to implement on their own dime. Custom requirements are a sucker's
game.

Re: Required Metrics

[Robert Dewar]

In article <mUfR4.8835$wb7.709941@news.flash.net>,
  "Ken Garlington" <Ken.Garlington@computer.org> wrote:

> IF IT'S A REQUIREMENT, IT SHOULD BE TREATED AS SUCH. THAT IS,
> IT SHOULD CAUSE THE IMPLEMENTOR TO DO SOMETHING.

Sounds reasonable to me! No one ever argued any differently.
In fact most people would go further and say that if something
is not testable, then it's not a requirement.

> IF IT'S NOT A REQUIREMENT, IT SHOULD NOT BE IDENTIFIED AS
> SUCH.

I think you mean "if it's not a requirement in the above
sense". Obviously requirements are precisely things that are
identified as requirements, so the above sentence stated as
you gave it is tautologically true.

> USERS (AND VENDORS) SHOULD BE CONFIDENT THEY KNOW WHICH PARTS
> OF THE STANDARD ARE IDENTIFIED AS REQUIREMENTS, BUT DO NOT
> CONSTRAIN BEHAVIOR.

Well I'm pretty confident, since it's pretty clear, sorry that
it continues to be unclear for you!

> Clearly, what you don't understand is that a requirement that
> is always met by definition, is not a requirement.

So Ken Garlington pronounces, but can Ken Garlington prove this
statement from the RM. I think not. Again, the "is not" here
is confusing, I think you mean "should not be".

It is quite clear in the RM what is and what is not a
requirement, from the headings. The fact that the definition
of requirement does not meet your definition is pretty much
irrelevant from the point of view of analyzing the existing
RM.

You could get lots of people to agree with a statement that
all requirements in the reference manual should be testable,
at least conceptually.


> REQUIREMENTS SHOULD CONSTRAIN BEHAVIOR. IF THEY DON'T, IT'S
> WRONG TO CALL THEM REQUIREMENTS.

Ah now that of course is a much better form, because now you
are giving Ken Garlington's opinion. Do I agree? Well I don't
understand "behavior" in a mathematical sense, it sounds more
like a software engineering sense. If the above statement is
equivalent to:

REQUIREMENTS SHOULD BE TESTABLE. IF THEY ARE NOT, IT'S WRONG
TO CALL THEM REQUIREMENTS.

Fits better into a language standard, and that of course I
agree with. Remember that Bob Duff and I both strongly objected
to all these ill-defined "requirements" in the reference manual.
Informally I agree with your statement, it is just not stated
in a well defined manner, because it uses terms "constrain" and
"behavior" that cannot be given formal semantic meaning.

Remember we are ALWAYS in the realm of the formal semantic
definition, so pronouncements about what this formal semantic
definition should or should not contain should try to avoid
making the mistake of themselves not being formally stated.

The standard is NOT about constraining implementors, in fact
the standard has nothing to do with implementors at all. It
is simply a definition of what it means to conform to the
ISO standard.

If someone constrains an implementor to conform to the standard,
then the language of the standard will be relevant. But it is
the someone, not the requirements, that are doing the
constraining.

That's an important difference, because for example, the ACVC
test suite IS all about constraining implementors and has quite
a different viewpoint than the standard itself.


> If that's the case, then all you have to do is identify
> which "requirements" fall into this category.

Nope, I don't *have* to do anything of the kind, it does not
seem important to me to do so at this stage for several
reasons:

1) I don't think any real users are affected by this one way
or another. Theoretical conversations on CLA don't have very
much to do with real users.

2) We went through the excercise during the design process, and
surprisingly lots of people wanted to keep in these ill-defined
"requirements". There is no reason to think that anything has
changed. The fact that Ken Garlington may or many not understand
things differently is irrelevant. Certainly Robert Dewar's
understanding has not been changed one iota by this exchange.
He still thinks that documentation requirements etc should never
have been included in the standard as normative, and nothing
that has been said here has changed his mind, but he never
makes the mistake of rearguing old points unless there is new
data!

3) The ARG has many more important things to do than sit around
in angels-on-a-pin-head discussions that have no actual effect
either on compiler writers or compiler users.

> Isn't this why you have a standards process? Users bring ideas
> to the table; they're discussed by other users, implementors,
> and other interested parties, and if THERE'S A CONSENSUS they
> get implemented?

Only for well defined semantic features. This discussion is
all about things not meeting that requirement. Go look up your
previous post where you mention sharing costs, you were not
talking about static pointers, but about metrics!

> "if you're going to implement this sort of
> feature, we'd like to have some level of standardization as to
> what it means. Is this possible?"

Just as I thought you were beginning to understand :-(

Now, not only are you defending requirements like the
documentation requirements for pragma Reviewable, you are
suggesting that this information be output in a standardized
manner -- worse and worse if you ask me! Do you want to
specify the font used :-)

> Fine. So which parts of the standard are this "formal
> English," and which
> aren't?

The entire normative part of the standard should be read
as formal english. I am surprised anyone would ask this
question. It just goes to show the danger of making standards
accessible :-)

Don't get me wrong, I think it is great for the standard to be
accessible, and I am one person who liked the less formal style
of the Ada 83 standard better than the more formal language of
the Ada 95 standard, but it does have this risk of people
reading things and interpreting them informally.

> For example, the following implementation requirement seems to
> dictate
> certain input-output properties that a compiler must have:

> "The implementation shall provide control- and data-flow
> information, both within each compilation unit and across the
> compilation units of the
> partition."

It might, except the phrase "control-flow information" is
undefined, and even as an expert compiler writer, I can only
guess what it might mean.

Actually I don't think it dictates any input-output properties,
since the RM makes clear that documentation on implementation
dependent features (most certainly this information is in that
category) can be provided by simply pointing to HOW to find out
the information. I assume this means that "control flow
information" can be determined by following a debugger, now
that's REAL useful.

Perhaps the writers of this paragraph had something specific
in mind, but they sure did not express it. I suspect if they
HAD tried to be more specific, no one would have put up with
it even in annex H (which is worse than other sections of the
RM in many respects when it comes to being formal and precise).

> I know it's not testable through an ACVC-style test, and that
people can
> differ on the meaning of the definition, and that this
requirement
> "distorts" the standard. I just don't see how it answers the
question.
> Please don't spend too much time explaining all of this to me
again. It's a
> simple "yes" or "no" question.

No it isn't, I can't find any simple yes or no question in
what you wrote.


> You've made it fully clear that a language standard does not
> exist to address user needs.

And that's a bit of a pathetic, but VERY revealing statement.
The language standard exists for one purpose and one purpose
only, to define exactly what it means for a compiler to conform
to the ISO standard.

Ken Garlington may not find that this narrow objective addresses
his "user needs", but I can *assure* you that the great majority
of our users find the existence of the standard and the
associated conformance testing vital, and it most certainly
addresses their needs.

Now of course they do NOT make Ken's mistake of thinking that
the standard can do any more than that, and that my some magic
rewording of the standard, he will no longer have to write
special requirements for his compilers, and will be able to
avoid spending unreasonable amounts of money getting the tools
they need.

It's been useful to have someone like Ken put into very clear
perspective the frustration that he and others feel that the
standard cannot guarantee them high level compilers, and you
can see from his posts how reluctant he is to abandon this
goal. Moreover, as he begins to see that this guarantee might
not be possible, he rushes to the conclusion that the standard
does "exist to meet user's needs".

This viewpoint of the standard as a complete requirements
document is an amazingly common one. But it was never and
will never be a useful one. We probably even still have
people around who hold the closely related fantasy that
validation testing of compilers will assure quality. This
is a closely related issue, and the natural consequence
of Ken's view of the standard is presumably that ACVC
testing does not "exist to meet user's needs", since it
shows partial conformance to the standard, nothing less
and nothing more.


Sent via Deja.com http://www.deja.com/
Before you buy.

Re: Required Metrics

[Robert Dewar]

In article <uegR4.8838$wb7.710909@news.flash.net>,
  "Ken Garlington" <Ken.Garlington@computer.org> wrote:
> Apparently, we're going to have to agree to disagree as to
> whether it's a problem if the vendor decides a requirement
> doesn't constrain his behavior
> (and doesn't even have to tell anyone which requirements he
> decided fell
> into this category).

Ken, always remember that I was one of the ones who think
it was a huge mistake to include ill-defined requirements
in the RM. I think it's definitely a problem, if for no
other reason than it leads you and others to be confused.

That's why I think it would have been FAR better to have
all such "requirements" be implementation advice instead
for the following reasons:

1. We can say far more information things under the IA
than in the requirements section. The non-normative
sections of the RM, including all IA, are not in formal
english, and can be read informally, which is far more
useful in this case.

2. We would word them with this in mind, so that they
would be clearer when interpreted this way.

3. Implementors would indeed have to say whether they
meet the IA in each case or not (OK, to T.E.D, yes this
is not a formal requirement, but in practice it is well
understood, and customarily followed, at least I assume
all decent Ada 95 compilers provide this useful information).

If it is important to you to have implementors tell you how
they deal with each such "requirement", you can only get
that if they are IA. For real requirements, you either meet
it or you don't, and there is not a hint of an obligation
to say *how* you met the requirement. For example, I don't
have to tell you how I compute the sqrt function. There is
no such requirement for information provision, even in an
informal woolly sense.

One would have thought that the people who demanded that
documentation be included in the RM would have also pushed
for requiring documentation documenting that documentation :-)

But they did not, or very properly, such a suggestion was
rejected in the context of formal requirements.

Ken, is this really all new to you? It is really a very familiar
issue to those involved in the language design. I am a bit
surprised by all this sturm and drang over an issue that was
laid to rest 5 years ago, after extensive discussion.





> Why does someone have to "make sure" they have to meet a
> requirement that they've decided does not constrain them in
> any way?

Sorry, I don't understand. Each requirement is addressed from
the point of view of whether the compiler meets it, not whether
it "constrains" you, whatever that means.

> Do people who don't
> drive cars spend much time worrying if they're driving over
> the speed limit?

This is not relevant at all, nice rhetoric, but completely
irrelevant.

A compiler vendor must look through the documentation
requirements, and make sure that they are satisfied according
to the vendors best reading of the reference manual. If the
vendor is relying on 1.1.3(19), they need to think about whether
this applies. It may take little thought, but little is not
zero.

> Now we're getting to the question that wen't unanswered
> before. How do you do this? Is there a checklist? What
> criteria do you use to determine if you've met all of the
> requirements that aren't part of the "formal
> semantics"?

This is simply not a hard job to do for people who understand
what the standard is about, and who know and understand the
standard well. Remember the DOC does NOT say you meet all
requirements (such a DOC would be unsignable, since as I
mentioned before, it would be a declaration that you had
no bugs). The DOC declares that you have no intentional
deviations from the standard. So the process of signing the
DOC is precisely to make sure that you have no intentional
deviations.

> If there are requirements that the vendor decides are
vacuously true, is
> there any "moral" responsibility to either (a) verify this is
the case,
> and/or (b) document the decision?

For requirements that are ill-specified and untestable, it is
clearly impossible to formally verify that the requirement has
been met. You simply do the best you can in such cases to make
sure that the requirement is met. There is no hard and fast
division into requirements that are or are not vacuously true.
You have seized on this phrase, but it is not useful to do so.

I have no idea what morality has to do with a formal language
definition. Most certainly there is no requirement at all
to document decisions, because no decisions have been made.
You paint a picture in which the vendor lines up all the
requirements, and then "decides" which are vacuously true
and can be ignored.

I know of know vendor who goes through such a process. Rather
you go through all the requirements, and as best you can, make
sure you have met them. That's all, nothing more, nothing less,
and you do NOT have to document the fact that you conform to
each individual decision, or how you conform to it.





> Oh, we always got what we wanted -- just before the vendor
> went out of business.

Well I guess you were not very good at picking vendors then.
Too bad, but it's not something the standard can help you with.

> We're just tired of having to go through the cycle over, and
> over, and over again.

Lots of Ada users do NOT have the experience of going over a
cycle of vendors going out of business over and over again, so
this is definitely avoidable. Perhaps you need to look to your
procurement procedures, they sound shaky, at least if judged
by results. In any case the standard is not going to be able
to prevent this I am afraid.

> You're quite correct that we have to shoulder our part of the
> blame. We thought that Ada would naturally win out in the
> marketplace, because it had all of these advantages over other
> languages, and so we were an early
> adopter.

Well many early adopters of Ada have had very good experience.
Maybe they know something you don't, or they have better
procurement procedures. I don't think you can blame all your
difficulties on early adoption.

>  Now, we know better. Rather than trying to drive the
> marketplace, we adapt our requirements to what's out there,
> and to what vendors are
> willing to implement on their own dime.

Well more accurately, what they are willing to implement to
meet general requirements of the Ada community. That's a little
different. Certainly all of ACT's development is paid for by our
paying users. We have no outside investment, so we have no
"our own dime". All the dimes come from customers, so we try
to spend them wisely in a way that corresponds to the general
needs that they have.

> Custom requirements are a sucker's game.

They certainly can be, and it sounds like you have been sucked
in, but that's largely your fault. Your new approach of looking
to see what's generally available, rather than demanding the
implementation of custom tools sounds like a FAR better way to
go. Of course it's not absolute. As I said before, we do many
small enhancements for our customers as part of our regular
support. We also implement special features on a contract basis
when there are special needs, but we will only do this if we
think the features are generally useful.

I definitely agree that the Ada 83 world was too full of
custom requirements. Isn't this a general problem in the
past history of the DoD? Aren't million dollar Ada compiler
tool sets pretty closely related to $500 toilet seats? :-)

Robert Dewar
Ada Core Technologies


Sent via Deja.com http://www.deja.com/
Before you buy.

Re: Required Metrics

[Ken Garlington]

"Robert Dewar" <robert_dewar@my-deja.com> wrote in message
news:8f49je$pti$1@nnrp1.deja.com...
> In article <mUfR4.8835$wb7.709941@news.flash.net>,
>   "Ken Garlington" <Ken.Garlington@computer.org> wrote:
>
> > IF IT'S A REQUIREMENT, IT SHOULD BE TREATED AS SUCH. THAT IS,
> > IT SHOULD CAUSE THE IMPLEMENTOR TO DO SOMETHING.
>
> Sounds reasonable to me! No one ever argued any differently.

Excellent! So, when you were implementing the requirement described in H.3.1
that says, "The implementation shall provide control- and data-flow
information, both within each compilation unit and across the compilation
units of the partition," what was the something you did in the
implementation to meet that requirement?

> > Clearly, what you don't understand is that a requirement that
> > is always met by definition, is not a requirement.
>
> So Ken Garlington pronounces, but can Ken Garlington prove this
> statement from the RM. I think not.

No, but I can "prove" this by referring to the nearest dictionary:
"requirement. something essential to the existence or occurrence of
something else." If there are statements in the RM that are not essential to
the existence of anything in the implementation, why would they be
considered requirements?

Perhaps the language specification does not use terms in either the software
engineering or English language sense?

I can, of course, also indirectly infer the meaning of requirement from
1.1.2:37.

> > IF IT'S NOT A REQUIREMENT, IT SHOULD NOT BE IDENTIFIED AS
> > SUCH.
>
> I think you mean "if it's not a requirement in the above
> sense". Obviously requirements are precisely things that are
> identified as requirements, so the above sentence stated as
> you gave it is tautologically true.

No more than "if an apple is identified as a banana, it's a banana ."

Apparently, however, this does not translate to language specifications. If
I understand you correctly, I could replace the word "requirement" with
"fodasugfdu" with no ill effects, or (more importantly) replace "advice"
with "requirement".

> > USERS (AND VENDORS) SHOULD BE CONFIDENT THEY KNOW WHICH PARTS
> > OF THE STANDARD ARE IDENTIFIED AS REQUIREMENTS, BUT DO NOT
> > CONSTRAIN BEHAVIOR.
>
> Well I'm pretty confident, since it's pretty clear, sorry that
> it continues to be unclear for you!

Can you then translate that confidence into words more precise than "formal
semantics" (which, as we've learned, is not related to the terminology used
in the ARM), or "a specification of input-output relationships" (for which
counter-examples exist; e.g. pragma Reviewable), so that we know which parts
of the RM are essential to the existence or occurence of something else?

> Nope, I don't *have* to do anything of the kind, it does not
> seem important to me to do so

OK, then why are you responding to this thread? Simply say, "Sorry -- your
problem is just not interesting enough to work" and be done with it?

It seems important to me to do so, because:

> I think it's definitely a problem, if for no other reason than it leads
you and others to be confused.

Given that, it is strange to conclude that:

> It is quite clear in the RM what is and what is not a
> requirement, from the headings. The fact that the definition
> of requirement does not meet your definition is pretty much
> irrelevant from the point of view of analyzing the existing
> RM.

Just as it is quite clear from the headings what is and is not formal
semantics, I presume?

Just as you continue to refer to implementation requirements as
documentation requirements, as in:

> Now, not only are you defending requirements like the
> documentation requirements for pragma Reviewable,

And yet, somehow the phrase "well defined" continues to appear, as in

> > Isn't this why you have a standards process? Users bring ideas
> > to the table; they're discussed by other users, implementors,
> > and other interested parties, and if THERE'S A CONSENSUS they
> > get implemented?
>
> Only for well defined semantic features.

Where each vendor not only gets to define those features, but in fact may
not be able to articulate which specific parts of the standard are in that
category? That doesn't seem very "well defined".

> Go look up your
> previous post where you mention sharing costs, you were not
> talking about static pointers, but about metrics!

Actually, I was talking about the "implementation requirements" of pragma
Reviewable -- the idea being that if users in a "special needs" area (like,
say Safety and Security) commonly needed some output from the compiler
(e.g., those commonly required by DO-178B), that it made sense to identify
that output in a standard, to the level of detail that the implementors
could stand (which, of course, would not include the report content or font,
or even if it were generated as a standard "report" at all).

I don't think I provided one word of input regarding metrics during the
standarization process.

However, now that I've learned that:

> The standard is NOT about constraining implementors, in fact
> the standard has nothing to do with implementors at all.

then I guess it doesn't make any sense to talk about the standard with
implementors!

[the usual strawmen of testing, the extent to which the standard addresses
all the users needs, etc. ignored]

Re: Required Metrics

[Robert Dewar]

I'm signing off from this thread now. I think I have
said everything I want to say (more than once), and
it is simply beginning to repeat itself :-)


Sent via Deja.com http://www.deja.com/
Before you buy.

Re: Required Metrics

[Ole-Hjalmar Kristensen]

Robert Dewar <robert_dewar@my-deja.com> writes:

> In article <24VQ4.8453$wb7.646902@news.flash.net>,
>   "Ken Garlington" <Ken.Garlington@computer.org> wrote:
> > By the way, while writing up a bug report, I found this in the
> gnatinfo.txt
> > file...
> >
> > "The Documentation for GNAT consists of two manuals, the GNAT
> User's Guide
> > and the GNAT Reference Manual. These are available in a number
> of formats
> > (ASCII, HTML, PostScript and info) and are bundled as a
> separate
> > documentation
> > distribution and can be found at the same places as the GNAT
> binary and
> > source distributions."
> >
> > ...and I just had to laugh!
> 
> 
> By the way, the documentation for GNAT very definitely
> includes the Ada Reference Manual -- but perhaps that's a
> surprise to Ken too ... hard to tell!
> 
> This thread is quite instructive.
> 
> I think the entire problem is that in the realm of software
> engineering, you define a set of "requirements", and then
> you can tell whether you have done a good job of implementing
> the software by seeing if it meets these requirements.
> 
> It is, I guess, quite understandable, if quite wrong, for people
> to make the mistake of thinking that a language definition is
> like such a requirements document, and that you will be able
> to tell if a vendor has done a good job of writing an Ada
> compiler by seeing if it has met these requirements.
> 
> That's quite wrong of course, the defining language document
> is not a set of requirements in this sense at all. Well more
> properly it is a small part of the requirements. Left out
> entirely are issues of performance, reliability, efficiency,
> usability, maintainability etc etc.

But it *is* the definition of the language. In that case, it is the
*minimum* requirement if you want a complete implementation.

> 
> By the way Ken, you questioned me saying that you were one of
> the people arguing for inclusion of these ill-defined
> requirements in the language. I may remember wrong, but I
> distinctly remember you arguing for this approach in the
> meetings we had on safety-critical requirements, and your
> previous post seems to confirm that memory (the one where
> you say that it was a waste of time attending that meeting).
> 
> I sure hope that SOME people reading this thread come away with
> a little bit better understanding of what language definitions
> are all about (and also an understanding of why validation
> cannot guarantee usability or quality).
> 
> 
> 
> Sent via Deja.com http://www.deja.com/
> Before you buy.

-- 
E pluribus Unix

Re: Required Metrics

[Ole-Hjalmar Kristensen]

"Robert I. Eachus" <rieachus@earthlink.net> writes:

<stuff deleted>

> 
>      Again, from a user's point of view, all this nit-picking is
> irrelevant.  A user who needs, say, a compiler which never takes more
> than 1 millisecond for a clock interrupt is not going to be satisfied
> with some random document.  He or she is going to write those
> requirements in the purchasing contract.  Any such requirements in the
> contract will not specify that this or that feature be documented, but
> the actual required performance.

Or is simply going to test it, and throw away those compilers that do
not meet his requirement.
Or implement the feature himself.
Or use another language, since the need for implementing it yourself
really removes much of the reason for using Ada in the first place.

-- 
E pluribus Unix

Re: Required Metrics

[Robert Dewar]

In article <umqzoq1h3u6.fsf@gong2.clustra.com>,
  Ole-Hjalmar Kristensen <ohk@clustra.com> wrote:
> Or use another language, since the need for implementing it
> yourself really removes much of the reason for using Ada in
> the first place.

No, that's MUCH too strong. It is still the case that many uses
of Ada simply avoid tasking completely. One case is safety
critical programming which wants to avoid the complexities
of tasking semantics. Another case is where the Ada programmer
*prefers* to use the underlying thread capabilities of the
system directly. There is nothing wrong with this.

Furthermore, there primary advantages of Ada to most users do
not stem from performance of rendez-vous, but rather many other
features.

Of course people might switch to other languages for many
reasons but the "much" in your sentence above does not hold
water.

In any case, I think we agree that the RM is not the issue here
:-)


Sent via Deja.com http://www.deja.com/
Before you buy.

Re: Required Metrics

[Robert Dewar]

In article <umqzoq1h3u6.fsf@gong2.clustra.com>,
  Ole-Hjalmar Kristensen <ohk@clustra.com> wrote:
> Or use another language, since the need for implementing it
> yourself really removes much of the reason for using Ada in
> the first place.

No, that's MUCH too strong. It is still the case that many uses
of Ada simply avoid tasking completely. One case is safety
critical programming which wants to avoid the complexities
of tasking semantics. Another case is where the Ada programmer
*prefers* to use the underlying thread capabilities of the
system directly. There is nothing wrong with this.

Furthermore, there primary advantages of Ada to most users do
not stem from performance of rendez-vous, but rather many other
features.

Of course people might switch to other languages for many
reasons but the "much" in your sentence above does not hold
water.

In any case, I think we agree that the RM is not the issue here
:-)


Sent via Deja.com http://www.deja.com/
Before you buy.

Re: Required Metrics

[Robert I. Eachus]

Robert Dewar wrote:
 
> Not till the ARG ruled clearly on this issue, and the AVO
> backed it up with the tests (which as I mention above, I
> helped write), in ACVC version 1.10, did a proper set of
> tests get written.

   I have no disagreement with what Robert Dwer wrote here (hmmm
enthusiastically agree?  applaud wildly?).  However, the Chapter 13
tests which RBKD wrote had less effect on Ada 95
than he and others including myself hoped, for several reasons.

   First, by the time some of these were required by the ACVC vendors
were all working
on Ada 9X compilers and were unwilling to make fundamental changes not
required by Ada 9X.

   Second, the conformance rules were changed to allow compilers not to
comply with up to ten tests, as long as the intent was to correct these
in the next validated version.  In practice, since most new tests in
1.10 and after were chapter 13 tests, it allowed vendors to be much more
agressive in stating that "this cannot be 'simply' implemented in this
compiler."  Of course, in many cases this was exactly right--it was the
tests that would have required major compiler changes that didn't get
made.

   Finally, since many of the Chapter 13 tests were, in effect, pushed
back by challenges to later ACVC versions, some of these tests were
never effectively required.  Since the design of Ada 9X started well
before ACVC 1.10 affected production compilers, the experience of users
during the early phases of the Ada 9X development process was with
compilers that in some cases  did not take representation clauses, etc.
seriously.

   So as Robert Dewar points out, while the requirements in Ada 95 are
much more clearly spelled out, they are definitely a subset of what was
finally required in Ada 83.  (Fortunately, the marketplace has tended to
require this 'additional' support in any case.  I hope 'de facto'
becomes 'de jure' with the next Ada standard...
 
> Interesting -- I had COMPLETELY forgotten this
> statement in the standard. It is of course an exact codification
> of some of the things that Bob Duff and I have argued. In
> particular, it provides an absolutely CLEAR basis for Bob Duff's
> observation that the behavior of the delivered binary compiler
> code, and the behavior of code generated by this compiler
> constitute documentation in the sense of the RM.
> 
> So the RM *did* try to define what documentation meant, and
> came up with something about as far from what Ken has in mind
> as one can imagine.
 
    Yes, the intent was to prevent documentation requirements from
limiting actual implementations.  In Ada 83, the ACVC tests effectively
required every Ada compiler to implement a "usable" maximum input line
length.  And the blasted ACVC tests checked that the compiler DID NOT
accept longer lines, whle requiring that you accept an identifier that
was the same length as a line.  So you had to set an arbitrary maximum
line length and enforce it even if in the absence of the ACVC test the
limit would only be imposed by memory size or file characteristics.  In
Ada 95 the requirement concerning line length is in RM 2.2(15): "...The
maximum supported line length and lexical element length are
implementation defined."  (The minimum for each is 200 characters.)

Re: Required Metrics

[Robert I. Eachus]

(I said:)
> >          Again, from a user's point of view, all this nit-picking is
> > irrelevant.  A user who needs, say, a compiler which never takes more
> > than 1 millisecond for a clock interrupt is not going to be satisfied
> > with some random document.  He or she is going to write those
> > requirements in the purchasing contract.  Any such requirements in the
> > contract will not specify that this or that feature be documented, but
> > the actual required performance.
 
    I was not saying that you should have a compiler built to your
special order, although
I have been in the position of writing such contracts (for hardware
vendors).  I was saying
that when you are setting the requirements for a compiler for YOUR
particular project, you look at what YOUR specific requirements are, and
then write a requirements document, possibly an RFP, and a purchase
contract.  If a compiler is available off the shelf that does meet your
requirements for a large, long-duration project, there is a major
advantage to writing those key requirements into the purchase order.  It
informs the vendor that those particular features are key requirements
for your project, and that you will need long term support for a
compiler that meets those requirements.

    The compiler vendor, when planning enhancements of his product, will
look at his support requirements and pay attention to them.  Of course,
if you write a detailed contract which documents every possible detail
of the expected product, the vendor will ignore those requirements when
planning significant revisions.  He will expect you to continue to use
the current compiler version, perhaps with some patches.  He can afford
to halt progress to support you and still be a viable competitor. 
(Actually, it sounds like Ken did get into this situation--imposing
enough requirements on the vendor to prevent him from staying
competitive.)

Re: Required Metrics

[Ken Garlington]

> (Actually, it sounds like Ken did get into this situation--imposing
> enough requirements on the vendor to prevent him from staying
> competitive.)

Not really - more that the vendor made poor business decisions (one couldn't
sell their product overseas, for example), and so lost out in the inevitable
Ada market contraction. Many of the "requirements" we imposed are now
described in the Ada standard. The real moral of the story, IMHO, is to
never plan on a long-term relationship with a single vendor, toolset,
platform or language.

Re: Required Metrics

[Robert A Duff]

"Robert I. Eachus" <rieachus@earthlink.net> writes:

>...And the blasted ACVC tests checked that the compiler DID NOT
> accept longer lines, whle requiring that you accept an identifier that
> was the same length as a line.

That test is clearly wrong.  I'm surprised nobody has challenged it.
There is no requirement that a compiler impose a maximum line-length
limit, and there never has been.

I guess it's easier to make your compiler pass a test than it is to go
through the challenge process.  Sigh.

- Bob

Re: Required Metrics

[Robert I. Eachus]

Robert A Duff wrote:
  
> That test is clearly wrong.  I'm surprised nobody has challenged it.
> There is no requirement that a compiler impose a maximum line-length
> limit, and there never has been.
 
> I guess it's easier to make your compiler pass a test than it is to go
> through the challenge process.  Sigh.

  If the test is still in the suite, it definitely should be
challenged.  The Ada 95 rules explicitly allow the maximum identifier
length to be different from the maximum line length.

  However, in the early days of Ada 83 validation--around ACVC 1.4--you
were faced with
an impossible Catch-22.  You could argue that the test had to be
modified for your implementation, but you still had to demonstrate
somehow, that an identifier of maximum
length was accepted, and an identifier of maximum length + 1 was either
rejected by the
compiler, or impossible to create in a file.

  And yes, we took the easy way out on the compiler for the Honeywell
DPS6.  The "real" maximum line length for {normal} text files was 255
characters.  It was MUCH easier to set the maximum Ada source file line
length to 253 and pass the ACVC test as written, than to try to
demonstrate that we couldn't create a non-standard file containing an
identifier longer than the maximum line length.  (The choice of 253 was
for consistancy with Ada Text_IO files, where we needed a couple extra
characters for new page and end of file makers.)

Re: Required Metrics

[Robert Dewar]

In article <wccbt23uaut.fsf@world.std.com>,
  Robert A Duff <bobduff@world.std.com> wrote:
> "Robert I. Eachus" <rieachus@earthlink.net> writes:
>
> >...And the blasted ACVC tests checked that the compiler DID
NOT
> > accept longer lines, whle requiring that you accept an
identifier that
> > was the same length as a line.
>
> That test is clearly wrong.  I'm surprised nobody has
challenged it.
> There is no requirement that a compiler impose a maximum
line-length
> limit, and there never has been.
>
> I guess it's easier to make your compiler pass a test than it
is to go
> through the challenge process.  Sigh.

You missed the point. There was a requirement that if you
support lines of length N, you must also support identifiers
of length N, including externally visible names of length N.
It is impractical in most environments to allow arbitrary
length for identifier names.

A compiler that limited names to 256 characters, but had no
limit on line length would NOT have been valid in Ada 83 (I
think we fixed this in Ada 95? but on the other hand we
recommend a line limit of 250 in Ada 95!)


Sent via Deja.com http://www.deja.com/
Before you buy.

Re: Required Metrics

[Robert I. Eachus]

Robert Dewar wrote:
 
> You missed the point. There was a requirement that if you
> support lines of length N, you must also support identifiers
> of length N, including externally visible names of length N.
> It is impractical in most environments to allow arbitrary
> length for identifier names.

   No, I didn't miss the point, since at Honeywell we did have a
compiler that ran into exactly this problem.  The compiler used a hash
table with the actual identifier strings stored in the heap. 
Originally, the only limit on identifier length was imposed by the type
of text file used for the source file.  This limit was always shorter
than the limit imposed by the compiler (none) or by the size of memory
on the machine used to compile the program.  We could pass the ACVC test
on the largest DPS-6 machines with the maximum line length set to
2**16-4, which was the longest line allowed if you used a special file
type.  However, on the low end machines, that ACVC test would fail to
compile when it couldn't allocate one of the long identifiers.  In this
case, we could have broken the test up into several parts, but it was
much easier to require a smaller maximum line length--I'd have to look
but I think we used 8K--so that all validations were identical.  (Later,
I think DDC changed their front-end to store the identifier strings in a
file and just keep the hash table in memory.  But the Line length limit
stayed.)

    So I do personally know that in at least one case, this ACVC test
did force a compiler limitation.  And we did fix this in Ada 95--you are
allowed to have different limits on identifier size and line length. 
(And I do mean we.  I remember giving an example where requiring 255
instead of 254 would cause problems, and you (Robert Dewar) suggested
250, since there was no particular reason not to choose a decimal, not
binary, round number.)

    Hmmm.  Just remembering a different effect of this rule in Ada 83. 
There were some ACVC floating point tests that had long numeric
literals.  The length of these literals depended on the number of digits
allowed in a floating point type declaration.  I remember at least one
compiler choosing not to support extended floating point since their
compiler couldn't support numeric literals as long as required by those
tests.  (Does anyone remember?  Was that the ALS?  It certainly was a
VAX targeted compiler.)  In any case choosing a "short enough" maximum
line length negated the problem.