Design your "Dream" Metric/Metric Tool

Design your "Dream" Metric/Metric Tool

[Ken Nelson]

Hello,

  From monitoring the debate about programmer
  productivity, I know that program and programmer measurement is a
  sensitive topic on which many have strongly held (usually 
  different) views.  
  
  My company is developing a new quality analysis
  and metrics tool. Naturally we want to make you happy, so here is your
  chance to all become product designers.

  Send me your spec for your dream metric and your
  dream metrics tool.  What phase of the lifecycle, what aspect of code,
  design, requirements etc... do you think is worth measuring, and how 
  would you measure.  How would you present/use the information?

  E-Mail responses to me.  I will summarize "dream tool" reponses on the
  net.

				  Ken Nelson
				  Project Manager, Quality/Testing Tools

				  Software Systems Design
				  3267 Padua Av
				  Claremont, CA (USA) 91711
				  (714) 625-6147

Re: Design your "Dream" Metric/Metric Tool

[Dave Jones]

From article <3305@jarthur.Claremont.EDU>, by ssdken@jarthur.Claremont.EDU (Ken Nelson):
> 
...
> 
>   Send me your spec for your dream metric and your
>   dream metrics tool.

  Oh, let me just ramble on here for a bit, okay?  (By the way, why is this
  posted to comp.lang.ada?)

>   What phase of the lifecycle, what aspect of code,
>   design, requirements etc... do you think is worth measuring, and how 
>   would you measure.  How would you present/use the information?
> 

I don't think I would ever take or keep a job where automated techniques
were used for more than collecting very general data. I'm not one to say,
"Machines will never do this or that."  I mean, _WE_ are machines, right?
And I must admit that somebody at the company is probably going to have
to decide how great a job I'm doing, even if he hasn't a clue. :-) But
every "software metric" program I have heard of is just out and out silly.
I imagine that I will be replaced by a program -- perhaps one that
I write? -- long before there is a program competent to judge the quality
of my programs.

As a starter, the program will absolutely need to read and understand the
program's functional spec, (or infer one if there is none), and decide to what
extent the program does what it is supposed to do. If you don't have a
good guess at that, you don't have anything. It should also take into
account such market considerations as cost of delayed entry into the market
versus the cost of user-discovered bugs. It should measure how well the
spec is implemented in terms of speed, size, accuracy. (Fast, small, and
bug-free are better.) It should know how to weigh these considerations,
knowing when bugs are disastrous, and when they are mere annoyances, when
speed is crucial and when moderate speed is plenty, etc..

A corollary is that no bonus should ever be given for _more_ lines written.
More _functionality_ is the goal, not higher disk usage. If functionality
and speed are constant, then _fewer_ lines is better. (I say this even
though, if you can believe what's been written here lately, my own
lines-of-code output over the last year is well over twenty times the norm.)

Style-checkers that count goto-statements, depth of loops, etc.. are
virtually worthless. If somebody's style is so bad that such a simplistic
program can figure out that it's bad  -- dubious premise -- then somebody
should be helping him to improve it. Code like that should never be judged
as a final product.

Re: Design your "Dream" Metric/Metric T

[ryer]

I am rather pleased with the note stream so far:  A request for
quality metric definitions, followed by several days of silence
(from thousands of people), followed by an intelligent refutation
of the premise.  However, I will venture to add a few cents
worth.  Here are some metrics of quality:

1.  How complex is the program in comparison to the simplest possible
program that performs the same function?

2.  Are the user-visible parts (inputs, parameters, listings, etc.)
"friendly" -- easy to use/remember/understand, idiot-proof, etc?

3.  Where complex combinations of features are used, is it
elegance or capriciousness?

4.  How fast will it execute in comparison to the optimal equivalent
version?

5.  Are the names of types, blocks and objects chosen to make the
semantics intuitive (without being cumbersome)?

6.  Do the comments answer the questions a maintainer is most likely
to ask?

7.  Has the programmer anticipated the type of changes/extensions that
are likely during maintenance and designed the program so that they can 
easily be added without breaking something else?

8.  Does the program do reasonable things when the inputs are unreasonable
and outside the intended scope of the program?

9.  Does the terminology used reflect conventional useage in the
application domain?

10.  Are there parts which have a high potential for reuse in other
programs?

11.  If a maintainer introduces an error in one part of the program,
will the other parts recover, detect it, or crash-and-burn?

12.  Does it use standard concepts, algorithms, and data structures
where appropriate (with identification of their names/origins) or
re-invent the wheel?

After you've implemented these, I will spend another half hour and give
you a dozen more.  :-)

Mike Ryer
Intermetrics