Software Metrics (was Re: official recommendations of Ada)

["Marin David Condic" <marin.condic.auntie.spam@pacemicro.com>]

I would like to ask a favor of those in this thread who have an interest in
productivity metrics: Please define and explain the metric(s) you would be
interested in collecting. (This is not an attempt to insult or deride
anyone - I'm professionally curious about what metrics have been used to
measure productivity and/or what metrics might be considered valuable for
measuring productivity. If you feel slammed, then submit a post to asuage
your guilt! :-)

In a past life, I measured productivity in a study of control system code.
(This was comparing Ada to other languages used in control system
development). The numbers may not be significant for other types of systems
as a result. It should be useful for other embedded/realtime apps with
all/most code being developed from the ground-up. (No we didn't have an
RTOS, or libraries of utilities, etc. We had (with Ada at least) the RTK
that was provided with the embedded compiler and that was about it.) Without
covering the development process in detail, this is basically what we did:

All software was under a CM system, so we could identify modules changed
between two times. We presumed that to add a new module, delete a module or
change a module (even a single line of code in it) you basically had to read
and understand that module, so you got credit for the whole thing.
Statistically, module size didn't vary hugely, so we didn't think we'd see
much skewing of results due to modifying a single 10,000 sloc module
regularly. Over time, we thought it would all average out enough to be
useful.

We could easily get man-hours/month for a given project. We could relatively
easily get modified modules (reduced to a semicolon count of all modules)
within a month. Hence we could calculate some version of SLOCs/Fortnight.
Introducing Ada and other factors moved our productivity from about 35
SLOCs/Hour to roughly 70 SLOCs/hour. (Not all credit could be given to Ada
alone - there were other high level tools introduced to go with it.)

Defects were another matter. We had a change request system tied to the CM
system, so it was easy to identify all change requests made in a month. The
submitter of a CR and the approver of a CR both made some kind of
determination of if the CR was asking for an enhancement/new feature or if
it was correcting some kind of deficiency in the code. Deficiencies were
chalked up as defects and counted. Introduction of Ada (and the tools)
resulted in reduction of defects by a factor of four. We largely credited
Ada for this (rather than the tools) because of the types of errors that
were being eliminated. Most of the errors that were not being made were the
kinds of things that the syntax/semantics of Ada were not allowing to occur.
(scaling errors, indexing errors, addressing errors, etc. Exactly the sort
of errors that C/C++ would *not* catch and eliminate.)

Anyway, that outlines my experience with collecting productivity numbers and
I would appreciate other ideas about how y'all would personally go about
doing it. This is because I'm interested in what people would consider to be
a *meaningful* measure of productivity. If we expect to state that Ada is
superior for productivity, we could maybe be clear about what we mean.

MDC
--
Marin David Condic
Senior Software Engineer
Pace Micro Technology Americas    www.pacemicro.com
Enabling the digital revolution
e-Mail:    marin.condic@pacemicro.com
Web:      http://www.mcondic.com/


"Colin Paul Gloster" <Colin_Paul_Gloster@ACM.org> wrote in message
news:slrn9lqaf7.k5j.Colin_Paul_Gloster@camac.dcu.ie...
>
> But if they just start collecting these data (which you have yet to
> satisfactorily (or even try to) exhibit that they work and that you
> understand anything about collecting them and exactly which sorts of data
> these are anyway) from right now as opposed to having already had a
> tradition of doing so, they will not have enough of a sample space for
> many years. This is not to denigrate harvesting, but you did not get Russ
> to back you up on such a practice already being installed therein so you
> have a problem there.