Software Development Environments (Was: What is wrong with OO ?)

[Stephen Riehm <sr@pc-plus.de>]

Robert C. Martin <rmartin@oma.com> wrote:

> As engineers we need a CAE system that combines the CASE tool,
> editor, compiler, documentation tool, debugger, test plan generator,
> simulator, etc into a single seamless facility.

> Such a system maintain traceability between requirements, use cases,
> design artifacts and modules.
> It would maintain lists of programmer and analysts notes that were
> germane to each other.
> It would facilitate communications between engineers working on the
> project regardless of their location.
> It would facilitate configuration management and version control.
> It would facilitate project management.

> No such tool exists today.  I think it will be a long time before
> such a tool does exist.  But I can dream.

Exactly, but there's more. The above system would also have to cope
with various languages. Especially when dealing with GUI's etc,
generators are used to generate C++ code, which then needs to be
compiled etc. In this case, you don't want to edit the generated code,
but the code that was used to generate it. (other examples include
lex and yacc files, scripts etc) Such an "all singing and dancing"
system can't possibly cater for ALL possibilities.

As Robert said, no such tool exists today, but there is no reason why
we can't move in that direction.

The Unix view of the world is not a bad one, but one which compiler
writer's seem to have discarded. Unix works because programs are
expected to communicate with each other, via well defined interfaces,
the principle being that the sum of many parts is much greater than
could ever be achieved by one whole. Robert's dream could be achieved
if the parts (compiler, editor, build system, debugger etc) could be
made to really fit together. Someone here said that this "isn't rocket
science", and he's right. It just takes a little logical thinking.

I'll try to identify some of the problems facing developers, regardless
of what language they are using. It would be nice to take the
suggested improvements below and integrate them into existing systems,
but the way I see it at the moment, the lack of communication from the
existing compilers provides a constant stumbling block. So, in classic
hacker style, I recommend that we "work around" the problem for the
time being, while constantly moving in the direction of Robert's
dream. (don't worry Robert - I share your dream too!)

One very large problem with software development practices is that
they aren't very scalable. What works for a 3 woman team almost
definitely doesn't work when you have 30 people working on a piece of
software. Unfortunately, in University, you learn the language, and
then you write a small program or two to prove that you know the
basics, but I have never seen a course where students are exposed to
practices which apply to software projects with many developers, huge
libraries of code, etc. Students enter the world thinking that they
can create a directory, start up their favorite editor and write a
program, possibly with the afterthought of doing a little design and
testing, documentation may also cross their minds if they've been
paying attention.

Languages aside, there is the problem of how software gets written. My
eyes have been opened by the (mis)fortune of working with teams of 50
developers or more, in which case problems such as code stability,
deterministic build mechanisms, communication between developers and
other factors become very apparent.

One step in the right direction is the introduction of Configuration
Management Systems (CMS), like CVS, RCS, SCCS, and the various
commercial
products available today (I know of 3 very good systems, but this is
not the place or time to advertise). These systems can be used to
force developers to think a little more carefully about what they are
doing, and above all, make it possible to trace what has been done,
and sometimes even why.

Another big help would be, in my view, the teaching of deterministic
development practices. Unfortunately I am yet to have found ANY
definitive work in this area, although some people are beginning to
see the relevance. I can best describe this with examples of what NOT
to do, but I am desperately searching for a mechanism to help improve
the situation.

In a small project, it is perfectly feasible for each developer to
have a complete copy of the software in her private domain
($HOME/project), where she can edit the code, and test it without
inadvertently effecting anybody else. When she's finished, she
publicises her changes, and moves on to the next task. In small
circles this works OK, as communication between developers is simple,
and normally verbal confirmation is adequate. Try this with more than
about 5 developers and you very soon get chaos. The chances that some
(or all!) developers are working with inconsistent code (because they
are working with copies) increases greatly. A big help in this regard
is the advent of CMS, which provides a central software repository,
and which allows no more than one developer to change a file at a time
(there are workable exceptions to this, but the CMS needs to be quite
clever). The developers still need to copy the files in order to
build (in most cases), but we're moving in the right direction.

The next problem is that of deterministic builds. Make (gmake etc) is
a tragic example of a hammer being used to build the Empire State
building (from digging the foundations, to installing the lights)!
Make works fine as long as all the files are neatly located in one
directory, and as long as no errors occur. In the real world of larger
software, source code is (sensibly) distributed throughout a tree of
directories, thus related software is kept together, while unrelated
code is kept well and truly apart. Errors often do occur (especially
during development) and parts of the system are often not even
complete. This is where large improvements can be made.

In the classic "make" environment, all files must be in the current
directory. A good enhancement of this is the introduction of gmake's
VPATH, where files can be found elsewhere, ie: in a central
repository, but in order to work out what is going to be done by make,
you need to be nothing short of a genius (OK, the rules are simple
enough, but if I had a cent for each time some one asked my "why did
make do that?" I would be a very rich man indeed). The next
improvement is to centralise the rules used for generating code. This
"can be achieved" via included makefiles, but there is still lots of
room for improvement.

At the moment I know of Christopher Siewald's "Jam" program, which is
a huge step in the right direction, and Bob Sidebotham's "Cons", which
extends on the ideas behind Jam, and adds some very interesting
concepts for determinability. Cons is still a little young, but when
it's completed it could be a major win.

So, my point is, that the language is a key factor when considering
software development, but the development environment is also very
important. What's the point of developing a program if you can't
reproduce what you've done? In my view, too much attention is being
given to the software lanuages and the source code itself, and not
enough to the environment in which the software is being written.
Class browsers, CASE tools, CM systems and deterministic build
environments can all greatly contribute to the reduction of bugs
written in the first place. Finding bugs takes up about half of the
software development cycle, and you will never get all of the bugs out
of a program, but if you don't put them there in the first place, you
end up with more stable, and easier to maintain software. (cheap and
reliable!)

I would like to hear your comments about production environments,
should shadow trees be used to provide simple access to a central
software repository? what about using hard or soft links? How do you
coordinate the work of 50+ developers all working on the same
project?

If anyone knows of any seminars, documents, research papers, web sites
etc etc that address these problems, I would be most grateful to hear
from you.

Thanks for your time, this article got a little longer than I had
expected. Sorry

Steve
Software Production Environment Analyst
pc-plus, Germany

//   __________________________ ._ o  ________________________________
\\\\ Stephen Riehm             /  //\.        Stephen.Riehm@pc-plus.de
//// pc-plus                  '  \>> |         Phone: +49 89 62030 148
  \\ 81539 Munich, Germany        \\ `           Fax: +49 89 62030 113