From mboxrd@z Thu Jan  1 00:00:00 1970
X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on polar.synack.me
X-Spam-Level: 
X-Spam-Status: No, score=0.7 required=5.0 tests=BAYES_00,INVALID_DATE,
	REPLYTO_WITHOUT_TO_CC autolearn=no autolearn_force=no version=3.4.4
Path: utzoo!utgpu!news-server.csri.toronto.edu!rpi!zaphod.mps.ohio-state.edu!wuarchive!uunet!bu.edu!encore!jcallen
From: jcallen@Encore.COM (Jerry Callen)
Newsgroups: comp.lang.ada
Subject: Re: Yearly Fees for Support of Compiler
Summary: Small test cases help!
Message-ID: <14774@encore.Encore.COM>
Date: 9 May 91 14:25:29 GMT
References: <2222@ac17.cs.nps.navy.mil> <3050@cod.NOSC.MIL>
Reply-To: jcallen@encore.Com (Jerry Callen)
Organization: Encore Computer Corp, Marlboro, MA
List-Id: <comp.lang.ada>

In article <3050@cod.NOSC.MIL> sampson@cod.NOSC.MIL (Charles H. Sampson) writes:
>     For over 10 years I was in charge of a compiler that had unlimited
>free user support.  (Paid for with your tax dollars.  It was for one of
>those strange DoD languages.)  A tremendous amount of time was spent try-
>ing to discern exactly what the user had done.  Most users, trying to be
>helpful, partially analyzed the problem themselves and filtered out "un-
>important" information.  They were almost always wrong about what the
>problem was ("array references don't work") and what was unimportant
>(after 30 minutes of probing, "Oh, is that important?  Yes, I did insert
>a line like that upstream.").  Furthermore, often the problem turned out
>to be the user's, not a compiler bug.

Let me offer the flip side to this discussion. I helped support an Ada compiler
and RTS that were used on a very large project a few years back. Problem
analysis was often incredibly difficult because the "test case" consisted
of >100 modules and >10,000 lines of code. Initially the customer made almost
no effort at all to produce a managable test case. The result was that I
was flown to the customer site and spent several hours trimming the test case
down to, say, 30 lines of code, which I brought back and either gave to the
compiler folks or myself (the RTS guy) to fix.

As the project went on I trained several of the folks at the customer site
how to do what I did; eventually we were getting good quality, managable
test cases for most bugs. The folks who WROTE the code generally know
what pieces can be stubbed out when producing a test case; it's harder
for a "hired gun" like me to come in and be pointed at a zillion lines
of source and be told "This doesn't compile. Why?"

>I hope this doesn't sound like I'm
>flaming my erstwhile users.  They were trying to be helpful and this situ-
>ation just goes with the territory.

Why do some people find it so hard to produce small test cases?

Now THIS is a skill I'd like to see taught by universities. It requires,
of course, access to a buggy compiler and RTS. I could recommend one,
but I won't. :-)

-- Jerry Callen
   jcallen@encore.com