From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on polar.synack.me X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham autolearn_force=no version=3.4.4 X-Google-Language: ENGLISH,ASCII-7-bit X-Google-Thread: 103376,de5dfd6df880dd3 X-Google-Attributes: gid103376,public From: "Robert I. Eachus" Subject: Re: Bugs or Mistakes? Date: 1999/11/16 Message-ID: <3831DA52.964E4748@mitre.org> X-Deja-AN: 549452283 Content-Transfer-Encoding: 7bit References: <19991112202656.2368.rocketmail@web216.mail.yahoo.com> <80i9la$i9e$1@nntp9.atl.mindspring.net> <383086EC.372545D1@mitre.org> <80q4jk$fvo$1@nntp2.atl.mindspring.net> X-Accept-Language: en Content-Type: text/plain; charset=us-ascii X-Complaints-To: usenet@news.mitre.org X-Trace: top.mitre.org 942790882 8629 129.83.41.77 (16 Nov 1999 22:21:22 GMT) Organization: The MITRE Corporation Mime-Version: 1.0 NNTP-Posting-Date: 16 Nov 1999 22:21:22 GMT Newsgroups: comp.lang.ada Date: 1999-11-16T22:21:22+00:00 List-Id: Richard D Riehle wrote: > >If the simulation does meet spec, but > >someone supplies or uses sub-standard materials or doesn't build to > >spec, then they are liable. (The suspended walkway collapse in St. > >Louis and the DC-10 crash on takeoff at O'Hare fall into this category.) > > In this case, I would say there is an error in the specification. Huh? Where was the error in the specification in either of these two cases? If the DC-10 had been maintained in accordance with specifications, the cracks that caused the crash would never have occurred. If the suspended walkway had been build in accordance with the engineering design, it probably would not have collapsed. In the software world, the first would have been a programming error, not a design error, and the second would have been an error introduced during maintenance. > Ignorance of the laws of physics is no excuse. :-) Seriously, this > kind of thing is quite common. In physical engineering, we are often > faced with incomplete understanding of the properties of common > materials under new conditions. Which is why the designer is often faced with choices between "tried and true" practices, and newer less expensive or better technologies which have not been well tested. > Is it a mistake to go ahead and > put people's lives at risk even when we are working off our best > guess? Yes, when you phrase it like that. But when there is an unknown risk, you have no other choice. Remember that an unknown risk is very different from an unquantifiable risk. Three elements, mercury, fluorine, and radium killed chemists before enough was known about their properties so that proper precautions could be designed. In each case there was an unknown risk, that once known could be treated as an unquantifiable risk with other new elements as they were discovered. (Fluorine is still the only element that can displace oxygen in compounds. Other metals besides mercury evaporate and are harmful when inhaled, and there are many radioactive compounds other than radium--but radium is the only one that gives off an invisible gas that is radioactive.) Unknown risks are, by their nature, unavoidable. > Really a different question. This is why insurance companies sometimes > use an "act of God" clause in their policies. > HmmmMMMMmmmm. God is a bug? No, God is a teacher, but some lessons are more emphatic than others. As those lessons are learned, the number of "act of God" cases decreases. Hurricanes, floods, earthquakes, and volcanic eruptions are no longer treated as acts of God in this country, but as part of nature that must be allowed for in designing buildings, dams, and highways. As a result, the loss of life and property when those events do happen is much less than in countries where they are still treated as acts of God. I hope we soon get to the point where software is subjected to the same standards, and most bugs are treated as avoidable. We can continue to disagree as to the number of bugs that are unavoidable, but we certainly agree that the number is very, very small. Reminds me of a study that we did, years ago on software for fault-tolerant systems. We basically concluded that if the software started with one bug per hundred lines, you couldn't afford the cost of getting it to a one per 10,000 standard. If you were above one bug per thousand lines, the cost of getting to one per 10,000 or one per million was the same. (But nothing could remove specification errors other than to redo the requirements analysis with a new team.) > No engineer would ever claim that that the data sheets were always > adequate. In some cases the engineer will make a mistake in using > the data. Other times, it will be insufficient testing. Which is why the testing is the engineer's responsibility. The cases I discussed involved cases where the material provided for testing did not match the material actually used. Especially when the only "true" test is a destructive one, you have to rely on sampling. > > Now we can discuss when a bug is not a mistake. A bug that never > >gets to the field can be the result of a mistake, but often a bug > >detected early is just that, a bug. > > In the words of William Gates, "We respectfully disagree..." It is > simply an error that goes through a succession of corrections. Hmmm. If I am not sure of the syntax, is it a bug to use the compiler as a reference? There are obscure areas, even in Ada ;-), where the compiler is the best available reference. Those cases I don't consider errors, rather that the compiler is being used as a tool. > >For example, in Ada it is much more > >productive to let the compiler worry about elaboration order, and only > >consider it as a problem if the compiler can't find a solution. > > The compilers I use give me error messages. If the program fails to > elaborate correctly, there is still a mistake in my design. I think you miss my point. If possible it is a good idea to put pragma Elaborate_Body on packages to protect future users from elaboration errors. I think most software engineers recommend doing that. But what about cases where it is not clear that the pragma is necessary, but may--stress on may--interact with other units to prevent the compiler from finding an elaboration order? In my judgement, the best approach is to compile with the pragma, and only if it prevents elaboration do the work to determine if it can be safely removed. > >If > >judicious use of elaboration pragmas can sort it out, was there a > >mistake? Not really. It is much more efficient to allow the compiler > >to select an order to the extent the order irrelevant as long as it is > >within the rules. > > The mistake, here, would be failure to test the results for correct > execution. Correct execution will probably not occur in an incorrectly > compiled [read elaboration order is incorrect] program. Now you are mixing the two cases. And maybe I wasn't clear. If you choose to err on the side of caution, and let the compiler tell you to do the extra work to show that the precaution is not necessary, and also to allow more opportunity for maintenance to introduce errors, that is not a mistake, but... > >If you leave off a necessary pragma Elaborate_Body, > >and as a result use an uninitialized variable, that is a mistake, even > >if you do catch it during unit test. > > No argument with this. I fully agree. ...an error in the other direction is always a mistake, even if it is caught. Basically, introducing "unnecessary" checking is always much better than omitting necessary checks. (I have never found it necessary to remove checks to meet real-time deadlines, in fact, there have been cases where I recommended that the government "give" the contractor some of the schedule reserve to so that the contractor would not run around supressing "unnecessary" checks.) > I fully agree. It is always counterproductive to engage in > the blame game during development. OTOH, it is not counterproductive > to simply acknowledge that, as someone else noted, there are defects > in the program. The defects did not create themselves. They were > put there by someone through ommission or commission. The defects > may be a function of inexperience, sloppiness, designing in an > entirely unknown domain, or ... There is still no creature > spontaneously springing to life, worrying its way through > the code. If they are defects, sure. But until they have been identified as such, they are just reported bugs. > Sometimes the mistake is all the way back in the compiler. We once > had a C compiler that had a strange behavior with certain uses of > pointer arithmetic. A bug in the compiler? No. A serious mistake > made by the developers of the compiler. Hmmm. When I was teaching programming courses, I used to instruct students that, if something went wrong, the last place to look was at the compiler. At work the next day, I would write, compile, and run programs and if I found a bug, the first place I looked was in the compiler. Of course, I was working as a compiler developer, and testing my changes... > It can > >also be counterproductive to label as mistakes errors in draft > >documents, schedules, or software. > > I think this is the very place they should be so identified. The > earlier in the process we identify errors as errors, the greater > the potential for eliminating them before they become part of the > actual design. I think we are in violent agreement here. They are errors or omissions and they should be fixed as soon as possible. But a TBD in a draft is not a mistake, it is just a mark indicating that there is something which must be fixed before this leaves draft status. (A TBD remaining in the documentation when it is shipped to the customer is a big mistake. ;-) > It does seem clear that we both agree that, once a software > system is deployed, it is too late to euphemize the defects > as bugs. At this level, given our increasingly litigious > society, we'd better get used to the idea that some parasitic > bottom dweller is going to rip open our brainchild, probing > for the rotting tissue that proves we made some mistake. In > this regard, your question, "Who was at fault," will not be > satisfied by the answer, "Oh, it was just another bug." Unless you work for Microsoft! The quality of the garbage that Microsoft sends out as beta software is often abominable. And not just beta releases. MITRE does a very good job of vetting supported software before approving it for general use. With Microsoft, this often requires critical patches before approval. And talk about using bugs as a euphemism! Microsoft has "hot fixes" and "Service Packs," not patches. -- Robert I. Eachus with Standard_Disclaimer; use Standard_Disclaimer; function Message (Text: in Clever_Ideas) return Better_Ideas is...