From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on polar.synack.me X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,FREEMAIL_FROM, LOTS_OF_MONEY autolearn=ham autolearn_force=no version=3.4.4 X-Google-Language: ENGLISH,ASCII-7-bit X-Google-Thread: 103376,a00006d3c4735d70 X-Google-Attributes: gid103376,public X-Google-Thread: f849b,b8d52151b7b306d2 X-Google-Attributes: gidf849b,public X-Google-ArrivalTime: 2004-01-10 10:34:36 PST Path: archiver1.google.com!news2.google.com!newsfeed2.dallas1.level3.net!news.level3.com!crtntx1-snh1.gtei.net!news.gtei.net!newsfeed1.easynews.com!easynews.com!easynews!cyclone1.gnilink.net!small1.nntp.aus1.giganews.com!border1.nntp.aus1.giganews.com!intern1.nntp.aus1.giganews.com!nntp.giganews.com!nntp.comcast.com!news.comcast.com.POSTED!not-for-mail NNTP-Posting-Date: Sat, 10 Jan 2004 12:34:34 -0600 Date: Sat, 10 Jan 2004 13:34:33 -0500 From: "Robert I. Eachus" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 Netscape/7.1 (ax) X-Accept-Language: en-us, en MIME-Version: 1.0 Newsgroups: comp.arch.embedded,comp.lang.ada Subject: Re: Certified C compilers for safety-critical embedded systems References: <3fe00b82.90228601@News.CIS.DFN.DE> <5802069.JsgInS3tXa@linux1.krischik.com> <1072464162.325936@master.nyc.kbcfp.com> <1563361.SfB03k3vvC@linux1.krischik.com> <11LvOkBBXw7$EAJw@phaedsys.demon.co.uk> <3ff0687f.528387944@News.CIS.DFN.DE> <1086072.fFeiH4ICbz@linux1.krischik.com> <3ff18d4d.603356952@News.CIS.DFN.DE> <1731094.1f7Irsyk1h@linux1.krischik.com> <3ff1b8ef.614528516@News.CIS.DFN.DE> <3FF1E06D.A351CCB4@yahoo.com> <3ff20cc8.635997032@News.CIS.DFN.DE> <40000150.3050305@noplace.com> In-Reply-To: Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Message-ID: NNTP-Posting-Host: 24.34.214.193 X-Trace: sv3-Ai4drqHd9lcWo+VqOl7p4/Us8mmQH9k5bz9kUDKQKRNIu+oUFR4ReMjUpr1E/Z2WEsDXTZjQ8HTGtF7!lL1RJ80X1sTQyu7NPY3OZvtih+eCAmPiSBnyl6VoDTzocchWqmUmbtl/9HDSrA== X-Complaints-To: abuse@comcast.net X-DMCA-Complaints-To: dmca@comcast.net X-Abuse-and-DMCA-Info: Please be sure to forward a copy of ALL headers X-Abuse-and-DMCA-Info: Otherwise we will be unable to process your complaint properly X-Postfilter: 1.1 Xref: archiver1.google.com comp.arch.embedded:7231 comp.lang.ada:4313 Date: 2004-01-10T13:34:33-05:00 List-Id: Mark Lorenzen wrote: > Indeed. I am a big proponent of formal methods and mathematical > proofs. But using these tools makes it even more important to get the > requirements correct, sound and complete. And that was exactly the problem in the Airbus 320 case. The grumblings of the software designers about the opacity of the formal language used were ignored as "the difficulty of learning a new methodology." The real problem was the lack of a formal method for converting the formal version of the requirements back into something that could be understood by the domain experts. In government contracting we run into this problem all the time. The contractor wants the "B"-specs, the mapping of the original specifications to the intended implementation, to be considered to replace the "A"-spec once the B-specs are accepted by the government. The government wants the testing done to the A-spec, not the B-spec. The usual resolution is to delay government acceptance of the B-spec until the test plan is available to bounce off the original A-spec. Once that requirements tracability is complete, then the government allows the B-spec to be signed off as the specification to be implemented by the contractor. But the government does not accept the B-spec requirements until they are sure that the test plan will result in coverage of the A-spec requirements. I can't count the number of times that the MITRE project team I was on was tasked by the government to check that changes to the test plan still covered all the original (A-spec) requirements. Often this was less than a day's work since the requirements mapping was still valid. In other cases, approval dragged on for months since requirements changes had to be mapped back to the A-spec level. What happened in the Ariane 501 case can be easily explained by referring to this model. The original test plan covered the "real" A-spec. But the full-up navigation system test called for in the test plan was later eliminated due to cost and schedule problems involving the test rig. But no one "looked back" to see which A-spec requirements were to be tested by the eliminated test. Of course, doing such an evaluation of the test plan change should have discovered the mismatch between the Ariane 4 derived requirements and the Ariane 5 system requirements. (The change to the test plan substituted system testing for the upgraded INS for the Ariane 4 for the planned Ariane 5 guidance system test.) This is what I like about the FFRDC (federally funded research and development corporation) model the US government uses. Technically, the FFRDC does not work for the particular government agency that contracts for the development of whatever new system. (Often the DoD, but also NASA, intelligence organizations, the DOE, and the FAA among others.) The FFRDC is charged by Congress with oversight of the project for technical and financial feasibility. But in real day-to-day operations, this usually means keeping their eyes focused on the original goal. If a project like the SSC (super-conducting supercollider) or the A-12 (Navy stealth attack aircraft) goes out of control and is canceled by Congress there is egg on the face of the FFRDC involved. (Fortunately not MITRE in those two cases.) On the other hand, the FFRDC can go "around" the contracting agency directly to Congress when it first becomes clear that the risk associated with a project is too great. I've been involved in several of those calls. Or the FFRDC can recommend spending and "extra" hundred million dollars or three now to ensure that the project doesn't flop later. Again I have been involved in a few of those calls. And once I even told the government (in this case the Air Force) that the risk was not that the contractor couldn't meet the current requirements, but that they would, and the government would be left buying an unusable system because the governments actual requirements had changed. That is the other side of the "cost overrun" story. Had the governments real requirements changed? Yes. Could the government have held the contractors feet to the fire and force them to meet the original requirements for the original price? Yes. But the government was better off negotiating a change order, knowing that the contractor would "recover" their previous cost overruns when pricing the change, than spending hundreds of millions for a now-useless system, and then putting the changes needed to fix it out for bid. In case you are wondering what changes caused the problem, it was Goldwater-Nichols. This bill changed the way the military responded to crises by decentralizing the responsibility to several commands. Currently the most well known is the Central Command, which was responsible for running the Gulf War, and invasions of Afghanistan and Iraq. What was the problem? The data presented to various commands and to the Pentagon in the original system could be different, since data would arrive at different locations via different routes. But you didn't want say Central Command seeing a major threat, and the Pentagon (and the President) seeing different data. We also didn't want to force all data to go through the national command authority so it was consistent. The changes meant that the individual commands needed to know whether they were seeing the "full" picture, or damage to the network resulted in different commands having access to different data (or the same data at significantly different times). In other words all the theater commands needed to see the network status, not just the one location responsible for maintaining the network. It sounds like a minor change, but it wasn't. The network of course was fault-tolerant, and maintenance, of course, needed to know which if any links had failed. The problem had two parts. First the commands needed to be able to compute the network connectivity, including for links which did not directly affect them. This affected network topology and message routing for status messages. Second, the commands needed to be able to do data fusion, and get the SAME results as everyone else, independent of network timing delays. This required new data fusion algorithms, where the results were independent of the order the data was received. -- Robert I. Eachus "The war on terror is a different kind of war, waged capture by capture, cell by cell, and victory by victory. Our security is assured by our perseverance and by our sure belief in the success of liberty." -- George W. Bush