From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on polar.synack.me X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=unavailable autolearn_force=no version=3.4.4 Path: eternal-september.org!reader01.eternal-september.org!reader02.eternal-september.org!.POSTED!not-for-mail From: "Alejandro R. Mosteo" Newsgroups: comp.lang.ada Subject: Re: Killing software and certification Date: Wed, 28 Mar 2018 15:54:04 +0200 Organization: A noiseless patient Spider Message-ID: References: <9ed9edb1-3342-4644-89e8-9bcf404970ee@googlegroups.com> <26a1fe54-750c-45d7-9006-b6fecaa41176@googlegroups.com> <656fb1d7-48a4-40fd-bc80-10ba9c4ad0a4@googlegroups.com> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Injection-Date: Wed, 28 Mar 2018 13:54:06 -0000 (UTC) Injection-Info: h2725194.stratoserver.net; posting-host="460fb46a4c0350d70ba2f75d850e57e2"; logging-data="5475"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX1+Vq9D3FliZnBMG4FRZWs3Q" User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.6.0 In-Reply-To: Content-Language: en-US Cancel-Lock: sha1:ShjCt8ya4Av68vN5CRK97pPVYs8= Xref: reader02.eternal-september.org comp.lang.ada:51227 Date: 2018-03-28T15:54:04+02:00 List-Id: On 27/03/18 21:25, Dmitry A. Kazakov wrote: > On 2018-03-27 20:32, Alejandro R. Mosteo wrote: >> On 23/03/18 10:05, Jeffrey R. Carter wrote: >> >>> Autopilots have to be certified to DO178B/C. They'll continue to be >>> written in Ada and not kill us. >>> >>> Self-driving cars, though operating in a much more complex >>> environment, don't seem to need any certification, and will probably >>> kill us all. >> >> I'd like to revisit this point in light of the recent Uber news, but >> also let's not forget for example this one which is simpler than fully >> autonomous cars: >> >> https://en.wikipedia.org/wiki/2009–11_Toyota_vehicle_recalls >> >> I'm not in the industry, and I'd be surprised that unverified software >> were allowed to run in civilian environments where failures basically >> amount to a very dangerous situation. > > Why should it surprise you? How are you going to verify it? Black box > test is impossible. White box test isn't either, assuming any NN > involved. There is nothing to prove. I can think of an spectrum of regulatory/practical positions between 'nothing can be done, so everyone brace' and 'this won't ever fail, under every [un]conceivable situation'. It's the apparent nonchalance of the general public that coexists with these testing cars, the brashness/recklessness of those expecting to get rich with it and the apparent willingness of politicians that I find fascinating (that's the first ones that comes to mind). I can understand the appeal for politicians to be the first city with a working fleet (or whatever contributions they're getting to favor live testing). As a technophile, I want autonomous cars to become reality, so I can understand that too. As a researcher familiar with the algorithms involved and with the kind of C/C++/Python heaps that implement them I get chills about thinking that a car can be on the highway with a semi-awake safety driver as the only fallback in a split second. All in all, I find the tension between all forces in conflict captivating to watch from a distance. > >> After a bit of googling around I see that there are automotive >> standards for certification (the one I see more often mentioned is ISO >> 26262). About enforcement, I also read that regulation varies by US >> state. I haven't found anything definite about Europe. > > If any certification will ever be set up, it will be certification of the > tools and developing processes/teams, not certification of the actual > software. That is the usual backdoor to go around any questions about > correctness. I understand that now. > >> Also, it's not the same software for a drive-by-wire part than for an >> autonomous car. >> >> I'm under the impression that these autonomous car outfits are at the >> time closer to a research environment than to that of a >> well-established industry. I.e., code is produced faster, hence bugs >> are more likely. > > The code used in the ECU and other car subsystems is not any better from > that point of view. It is much simpler, deploys well-established > algorithms and, importantly, is testable with a large set of test > hardware and software available. That is the reason why it works better. > But otherwise, it is just same. There is no any guarantee for it to work. > >> In the end I'm not sure where I want to go with this post. It's simply >> that I find the topic very interesting. If anyone with actual knowledge >> on the status of automotive software certification (or any informed >> ideas) would share some thoughts I'll be eager to read. > > My understanding is that it is possible to certify about anything > regardless the correctness. That's something to put in a frame :-)