Re: Killing software and certification

["Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de>]

On 2018-03-27 20:32, Alejandro R. Mosteo wrote:
> On 23/03/18 10:05, Jeffrey R. Carter wrote:
> 
>> Autopilots have to be certified to DO178B/C. They'll continue to be 
>> written in Ada and not kill us.
>>
>> Self-driving cars, though operating in a much more complex 
>> environment, don't seem to need any certification, and will probably 
>> kill us all.
> 
> I'd like to revisit this point in light of the recent Uber news, but 
> also let's not forget for example this one which is simpler than fully 
> autonomous cars:
> 
> https://en.wikipedia.org/wiki/2009–11_Toyota_vehicle_recalls
> 
> I'm not in the industry, and I'd be surprised that unverified software 
> were allowed to run in civilian environments where failures basically 
> amount to a very dangerous situation.

Why should it surprise you? How are you going to verify it? Black box 
test is impossible. White box test isn't either, assuming any NN 
involved. There is nothing to prove.

> After a bit of googling around I see that there are automotive standards 
> for certification (the one I see more often mentioned is ISO 26262). 
> About enforcement, I also read that regulation varies by US state. I 
> haven't found anything definite about Europe.

If any certification will ever be set up, it will be certification of 
the tools and developing processes/teams, not certification of the 
actual software. That is the usual backdoor to go around any questions 
about correctness.

> Also, it's not the same software for a drive-by-wire part than for an 
> autonomous car.
> 
> I'm under the impression that these autonomous car outfits are at the 
> time closer to a research environment than to that of a well-established 
> industry. I.e., code is produced faster, hence bugs are more likely.

The code used in the ECU and other car subsystems is not any better from 
that point of view. It is much simpler, deploys well-established 
algorithms and, importantly, is testable with a large set of test 
hardware and software available. That is the reason why it works better. 
But otherwise, it is just same. There is no any guarantee for it to work.

> In the end I'm not sure where I want to go with this post. It's simply 
> that I find the topic very interesting. If anyone with actual knowledge 
> on the status of automotive software certification (or any informed 
> ideas) would share some thoughts I'll be eager to read.

My understanding is that it is possible to certify about anything 
regardless the correctness.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de