Re: Killing software and certification (was: Ada-Oriented GUI)

["Dan'l Miller" <optikos@verizon.net>]

On Tuesday, March 27, 2018 at 1:32:32 PM UTC-5, Alejandro R. Mosteo wrote:
> On 23/03/18 10:05, Jeffrey R. Carter wrote:
> 
> > Autopilots have to be certified to DO178B/C. They'll continue to be 
> > written in Ada and not kill us.
> > 
> > Self-driving cars, though operating in a much more complex environment, 
> > don't seem to need any certification, and will probably kill us all.
> 
> I'd like to revisit this point in light of the recent Uber news, but 
> also let's not forget for example this one which is simpler than fully 
> autonomous cars:
> 
> https://en.wikipedia.org/wiki/2009–11_Toyota_vehicle_recalls
> 
> I'm not in the industry, and I'd be surprised that unverified software 
> were allowed to run in civilian environments where failures basically 
> amount to a very dangerous situation. True, a human driver is still 
> required to be ready, but let's not be disingenuous about how well that 
> can work for several hours of testing/driving.
> 
> After a bit of googling around I see that there are automotive standards 
> for certification (the one I see more often mentioned is ISO 26262). 
> About enforcement, I also read that regulation varies by US state. I 
> haven't found anything definite about Europe.

https://en.wikipedia.org/wiki/ISO/TS_16949

IATF 16949:2016 is the active quality-management regime for the automotive industry that replaces QS9000 (North-American-only analogue of the aerospace industry's AS9000 and the telecom industry's TL9000) and prior ISO 16949 editions.  Just as AS9000 and TL9000 and the predecessor QS9000, IATF 16949 is ISO9001 with some teeth (e.g., transparent publicly-visible anonymized degree-of-compliance-versus-competitors statistics; effectively the same recursive requirements on all subcontractors and suppliers as on the top-level automobile-assembly manufacturer).

That said, USA law is effectively an extrapolated variant of pre-1776 British common law (with the exceptions of Louisiana which is effectively extrapolated Napoleon-era French law at the time of the Louisiana Purchase and a few states that are derivatives of the then Spanish Empire).  What that means is that generally we have a collection of systems of law that demand that companies (and individuals) act responsibly a priori on their own volition to avoid being negligent and thus liable in civil lawsuits that a-posteriori-ly claw back money for the cascading ramifications of negligence.  In particular, most of the USA (neither at the state nor federal nor municipal levels) has nothing much that resembles socialist system of laws that criminalize noncompliance with (our voluntary-membership industry-/guild-internal) regulations.  Indeed, being a member of an inferior “guild” of unwise industry ‘best practices’ (e.g., joining a nonrigorous industry-standard organization) or lack thereof is generally not illegal, hence why Silicon Valley often declares that they follow different/looser regimes of industry best practices than joining the more rigorous modern equivalent of guilds.  (I use guild there to call to mind the state of British law's thinking in the centuries leading up to 1776.)  The primary laws in that criminalize bad-acts themselves (as opposed to mere civil-law clawback/fine penalties for perceived negligence) are in the financial/embezzlement domain, not the technical engineering domain (except in the domains regulated by licensed professional engineers providing services directly to the public).  Indeed, compliance as an industry/guild to best practices in the industry can be almost tantamount to a safe harbor from judgements in civil-law prosecution, especially when a jury is taught about the company's meticulous adherence to the regime of best practices.  Given the existence of that de facto safe harbor in lawsuits, it is unclear why Silicon Valley companies tend to abhor joining such quality-management organizations and abhor performing their work under a regime of rigor that would not only improve their quality but also insulate them to a degree from lawsuits.  [I am not a lawyer; do not rely on these statements in any legalistic way.]

In other words, in the USA, our laws are effectively historically libertarian, not socialist:  it is up to the consumer to read the label on a product to see whether the product has been designed and manufactured in accordance with various voluntary regimes of best-practices in industry-standards (i.e., the ones that that individual holds dear at a personal level as their own personal belief system), not on the manufacturer to be forced to join those organizations, not on the management within those companies to obey those industry standards or be prosecuted, not at a one-size-fits-all societal level.  If you as a consumer buy an inferior-designed product, then it sucks to be you for not engaging in caveat-emptor/buyer-beware, except to claw back wrongs after a mishap/wreck/death/maiming, and except certain categories of consumer goods for children that are more tightly regulated since children cannot speak for themselves due to being minors.