Re: Killing software and certification

["Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de>]

On 2018-03-28 19:06, Alejandro R. Mosteo wrote:

> This line of thinking is often brought by a colleague working on 
> "classical" solutions to problems that are nowadays trendy on deep 
> learning circles. When feeling optimistic I see it as the complexity of 
> the simplex method: linear on average but worst case exponential. I tend 
> to think that the same you could have a watchdog for a stray simplex, 
> you could have some fallback for a DNN behaving badly (if you can detect 
> it in the first place :P).

If you are sure that it converges to the "right" thing. Does it? How do 
you know when the "right thing" is not even defined?

> In other words: if DNNs prove to be as useful as they promise, they'll 
> find a way to statistically cover the desired percent of reliability. 
> Maybe this is another field of research in the making.

Huh, you cannot use mathematical statistics here. You don't know what 
are elementary events here. You cannot use models because any sample set 
will be too small for testing any hypothesis. That is for the case of 
using brute force. In other words, to make mathematical statistics work 
there must some solid set of known laws describing things, but there is 
none.

> I'm also told 
> that "we solve X with a DNN" is no longer acceptable in the main 
> conferences, that you have to also provide some insight on the DNN 
> workings. But this is hearsay that I pass along.

This is another can of worms: knowledge extraction. If you can extract 
knowledge from the trained system, then you can build another system 
based on that knowledge that would deploy a direct decision making 
method without any training. If you cannot, you still have no idea what 
the system does now and what it will do next time.

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de