AI and Machine Learning today: A reality check

Machine Learning and AI applications in general are moving forward in a fast pace and companies and organizations are trying their best to deploy state of the art models in production. But how good are the results and are we really making sure that our models are doing the job we think they are? Is it safe? If not, how do we fix it moving forward?

Transcript

1. AI and Machine learning today: A reality check Dr. Michael Green 2018-09-26
2. Agenda What can AI do today? What are the pitfalls? What do we do about it? · · · 2/23
3. Overview of AI and Machine learning
4. AI is the behaviour shown by an agent in an environment that seems to optimize the concept of future freedom “ 4/23
5. What is Artificial Intelligence? Artificial Narrow Intelligence Artificial General Intelligence Artificial Super Intelligence Classifying disease Self driving cars Playing Go · · · Using the knowledge of driving a car and applying it to another domain specific task In general transcending domains · · Scaling intelligence and moving beyond human capabilities in all fields Far away? · · 5/23
6. The AI algorithmic landscape 6/23
7. The probabilistic formulation What is actually being done A simple example: Linear regression y μ ∼ = N (μ, σ) βx + α y μ β, α σ ∼ = ∼ ∼ N (μ, σ) βx + α U(−∞, ∞) U(ϵ, ∞) 7/23
8. Just because you can assign names … Simple linear regression Robust noise regression data { int<lower=0> N; vector[N] x; vector[N] y; } parameters { real alpha; real beta; real<lower=0> sigma; } model { real mu[N] = alpha + beta * x; y ~ normal(mu, sigma); } data { int<lower=0> N; vector[N] x; vector[N] y; } parameters { real alpha; real beta; real<lower=0> nu; real<lower=0> sigma; } model { real mu[N] = alpha + beta * x; y ~ student_t(nu, mu, sigma); } 8/23
9. Does not mean you should.. Linear regression Negative binomial regression data { int<lower=0> N; vector[N] x; vector[N] y; } parameters { real alpha; real beta; real<lower=0> sigma; } model { real mu[N] = alpha + beta * x; y ~ normal(mu, sigma); } data { int<lower=0> N; vector[N] x; vector[N] y; } parameters { real alpha; real beta; real<lower=0> sigma; } model { vector[N] mu = exp(x*beta+alpha); y ~ neg_binomial_2(mu, sigma); } 9/23
10. Pitfalls
11. Machine learning can only take us so far Why is that? Data: Data is not available in cardinality needed for many real world interesting applications Structure: Problem structure is hard to detect without domain knowledge Identifiability: For any given data set there are many possible models that fit really well to it with fundamentally different interpretations Priors: The ability to add prior knowledge about a problem is crucial as it is the only way to do science Uncertainty: Machine learning application based on maximum likelihood cannot express uncertainty about it's model · · · · · 11/23
12. The Bayesian brain Domain space Machine learning Inference p (x, y, θ) p (y|θ, x) p (θ|y, x) = p (y|θ, x) p (θ|x) ∫ p (y, θ|x) dθ 12/23
13. The importance of uncertainty It's my belief that uncertainty is a key missing piece in most AI applications today A probabilistic framework readily includes this It does not fix all kinds of uncertainties though! · · · 13/23
14. You cannot do science without assumption! “ 14/23
15. Enter the Datasaurus All datasets, and all frames of the animations, have the same summary statistics ( , , , , ). = 54.26 μx = 47.83 μy = 16.76 σ x = 26.93 σ y = −0.06 ρ x,y 15/23
16. Visualization matters! Seven distributions of data, shown as raw data points (or strip-plots), as box- plots, and as violin-plots. 16/23
17. So what's my point? The point is that these spurious patterns will be realized in most if not all neural networks and their representation of the reality they're trying to predict will be inherently wrong. Read the paper by Nguyen A, Yosinski J, Clune J 17/23
18. Further sillyness AI cannot read numbers 18/23
19. Further sillyness AI does not understand stop signs 19/23
20. A Remedy
21. If you live and breathe AI If you don't How to do AI today then? Build an in-house team to do Narrow AI solutions for you and your product Use a proper probabilistic framework Always take bigger interest in the failures than the success Quantify uncertainty Be honest and responsible Set up continuous learning · · · · · · Partner up with someone that actually knows what they're doing.. Make sure you don't pay for one off solution: An AI is a living thing Verify that you can deploy the solution where you need Be wary of the bullshitter effect Make sure the communication is technology agnostic · · · · · 21/23
22. Customer API Behind the scenes Desupervised Architecture 22/23
23. Take home messages Doing AI safely and correctly is hard Doing it is most likely a game changer for your products If you don't do it your business will be heading towards a tough time Familiarize yourself with the topic Socialize with people who are leaders in this field · · · · · 23/23