[PyCon JP 2018] Interpretable Machine Learning, making black box models explainable with Python!

Ever wonder how does a Machine Learning model make predictions? In particularly a 256-layers deep neural network, how does it distinguish a Corgi from a Husky puppy? Come to my talk and I’ll enlighten you by demystifying black-box ML models with some Python magic!

Machine learning models are increasingly complex due to the advancements of model architectures such as deep neural networks and ensemble models. While these sophisticated models have achieved higher accuracy, they are like black boxes which how the decision was made couldn’t be fully understood. There are potential risks of misrepresentation, discrimination or overfitting. Furthermore, the need of interpretability is crucial to gain the trusts of regulators and users towards ML models.

Transcript

1. Interpretable Machine Learning: Making black-box models explainable with Python! David Low Co-founder / CDS
2. Bio • • • • •
3. • • ◦ ◦ ◦ • ◦ ◦ Overview
4. Machine Learning is everywhere! Source: Nvidia
5. Source: xkcd
6. Is there a need to explain ML model? • Safety Make sure the system is making sound decisions. • Debugging Understand why a system doesn't work, so we can fix that. • Science Enable new discovery. • Mismatched Objectives and multi-objectives trade-offs: The system may not be optimizing the true objective. • Legal / Ethics: Legally required to provide an explanation and/or avoid discriminate against particular groups due to bias in data.
7. EU General Data Protection Regulation (GDPR) • Article 15 and 22 from GDPR suggest that: Customer/User has the rights to request for information/explanation pertaining to decision made by the automated system. • Such a right would enable people to ask how a specific decision (e.g. being declined insurance or being denied a promotion) was reached.
8. A highly accurate model to classify Wolf and Husky
9. A highly accurate model BUT... • Does the model learn the RIGHT things? The answer is “NO”. Instead of learning the appearance feature of the dogs, the model picks up the signal from the background (Snow in, this case)
10. Machine Learning Model • ML model is a function that takes the input and produce output f(x) x (Data) y (Output)
11. Complexity of learned function 1. Linear + Monotonic function 2. Non-linear + Monotonic function 3. Non-linear + Non-monotonic function Increased complexity, hence harder to interpret... Eg of Linear model: • Linear regression, Logistic Regression, Naives Bayes... Eg of Non-linear model • Neural Network, Tree-based (Random Forest, Gradient Boosting)...
12. Monotonicity Non-monotonic x y Monotonic x y
13. Scope of Interpretability • Global Interpretability ◦ How do parts of the model influence predictions? • Local Interpretability ◦ Why did the model make a specific decision for an instance? ◦ What factors contributed to a particular decision impacting a specific person
14. Approaches • Interpretable Models • Model-Agnostic Methods ◦ Partial Dependence Plot (PDP) ◦ Individual Conditional Expectation (ICE) ◦ Feature Importance ◦ Surrogate Models
15. Partial Dependence Plot (PDP) I • Shows the marginal effect of a feature on the predicted outcome of a previously fit model • Steps ◦ Select a feature: Temperature ◦ Identify a list of value for that feature: 0 to 35 Celcius ◦ Iterate over the list ▪ Replace Temperature one value at a time ▪ Take the average of the prediction outputs ◦ Repeat it for other features
16. Partial Dependence Plot (PDP) II
17. Visualize in 3D
18. Individual Conditional Expectations (ICE) • Visualizes the dependence of the predicted response on a feature for EACH instance separately, resulting in multiple lines, one for each instance.
19. • Determined by the changes in the model’s prediction error after permuting the feature’s values • Steps ◦ For each feature ▪ Replace the selected feature with noise (random) values ▪ Measure the changes in prediction error • Important feature → Decline in accuracy Less important feature → No changes / Increase of accuracy Feature Importance I
20. Feature Importance II
21. Local Interpretable Model-Agnostic Explanations (LIME) • Local surrogate models that can explain single predictions of any black-box machine learning model • Surrogate models are interpretable models (Linear mode / Decision Tree) that are learned on the predictions of the original black box model. • How it works ◦ Generate a artificial dataset from the example we’re going to explain. ◦ Use original model to get target values for each example in a generated dataset ◦ Train a new interpretable model, using generated dataset and generated labels as training data. ◦ Explain the original example through weights/rules of this new model. *Prediction quality of a white-box classifier shows how well it approximates the original model. If the quality is low then explanation shouldn’t be trusted.
22. No Perfect Solution • • ◦ ▪
23. Do it with Python
24. Libraries • Scikit-Learn ◦ https://scikit-learn.org/ • Local Interpretable Model-Agnostic Explanations (LIME) ◦ https://github.com/marcotcr/lime • ELI5 ◦ https://github.com/TeamHG-Memex/eli5
26. Dog Breeds Classification • Stanford Dogs Dataset (subset of ImageNet) ◦ http://vision.stanford.edu/aditya86/ImageNetDogs/ • Summary ◦ 120 dog breeds ◦ Around 150 images per class ◦ In total, 20580 images
27. DEMO
28. Further readings • ◦ • ◦ • ◦
29. We’re hiring Front-end Developer and SW Engineer! Send your CV to david@pand.ai