MACHINE LEARNING INTERPRETABILITY: WHY AND HOW!

With the increasing adoption of machine learning-based solutions in different domains, systems that use black-box algorithms are getting used more often with the promise of providing higher accuracy. However, this accuracy comes at the cost of interpretability, which introduces a barrier against wider adoption of such algorithms in crucial areas and raises the skepticism of the impacted individuals. This talk focuses on the importance of interpretable machine learning, why it is crucial from technical and ethical perspectives and its current limitations. In addition, it gives an overview of some of the relevant tools and packages (e.g. LIME, SHAPLEY).

Transcript

1. MACHINE LEARNING INTERPRETABILITY: WHY AND HOW! OMAYMA SAID Dec 2019
2. INTERPRETABLE ML/AI EXPLAINABLE ML/AI FEATURE ATTRIBUTIONS WHAT DO YOU KNOW ABOUT ?
3. • You will be given two labels with the corresponding mapping to 0/1. • You will be asked to classify some images. • You are a classifier. HOW WOULD YOUR MENTAL MODEL LABEL THIS IMAGE? RULES
4. CAT → 0 DOG → 1 OR HOW WOULD YOUR MENTAL MODEL LABEL THIS IMAGE?
5. CAT → 0 DOG → 1 OR HOW WOULD YOUR MENTAL MODEL LABEL THIS IMAGE?
6. DUCK → 0 RABBIT → 1 OR HOW WOULD YOUR MENTAL MODEL LABEL THIS IMAGE?
7. WHY ? HOW WOULD YOUR MENTAL MODEL LABEL THIS IMAGE?
8. https:/ /github.com/minimaxir/optillusion-animation Max Woolf
9. INTERPRETABILITY TRADEOFF COMPLEXITY
10. AI-POWERED [----] ML-ENABLED [----] MORE AND MORE
11. BUT LET’S FIRST EXCLUDE THE NONSENSE
12. “Uses AI to give you more insight into candidates, so you can make better decisions.” $#&T NONSENSE Source: Business Insider Video (2017) 25000 FEATURES → INSIGHT SCORE
14. $#&@!^&$$& !!!!!!!!!!!!!!!!!!!!! https:/ /www.faception.com/
15. HOW ABOUT THE OTHER APPLICATIONS?
20. WHAT IF IT WAS NOT GENDER OR A PROXY TO GENDER ?
21. WHY? TAMMY DOBBS, ARKANSAS REDUCED HOURS OF HOME CARE VISITS PER WEEK
22. TAMMY DOBBS, ARKANSAS WHY? REDUCED HOURS OF HOME CARE VISITS PER WEEK WHO IS ACCOUNTABLE?
23. COLLECT/LABEL DATA IT IS HUMANS WHO WRITE ALGORITHMS DEFINE METRICS
24. COLLECT/LABEL DATA IT IS HUMANS WHO BIAS IN: - REPRESENTATION - DISTRIBUTION - LABELS AND MORE….. WRITE ALGORITHMS DEFINE METRICS
25. IT IS HUMANS WHO DEFINE METRICS WRITE ALGORITHMS COLLECT/LABEL DATA - TRAIN/TEST SPLIT - FEATURES/PROXIES - MODEL COMPLEXITY AND MORE…..
26. IT IS HUMANS WHO COLLECT/LABEL DATA DEFINE METRICS WRITE ALGORITHMS - WHAT IS THE IMPACT OF DIFFERENT ERROR TYPES ON DIFFERENT GROUPS? - WHAT DO YOU OPTIMIZE FOR?
27. PRACTITIONERS CONSISTENTLY: - OVERESTIMATE THEIR MODEL’S ACCURACY. - PROPAGATE FEEDBACK LOOPS. - FAIL TO NOTICE DATA LEAKS. “ ” “Why Should I Trust You?” Explaining the Predictions of Any Classifier
28. MACHINE LEARNING INTERPRETABILITY: WHY?
29. TRANSPARENCY MACHINE LEARNING INTERPRETABILITY: WHY?
30. TRANSPARENCY DEBUGGING MODELS MACHINE LEARNING INTERPRETABILITY: WHY?
31. TRANSPARENCY DEBUGGING MODELS ROBUSTNESS MACHINE LEARNING INTERPRETABILITY: WHY? (e.g. Dealing with adversarial attacks)
32. TRANSPARENCY DEBUGGING MODELS ROBUSTNESS (e.g. Dealing with adversarial attacks) MACHINE LEARNING INTERPRETABILITY: WHY?
33. MACHINE LEARNING INTERPRETABILITY: HOW?
34. https:/ /github.com/marcotcr/lime Local Interpretable Model-agnostic Explanations LIME https:/ /github.com/thomasp85/lime
35. LIME Explain individual predictions of black-box models using a local interpretable model. “ ”
36. this documentary was boring and quite stupid….. I-Tabular Data II-Images III-Text LIME
37. 1- Select a point to explain (red). Based on an example in “Interpretable Machine Learning” Book by Christoph Molnar LIME (Tabular Data)
38. 2- Sample data points. LIME (Tabular Data) Based on an example in “Interpretable Machine Learning” Book by Christoph Molnar
39. 3- Weight points according to their proximity to the selected point. LIME (Tabular Data) Based on an example in “Interpretable Machine Learning” Book by Christoph Molnar
40. 4- Train a weighted, interpretable local model. LIME (Tabular Data) Based on an example in “Interpretable Machine Learning” Book by Christoph Molnar
41. 5- Explain the black-box model prediction using the local model. LIME (Tabular Data) Based on an example in “Interpretable Machine Learning” Book by Christoph Molnar
42. LIME (Tabular Data)
43. LOCAL → Unique explanation for each prediction LIME (Tabular Data)
44. LIME (Tabular Data)
45. Label: toy terrier Probability: 0.81 Explanation Fit: 0.38 LIME (Images) Original Model: pre-trained ImageNet model
46. LIME (Images) superpixels 50 100 150 Original Model: pre-trained ImageNet model
47. Label: tabby, tabby cat Probability: 0.29 Explanation Fit: 0.77 Label: Egyptian Cat Probability: 0.28 Explanation Fit: 0.69 LIME (Images) Original Model: pre-trained ImageNet model
48. Label: tabby, tabby cat Probability: 0.29 Explanation Fit: 0.77 Label: Egyptian Cat Probability: 0.28 Explanation Fit: 0.69 Type: Supports Type: Contradicts LIME (Images) Original Model: pre-trained ImageNet model
49. LIME (Images) “Why Should I Trust You?” Explaining the Predictions of Any Classifier https:/ /arxiv.org/pdf/1602.04938.pdf
50. LIME (Images) “Why Should I Trust You?” Explaining the Predictions of Any Classifier SNOW
51. IMDB reviews sentiment classification LIME (Text) Original Model: Keras model (CNN+LSTM)
52. Boring Stupid Dumb Waste information Label predicted: -ve sentiment LIME (Text) Original Model: Keras model (CNN+LSTM)
53. Pros LIME - Provides human-friendly explanations. - Gives a fidelity measure. - Can use other features than the black-box model.
54. Pros LIME Cons - Provides human-friendly explanations. - Gives a fidelity measure. - Can use other features than the original model. - The definition of proximity is not totally resolved in tabular data. - Instability of explanations.
55. Pros - Provides human-friendly explanations. - Gives a fidelity measure. - Can use other features than the original model. Cons - Instability of explanations. LIME - The definition of proximity is not totally resolved in tabular data.
56. SHAPLEY VALUES
57. SHAPLEY VALUES Explain the difference between the actual prediction and the average/baseline prediction of the black-box model. coalitional game theory “ ”
58. Pros - Solid theory. - The difference between the prediction and the average prediction is fairly distributed among the feature values of the instance. SHAPLEY VALUES
59. Pros Cons - Solid theory - The difference between the prediction and the average prediction is fairly distributed among the feature values of the instance. - Computationally expensive. - Can be misinterpreted. SHAPLEY VALUES - Uses all features (not ideal for explanations that contain few features).
60. Pros Cons - Solid theory. - The difference between the prediction and the average prediction is fairly distributed among the feature values of the instance. - Computationally expensive. - Can be misinterpreted. SHAPLEY VALUES - Uses all features (not ideal for explanations that contain few features).
61. https://github.com/slundberg/shap SHAP
62. Explainable AI for Trees: From Local Explanations to Global Understanding Bar chart (left) and SHAP summary plot (right) for a gradient boosted decision tree model trained on the mortality dataset. SHAP
63. https:/ /github.com/ModelOriented/DALEX https:/ /github.com/ModelOriented/DALEXtra
64. https://www.sicara.ai/blog/2019-07-31-tf-explain-interpretability-tensorflow
65. https://github.com/SeldonIO/alibi
67. https:/ /cloud.google.com/blog/products/ai-machine-learning/google-cloud-ai-explanations-to-increase-fairness-responsibility-and-trust
68. https:/ /cloud.google.com/ml-engine/docs/ai-explanations/overview AI Explanations for AI Platform
69. ADDED FEATURE ATTRIBUTIONS Google AI Explainability Whitepaper
71. • Deploy the model. • Log feature attributions for certain predictions (e.g. rare class). • Monitor model training/live skew. • Monitor attribution skew and focus on the most influential features. PREDICTION AUDITING AND MODEL MONITORING Explainable AI for Trees: From Local Explanations to Global Understanding Google AI Explainability Whitepaper
72. https:/ /cloud.google.com/ml-engine/docs/ai-explanations/limitations • Explanations are LOCAL (Each attribution only shows how much the feature affected the prediction for that particular example). • Explanations/Feature attributions are subject to adversarial attacks as predictions in complex models. • Explanations alone cannot tell if your model is fair, unbiased, or of sound quality • Different methods are just complementary tools to be combined with other approaches and the practitioners’ best judgement. • Explanations might be misinterpreted in some cases. LIMITATIONS Google AI Explainability Whitepaper Limitations of Interpretable Machine Learning Methods
73. EXTRA READINGS