MACHINE LEARNING INTERPRETABILITY: WHY AND HOW!

[Invited Talk]
Düsseldorf Data Science Meetup
Feb 2020

Transcript

1. MACHINE LEARNING INTERPRETABILITY: WHY AND HOW! OMAYMA SAID
3. INTERPRETABLE ML/AI EXPLAINABLE ML/AI FEATURE ATTRIBUTIONS WHAT DO YOU KNOW ABOUT ?
4. • You will be given two labels with the corresponding mapping to 0/1. • You will be asked to classify some images. • You are a classifier. HOW WOULD YOUR MENTAL MODEL LABEL THIS IMAGE? RULES
5. CAT → 0 DOG → 1 OR HOW WOULD YOUR MENTAL MODEL LABEL THIS IMAGE?
6. CAT → 0 DOG → 1 OR HOW WOULD YOUR MENTAL MODEL LABEL THIS IMAGE?
7. DUCK → 0 RABBIT → 1 OR HOW WOULD YOUR MENTAL MODEL LABEL THIS IMAGE?
8. WHY ? HOW WOULD YOUR MENTAL MODEL LABEL THIS IMAGE?
9. https:/ /github.com/minimaxir/optillusion-animation Max Woolf
10. INTERPRETABILITY TRADEOFF COMPLEXITY
11. AI-POWERED [----] ML-ENABLED [----] MORE AND MORE
12. FIRST OF ALL LET’S EXCLUDE THE NONSENSE
13. “Uses AI to give you more insight into candidates, so you can make better decisions.” $#&T NONSENSE Source: Business Insider Video (2017) 25000 FEATURES → INSIGHT SCORE
15. https:/ /twitter.com/kmlefranc/status/1223418818917060610
16. -ve +ve
17. $#&@!^&$$& !!!!!!!!!!!!!!!!!!!!! https:/ /www.faception.com/
18. HOW ABOUT OTHER APPLICATIONS?
23. WHAT IF IT WAS NOT GENDER OR A PROXY TO GENDER ?
24. WHY? TAMMY DOBBS, ARKANSAS REDUCED HOURS OF HOME CARE VISITS PER WEEK
25. TAMMY DOBBS, ARKANSAS WHY? REDUCED HOURS OF HOME CARE VISITS PER WEEK WHO IS ACCOUNTABLE?
26. COLLECT/LABEL DATA IT IS HUMANS WHO WRITE ALGORITHMS DEFINE METRICS
27. COLLECT/LABEL DATA IT IS HUMANS WHO BIAS IN: - REPRESENTATION - DISTRIBUTION - LABELS AND MORE….. WRITE ALGORITHMS DEFINE METRICS
28. IT IS HUMANS WHO DEFINE METRICS WRITE ALGORITHMS COLLECT/LABEL DATA - TRAIN/TEST SPLIT - FEATURES/PROXIES - MODEL COMPLEXITY AND MORE…..
29. IT IS HUMANS WHO COLLECT/LABEL DATA DEFINE METRICS WRITE ALGORITHMS - WHAT IS THE IMPACT OF DIFFERENT ERROR TYPES ON DIFFERENT GROUPS? - WHAT DO YOU OPTIMIZE FOR?
30. PRACTITIONERS CONSISTENTLY: - OVERESTIMATE THEIR MODEL’S ACCURACY. - PROPAGATE FEEDBACK LOOPS. - FAIL TO NOTICE DATA LEAKS. “ ” “Why Should I Trust You?” Explaining the Predictions of Any Classifier
31. MACHINE LEARNING INTERPRETABILITY: WHY?
32. TRANSPARENCY MACHINE LEARNING INTERPRETABILITY: WHY?
33. TRANSPARENCY DEBUGGING MODELS MACHINE LEARNING INTERPRETABILITY: WHY?
34. TRANSPARENCY DEBUGGING MODELS ROBUSTNESS MACHINE LEARNING INTERPRETABILITY: WHY? (e.g. Dealing with adversarial attacks)
35. TRANSPARENCY DEBUGGING MODELS ROBUSTNESS (e.g. Dealing with adversarial attacks) MACHINE LEARNING INTERPRETABILITY: WHY?
36. MACHINE LEARNING INTERPRETABILITY: HOW?
37. https:/ /github.com/marcotcr/lime Local Interpretable Model-agnostic Explanations LIME https:/ /github.com/thomasp85/lime
38. LIME Explain individual predictions of black-box models using a local interpretable model. “ ”
39. this documentary was boring and quite stupid….. I-Tabular Data II-Images III-Text LIME
40. Label: toy terrier Probability: 0.81 Explanation Fit: 0.38 LIME (Images) Original Model: pre-trained ImageNet model
41. LIME (Images) superpixels 50 100 150 Original Model: pre-trained ImageNet model
42. Label: tabby, tabby cat Probability: 0.29 Explanation Fit: 0.77 Label: Egyptian Cat Probability: 0.28 Explanation Fit: 0.69 LIME (Images) Original Model: pre-trained ImageNet model
43. Label: tabby, tabby cat Probability: 0.29 Explanation Fit: 0.77 Label: Egyptian Cat Probability: 0.28 Explanation Fit: 0.69 Type: Supports Type: Contradicts LIME (Images) Original Model: pre-trained ImageNet model
44. LIME (Images) “Why Should I Trust You?” Explaining the Predictions of Any Classifier https:/ /arxiv.org/pdf/1602.04938.pdf
45. LIME (Images) “Why Should I Trust You?” Explaining the Predictions of Any Classifier SNOW
46. IMDB reviews sentiment classification LIME (Text) Original Model: Keras model (CNN+LSTM)
47. Boring Stupid Dumb Waste information Label predicted: -ve sentiment LIME (Text) Original Model: Keras model (CNN+LSTM)
48. 1- Select a point to explain (red). Based on an example in “Interpretable Machine Learning” Book by Christoph Molnar LIME (Tabular Data)
49. 2- Sample data points. LIME (Tabular Data) Based on an example in “Interpretable Machine Learning” Book by Christoph Molnar
50. 3- Weight points according to their proximity to the selected point. LIME (Tabular Data) Based on an example in “Interpretable Machine Learning” Book by Christoph Molnar
51. 4- Train a weighted, interpretable local model. LIME (Tabular Data) Based on an example in “Interpretable Machine Learning” Book by Christoph Molnar
52. 5- Explain the black-box model prediction using the local model. LIME (Tabular Data) Based on an example in “Interpretable Machine Learning” Book by Christoph Molnar
53. LIME (Tabular Data)
54. LOCAL → Unique explanation for each prediction LIME (Tabular Data)
55. LIME (Tabular Data)
56. Pros LIME - Provides human-friendly explanations. - Gives a fidelity measure. - Can use other features than the black-box model.
57. Pros LIME Cons - Provides human-friendly explanations. - Gives a fidelity measure. - Can use other features than the original model. - The definition of proximity is not totally resolved in tabular data. - Instability of explanations.
58. Pros - Provides human-friendly explanations. - Gives a fidelity measure. - Can use other features than the original model. Cons - Instability of explanations. LIME - The definition of proximity is not totally resolved in tabular data.
59. SHAPLEY VALUES
60. SHAPLEY VALUES Explain the difference between the actual prediction and the average/baseline prediction of the black-box model. coalitional game theory “ ”
61. Pros - Solid theory. - The difference between the prediction and the average prediction is fairly distributed among the feature values of the instance. SHAPLEY VALUES
62. Pros Cons - Solid theory - The difference between the prediction and the average prediction is fairly distributed among the feature values of the instance. - Computationally expensive. - Can be misinterpreted. SHAPLEY VALUES - Uses all features (not ideal for explanations that contain few features).
63. Pros Cons - Solid theory. - The difference between the prediction and the average prediction is fairly distributed among the feature values of the instance. - Computationally expensive. - Can be misinterpreted. SHAPLEY VALUES - Uses all features (not ideal for explanations that contain few features).
64. https://github.com/slundberg/shap SHAP
65. Explainable AI for Trees: From Local Explanations to Global Understanding Bar chart (left) and SHAP summary plot (right) for a gradient boosted decision tree model trained on the mortality dataset. SHAP
66. Explainable AI for Trees: From Local Explanations to Global Understanding Bar chart (left) and SHAP summary plot (right) for a gradient boosted decision tree model trained on the mortality dataset. SHAP
67. Explainable AI for Trees: From Local Explanations to Global Understanding Bar chart (left) and SHAP summary plot (right) for a gradient boosted decision tree model trained on the mortality dataset. SHAP
68. https:/ /github.com/ModelOriented/DALEX https:/ /github.com/ModelOriented/DALEXtra
69. https://www.sicara.ai/blog/2019-07-31-tf-explain-interpretability-tensorflow
70. https://github.com/SeldonIO/alibi
72. https:/ /cloud.google.com/blog/products/ai-machine-learning/google-cloud-ai-explanations-to-increase-fairness-responsibility-and-trust
73. https:/ /cloud.google.com/ml-engine/docs/ai-explanations/overview AI Explanations for AI Platform
74. ADDED FEATURE ATTRIBUTIONS Google AI Explainability Whitepaper
75. https:/ /cloud.google.com/ml-engine/docs/ai-explanations/limitations Explanations are LOCAL (Each attribution only shows how much the feature affected the prediction for that particular example). LIMITATIONS Google AI Explainability Whitepaper Limitations of Interpretable Machine Learning Methods
76. https:/ /cloud.google.com/ml-engine/docs/ai-explanations/limitations LIMITATIONS Google AI Explainability Whitepaper Limitations of Interpretable Machine Learning Methods Explanations are subject to adversarial attacks as predictions in complex models. Explanations are LOCAL (Each attribution only shows how much the feature affected the prediction for that particular example).
77. https:/ /cloud.google.com/ml-engine/docs/ai-explanations/limitations Explanations are LOCAL (Each attribution only shows how much the feature affected the prediction for that particular example). LIMITATIONS Google AI Explainability Whitepaper Limitations of Interpretable Machine Learning Methods Explanations are subject to adversarial attacks as predictions in complex models. Explanations alone cannot tell if your model is fair, unbiased, or of sound quality.
78. https:/ /cloud.google.com/ml-engine/docs/ai-explanations/limitations Explanations are LOCAL (Each attribution only shows how much the feature affected the prediction for that particular example). LIMITATIONS Google AI Explainability Whitepaper Limitations of Interpretable Machine Learning Methods Explanations are subject to adversarial attacks as predictions in complex models. Explanations alone cannot tell if your model is fair, unbiased, or of sound quality. Different methods are just complementary tools to be combined with the practitioners’ best judgement.
79. Explanations are LOCAL (Each attribution only shows how much the feature affected the prediction for that particular example). LIMITATIONS Explanations are subject to adversarial attacks as predictions in complex models. Explanations alone cannot tell if your model is fair, unbiased, or of sound quality. Different methods are just complementary tools to be combined with other approaches and the practitioners’ best judgement. https:/ /cloud.google.com/ml-engine/docs/ai-explanations/limitations Google AI Explainability Whitepaper Limitations of Interpretable Machine Learning Methods
80. EXTRA READINGS
81. EXTRA READINGS
82. EXTRA READINGS
83. EXTRA READINGS
84. MACHINE LEARNING INTERPRETABILITY: WHY AND HOW! OMAYMA SAID Feb 2020