MACHINE LEARNING INTERPRETABILITY: WHY AND HOW!

Slide 1

Slide 1 text

MACHINE LEARNING INTERPRETABILITY: WHY AND HOW! OMAYMA SAID

Slide 2

Slide 2 text

No content

Slide 3

Slide 3 text

INTERPRETABLE ML/AI EXPLAINABLE ML/AI FEATURE ATTRIBUTIONS WHAT DO YOU KNOW ABOUT ?

Slide 4

Slide 4 text

● You will be given two labels with the corresponding mapping to 0/1. ● You will be asked to classify some images. ● You are a classiﬁer. HOW WOULD YOUR MENTAL MODEL LABEL THIS IMAGE? RULES

Slide 5

Slide 5 text

CAT → 0 DOG → 1 OR HOW WOULD YOUR MENTAL MODEL LABEL THIS IMAGE?

Slide 6

Slide 6 text

CAT → 0 DOG → 1 OR HOW WOULD YOUR MENTAL MODEL LABEL THIS IMAGE?

Slide 7

Slide 7 text

DUCK → 0 RABBIT → 1 OR HOW WOULD YOUR MENTAL MODEL LABEL THIS IMAGE?

Slide 8

Slide 8 text

WHY ? HOW WOULD YOUR MENTAL MODEL LABEL THIS IMAGE?

Slide 9

Slide 9 text

https:/ /github.com/minimaxir/optillusion-animation Max Woolf

Slide 10

Slide 10 text

INTERPRETABILITY TRADEOFF COMPLEXITY

Slide 11

Slide 11 text

AI-POWERED [----] ML-ENABLED [----] MORE AND MORE

Slide 12

Slide 12 text

FIRST OF ALL LET’S EXCLUDE THE NONSENSE

Slide 13

Slide 13 text

“Uses AI to give you more insight into candidates, so you can make better decisions.” $#&T NONSENSE Source: Business Insider Video (2017) 25000 FEATURES → INSIGHT SCORE

Slide 14

Slide 14 text

No content

Slide 15

Slide 15 text

https:/ /twitter.com/kmlefranc/status/1223418818917060610

Slide 16

Slide 16 text

-ve +ve

Slide 17

Slide 17 text

$#&@!^&$$& !!!!!!!!!!!!!!!!!!!!! https:/ /www.faception.com/

Slide 18

Slide 18 text

HOW ABOUT OTHER APPLICATIONS?

Slide 19

Slide 19 text

No content

Slide 20

Slide 20 text

No content

Slide 21

Slide 21 text

No content

Slide 22

Slide 22 text

No content

Slide 23

Slide 23 text

WHAT IF IT WAS NOT GENDER OR A PROXY TO GENDER ?

Slide 24

Slide 24 text

WHY? TAMMY DOBBS, ARKANSAS REDUCED HOURS OF HOME CARE VISITS PER WEEK

Slide 25

Slide 25 text

TAMMY DOBBS, ARKANSAS WHY? REDUCED HOURS OF HOME CARE VISITS PER WEEK WHO IS ACCOUNTABLE?

Slide 26

Slide 26 text

COLLECT/LABEL DATA IT IS HUMANS WHO WRITE ALGORITHMS DEFINE METRICS

Slide 27

Slide 27 text

COLLECT/LABEL DATA IT IS HUMANS WHO BIAS IN: - REPRESENTATION - DISTRIBUTION - LABELS AND MORE….. WRITE ALGORITHMS DEFINE METRICS

Slide 28

Slide 28 text

IT IS HUMANS WHO DEFINE METRICS WRITE ALGORITHMS COLLECT/LABEL DATA - TRAIN/TEST SPLIT - FEATURES/PROXIES - MODEL COMPLEXITY AND MORE…..

Slide 29

Slide 29 text

IT IS HUMANS WHO COLLECT/LABEL DATA DEFINE METRICS WRITE ALGORITHMS - WHAT IS THE IMPACT OF DIFFERENT ERROR TYPES ON DIFFERENT GROUPS? - WHAT DO YOU OPTIMIZE FOR?

Slide 30

Slide 30 text

PRACTITIONERS CONSISTENTLY: - OVERESTIMATE THEIR MODEL’S ACCURACY. - PROPAGATE FEEDBACK LOOPS. - FAIL TO NOTICE DATA LEAKS. “ ” “Why Should I Trust You?” Explaining the Predictions of Any Classiﬁer

Slide 31

Slide 31 text

MACHINE LEARNING INTERPRETABILITY: WHY?

Slide 32

Slide 32 text

TRANSPARENCY MACHINE LEARNING INTERPRETABILITY: WHY?

Slide 33

Slide 33 text

TRANSPARENCY DEBUGGING MODELS MACHINE LEARNING INTERPRETABILITY: WHY?

Slide 34

Slide 34 text

TRANSPARENCY DEBUGGING MODELS ROBUSTNESS MACHINE LEARNING INTERPRETABILITY: WHY? (e.g. Dealing with adversarial attacks)

Slide 35

Slide 35 text

TRANSPARENCY DEBUGGING MODELS ROBUSTNESS (e.g. Dealing with adversarial attacks) MACHINE LEARNING INTERPRETABILITY: WHY?

Slide 36

Slide 36 text

MACHINE LEARNING INTERPRETABILITY: HOW?

Slide 37

Slide 37 text

https:/ /github.com/marcotcr/lime Local Interpretable Model-agnostic Explanations LIME https:/ /github.com/thomasp85/lime

Slide 38

Slide 38 text

LIME Explain individual predictions of black-box models using a local interpretable model. “ ”

Slide 39

Slide 39 text

this documentary was boring and quite stupid….. I-Tabular Data II-Images III-Text LIME

Slide 40

Slide 40 text

Label: toy terrier Probability: 0.81 Explanation Fit: 0.38 LIME (Images) Original Model: pre-trained ImageNet model

Slide 41

Slide 41 text

LIME (Images) superpixels 50 100 150 Original Model: pre-trained ImageNet model

Slide 42

Slide 42 text

Label: tabby, tabby cat Probability: 0.29 Explanation Fit: 0.77 Label: Egyptian Cat Probability: 0.28 Explanation Fit: 0.69 LIME (Images) Original Model: pre-trained ImageNet model

Slide 43

Slide 43 text

Label: tabby, tabby cat Probability: 0.29 Explanation Fit: 0.77 Label: Egyptian Cat Probability: 0.28 Explanation Fit: 0.69 Type: Supports Type: Contradicts LIME (Images) Original Model: pre-trained ImageNet model

Slide 44

Slide 44 text

LIME (Images) “Why Should I Trust You?” Explaining the Predictions of Any Classiﬁer https:/ /arxiv.org/pdf/1602.04938.pdf

Slide 45

Slide 45 text

LIME (Images) “Why Should I Trust You?” Explaining the Predictions of Any Classiﬁer SNOW

Slide 46

Slide 46 text

IMDB reviews sentiment classiﬁcation LIME (Text) Original Model: Keras model (CNN+LSTM)

Slide 47

Slide 47 text

Boring Stupid Dumb Waste information Label predicted: -ve sentiment LIME (Text) Original Model: Keras model (CNN+LSTM)

Slide 48

Slide 48 text

1- Select a point to explain (red). Based on an example in “Interpretable Machine Learning” Book by Christoph Molnar LIME (Tabular Data)

Slide 49

Slide 49 text

2- Sample data points. LIME (Tabular Data) Based on an example in “Interpretable Machine Learning” Book by Christoph Molnar

Slide 50

Slide 50 text

3- Weight points according to their proximity to the selected point. LIME (Tabular Data) Based on an example in “Interpretable Machine Learning” Book by Christoph Molnar

Slide 51

Slide 51 text

4- Train a weighted, interpretable local model. LIME (Tabular Data) Based on an example in “Interpretable Machine Learning” Book by Christoph Molnar

Slide 52

Slide 52 text

5- Explain the black-box model prediction using the local model. LIME (Tabular Data) Based on an example in “Interpretable Machine Learning” Book by Christoph Molnar

Slide 53

Slide 53 text

LIME (Tabular Data)

Slide 54

Slide 54 text

LOCAL → Unique explanation for each prediction LIME (Tabular Data)

Slide 55

Slide 55 text

LIME (Tabular Data)

Slide 56

Slide 56 text

Pros LIME - Provides human-friendly explanations. - Gives a ﬁdelity measure. - Can use other features than the black-box model.

Slide 57

Slide 57 text

Pros LIME Cons - Provides human-friendly explanations. - Gives a ﬁdelity measure. - Can use other features than the original model. - The deﬁnition of proximity is not totally resolved in tabular data. - Instability of explanations.

Slide 58

Slide 58 text

Pros - Provides human-friendly explanations. - Gives a ﬁdelity measure. - Can use other features than the original model. Cons - Instability of explanations. LIME - The deﬁnition of proximity is not totally resolved in tabular data.

Slide 59

Slide 59 text

SHAPLEY VALUES

Slide 60

Slide 60 text

SHAPLEY VALUES Explain the diﬀerence between the actual prediction and the average/baseline prediction of the black-box model. coalitional game theory “ ”

Slide 61

Slide 61 text

Pros - Solid theory. - The diﬀerence between the prediction and the average prediction is fairly distributed among the feature values of the instance. SHAPLEY VALUES

Slide 62

Slide 62 text

Pros Cons - Solid theory - The diﬀerence between the prediction and the average prediction is fairly distributed among the feature values of the instance. - Computationally expensive. - Can be misinterpreted. SHAPLEY VALUES - Uses all features (not ideal for explanations that contain few features).

Slide 63

Slide 63 text

Pros Cons - Solid theory. - The diﬀerence between the prediction and the average prediction is fairly distributed among the feature values of the instance. - Computationally expensive. - Can be misinterpreted. SHAPLEY VALUES - Uses all features (not ideal for explanations that contain few features).

Slide 64

Slide 64 text

https://github.com/slundberg/shap SHAP

Slide 65

Slide 65 text

Explainable AI for Trees: From Local Explanations to Global Understanding Bar chart (left) and SHAP summary plot (right) for a gradient boosted decision tree model trained on the mortality dataset. SHAP

Slide 66

Slide 66 text

Slide 67

Slide 67 text

Slide 68

Slide 68 text

https:/ /github.com/ModelOriented/DALEX https:/ /github.com/ModelOriented/DALEXtra

Slide 69

Slide 69 text

https://www.sicara.ai/blog/2019-07-31-tf-explain-interpretability-tensorflow

Slide 70

Slide 70 text

https://github.com/SeldonIO/alibi

Slide 71

Slide 71 text

No content

Slide 72

Slide 72 text

https:/ /cloud.google.com/blog/products/ai-machine-learning/google-cloud-ai-explanations-to-increase-fairness-responsibility-and-trust

Slide 73

Slide 73 text

https:/ /cloud.google.com/ml-engine/docs/ai-explanations/overview AI Explanations for AI Platform

Slide 74

Slide 74 text

ADDED FEATURE ATTRIBUTIONS Google AI Explainability Whitepaper

Slide 75

Slide 75 text

Slide 76

Slide 76 text

https:/ /cloud.google.com/ml-engine/docs/ai-explanations/limitations LIMITATIONS Google AI Explainability Whitepaper Limitations of Interpretable Machine Learning Methods Explanations are subject to adversarial attacks as predictions in complex models. Explanations are LOCAL (Each attribution only shows how much the feature aﬀected the prediction for that particular example).

Slide 77

Slide 77 text

https:/ /cloud.google.com/ml-engine/docs/ai-explanations/limitations Explanations are LOCAL (Each attribution only shows how much the feature aﬀected the prediction for that particular example). LIMITATIONS Google AI Explainability Whitepaper Limitations of Interpretable Machine Learning Methods Explanations are subject to adversarial attacks as predictions in complex models. Explanations alone cannot tell if your model is fair, unbiased, or of sound quality.

Slide 78

Slide 78 text

Slide 79

Slide 79 text

Explanations are LOCAL (Each attribution only shows how much the feature aﬀected the prediction for that particular example). LIMITATIONS Explanations are subject to adversarial attacks as predictions in complex models. Explanations alone cannot tell if your model is fair, unbiased, or of sound quality. Diﬀerent methods are just complementary tools to be combined with other approaches and the practitioners’ best judgement. https:/ /cloud.google.com/ml-engine/docs/ai-explanations/limitations Google AI Explainability Whitepaper Limitations of Interpretable Machine Learning Methods