Interpreting Machine Learning Models: Why and How!

Invited talk at SatRday Johannesburg #satRdayJoburg
https://joburg2019.satrdays.org/

Transcript

1. INTERPRETING MACHINE LEARNING MODELS: WHY AND HOW! OMAYMA SAID
3. HOW WOULD YOUR MENTAL MODEL LABEL THIS IMAGE? CAT → 0 DOG → 1 OR
4. HOW WOULD YOUR MENTAL MODEL LABEL THIS IMAGE? CAT → 0 DOG → 1 OR
5. HOW WOULD YOUR MENTAL MODEL LABEL THIS IMAGE? DUCK → 0 RABBIT → 1 OR
6. WHY ? HOW WOULD YOUR MENTAL MODEL LABEL THIS IMAGE?
7. https:/ /github.com/minimaxir/optillusion-animation Max Woolf
8. INTERPRETABILITY TRADEOFF COMPLEXITY
9. AI-POWERED [----] ML-ENABLED [----] MORE AND MORE
11. TAMMY DOBBS, Arkansas CEREBRAL PALSY REDUCED HOURS OF HOME CARE VISITS PER WEEK
12. TAMMY DOBBS, Arkansas CEREBRAL PALSY REDUCED HOURS OF HOME CARE VISITS PER WEEK WHY?
13. “ ” [In Idaho], the state declined to disclose the formula it was using, saying that its math qualified as a TRADE SECRET.
15. “AUTOMATED REDACTION, TRANSCRIPTION, REPORTING”
16. WHAT ELSE ? I'm In to Connect and Serve* “AUTOMATED REDACTION, TRANSCRIPTION, REPORTING”
18. Amazon’s system TAUGHT ITSELF that male candidates were preferable. It penalized resumes that included the word “women’s,” as in “women’s chess club captain.” And it downgraded graduates of two all-women’s colleges, according to people familiar with the matter. They did not specify the names of the schools. “
19. Amazon’s system TAUGHT ITSELF that male candidates were preferable. It penalized resumes that included the word “women’s,” as in “women’s chess club captain.” And it downgraded graduates of two all-women’s colleges, according to people familiar with the matter. They did not specify the names of the schools. “ LEARNED FROM HUMANS
20. COLLECT/LABEL DATA IT IS HUMANS WHO WRITE ALGORITHMS DEFINE METRICS
21. COLLECT/LABEL DATA IT IS HUMANS WHO BIAS IN: - REPRESENTATION - DISTRIBUTION - LABELS AND MORE….. WRITE ALGORITHMS DEFINE METRICS
22. IT IS HUMANS WHO DEFINE METRICS WRITE ALGORITHMS COLLECT/LABEL DATA - TRAIN/TEST SPLIT - FEATURES/PROXIES - BLACK-BOX MODELS AND MORE…..
23. IT IS HUMANS WHO COLLECT/LABEL DATA DEFINE METRICS WRITE ALGORITHMS - WHAT IS THE IMPACT OF DIFFERENT ERROR TYPES ON DIFFERENT GROUPS? - WHAT DO YOU OPTIMIZE FOR?
24. Practitioners consistently: - overestimate their model’s accuracy. - propagate feedback loops. - fail to notice data leaks. “ ” “Why Should I Trust You?” Explaining the Predictions of Any Classifier https:/ /arxiv.org/pdf/1602.04938.pdf
25. ETHICAL IMPLICATIONS INTERPRETABLE MACHINE LEARNING: WHY?
26. TRANSPARENCY INTERPRETABLE MACHINE LEARNING: WHY?
27. DEBUGGING MODELS INTERPRETABLE MACHINE LEARNING: WHY?
28. SECURITY (e.g. Dealing with adversarial attacks) INTERPRETABLE MACHINE LEARNING: WHY?
29. 1 LIME https:/ /github.com/thomasp85/lime Local Interpretable Model-agnostic Explanations iml https:/ /github.com/christophM/iml INTERPRETABLE MACHINE LEARNING: HOW?
30. LIME Explain individual predictions of black-box models using a local interpretable model. “ ”
31. this documentary was boring and quite stupid….. I-Tabular Data II-Images III-Text LIME
32. 1- Select a point to explain (red). Based on an example in “Interpretable Machine Learning” Book by Christoph Molnar LIME (Tabular Data)
33. 2- Sample data points. LIME (Tabular Data) Based on an example in “Interpretable Machine Learning” Book by Christoph Molnar
34. 3- Weight points according to their proximity to the selected point. LIME (Tabular Data) Based on an example in “Interpretable Machine Learning” Book by Christoph Molnar
35. 4- Train a weighted, interpretable local model. LIME (Tabular Data) Based on an example in “Interpretable Machine Learning” Book by Christoph Molnar
36. 5- Explain the black-box model prediction using the local model. LIME (Tabular Data) Based on an example in “Interpretable Machine Learning” Book by Christoph Molnar
37. LIME (Tabular Data) https:/ /github.com/OmaymaS/lime_explainer
38. LIME (Tabular Data) GUESS THE DATASET ?
39. LIME (Tabular Data) Iris (Classification)
40. set.seed(5658) ## load libraries library(caret) library(lime) ## partition the data intrain <- createDataPartition(y = iris$Species,p = 0.8, list = F) ## create train and test data train_data <- iris[intrain, ] test_data <- iris[-intrain, ] ## train Random Forest model on train_data model <- train(x = train_data[, 1:4], y = train_data[, 5], method = 'rf') TRAIN
41. set.seed(5658) ## load libraries library(caret) library(lime) ## partition the data intrain <- createDataPartition(y = iris$Species, p = 0.8, list = F) ## create train and test data train_data <- iris[intrain, ] test_data <- iris[-intrain, ] ## train Random Forest model on train_data model <- train(x = train_data[, 1:4], y = train_data[, 5], method = 'rf') ## create an explainer object using train_data explainer <- lime(train_data, model) EXPLAIN
42. set.seed(5658) ## load libraries library(caret) library(lime) ## partition the data intrain <- createDataPartition(y = iris$Species, p = 0.8, list = F) ## create train and test data train_data <- iris[intrain, ] test_data <- iris[-intrain, ] ## train Random Forest model on train_data model <- train(x = train_data[, 1:4], y = train_data[, 5], method = 'rf') ## create an explainer object using train_data explainer <- lime(train_data, model) ## explain new observations in test data explanation <- explain(test_data[, 1], explainer, n_labels = 1, n_features = 4) EXPLAIN
43. set.seed(5658) ## load libraries library(caret) library(lime) ## partition the data intrain <- createDataPartition(y = iris$Species, p = 0.8, list = F) ## create train and test data train_data <- iris[intrain, ] test_data <- iris[-intrain, ] ## train Random Forest model on train_data model <- train(x = train_data[, 1:4], y = train_data[, 5], method = 'rf') ## create an explainer object using train_data explainer <- lime(train_data, model) ## explain new observations in test data explanation <- explain(test_data[, 1:4], explainer, n_labels = 1, n_features = 4) https:/ /github.com/OmaymaS/satRday2019_talk_scripts/blob/master/R/lime_tabular.R
44. ## plot features plot_features(explanation) LIME (Tabular Data)
45. LIME (Tabular Data)
46. LIME (Tabular Data)
47. Label: toy terrier Probability: 0.81 Explanation Fit: 0.38 LIME (Images) pre-trained ImageNet model
48. LIME (Images) superpixels 50 100 150
49. Label: tabby, tabby cat Probability: 0.29 Explanation Fit: 0.77 Label: Egyptian Cat Probability: 0.28 Explanation Fit: 0.69 LIME (Images) pre-trained ImageNet model
50. Label: tabby, tabby cat Probability: 0.29 Explanation Fit: 0.77 Label: Egyptian Cat Probability: 0.28 Explanation Fit: 0.69 Type: Supports Type: Contradicts LIME (Images) pre-trained ImageNet model
51. LIME (Images) “Why Should I Trust You?” Explaining the Predictions of Any Classifier https:/ /arxiv.org/pdf/1602.04938.pdf
52. LIME (Images) “Why Should I Trust You?” Explaining the Predictions of Any Classifier https:/ /arxiv.org/pdf/1602.04938.pdf SNOW
53. IMDB reviews sentiment classification LIME (Text) Keras model (CNN+LSTM)
54. Boring Stupid Dumb Waste information Label predicted: -ve sentiment LIME (Text)
55. Pros LIME - Provides human-friendly explanations. - Gives a fidelity measure. - Can use other features than the black-box model.
56. Pros LIME Cons - Provides human-friendly explanations. - Gives a fidelity measure. - Can use other features than the original model. - The definition of proximity is not totally resolved in tabular data. - Instability of explanations.
57. Pros - Provides human-friendly explanations. - Gives a fidelity measure. - Can use other features than the original model. Cons - Instability of explanations. LIME - The definition of proximity is not totally resolved in tabular data.
58. 2 SHAPLEY VALUES iml https:/ /github.com/christophM/iml INTERPRETABLE MACHINE LEARNING: HOW?
59. SHAPLEY VALUES Explain the difference between the actual prediction and the average prediction of the black-box model. coalitional game theory “ ”
60. UCI Bike Sharing (Regression) SHAPLEY VALUES
61. library(tidyverse) library(caret) library(iml) ## partition the data intrain <- createDataPartition(y = bike$cnt, p = 0.9, list = F) ## create train and test data train_data <- bike[intrain, ] test_data <- bike[-intrain, ] train_x <- select(train_data, -cnt) test_x <- select(test_data, -cnt) ## train model model <- train(x = train_x, y = train_data$cnt, method = 'rf', ntree = 30, maximise = FALSE) TRAIN
62. library(tidyverse) library(caret) library(iml) ## partition the data intrain <- createDataPartition(y = bike$cnt, p = 0.9, list = F) ## create train and test data train_data <- bike[intrain, ] test_data <- bike[-intrain, ] train_x <- select(train_data, -cnt) test_x <- select(test_data, -cnt) ## train model model <- train(x = train_x, y = train_data$cnt, method = 'rf', ntree = 30, maximise = FALSE) ## create predictor predictor <- Predictor$new(model, data = train_x) EXPLAIN
63. library(tidyverse) library(caret) library(iml) ## partition the data intrain <- createDataPartition(y = bike$cnt, p = 0.9, list = F) ## create train and test data train_data <- bike[intrain, ] test_data <- bike[-intrain, ] train_x <- select(train_data, -cnt) test_x <- select(test_data, -cnt) ## train model model <- train(x = train_x, y = train_data$cnt, method = 'rf', ntree = 30, maximise = FALSE) ## create predictor predictor <- Predictor$new(model, data = train_x) ## calculate shapley values for a new instance shapley_values <- Shapley$new(predictor, x.interest = test_x[10,]) EXPLAIN
64. library(tidyverse) library(caret) library(iml) ## partition the data intrain <- createDataPartition(y = bike$cnt, p = 0.9, list = F) ## create train and test data train_data <- bike[intrain, ] test_data <- bike[-intrain, ] train_x <- select(train_data, -cnt) test_x <- select(test_data, -cnt) ## train model model <- train(x = train_x, y = train_data$cnt, method = 'rf', ntree = 30, maximise = FALSE) ## create predictor predictor <- Predictor$new(model, data = train_x) ## calculate shapley values for a new instance shapley_values <- Shapley$new(predictor, x.interest = new_istance) https:/ /github.com/OmaymaS/satRday2019_talk_scripts/blob/master/R/shapley_tabular.R
65. SHAPLEY VALUES
66. SHAPLEY VALUES The contribution of temp value (4.416) to the difference between the actual prediction and the mean prediction is the estimated Shapley value (~ -1000).
67. SHAPLEY VALUES
68. Pros - Solid theory. - The difference between the prediction and the average prediction is fairly distributed among the feature values of the instance. SHAPLEY VALUES
69. Pros Cons - Solid theory - The difference between the prediction and the average prediction is fairly distributed among the feature values of the instance. - Computationally expensive. - Can be misinterpreted. SHAPLEY VALUES - Uses all features (not ideal for explanations that contain few features).
70. Pros Cons - Solid theory. - The difference between the prediction and the average prediction is fairly distributed among the feature values of the instance. - Computationally expensive. - Can be misinterpreted. SHAPLEY VALUES - Uses all features (not ideal for explanations that contain few features).
71. > LIME > SHAPLEY VALUES > ANCHORS …. And more iml METHODS AND PACKAGES
72. EXTRA READINGS
74. INTERPRETING MACHINE LEARNING MODELS: WHY AND HOW! OMAYMA SAID