Interpreting Machine Learning Models: Why and How!

62321e5935c9c0731462b8178a7423f8?s=47 OmaymaS
April 06, 2019

Interpreting Machine Learning Models: Why and How!

Invited talk at SatRday Johannesburg #satRdayJoburg
https://joburg2019.satrdays.org/

62321e5935c9c0731462b8178a7423f8?s=128

OmaymaS

April 06, 2019
Tweet

Transcript

  1. INTERPRETING MACHINE LEARNING MODELS: WHY AND HOW! OMAYMA SAID

  2. None
  3. HOW WOULD YOUR MENTAL MODEL LABEL THIS IMAGE? CAT →

    0 DOG → 1 OR
  4. HOW WOULD YOUR MENTAL MODEL LABEL THIS IMAGE? CAT →

    0 DOG → 1 OR
  5. HOW WOULD YOUR MENTAL MODEL LABEL THIS IMAGE? DUCK →

    0 RABBIT → 1 OR
  6. WHY ? HOW WOULD YOUR MENTAL MODEL LABEL THIS IMAGE?

  7. https:/ /github.com/minimaxir/optillusion-animation Max Woolf

  8. INTERPRETABILITY TRADEOFF COMPLEXITY

  9. AI-POWERED [----] ML-ENABLED [----] MORE AND MORE

  10. None
  11. TAMMY DOBBS, Arkansas CEREBRAL PALSY REDUCED HOURS OF HOME CARE

    VISITS PER WEEK
  12. TAMMY DOBBS, Arkansas CEREBRAL PALSY REDUCED HOURS OF HOME CARE

    VISITS PER WEEK WHY?
  13. “ ” [In Idaho], the state declined to disclose the

    formula it was using, saying that its math qualified as a TRADE SECRET.
  14. None
  15. “AUTOMATED REDACTION, TRANSCRIPTION, REPORTING”

  16. WHAT ELSE ? I'm In to Connect and Serve* “AUTOMATED

    REDACTION, TRANSCRIPTION, REPORTING”
  17. None
  18. Amazon’s system TAUGHT ITSELF that male candidates were preferable. It

    penalized resumes that included the word “women’s,” as in “women’s chess club captain.” And it downgraded graduates of two all-women’s colleges, according to people familiar with the matter. They did not specify the names of the schools. “
  19. Amazon’s system TAUGHT ITSELF that male candidates were preferable. It

    penalized resumes that included the word “women’s,” as in “women’s chess club captain.” And it downgraded graduates of two all-women’s colleges, according to people familiar with the matter. They did not specify the names of the schools. “ LEARNED FROM HUMANS
  20. COLLECT/LABEL DATA IT IS HUMANS WHO WRITE ALGORITHMS DEFINE METRICS

  21. COLLECT/LABEL DATA IT IS HUMANS WHO BIAS IN: - REPRESENTATION

    - DISTRIBUTION - LABELS AND MORE….. WRITE ALGORITHMS DEFINE METRICS
  22. IT IS HUMANS WHO DEFINE METRICS WRITE ALGORITHMS COLLECT/LABEL DATA

    - TRAIN/TEST SPLIT - FEATURES/PROXIES - BLACK-BOX MODELS AND MORE…..
  23. IT IS HUMANS WHO COLLECT/LABEL DATA DEFINE METRICS WRITE ALGORITHMS

    - WHAT IS THE IMPACT OF DIFFERENT ERROR TYPES ON DIFFERENT GROUPS? - WHAT DO YOU OPTIMIZE FOR?
  24. Practitioners consistently: - overestimate their model’s accuracy. - propagate feedback

    loops. - fail to notice data leaks. “ ” “Why Should I Trust You?” Explaining the Predictions of Any Classifier https:/ /arxiv.org/pdf/1602.04938.pdf
  25. ETHICAL IMPLICATIONS INTERPRETABLE MACHINE LEARNING: WHY?

  26. TRANSPARENCY INTERPRETABLE MACHINE LEARNING: WHY?

  27. DEBUGGING MODELS INTERPRETABLE MACHINE LEARNING: WHY?

  28. SECURITY (e.g. Dealing with adversarial attacks) INTERPRETABLE MACHINE LEARNING: WHY?

  29. 1 LIME https:/ /github.com/thomasp85/lime Local Interpretable Model-agnostic Explanations iml https:/

    /github.com/christophM/iml INTERPRETABLE MACHINE LEARNING: HOW?
  30. LIME Explain individual predictions of black-box models using a local

    interpretable model. “ ”
  31. this documentary was boring and quite stupid….. I-Tabular Data II-Images

    III-Text LIME
  32. 1- Select a point to explain (red). Based on an

    example in “Interpretable Machine Learning” Book by Christoph Molnar LIME (Tabular Data)
  33. 2- Sample data points. LIME (Tabular Data) Based on an

    example in “Interpretable Machine Learning” Book by Christoph Molnar
  34. 3- Weight points according to their proximity to the selected

    point. LIME (Tabular Data) Based on an example in “Interpretable Machine Learning” Book by Christoph Molnar
  35. 4- Train a weighted, interpretable local model. LIME (Tabular Data)

    Based on an example in “Interpretable Machine Learning” Book by Christoph Molnar
  36. 5- Explain the black-box model prediction using the local model.

    LIME (Tabular Data) Based on an example in “Interpretable Machine Learning” Book by Christoph Molnar
  37. LIME (Tabular Data) https:/ /github.com/OmaymaS/lime_explainer

  38. LIME (Tabular Data) GUESS THE DATASET ?

  39. LIME (Tabular Data) Iris (Classification)

  40. set.seed(5658) ## load libraries library(caret) library(lime) ## partition the data

    intrain <- createDataPartition(y = iris$Species,p = 0.8, list = F) ## create train and test data train_data <- iris[intrain, ] test_data <- iris[-intrain, ] ## train Random Forest model on train_data model <- train(x = train_data[, 1:4], y = train_data[, 5], method = 'rf') TRAIN
  41. set.seed(5658) ## load libraries library(caret) library(lime) ## partition the data

    intrain <- createDataPartition(y = iris$Species, p = 0.8, list = F) ## create train and test data train_data <- iris[intrain, ] test_data <- iris[-intrain, ] ## train Random Forest model on train_data model <- train(x = train_data[, 1:4], y = train_data[, 5], method = 'rf') ## create an explainer object using train_data explainer <- lime(train_data, model) EXPLAIN
  42. set.seed(5658) ## load libraries library(caret) library(lime) ## partition the data

    intrain <- createDataPartition(y = iris$Species, p = 0.8, list = F) ## create train and test data train_data <- iris[intrain, ] test_data <- iris[-intrain, ] ## train Random Forest model on train_data model <- train(x = train_data[, 1:4], y = train_data[, 5], method = 'rf') ## create an explainer object using train_data explainer <- lime(train_data, model) ## explain new observations in test data explanation <- explain(test_data[, 1], explainer, n_labels = 1, n_features = 4) EXPLAIN
  43. set.seed(5658) ## load libraries library(caret) library(lime) ## partition the data

    intrain <- createDataPartition(y = iris$Species, p = 0.8, list = F) ## create train and test data train_data <- iris[intrain, ] test_data <- iris[-intrain, ] ## train Random Forest model on train_data model <- train(x = train_data[, 1:4], y = train_data[, 5], method = 'rf') ## create an explainer object using train_data explainer <- lime(train_data, model) ## explain new observations in test data explanation <- explain(test_data[, 1:4], explainer, n_labels = 1, n_features = 4) https:/ /github.com/OmaymaS/satRday2019_talk_scripts/blob/master/R/lime_tabular.R
  44. ## plot features plot_features(explanation) LIME (Tabular Data)

  45. LIME (Tabular Data)

  46. LIME (Tabular Data)

  47. Label: toy terrier Probability: 0.81 Explanation Fit: 0.38 LIME (Images)

    pre-trained ImageNet model
  48. LIME (Images) superpixels 50 100 150

  49. Label: tabby, tabby cat Probability: 0.29 Explanation Fit: 0.77 Label:

    Egyptian Cat Probability: 0.28 Explanation Fit: 0.69 LIME (Images) pre-trained ImageNet model
  50. Label: tabby, tabby cat Probability: 0.29 Explanation Fit: 0.77 Label:

    Egyptian Cat Probability: 0.28 Explanation Fit: 0.69 Type: Supports Type: Contradicts LIME (Images) pre-trained ImageNet model
  51. LIME (Images) “Why Should I Trust You?” Explaining the Predictions

    of Any Classifier https:/ /arxiv.org/pdf/1602.04938.pdf
  52. LIME (Images) “Why Should I Trust You?” Explaining the Predictions

    of Any Classifier https:/ /arxiv.org/pdf/1602.04938.pdf SNOW
  53. IMDB reviews sentiment classification LIME (Text) Keras model (CNN+LSTM)

  54. Boring Stupid Dumb Waste information Label predicted: -ve sentiment LIME

    (Text)
  55. Pros LIME - Provides human-friendly explanations. - Gives a fidelity

    measure. - Can use other features than the black-box model.
  56. Pros LIME Cons - Provides human-friendly explanations. - Gives a

    fidelity measure. - Can use other features than the original model. - The definition of proximity is not totally resolved in tabular data. - Instability of explanations.
  57. Pros - Provides human-friendly explanations. - Gives a fidelity measure.

    - Can use other features than the original model. Cons - Instability of explanations. LIME - The definition of proximity is not totally resolved in tabular data.
  58. 2 SHAPLEY VALUES iml https:/ /github.com/christophM/iml INTERPRETABLE MACHINE LEARNING: HOW?

  59. SHAPLEY VALUES Explain the difference between the actual prediction and

    the average prediction of the black-box model. coalitional game theory “ ”
  60. UCI Bike Sharing (Regression) SHAPLEY VALUES

  61. library(tidyverse) library(caret) library(iml) ## partition the data intrain <- createDataPartition(y

    = bike$cnt, p = 0.9, list = F) ## create train and test data train_data <- bike[intrain, ] test_data <- bike[-intrain, ] train_x <- select(train_data, -cnt) test_x <- select(test_data, -cnt) ## train model model <- train(x = train_x, y = train_data$cnt, method = 'rf', ntree = 30, maximise = FALSE) TRAIN
  62. library(tidyverse) library(caret) library(iml) ## partition the data intrain <- createDataPartition(y

    = bike$cnt, p = 0.9, list = F) ## create train and test data train_data <- bike[intrain, ] test_data <- bike[-intrain, ] train_x <- select(train_data, -cnt) test_x <- select(test_data, -cnt) ## train model model <- train(x = train_x, y = train_data$cnt, method = 'rf', ntree = 30, maximise = FALSE) ## create predictor predictor <- Predictor$new(model, data = train_x) EXPLAIN
  63. library(tidyverse) library(caret) library(iml) ## partition the data intrain <- createDataPartition(y

    = bike$cnt, p = 0.9, list = F) ## create train and test data train_data <- bike[intrain, ] test_data <- bike[-intrain, ] train_x <- select(train_data, -cnt) test_x <- select(test_data, -cnt) ## train model model <- train(x = train_x, y = train_data$cnt, method = 'rf', ntree = 30, maximise = FALSE) ## create predictor predictor <- Predictor$new(model, data = train_x) ## calculate shapley values for a new instance shapley_values <- Shapley$new(predictor, x.interest = test_x[10,]) EXPLAIN
  64. library(tidyverse) library(caret) library(iml) ## partition the data intrain <- createDataPartition(y

    = bike$cnt, p = 0.9, list = F) ## create train and test data train_data <- bike[intrain, ] test_data <- bike[-intrain, ] train_x <- select(train_data, -cnt) test_x <- select(test_data, -cnt) ## train model model <- train(x = train_x, y = train_data$cnt, method = 'rf', ntree = 30, maximise = FALSE) ## create predictor predictor <- Predictor$new(model, data = train_x) ## calculate shapley values for a new instance shapley_values <- Shapley$new(predictor, x.interest = new_istance) https:/ /github.com/OmaymaS/satRday2019_talk_scripts/blob/master/R/shapley_tabular.R
  65. SHAPLEY VALUES

  66. SHAPLEY VALUES The contribution of temp value (4.416) to the

    difference between the actual prediction and the mean prediction is the estimated Shapley value (~ -1000).
  67. SHAPLEY VALUES

  68. Pros - Solid theory. - The difference between the prediction

    and the average prediction is fairly distributed among the feature values of the instance. SHAPLEY VALUES
  69. Pros Cons - Solid theory - The difference between the

    prediction and the average prediction is fairly distributed among the feature values of the instance. - Computationally expensive. - Can be misinterpreted. SHAPLEY VALUES - Uses all features (not ideal for explanations that contain few features).
  70. Pros Cons - Solid theory. - The difference between the

    prediction and the average prediction is fairly distributed among the feature values of the instance. - Computationally expensive. - Can be misinterpreted. SHAPLEY VALUES - Uses all features (not ideal for explanations that contain few features).
  71. > LIME > SHAPLEY VALUES > ANCHORS …. And more

    iml METHODS AND PACKAGES
  72. EXTRA READINGS

  73. None
  74. INTERPRETING MACHINE LEARNING MODELS: WHY AND HOW! OMAYMA SAID