Upgrade to Pro — share decks privately, control downloads, hide ads and more …

MACHINE LEARNING INTERPRETABILITY: WHY AND HOW!

62321e5935c9c0731462b8178a7423f8?s=47 OmaymaS
February 27, 2020

MACHINE LEARNING INTERPRETABILITY: WHY AND HOW!

[Invited Talk]
Düsseldorf Data Science Meetup
Feb 2020

62321e5935c9c0731462b8178a7423f8?s=128

OmaymaS

February 27, 2020
Tweet

Transcript

  1. MACHINE LEARNING INTERPRETABILITY: WHY AND HOW! OMAYMA SAID

  2. None
  3. INTERPRETABLE ML/AI EXPLAINABLE ML/AI FEATURE ATTRIBUTIONS WHAT DO YOU KNOW

    ABOUT ?
  4. • You will be given two labels with the corresponding

    mapping to 0/1. • You will be asked to classify some images. • You are a classifier. HOW WOULD YOUR MENTAL MODEL LABEL THIS IMAGE? RULES
  5. CAT → 0 DOG → 1 OR HOW WOULD YOUR

    MENTAL MODEL LABEL THIS IMAGE?
  6. CAT → 0 DOG → 1 OR HOW WOULD YOUR

    MENTAL MODEL LABEL THIS IMAGE?
  7. DUCK → 0 RABBIT → 1 OR HOW WOULD YOUR

    MENTAL MODEL LABEL THIS IMAGE?
  8. WHY ? HOW WOULD YOUR MENTAL MODEL LABEL THIS IMAGE?

  9. https:/ /github.com/minimaxir/optillusion-animation Max Woolf

  10. INTERPRETABILITY TRADEOFF COMPLEXITY

  11. AI-POWERED [----] ML-ENABLED [----] MORE AND MORE

  12. FIRST OF ALL LET’S EXCLUDE THE NONSENSE

  13. “Uses AI to give you more insight into candidates, so

    you can make better decisions.” $#&T NONSENSE Source: Business Insider Video (2017) 25000 FEATURES → INSIGHT SCORE
  14. None
  15. https:/ /twitter.com/kmlefranc/status/1223418818917060610

  16. -ve +ve

  17. $#&@!^&$$& !!!!!!!!!!!!!!!!!!!!! https:/ /www.faception.com/

  18. HOW ABOUT OTHER APPLICATIONS?

  19. None
  20. None
  21. None
  22. None
  23. WHAT IF IT WAS NOT GENDER OR A PROXY TO

    GENDER ?
  24. WHY? TAMMY DOBBS, ARKANSAS REDUCED HOURS OF HOME CARE VISITS

    PER WEEK
  25. TAMMY DOBBS, ARKANSAS WHY? REDUCED HOURS OF HOME CARE VISITS

    PER WEEK WHO IS ACCOUNTABLE?
  26. COLLECT/LABEL DATA IT IS HUMANS WHO WRITE ALGORITHMS DEFINE METRICS

  27. COLLECT/LABEL DATA IT IS HUMANS WHO BIAS IN: - REPRESENTATION

    - DISTRIBUTION - LABELS AND MORE….. WRITE ALGORITHMS DEFINE METRICS
  28. IT IS HUMANS WHO DEFINE METRICS WRITE ALGORITHMS COLLECT/LABEL DATA

    - TRAIN/TEST SPLIT - FEATURES/PROXIES - MODEL COMPLEXITY AND MORE…..
  29. IT IS HUMANS WHO COLLECT/LABEL DATA DEFINE METRICS WRITE ALGORITHMS

    - WHAT IS THE IMPACT OF DIFFERENT ERROR TYPES ON DIFFERENT GROUPS? - WHAT DO YOU OPTIMIZE FOR?
  30. PRACTITIONERS CONSISTENTLY: - OVERESTIMATE THEIR MODEL’S ACCURACY. - PROPAGATE FEEDBACK

    LOOPS. - FAIL TO NOTICE DATA LEAKS. “ ” “Why Should I Trust You?” Explaining the Predictions of Any Classifier
  31. MACHINE LEARNING INTERPRETABILITY: WHY?

  32. TRANSPARENCY MACHINE LEARNING INTERPRETABILITY: WHY?

  33. TRANSPARENCY DEBUGGING MODELS MACHINE LEARNING INTERPRETABILITY: WHY?

  34. TRANSPARENCY DEBUGGING MODELS ROBUSTNESS MACHINE LEARNING INTERPRETABILITY: WHY? (e.g. Dealing

    with adversarial attacks)
  35. TRANSPARENCY DEBUGGING MODELS ROBUSTNESS (e.g. Dealing with adversarial attacks) MACHINE

    LEARNING INTERPRETABILITY: WHY?
  36. MACHINE LEARNING INTERPRETABILITY: HOW?

  37. https:/ /github.com/marcotcr/lime Local Interpretable Model-agnostic Explanations LIME https:/ /github.com/thomasp85/lime

  38. LIME Explain individual predictions of black-box models using a local

    interpretable model. “ ”
  39. this documentary was boring and quite stupid….. I-Tabular Data II-Images

    III-Text LIME
  40. Label: toy terrier Probability: 0.81 Explanation Fit: 0.38 LIME (Images)

    Original Model: pre-trained ImageNet model
  41. LIME (Images) superpixels 50 100 150 Original Model: pre-trained ImageNet

    model
  42. Label: tabby, tabby cat Probability: 0.29 Explanation Fit: 0.77 Label:

    Egyptian Cat Probability: 0.28 Explanation Fit: 0.69 LIME (Images) Original Model: pre-trained ImageNet model
  43. Label: tabby, tabby cat Probability: 0.29 Explanation Fit: 0.77 Label:

    Egyptian Cat Probability: 0.28 Explanation Fit: 0.69 Type: Supports Type: Contradicts LIME (Images) Original Model: pre-trained ImageNet model
  44. LIME (Images) “Why Should I Trust You?” Explaining the Predictions

    of Any Classifier https:/ /arxiv.org/pdf/1602.04938.pdf
  45. LIME (Images) “Why Should I Trust You?” Explaining the Predictions

    of Any Classifier SNOW
  46. IMDB reviews sentiment classification LIME (Text) Original Model: Keras model

    (CNN+LSTM)
  47. Boring Stupid Dumb Waste information Label predicted: -ve sentiment LIME

    (Text) Original Model: Keras model (CNN+LSTM)
  48. 1- Select a point to explain (red). Based on an

    example in “Interpretable Machine Learning” Book by Christoph Molnar LIME (Tabular Data)
  49. 2- Sample data points. LIME (Tabular Data) Based on an

    example in “Interpretable Machine Learning” Book by Christoph Molnar
  50. 3- Weight points according to their proximity to the selected

    point. LIME (Tabular Data) Based on an example in “Interpretable Machine Learning” Book by Christoph Molnar
  51. 4- Train a weighted, interpretable local model. LIME (Tabular Data)

    Based on an example in “Interpretable Machine Learning” Book by Christoph Molnar
  52. 5- Explain the black-box model prediction using the local model.

    LIME (Tabular Data) Based on an example in “Interpretable Machine Learning” Book by Christoph Molnar
  53. LIME (Tabular Data)

  54. LOCAL → Unique explanation for each prediction LIME (Tabular Data)

  55. LIME (Tabular Data)

  56. Pros LIME - Provides human-friendly explanations. - Gives a fidelity

    measure. - Can use other features than the black-box model.
  57. Pros LIME Cons - Provides human-friendly explanations. - Gives a

    fidelity measure. - Can use other features than the original model. - The definition of proximity is not totally resolved in tabular data. - Instability of explanations.
  58. Pros - Provides human-friendly explanations. - Gives a fidelity measure.

    - Can use other features than the original model. Cons - Instability of explanations. LIME - The definition of proximity is not totally resolved in tabular data.
  59. SHAPLEY VALUES

  60. SHAPLEY VALUES Explain the difference between the actual prediction and

    the average/baseline prediction of the black-box model. coalitional game theory “ ”
  61. Pros - Solid theory. - The difference between the prediction

    and the average prediction is fairly distributed among the feature values of the instance. SHAPLEY VALUES
  62. Pros Cons - Solid theory - The difference between the

    prediction and the average prediction is fairly distributed among the feature values of the instance. - Computationally expensive. - Can be misinterpreted. SHAPLEY VALUES - Uses all features (not ideal for explanations that contain few features).
  63. Pros Cons - Solid theory. - The difference between the

    prediction and the average prediction is fairly distributed among the feature values of the instance. - Computationally expensive. - Can be misinterpreted. SHAPLEY VALUES - Uses all features (not ideal for explanations that contain few features).
  64. https://github.com/slundberg/shap SHAP

  65. Explainable AI for Trees: From Local Explanations to Global Understanding

    Bar chart (left) and SHAP summary plot (right) for a gradient boosted decision tree model trained on the mortality dataset. SHAP
  66. Explainable AI for Trees: From Local Explanations to Global Understanding

    Bar chart (left) and SHAP summary plot (right) for a gradient boosted decision tree model trained on the mortality dataset. SHAP
  67. Explainable AI for Trees: From Local Explanations to Global Understanding

    Bar chart (left) and SHAP summary plot (right) for a gradient boosted decision tree model trained on the mortality dataset. SHAP
  68. https:/ /github.com/ModelOriented/DALEX https:/ /github.com/ModelOriented/DALEXtra

  69. https://www.sicara.ai/blog/2019-07-31-tf-explain-interpretability-tensorflow

  70. https://github.com/SeldonIO/alibi

  71. None
  72. https:/ /cloud.google.com/blog/products/ai-machine-learning/google-cloud-ai-explanations-to-increase-fairness-responsibility-and-trust

  73. https:/ /cloud.google.com/ml-engine/docs/ai-explanations/overview AI Explanations for AI Platform

  74. ADDED FEATURE ATTRIBUTIONS Google AI Explainability Whitepaper

  75. https:/ /cloud.google.com/ml-engine/docs/ai-explanations/limitations Explanations are LOCAL (Each attribution only shows how

    much the feature affected the prediction for that particular example). LIMITATIONS Google AI Explainability Whitepaper Limitations of Interpretable Machine Learning Methods
  76. https:/ /cloud.google.com/ml-engine/docs/ai-explanations/limitations LIMITATIONS Google AI Explainability Whitepaper Limitations of Interpretable

    Machine Learning Methods Explanations are subject to adversarial attacks as predictions in complex models. Explanations are LOCAL (Each attribution only shows how much the feature affected the prediction for that particular example).
  77. https:/ /cloud.google.com/ml-engine/docs/ai-explanations/limitations Explanations are LOCAL (Each attribution only shows how

    much the feature affected the prediction for that particular example). LIMITATIONS Google AI Explainability Whitepaper Limitations of Interpretable Machine Learning Methods Explanations are subject to adversarial attacks as predictions in complex models. Explanations alone cannot tell if your model is fair, unbiased, or of sound quality.
  78. https:/ /cloud.google.com/ml-engine/docs/ai-explanations/limitations Explanations are LOCAL (Each attribution only shows how

    much the feature affected the prediction for that particular example). LIMITATIONS Google AI Explainability Whitepaper Limitations of Interpretable Machine Learning Methods Explanations are subject to adversarial attacks as predictions in complex models. Explanations alone cannot tell if your model is fair, unbiased, or of sound quality. Different methods are just complementary tools to be combined with the practitioners’ best judgement.
  79. Explanations are LOCAL (Each attribution only shows how much the

    feature affected the prediction for that particular example). LIMITATIONS Explanations are subject to adversarial attacks as predictions in complex models. Explanations alone cannot tell if your model is fair, unbiased, or of sound quality. Different methods are just complementary tools to be combined with other approaches and the practitioners’ best judgement. https:/ /cloud.google.com/ml-engine/docs/ai-explanations/limitations Google AI Explainability Whitepaper Limitations of Interpretable Machine Learning Methods
  80. EXTRA READINGS

  81. EXTRA READINGS

  82. EXTRA READINGS

  83. EXTRA READINGS

  84. MACHINE LEARNING INTERPRETABILITY: WHY AND HOW! OMAYMA SAID Feb 2020