Upgrade to Pro — share decks privately, control downloads, hide ads and more …

MACHINE LEARNING INTERPRETABILITY: WHY AND HOW!

62321e5935c9c0731462b8178a7423f8?s=47 OmaymaS
December 10, 2019
260

MACHINE LEARNING INTERPRETABILITY: WHY AND HOW!

With the increasing adoption of machine learning-based solutions in different domains, systems that use black-box algorithms are getting used more often with the promise of providing higher accuracy. However, this accuracy comes at the cost of interpretability, which introduces a barrier against wider adoption of such algorithms in crucial areas and raises the skepticism of the impacted individuals. This talk focuses on the importance of interpretable machine learning, why it is crucial from technical and ethical perspectives and its current limitations. In addition, it gives an overview of some of the relevant tools and packages (e.g. LIME, SHAPLEY).

62321e5935c9c0731462b8178a7423f8?s=128

OmaymaS

December 10, 2019
Tweet

Transcript

  1. MACHINE LEARNING INTERPRETABILITY: WHY AND HOW! OMAYMA SAID Dec 2019

  2. INTERPRETABLE ML/AI EXPLAINABLE ML/AI FEATURE ATTRIBUTIONS WHAT DO YOU KNOW

    ABOUT ?
  3. • You will be given two labels with the corresponding

    mapping to 0/1. • You will be asked to classify some images. • You are a classifier. HOW WOULD YOUR MENTAL MODEL LABEL THIS IMAGE? RULES
  4. CAT → 0 DOG → 1 OR HOW WOULD YOUR

    MENTAL MODEL LABEL THIS IMAGE?
  5. CAT → 0 DOG → 1 OR HOW WOULD YOUR

    MENTAL MODEL LABEL THIS IMAGE?
  6. DUCK → 0 RABBIT → 1 OR HOW WOULD YOUR

    MENTAL MODEL LABEL THIS IMAGE?
  7. WHY ? HOW WOULD YOUR MENTAL MODEL LABEL THIS IMAGE?

  8. https:/ /github.com/minimaxir/optillusion-animation Max Woolf

  9. INTERPRETABILITY TRADEOFF COMPLEXITY

  10. AI-POWERED [----] ML-ENABLED [----] MORE AND MORE

  11. BUT LET’S FIRST EXCLUDE THE NONSENSE

  12. “Uses AI to give you more insight into candidates, so

    you can make better decisions.” $#&T NONSENSE Source: Business Insider Video (2017) 25000 FEATURES → INSIGHT SCORE
  13. None
  14. $#&@!^&$$& !!!!!!!!!!!!!!!!!!!!! https:/ /www.faception.com/

  15. HOW ABOUT THE OTHER APPLICATIONS?

  16. None
  17. None
  18. None
  19. None
  20. WHAT IF IT WAS NOT GENDER OR A PROXY TO

    GENDER ?
  21. WHY? TAMMY DOBBS, ARKANSAS REDUCED HOURS OF HOME CARE VISITS

    PER WEEK
  22. TAMMY DOBBS, ARKANSAS WHY? REDUCED HOURS OF HOME CARE VISITS

    PER WEEK WHO IS ACCOUNTABLE?
  23. COLLECT/LABEL DATA IT IS HUMANS WHO WRITE ALGORITHMS DEFINE METRICS

  24. COLLECT/LABEL DATA IT IS HUMANS WHO BIAS IN: - REPRESENTATION

    - DISTRIBUTION - LABELS AND MORE….. WRITE ALGORITHMS DEFINE METRICS
  25. IT IS HUMANS WHO DEFINE METRICS WRITE ALGORITHMS COLLECT/LABEL DATA

    - TRAIN/TEST SPLIT - FEATURES/PROXIES - MODEL COMPLEXITY AND MORE…..
  26. IT IS HUMANS WHO COLLECT/LABEL DATA DEFINE METRICS WRITE ALGORITHMS

    - WHAT IS THE IMPACT OF DIFFERENT ERROR TYPES ON DIFFERENT GROUPS? - WHAT DO YOU OPTIMIZE FOR?
  27. PRACTITIONERS CONSISTENTLY: - OVERESTIMATE THEIR MODEL’S ACCURACY. - PROPAGATE FEEDBACK

    LOOPS. - FAIL TO NOTICE DATA LEAKS. “ ” “Why Should I Trust You?” Explaining the Predictions of Any Classifier
  28. MACHINE LEARNING INTERPRETABILITY: WHY?

  29. TRANSPARENCY MACHINE LEARNING INTERPRETABILITY: WHY?

  30. TRANSPARENCY DEBUGGING MODELS MACHINE LEARNING INTERPRETABILITY: WHY?

  31. TRANSPARENCY DEBUGGING MODELS ROBUSTNESS MACHINE LEARNING INTERPRETABILITY: WHY? (e.g. Dealing

    with adversarial attacks)
  32. TRANSPARENCY DEBUGGING MODELS ROBUSTNESS (e.g. Dealing with adversarial attacks) MACHINE

    LEARNING INTERPRETABILITY: WHY?
  33. MACHINE LEARNING INTERPRETABILITY: HOW?

  34. https:/ /github.com/marcotcr/lime Local Interpretable Model-agnostic Explanations LIME https:/ /github.com/thomasp85/lime

  35. LIME Explain individual predictions of black-box models using a local

    interpretable model. “ ”
  36. this documentary was boring and quite stupid….. I-Tabular Data II-Images

    III-Text LIME
  37. 1- Select a point to explain (red). Based on an

    example in “Interpretable Machine Learning” Book by Christoph Molnar LIME (Tabular Data)
  38. 2- Sample data points. LIME (Tabular Data) Based on an

    example in “Interpretable Machine Learning” Book by Christoph Molnar
  39. 3- Weight points according to their proximity to the selected

    point. LIME (Tabular Data) Based on an example in “Interpretable Machine Learning” Book by Christoph Molnar
  40. 4- Train a weighted, interpretable local model. LIME (Tabular Data)

    Based on an example in “Interpretable Machine Learning” Book by Christoph Molnar
  41. 5- Explain the black-box model prediction using the local model.

    LIME (Tabular Data) Based on an example in “Interpretable Machine Learning” Book by Christoph Molnar
  42. LIME (Tabular Data)

  43. LOCAL → Unique explanation for each prediction LIME (Tabular Data)

  44. LIME (Tabular Data)

  45. Label: toy terrier Probability: 0.81 Explanation Fit: 0.38 LIME (Images)

    Original Model: pre-trained ImageNet model
  46. LIME (Images) superpixels 50 100 150 Original Model: pre-trained ImageNet

    model
  47. Label: tabby, tabby cat Probability: 0.29 Explanation Fit: 0.77 Label:

    Egyptian Cat Probability: 0.28 Explanation Fit: 0.69 LIME (Images) Original Model: pre-trained ImageNet model
  48. Label: tabby, tabby cat Probability: 0.29 Explanation Fit: 0.77 Label:

    Egyptian Cat Probability: 0.28 Explanation Fit: 0.69 Type: Supports Type: Contradicts LIME (Images) Original Model: pre-trained ImageNet model
  49. LIME (Images) “Why Should I Trust You?” Explaining the Predictions

    of Any Classifier https:/ /arxiv.org/pdf/1602.04938.pdf
  50. LIME (Images) “Why Should I Trust You?” Explaining the Predictions

    of Any Classifier SNOW
  51. IMDB reviews sentiment classification LIME (Text) Original Model: Keras model

    (CNN+LSTM)
  52. Boring Stupid Dumb Waste information Label predicted: -ve sentiment LIME

    (Text) Original Model: Keras model (CNN+LSTM)
  53. Pros LIME - Provides human-friendly explanations. - Gives a fidelity

    measure. - Can use other features than the black-box model.
  54. Pros LIME Cons - Provides human-friendly explanations. - Gives a

    fidelity measure. - Can use other features than the original model. - The definition of proximity is not totally resolved in tabular data. - Instability of explanations.
  55. Pros - Provides human-friendly explanations. - Gives a fidelity measure.

    - Can use other features than the original model. Cons - Instability of explanations. LIME - The definition of proximity is not totally resolved in tabular data.
  56. SHAPLEY VALUES

  57. SHAPLEY VALUES Explain the difference between the actual prediction and

    the average/baseline prediction of the black-box model. coalitional game theory “ ”
  58. Pros - Solid theory. - The difference between the prediction

    and the average prediction is fairly distributed among the feature values of the instance. SHAPLEY VALUES
  59. Pros Cons - Solid theory - The difference between the

    prediction and the average prediction is fairly distributed among the feature values of the instance. - Computationally expensive. - Can be misinterpreted. SHAPLEY VALUES - Uses all features (not ideal for explanations that contain few features).
  60. Pros Cons - Solid theory. - The difference between the

    prediction and the average prediction is fairly distributed among the feature values of the instance. - Computationally expensive. - Can be misinterpreted. SHAPLEY VALUES - Uses all features (not ideal for explanations that contain few features).
  61. https://github.com/slundberg/shap SHAP

  62. Explainable AI for Trees: From Local Explanations to Global Understanding

    Bar chart (left) and SHAP summary plot (right) for a gradient boosted decision tree model trained on the mortality dataset. SHAP
  63. https:/ /github.com/ModelOriented/DALEX https:/ /github.com/ModelOriented/DALEXtra

  64. https://www.sicara.ai/blog/2019-07-31-tf-explain-interpretability-tensorflow

  65. https://github.com/SeldonIO/alibi

  66. None
  67. https:/ /cloud.google.com/blog/products/ai-machine-learning/google-cloud-ai-explanations-to-increase-fairness-responsibility-and-trust

  68. https:/ /cloud.google.com/ml-engine/docs/ai-explanations/overview AI Explanations for AI Platform

  69. ADDED FEATURE ATTRIBUTIONS Google AI Explainability Whitepaper

  70. None
  71. • Deploy the model. • Log feature attributions for certain

    predictions (e.g. rare class). • Monitor model training/live skew. • Monitor attribution skew and focus on the most influential features. PREDICTION AUDITING AND MODEL MONITORING Explainable AI for Trees: From Local Explanations to Global Understanding Google AI Explainability Whitepaper
  72. https:/ /cloud.google.com/ml-engine/docs/ai-explanations/limitations • Explanations are LOCAL (Each attribution only shows

    how much the feature affected the prediction for that particular example). • Explanations/Feature attributions are subject to adversarial attacks as predictions in complex models. • Explanations alone cannot tell if your model is fair, unbiased, or of sound quality • Different methods are just complementary tools to be combined with other approaches and the practitioners’ best judgement. • Explanations might be misinterpreted in some cases. LIMITATIONS Google AI Explainability Whitepaper Limitations of Interpretable Machine Learning Methods
  73. EXTRA READINGS