Share
Embed
Transcript
18 slides
-
Talk Outline ๏ Introduction ๏ Interpretability ๏ Intrinsically Interpretable Models
๏ Post-hoc Interpretation Methods ๏ Local Interpretable Model-Agnostic Explanations (LIME) ๏ Demo 2 -
Talk Outline ๏ Introduction ๏ Interpretability ๏ Intrinsically Interpretable Models
๏ Post-hoc Interpretation Methods ๏ Local Interpretable Model-Agnostic Explanations (LIME) ๏ Demo 3 -
Introduction ๏ Machine learning algorithms build mathematical models using training
data, in order to make predictions or inference. ๏ A model is a mapping ๏ A internal workings of a black-box model is opaque e.g. neural networks ๏ The internal workings of a white-box model is available for inspection e.g. linear regression, logistic regression f : X → Y 4 -
Talk Outline ๏ Introduction ๏ Interpretability ๏ Intrinsically Interpretable Models
๏ Post-hoc Interpretation Methods ๏ Local Interpretable Model-Agnostic Explanations (LIME) ๏ Demo 5 -
Interpretability ”Interpretability is the degree to which a human can
understand the cause of a decision” Miller, Tim. “Explanation in artificial intelligence: Insights from the social sciences.” arXiv Preprint arXiv:1706.07269. (2017) ”Interpretability is the degree to which a human can consistently predict the model’s result” Kim, Been, Rajiv Khanna, and Oluwasanmi O. Koyejo. “Examples are not enough, learn to criticize! Criticism for interpretability.” Advances in Neural Information Processing Systems (2016) 6 -
More interpretable Less interpretable Less complex More complex Model Complexity
Model Interpretability 7 Less performant More performant Prediction Performance -
Why do we care about interpretability? ๏ Ethical and legal
reasons ๏ Social acceptance and trust ๏ Auditing and debugging 8 -
Talk Outline ๏ Introduction ๏ Interpretability ๏ Intrinsically Interpretable Models
๏ Post-hoc Interpretation Methods ๏ Local Interpretable Model-Agnostic Explanations (LIME) ๏ Demo 9 -
Intrinsically Interpretable Models ๏ The easiest way to achieve interpretability
is to use machine learning algorithms that produce interpretable models. ๏ Linear regression, Logistic regression, Decision trees are examples. 10 -
11 Linear regression Logistic regression y = β0 + β1
x1 + … + βp xp + ϵ log ( p(X) 1 − p(X)) = β0 + β1 X1 + ⋯ + βp Xp p(X) = 1 1 + e(−β0 +β1 X1 +⋯+βp Xp ) -
Talk Outline ๏ Introduction ๏ Interpretability ๏ Intrinsically Interpretable Models
๏ Post-hoc Interpretation Methods ๏ Local Interpretable Model-Agnostic Explanations (LIME) ๏ Demo 12 -
Post-hoc Interpretation Methods ๏ Model-agnostic methods applied to analyse models
after training ๏ Global surrogates ๏ Local interpretable model-agnostic explanations (LIME) ๏ Shapley Additive Explanations (SHAP) ๏ Individual Conditional Expectation (ICE) ๏ Permutation Feature Importance 13 -
Talk Outline ๏ Introduction ๏ Interpretability ๏ Intrinsically Interpretable Models
๏ Post-hoc Interpretation Methods ๏ Local Interpretable Model-Agnostic Explanations (LIME) ๏ Demo 14 -
Local Interpretable Model-agnostic Explanations (LIME) ๏ Local surrogate model used
to explain the individual predictions of a black-box model. ๏ Local surrogate model should have local fidelity. 15 Source: https://github.com/marcotcr/lime -
Local Interpretable Model-agnostic Explanations (LIME) ๏ Mathematically, where is the
model being explained is a interpretable model measures how unfaithful is in approximating in the locality defined by measures the complexity ๏ There is a fidelity-interpretability trade-off ξ(x) = arg min g∈G ℒ(f, g, πx ) + Ω(x) f g ∈ G ℒ(f, g, πx ) g f πx Ω(x) 16 -
Caveats ๏ The neighbourhood is really hard to define. Currently
uses exponential smoothing kernel. ๏ Sampling could be improved. ๏ The complexity of the explanation model has to be defined in advance. 17 -
Talk Outline ๏ Introduction ๏ Interpretability ๏ Intrinsically Interpretable Models
๏ Post-hoc Interpretation Methods ๏ Local Interpretable Model-Agnostic Explanations (LIME) ๏ Demo 18