Keep calm and trust your model - On Explainability of Machine Learning Models

The accuracy of Machine Learning models is going up by the day with advances in Deep Learning. But this comes at a cost of explainability of these models. There is a need to uncover these black boxes for the Business users. This is very essential especially for heavily regulated industries like Finance, Medicine, Defence and the likes

A lot of research is going on to make ML models interpretable and explainable. In this talk we will be going through the various approaches taken to unravel machine learning models and explain the reason behind their predictions.

We’ll see the different approaches being taken by discussing the latest research literature, the ‘behind the scenes’ view of what is happening inside these approaches with enough mathematical depth and intuition.

Finally, the aim is to leave the audience with the practical know-how on how to use these approaches in understanding deep learning and classical machine learning models using open source tools in Python.

Transcript

1. *notice issued in public interest by the Queen of NeuralNetworkLand
2. On Explainability of Machine Learning Models Praveen Sridhar @psbots praveen.ai
3. About me Machine Learning R&D for Machine Learning Engineer at Praveen Sridhar @psbots praveen.ai
4. As we improve the accuracy with complex models, it is becoming increasingly difficult to explain how it is making those predictions Problem : source : DARPA
5. Interpretability is of prime importance in regulated industries Clinical & Pharmacy Finance Medical Diagnosis Insurance Legal Defence source : Lipton 2017
6. Lets start with the basics Which models are explainable in the first place?
7. Linear, monotonic functions For any change in any independent variable, the response function : ✦ changes in only one direction ✦ and at a magnitude represented by a readily available co-efficient
8. The culprit : Non-Linear, non-monotonic functions Most algorithms create functions that change in any direction and at varying rate Lets see how these can be explained
9. Model Specific approaches These are special methods for explainability very specific to the model itself. ✦ Tree Interpreter ✦ Bayesian Rule Lists ✦ Model specific Visualisations ✦ Attention mechanism as explanations ✦ Generating explanations as part of model
10. Tree Interpreter intuitively, for each decision that a tree (or a forest) makes there is a path (or paths) from the root of the tree to the leaf, consisting of a series of decisions, guarded by a particular feature, each of which contribute to the final predictions. source : treeinterpreter
11. Tree Interpreter Behind the scenes : for a Decision Tree, the prediction function can be written as K : number of features, c : value at the root of the node contrib(x,k) : contribution from the kth feature for a Random Forest, this can be expanded as simply the average of the bias terms plus the average contribution of each feature
12. Bayesian Rule Lists BRLs are interpretability taken to its extreme! You can just read the model and understand its predictions. Take a look at this BRL classifier ouput for the Titanic dataset : source : Letham et. al 2015
13. Bayesian Rule Lists Behind the scenes : The task of a BRL is to create an ordered list of assertions ‘a’ (eg “if x is a male and an adult”) and then find the probability of a predicted label ‘y’ source : Letham et. al 2015
14. Model Specific Visualisations Andrew NG’s famous ‘cat’ neuron FC8 Layer ‘Deep’ Visualisation of AlexNet source : Jason et. al 2015
15. Salesforce Einstein AI’s abstractive summarization Attention Mechanism as Explanations source : Salesforce einstein.ai
16. Attention Mechanism as Explanations
17. Attention Mechanism as Explanations Behind the scenes : An attention mechanism ✦ takes ’n’ arguments ‘y1,y2….,yn’ ✦ and a context vector ‘c’ to return a vector z which is the ‘summary’ of ‘y’ focusing on the information linked to the context ‘c’ more formally, it returns a weighted mean of ‘y’ with the weights chosen as per the relevance of each ‘yi’ given the context ‘c’
18. Generating explanations as part of model MIT’s CSAIL Lab’s research on “Making computers explain themselves” source : Lei et. al 2016
19. Generating explanations as part of model Berkeley AI Lab’s research on “Generating Visual Explanations” source : Lisa Anne et. al 2016
20. Model Agnostic approaches These are general methods for explainability ✦ Global Scope ➡ Variable Importance measure ➡ Residual plots ➡ Partial Dependance plots ➡ Surrogate models ✦ Local Scope ➡ LIME (Local Interpretable Model-agnostic Explanations)
21. Mushroom source : Reibero et. al 2016 LIME (Locally Interpretable Model-agnostic Explanations)
22. Behind the scenes : Idea : while treating the model as a black box, perturb the instance we want to explain and learn a sparse linear model around it, as an explanation LIME (Locally Interpretable Model-agnostic Explanations) Intuitively, an “explanation” is a local linear approximation of the model’s behaviour. source : Reibero et. al 2016
23. Layerwise Relevance Propagation Technique for determining which inputs in feature vector contribute the strongest to the Neural Network output
24. Layerwise Relevance Propagation Behind the scenes : The goal of LRP is to define some relevance measure R over the input vector such that we can express the network output as the sum of the values of R if we can decompose this function in terms of its partial derivatives, we can use that decomposition to approximate the relevance propagation function.
25. Layerwise Relevance Propagation Behind the scenes : The process is similar in spirit to BackPropagation Deep Taylor Decomposition we can use a Taylor series to approximate the value of a function f(x) near a point x0 with The closer that x is to x0, the better the approximation. One clever thing that we can do is set x0 to be a “root point” of the forward propagation function, that is, a point such that f(x0)=0. This simplifies the above Taylor expression to
26. Layerwise Relevance Propagation Behind the scenes : Root points of the forward propagation function are located at the local decision boundary, so the gradients along that boundary point give us the most information about how the function separates the input by class.
27. LIVE DEMO Let’s apply some of the approaches we discussed. Thanks to the following open source packages : ✦ Tree Interpreter : https://github.com/andosa/treeinterpreter ✦ LIME : https://github.com/marcotcr/lime ✦ ELI5 : https://github.com/TeamHG-Memex/eli5 ✦ BRL : https://github.com/tmadl/sklearn-expertsys https://github.com/psbots/explainableML
28. Thank you! Praveen Sridhar @psbots praveen.ai ps@praveen.ai mail me at