Demystifying the neural network black box

Convolutional Neural Networks (CNN) are state of the art when it comes to computer vision tasks, such as image recognition and object detection. However, due to the high amount of architectural complexity, it is often difficult to interpret the decisions made by these networks. Luckily, there are several techniques available which can enhance our comprehension of CNN decisions. These techniques are generally divided into attribution and visualisation methods. We have talked about several such techniques in this presentation.

Transcript

1. 1 Demystifying the neural network black box Christopher Lennan (Senior Data Scientist) @chris_lennan Tanuj Jain (Data Scientist) @tjainn #idealoTech
2. Motivation 2
3. 3 Motivation Convolutional Neural Networks (CNN) • CNNs are at the heart of most computer vision applications these days • They have also established themselves as successful for various NLP tasks • But: It is notoriously difficult to interpret/understand predictions! • Problematic, because o Understanding is just as important as performance o Sensitive applications (medical or autonomous driving) require a thorough understanding of its inner workings o CNNs can be improved if their predictions are properly understood
4. idealo Use Case • Why are these misclassified as bathrooms? 4 Use Case: Identify the area of Hotel Property
5. Convolutional Neural Network (CNN) 5 source: Zhou et. al., “Learning Deep Features for Discriminative Localization”
6. Convolutional Neural Network (CNN) 6 source: http://cs231n.github.io/convolutional-networks/
7. Convolutional Neural Network (CNN) 7 source: Zhou et. al., “Learning Deep Features for Discriminative Localization”
8. Interpretability 8
9. 9 Interpreting CNNs • The available methods for interpreting CNNs can be broadly grouped into 1. Attribution methods • attributing a classification result back to the input pixels • output is usually a heatmap over input pixels that indicates how much each pixel contributed to the classification outcome 2. Visualization methods • Visualize the patterns that filters have learned • Generate images that cause maximum neuron activation • Visualize neuron interactions
10. Attribution methods 1. Perturbation based approaches - Occlude area of interest to test its effectiveness in changing predictions 2. Gradient based approaches - Calculation of gradients of output w.r.t. some network variable a. Saliency Maps b. Guided Backpropagation c. Deconvolution 3. Relevance Score approaches a. Class Activation Map (CAM) b. Grad- CAM c. Layerwise Relevance Propagation (LRP) Relative importance of areas of the input image for predictions
11. Visualization methods Neurons and parameters can be visualized at all levels of the network: 11 Looking at the learned parameters of the neural network source: https://distill.pub/2017/feature-visualization/
12. Coding Time! 12
13. Idealo Use Cases 13
14. CAM-Analysis
15. CAM-Analysis
16. Insights With CAM Swimming Pool misclassified as Bathroom 16 CAM
17. Insights With CAM Swimming Pool misclassified as Bathroom 17 CAM
18. Insights With CAM Swimming Pool misclassified as Bathroom 18 CAM
19. Insights With CAM Swimming Pool misclassified as Bathroom Using rails to misidentify Pool as Bathroom. 19
20. Insights With CAM Bathroom correct classification 20 CAM
21. Insights With CAM Bathroom correct classification 21 CAM
22. Insights With CAM Bathroom correct classification 22 CAM
23. Insights With CAM Bathroom correct classification Using faucets to correctly identify Bathroom. 23
24. Image Aesthetics
25. Convolutional filter visualisations Layer 23 MobileNet original MobileNet Aesthetic 25
26. Convolutional filter visualisations Layer 51 MobileNet original MobileNet Aesthetic 26
27. Convolutional filter visualisations Layer 79 MobileNet original MobileNet Aesthetic 27
28. Image-ATM Python package we developed for tagging images 28
29. Check us out! #idealoTech 29 https://github.com/idealo https://medium.com/idealo-tech-blog
30. Data Science Team 30
31. Tanuj Jain tanuj.jain@idealo.de @tjainn Christopher Lennan christopher.lennan@idealo.de @chris_lennan 31
32. THE END 32