[JSAI24] C3-LRP: Visual Explanation Generation based on Layer-Wise Relevance Propagation for ResNet

1. 𝐶!-LRP : Visual Explanation Generation based on Layer-Wise Relevance Propagation for ResNet Felix Doublet, Seitaro Otsuki, Tsumugi Iida, Komei Sugiura Keio University
2. - 2 - Summary : We adapted LRP to handle ResNet models • A new Layer-wise Relevance Propagation (LRP) rule to handle residual connections of ResNet • 𝑪𝟑-LRP that selects the most noteworthy area based on the generated relevance regions. We introduce : LRP [Bach+, PLoS15] GradCAM [Selvaraju+, ICCV17] Ours
3. - 3 - Background : The need for an eXplainable AI Deep Neural Networks (DNNs) = black box nature : • Lack of trust by end-users (medical analysis, autonomous vehicles) • Legal need (RGPD, AI Act) to explain decisions Anastasia Grivia : https://impact.universityofgalway.ie/articles/to- black-box-or-not-to-black-box/ ⇒ Explainable AI (XAI) Layer-Wise Relevance Propagation [Bach+, PLoS15] : • Strong theoretical background • Transparency in Decision-Making • Interpretable Relevance scores ⇒ White Box Analysis
4. - 4 - Background : Generating visual explanations for ResNet with LRP Input Image n Layer-Wise Relevance Propagation (LRP) fails to generate insightful explanations for ResNet models LRP Output LRP FAIL !
5. - 5 - n Layer-Wise Relevance Propagation (LRP) fails to generate insightful explanations for ResNet models LRP Output Residual connections ResNet[He+, CVPR16] No designed rule ✕ Background : Generating visual explanations for ResNet with LRP ???
6. - 6 - n Layer-Wise Relevance Propagation (LRP) fails to generate insightful explanations for ResNet models LRP Output Residual connections ResNet[He+, CVPR16] Tailored rule ✓ Our method Background : Generating visual explanations for ResNet with LRP
7. - 7 - GradCAM [Selvaraju+, ICCV17] Related work : Back-Propagation methods GradCAM [Selvaraju+, ICCV17] Gradients flowing to produce a coarse localization map. Input x Gradient [Shrikumar+, ICLR18] Multiplying the input image by the gradient of the output. LRP [Bach+, PLoS15] Propagating the prediction backward through the network layers.
8. - 8 - Related work : Layer-Wise Relevance Propagation - an established theoretical framework LRP [Bach+, PloS15] n Layer-wise Relevance Propagation [Bach+, PLoS15] n Established theoretical framework n Transparent computational processes
9. - 9 - : the relevance score of neuron 𝑘 at layer 𝑙 + 1 : activation of neuron 𝑘 : weight of the connection between neuron 𝑖 and neuron 𝑘 Related work : Layer-Wise Relevance Propagation - an established theoretical framework LRP [Bach+, PloS15]
10. - 10 - : the relevance score of neuron 𝑘 at layer 𝑙 + 1 : activation of neuron 𝑘 : weight of the connection between neuron 𝑖 and neuron 𝑘 Related work : Layer-Wise Relevance Propagation - an established theoretical framework LRP [Bach+, PloS15] A rule must be defined for each type of layer
11. - 11 - Proposed Method : The need for a new rule for the ResNet models ResNet50 architecture :
12. - 12 - Proposed Method : The need for a new rule for the ResNet models ResNet50 architecture :
13. - 13 - One bottleneck block: ResNet50 architecture : Proposed Method : The need for a new rule for the ResNet models Need for a tailored backpropagation rule
14. - 14 - IDEA : Considering each bottleneck block as a single dense layer Proposed Method : Relevance backpropagation for Bottleneck layers
15. - 15 - : Relevance of the layer L IDEA : Considering each bottleneck block as a single dense layer Proposed Method : Relevance backpropagation for Bottleneck layers
16. - 16 - : Input of the Bottleneck block : Output of the Bottleneck block 𝐶 : Output channels 𝑈 : Height 𝑉 : Width : Avoiding a zero division IDEA : Considering each bottleneck block as a single dense layer : Relevance of the layer L Proposed Method : Relevance backpropagation for Bottleneck layers
17. - 17 - Choice Contour Component (𝐶") : Removing unnecessary information Relevance map (LRP output) Proposed Method : Choice Contour Component : 𝑪𝟑
18. - 18 - Choice Contour Component (𝐶") : Removing unnecessary information Relevance map (LRP output) Proposed Method : Choice Contour Component : 𝑪𝟑 1st Contour (findContours) 1st Component (connectedComponents) C1C [Iida+, SIG-AM23] TRUE Two binary masks are created
19. - 19 - Choice Contour Component (𝐶") : Removing unnecessary information Relevance map (LRP output) Proposed Method : Choice Contour Component : 𝑪𝟑 1st Contour (findContours) 1st Component (connectedComponents) C1C [Iida+, SIG-AM23] AND TRUE
20. - 20 - Choice Contour Component (𝐶") : Removing unnecessary information Relevance map (LRP output) Proposed Method : Choice Contour Component : 𝑪𝟑 1st Contour (findContours) 1st Component (connectedComponents) C1C [Iida+, SIG-AM23] AND Final Relevance map Only the pixels from BOTH the 1st contour and the 1st component are kept TRUE
21. - 21 - Experimental setup : The CUB-200-2011 dataset n The CUB-200-2011 dataset [Wah+, 11] : n 11,788 images of birds n 200 classes n Split : 5,794:200:5,794 (train:val:test) Laysan Albatros Least Auklet
22. Method Acc. ↑ Insertion↑ Deletion↓ ID score↑ RISE [Petsuik+, RMVC18] 0.815±0.001 0.371±0.015 0.043±0.004 0.328±0.004 GradCAM [Selvaraju+, ICCV19] 0.815±0.001 0.466±0.019 0.156±0.008 0.310±0.020 LRP [Bach+, PLoS15] 0.815±0.001 0.063±0.007 0.051±0.006 0.011±0.001 ABN [Fukui+, CVPR19] 0.642±0.009 0.282±0.052 0.075±0.011 0.207±0.054 Ours 0.815±0.001 0.685±0.015 0.017±0.001 0.668±0.015 - 22 - Quantitative results : Our method outperformed the baseline methods
23. - 23 - +𝟎. 𝟐𝟏𝟗 Quantitative results : Our method outperformed the baseline methods
24. - 24 - −𝟎. 𝟎𝟐𝟔 Quantitative results : Our method outperformed the baseline methods
25. - 25 - +𝟎. 𝟑𝟑𝟎 Quantitative results : Our method outperformed the baseline methods
26. - 26 - Qualitative results : Our method provided the most insightful explanations Original Ours ABN [Fukui+, CVPR19] LRP [Bach+, PloS15] GradCAM [Selvaraju+, ICCV17] RISE [Petsuik+, BMVC18]
27. - 27 - Ours Qualitative results : Our method provided the most insightful explanations § Only one wing is attended § The attended area is vague Original ABN [Fukui+, CVPR19] LRP [Bach+, PloS15] GradCAM [Selvaraju+, ICCV17] RISE [Petsuik+, BMVC18]
28. - 28 - Ours Qualitative results : Our method provided the most insightful explanations § The bird is correctly attended § The attended area is vague Original ABN [Fukui+, CVPR19] LRP [Bach+, PloS15] GradCAM [Selvaraju+, ICCV17] RISE [Petsuik+, BMVC18]
29. - 29 - Ours Qualitative results : Our method provided the most insightful explanations § Almost no pixel is attended § Not an insightful explanation Original ABN [Fukui+, CVPR19] LRP [Bach+, PloS15] GradCAM [Selvaraju+, ICCV17] RISE [Petsuik+, BMVC18]
30. - 30 - Ours Qualitative results : Our method provided the most insightful explanations § The bird is correctly attended § The background is also attended Original ABN [Fukui+, CVPR19] LRP [Bach+, PloS15] GradCAM [Selvaraju+, ICCV17] RISE [Petsuik+, BMVC18]
31. - 31 - Ours Qualitative results : Our method provided the most insightful explanations § The bird is correctly attended § The shape of the bird’s body is precisely attended Original ABN [Fukui+, CVPR19] LRP [Bach+, PloS15] GradCAM [Selvaraju+, ICCV17] RISE [Petsuik+, BMVC18]
32. - 32 - Ablation study : 𝑪𝟑 effectively generated high quality explanations +𝟎. 𝟒𝟑𝟔
33. - 33 - Ablation study : 𝑪𝟑 effectively generated high quality explanations +𝟎. 𝟎𝟔𝟖
34. - 34 - Conclusion : We successfully generated high quality explanations Contributions: • A method for calculating LRP in models with residual connections • 𝐶!-LRP, which improves the quality of explanations 𝑪𝟑-LRP LRP [Bach+, PloS15] output 𝐶!-LRP output
35. - 35 - Appendix : Failure case example – All the different methods failed to generate an insightful explanation Original Ours ABN [Fukui 19] LRP [Bach 15] GradCAM [Selvaraju 18] RISE [Petsuik 18] ⇒ None of the different methods correctly attend the bird
36. - 36 - Appendix : Error analysis - The method focusing on an insufficient part of the image Error Type IA OA WA #Error 63 21 16 Insufficiently Attended (IA) Over-Attended (OA) Wrongly-Attended (WA) • IA : The area of attention is too small. • OA : The area of relevance is excessively large • WA : The relevance is given to pixels that do not directly contribute to the classification.