Accumulated Local Effects（ALE）で機械学習モデルを解釈する / TokyoR95

2021年10月30日に行われた、第95回R勉強会@東京（#TokyoR）での発表資料です。
https://tokyor.connpass.com/event/225967/

コードはこちらになります。
https://github.com/dropout009/tokyor95

Transcript

1. Accumulated Local Effects 2021/10/30 95 R @ #TokyoR @dropout009
2. TVISION INSIGHTS Twitter: @dropout009 Speaker Deck: dropout009 Blog: https://dropout009.hatenablog.com/
4. RF/GBDT/NN +
5. Partial Dependence
6. Partial Dependence PD PD! 𝑥" = 𝔼 & 𝑓 𝑥" , 𝑿∖" = * & 𝑓 𝑥" , 𝒙∖" 𝑝 𝒙∖" 𝑑𝒙∖" ! 𝑓(𝑿) 𝑋!
7. PD ' PD! 𝑥! = 1 𝑁 . "#$ % ! 𝑓(𝑥! , 𝒙",∖! )
8. PD R
9. PD 𝑌 = 𝑋! + 𝑋" " + 𝜖 𝑋! , 𝑋" ∼ Uniform 0, 1 𝜖 ∼ 𝒩(0, 0.01") CDF
10. tidymodels tidymodels rand_forest() Random Forest
11. = 0 PD PD
12. = 0.99 PD PD
13. PD ! 𝑓 𝑋$ , 𝑋( = 𝑋$ + 𝑋( ( PD! 𝑥! = 𝔼 9 𝑓 𝑥! , 𝑋" = 𝔼 𝑥! + 𝑋" " = ! # + 𝑥! PD" 𝑥" = 𝔼[ 9 𝑓 𝑋! , 𝑥" ] = 𝔼 𝑋! + 𝑥" " = ! " + 𝑥" " PD
14. PD
15. PD 𝑋! 𝑥! 𝑋" PD! 𝑥! = 𝔼 & 𝑓 𝑥! , 𝑋" = ∫ & 𝑓 𝑥!, 𝑥" 𝑝 𝑥" 𝑑𝑥" PD 𝑋!
16. 4 PD 𝑋! 𝑥! 𝑋" (𝑥!,# , 𝑥!,! ) (𝑥$,# , 𝑥$,! ) (𝑥%,# , 𝑥%,! ) 𝑥#,# , 𝑥#,!
17. 𝑋! 𝑥! 𝑋" 𝑥#,# , 𝑥#,! (𝑥!,# , 𝑥!,! ) (𝑥$,# , 𝑥$,! ) - PD! 𝑥! = ! ' & 𝑓 𝑥!, 𝑥!," + ! ' & 𝑓 𝑥!, 𝑥"," + ! ' & 𝑓 𝑥! , 𝑥(," + ! ' & 𝑓 𝑥! , 𝑥'," (𝑥%,# , 𝑥%,! ) ) 𝑓(𝑥# , 𝑥!,! ) ) 𝑓(𝑥# , 𝑥!,! ) ) 𝑓(𝑥# , 𝑥%,! ) ) 𝑓(𝑥# , 𝑥#,! ) PD
18. Conditional Dependence (Marginal Plot)
19. PD 𝑋! 𝑥! 𝑋" PD! 𝑥! = 𝔼 & 𝑓 𝑥! , 𝑋" = ∫ & 𝑓 𝑥!, 𝑥" 𝑝 𝑥" 𝑑𝑥" PD 𝑋!
20. 築 𝑋! 𝑥! 𝑋" PD! 𝑥! = 𝔼 & 𝑓 𝑥! , 𝑋" = ∫ & 𝑓 𝑥!, 𝑥" 𝑝 𝑥" 𝑑𝑥" CD! 𝑥! = 𝔼 & 𝑓 𝑥! , 𝑋" ∣ 𝑋! = 𝑥! = ∫ & 𝑓 𝑥!, 𝑥" 𝑝 𝑥" ∣ 𝑥! 𝑑𝑥"
21. Conditional Dependence 𝑋! 𝑥! (") 𝑋" 𝑥! (() 𝑥! (') 𝑥! (+) 𝑥! (,) 𝑥! (!)
22. CD * 𝑓( 𝑥" ($%&) + 𝑥" ($) 2 , 𝒙(,∖" )
23. CD CD
24. CD 𝑓 𝑋! , 𝑋" = 𝑋! + 𝑋" " 𝑋! = 𝑋" CD! 𝑥! = 𝔼 & 𝑓 𝑥!, 𝑋" ∣ 𝑋! = 𝑥! = 𝔼 𝑥! + 𝑋" " ∣ 𝑋! = 𝑥! = 𝑥! + 𝔼 𝑋" " ∣ 𝑋! = 𝑥! = 𝑥! + 𝑥! " CD" 𝑥" = 𝔼 & 𝑓 𝑋!, 𝑥" ∣ 𝑋" = 𝑥" = 𝔼 𝑋! + 𝑥" " ∣ 𝑋" = 𝑥" = 𝔼 𝑋! ∣ 𝑋" = 𝑥" + 𝑥" " = 𝑥" + 𝑥" " CD 𝑋! 𝑋"
25. Accumulated Local Effects
26. ALE CD 𝑋! 𝑥! (") 𝑋" 𝑥! (() 𝑥! (') 𝑥! (+) 𝑥! (,) 𝑥! (!) 𝑥#,# , 𝑥#,! 𝑥!,# , 𝑥!,! 𝑥$,# , 𝑥$,!
27. ALE Local Effect Accumulate 𝑋! 𝑥! (") 𝑋" 𝑥! (() 𝑥! (') 𝑥! (+) 𝑥! (,) 𝑥! (!) ) 𝑓 𝑥# ($), 𝑥#,! ) 𝑓 𝑥# ($) , 𝑥!,! ) 𝑓 𝑥# ($), 𝑥!,! ) 𝑓 𝑥# (%), 𝑥#,! ) 𝑓 𝑥# (%), 𝑥!,! ) 𝑓 𝑥# (%), 𝑥!,! Local Effect 1 3 ) 𝑓 𝑥# (%), 𝑥#,! − ) 𝑓 𝑥# ($), 𝑥#,! + 1 3 ) 𝑓 𝑥# (%), 𝑥!,! − ) 𝑓 𝑥# ($), 𝑥!,! + 1 3 ) 𝑓 𝑥# (%), 𝑥$,! − ) 𝑓 𝑥# ($), 𝑥$,! Local Effect
28. ALE Local Effect Local Effect
29. ALE ALE
30. ALE 𝑓 𝑋! , 𝑋" = 𝑋! + 𝑋" " 𝑖 𝑋! [𝑥! (12!), 𝑥! (1)) Local Effect & 𝑓 𝑥! (1), 𝑥3," − & 𝑓 𝑥! 12! , 𝑥3," = 𝑥! 1 + 𝑥3," " − 𝑥! 12! + 𝑥3," " = 𝑥! 1 − 𝑥! 12! 𝑋" 𝑋! [𝑥! (12!), 𝑥! (1))
31. ALE ALE Local Effect 𝑋4 𝜕 * 𝑓(𝑋", 𝑿∖") 𝜕𝑋" 𝔼 𝜕 * 𝑓(𝑋", 𝑿∖") 𝜕𝑋" ∣ 𝑋" = 𝑧" ALE" 𝑥" = ; + ! (#$%) +! 𝔼 𝜕 * 𝑓(𝑋", 𝑿∖") 𝜕𝑋" ∣ 𝑋" = 𝑧" 𝑑𝑧" 𝑋4 = 𝑧4 Local Effect Local Effect 𝑥4 (567) ALE
32. ALE ! 𝑓 𝑋$ + 𝑋( ( = 𝑋$ + 𝑋( ( ALE ALE! 𝑥! = 9 8 9! 𝔼 𝜕(𝑋! + 𝑋" ") 𝜕𝑋! ∣ 𝑋! = 𝑧! 𝑑𝑧! = 9 8 9! 𝔼 1 ∣ 𝑋! = 𝑧! 𝑑𝑧! = 9 8 9! 1𝑑𝑧! = 𝑥! ALE" 𝑥" = 9 8 9" 𝔼 𝜕(𝑋! + 𝑋" ") 𝜕𝑋" ∣ 𝑋" = 𝑧" 𝑑𝑧" = 9 8 9" 𝔼 2𝑋" ∣ 𝑋" = 𝑧" 𝑑𝑧" = 9 8 9" 2𝑧"𝑑𝑧" = 𝑥 " "
34. • • Partial Dependence ⾒ • Accumulated Local Effects ⾒ • ALE
35. • Friedman, Jerome H. "Greedy function approximation: a gradient boosting machine." Annals of statistics (2001): 1189-1232. • Hooker, Giles, and Lucas Mentch. "Please Stop Permuting Features: An Explanation and Alternatives." arXiv preprint arXiv:1905.03151 (2019). • Apley, Daniel W., and Jingyu Zhu. "Visualizing the effects of predictor variables in black box supervised learning models." Journal of the Royal Statistical Society: Series B (Statistical Methodology) 82.4 (2020): 1059- 1086. • Molnar, Christoph. "Interpretable machine learning. A Guide for Making Black Box Models Explainable." (2019). https://christophm.github.io/interpretable-ml-book/. • Biecek, Przemyslaw and Tomasz Burzykowski. "Explanatory Model Analysis. Chapman and Hall/CRC (2021). https://pbiecek.github.io/ema/. • . . . (2021). https://is.gd/nkYPPG
36. R