DALEXでexplainerを作成する explainer = model_trained %>% DALEX::explain(data = df %>% select(-log_price), y = df %>% pull(log_price), label = "Random Forest") Preparation of a new explainer is initiated -> model label : Random Forest -> data : 53940 rows 6 cols -> data : tibbble converted into a data.frame -> target variable : 53940 values -> predict function : yhat.model_fit will be used ( default ) -> predicted values : numerical, min = 5.945424 , mean = 7.786629 , max = 9.784551 -> residual function : difference between y and yhat ( default ) -> residuals : numerical, min = -0.3707432 , mean = 0.0001399497 , max = 0.718558 -> model_info : package parsnip , ver. 0.0.5 , task regression ( default ) A new explainer has been created! 解釈⼿法を使うための準備
重要度をどう定義する︖PFIのアイデア • 各特徴量をシャッフルして予測精度の減少率を計算する • 予測精度が⼤きく落ちる → モデルとって重要な特徴量︕ 予測精度を計算 予測精度を計算 予測精度を⽐較 シャッフル 特徴量の重要度の求め⽅（caratの場合） 元データ シャッフル後データ carat cut color clarity 1.04 Ideal G VS2 0.59 Very Good E SI2 0.53 Ideal G VVS1 1.51 Premium H VS2 carat cut color clarity 0.59 Ideal G VS2 1.51 Very Good E SI2 1.04 Ideal G VVS1 0.53 Premium H VS2
carat cut color clarity ICE 0.5 Ideal G VS2 𝑓!"#"$,& (0.5) 0.5 Very Good E SI2 𝑓!"#"$,' (0.5) 0.5 Ideal G VVS1 𝑓!"#"$,( (0.5) 0.5 Premium H VS2 𝑓!"#"$,) (0.5) ICEの計算⽅法は︖ PDの推定に使った予測値を平均せずに使⽤ 元データ carat cut color clarity 1.04 Ideal G VS2 0.59 Very Good E SI2 0.53 Ideal G VVS1 1.51 Premium H VS2 carat cut color clarity ICE 1 Ideal G VS2 𝑓!"#"$,& (1.0) 1 Very Good E SI2 𝑓!"#"$,' (1.0) 1 Ideal G VVS1 𝑓!"#"$,( (1.0) 1 Premium H VS2 𝑓!"#"$,) (1.0) carat cut color clarity ICE 1.5 Ideal G VS2 𝑓!"#"$,& (1.5) 1.5 Very Good E SI2 𝑓!"#"$,' (1.5) 1.5 Ideal G VVS1 𝑓!"#"$,( (1.5) 1.5 Premium H VS2 𝑓!"#"$,) (1.5) 𝑓!,, 𝑥! = 𝑓(𝑥!, 𝑿#,) ICEの推定式
参考⽂献︓機械学習の解釈性全般 • Exploratory data analysis using xgboost package in R https://www.slideshare.net/kato_kohaku/exploratory-data-analysis-using-xgboost- package-in-r-146048320 • How to use in R model-agnostic data explanation with DALEX & iml https://www.slideshare.net/kato_kohaku/how-to-use-in-r-modelagnostic-data- explanation-with-dalex-iml • 機械学習と解釈可能性 / Machine Learning and Interpretability https://speakerdeck.com/line_developers/machine-learning-and-interpretability • 機械学習モデルの判断根拠の説明 https://www.slideshare.net/SatoshiHara3/ss-126157179 • BlackBox モデルの説明性・解釈性技術の実装 https://www.slideshare.net/DeepLearningLab/blackbox-198324328
参考⽂献︓PFI • Fisher, Aaron, Cynthia Rudin, and Francesca Dominici. "All models are wrong, but many are useful: Learning a variable's importance by studying an entire class of prediction models simultaneously." arXiv preprint arXiv:1801.01489 (2018). • Hooker, Giles, and Lucas Mentch. "Please Stop Permuting Features: An Explanation and Alternatives." arXiv preprint arXiv:1905.03151 (2019). • Beware Default Random Forest Importances https://explained.ai/rf-importance/index.html • 特徴量重要度にバイアスが⽣じる状況ご存知ですか︖ https://aotamasaki.hatenablog.com/entry/bias_in_feature_importances
参考⽂献︓PD/ICE • Friedman, Jerome H. "Greedy function approximation: a gradient boosting machine." Annals of statistics (2001): 1189-1232. • Goldstein, Alex, et al. "Peeking inside the black box: Visualizing statistical learning with plots of individual conditional expectation." Journal of Computational and Graphical Statistics 24.1 (2015): 44-65. • Hastie, Trevor, et al. "The elements of statistical learning: data mining, inference and prediction." The Mathematical Intelligencer 27.2 (2005): 83-85. • Zhao, Qingyuan, and Trevor Hastie. "Causal interpretations of black-box models." Journal of Business & Economic Statistics just-accepted (2019): 1-19. • Zhao, Xilei, Xiang Yan, and Pascal Van Hentenryck. "Modeling heterogeneity in mode- switching behavior under a mobility-on-demand transit system: An interpretable machine learning approach." arXiv preprint arXiv:1902.02904 (2019).
参考⽂献︓SHAP • Lundberg, Scott M., and Su-In Lee. "A unified approach to interpreting model predictions." Advances in Neural Information Processing Systems. 2017. • Lundberg, Scott M., Gabriel G. Erion, and Su-In Lee. "Consistent individualized feature attribution for tree ensembles." arXiv preprint arXiv:1802.03888 (2018). • Lundberg, Scott M., et al. "Explainable AI for Trees: From Local Explanations to Global Understanding." arXiv preprint arXiv:1905.04610 (2019). • Sundararajan, Mukund, and Amir Najmi. "The many Shapley values for model explanation." arXiv preprint arXiv:1908.08474 (2019). • Janzing, Dominik, Lenon Minorics, and Patrick Blöbaum. "Feature relevance quantification in explainable AI: A causality problem." arXiv preprint arXiv:1910.13413 (2019). • GitHub - slundberg/shap: A game theoretic approach to explain the output of any machine learning model. https://github.com/slundberg/shap. • 岡⽥ 卓. "ゲーム理論 新版. " 有斐閣. (2011).
dropout009
lcolladotor
gaelvaroquaux
damberg
mlarrieu
ianozsvald
ssii
nkmr_rl
yuya4
bob3bob3
shimacos
aarron
3n
palkan
jennybc
vanstee
samanthasiow
addyosmani
jcasabona
ddemaree
eileencodes
zakiyama
destraynor