KDD2019 - A/Bテストに関する論文など ● Tutorial : Challenge, Best Practices and Pitfalls in Evaluating Results of Online Controlled Experiments ● Tutorial : Fundamentals of large-scale sequential experimentation ● Shrinkage Estimators in Online Experiments ● The Identification and Estimation of Direct and Indirect Effects in A/B Tests through Causal Mediation Analysis ● Diagnosing Sample Ratio Mismatch in Online Controlled Experiments 5 *オレンジを1スライド / 青をメインで紹介
The Identification and Estimation of Direct and Indirect Effects in A/B Tests through Causal Mediation Analysis (Etsy, Inc.) Treatment Effect = Indirect Effect + Direct Effect で分解して、New Featureに よる効果を知りたい 7 T Y :Treatment(Recommend Moduleがある)かどう か(0 or 1) 事例:Recommend ModuleのA/Bテストで有 意差が検出されなかった M Direct Effect :アウトカム(CV) :Organic Searchのクリック数 ユーザiのTreatment Effect = + Indirect Effect = Direct Effect = Recommend Moduleの効果 を確認 Indirect Effect Baron-Kenny法の発展 - 一般化回帰モデル Treatment ではないなら Treatment であるなら