Off-Policy Evaluation of Ranking Policies under Diverse User Behavior Haruka Kiyohara, Masatoshi Uehara, Yusuke Narita, Nobuyuki Shimizu, Yasuo Yamamoto, Yuta Saito 清原 明加（Haruka Kiyohara） https://sites.google.com/view/harukakiyohara August 2023 Adaptive OPE of Ranking Policies @ KDDʼ23 1
ランキング推薦はどのように成り⽴っている？ August 2023 Adaptive OPE of Ranking Policies @ KDDʼ23 3 ranking with 𝑲 items a coming user context clicks reward(s) a ranking policy … ▼ これを評価したい
オフ⽅策評価（Off-Policy Evaluation; OPE） August 2023 Adaptive OPE of Ranking Policies @ KDDʼ23 4 ranking with 𝑲 items a coming user context clicks reward(s) a logging policy an evaluation policy …
ナイーブな⼿法: Inverse Propensity Scoring [Strehl+,10] August 2023 Adaptive OPE of Ranking Policies @ KDDʼ23 8 importance weight evaluation logging ranking A ranking B more less less more ・unbiased ・variance
ナイーブな⼿法: Inverse Propensity Scoring [Strehl+,10] August 2023 Adaptive OPE of Ranking Policies @ KDDʼ23 9 importance weight evaluation logging ranking A ranking B more less less more ・unbiased ・variance 分布シフトを補正している
ナイーブな⼿法: Inverse Propensity Scoring [Strehl+,10] August 2023 Adaptive OPE of Ranking Policies @ KDDʼ23 11 importance weight ・unbiased ・variance When 𝜋0 is the uniform random policy,
ナイーブな⼿法: Inverse Propensity Scoring [Strehl+,10] August 2023 Adaptive OPE of Ranking Policies @ KDDʼ23 12 importance weight ・unbiased ・variance!! When 𝜋0 is the uniform random policy, （組み合わせ⾏動による）
AIPSは多様なユーザー⾏動下で正確な推定を可能に IPS (⾚)：どんなユーザー⾏動に対しても分散が⼤きい August 2023 Adaptive OPE of Ranking Policies @ KDDʼ23 45 performance: a lower value is better (simple) (diverse) (complex) user behavior distributions
AIPSは多様なユーザー⾏動下で正確な推定を可能に IIPS (⻘) RIPS (紫)：分散は⼩さいが、複雑なユーザー⾏動で偏りが⼤きい August 2023 Adaptive OPE of Ranking Policies @ KDDʼ23 46 performance: a lower value is better (simple) (diverse) (complex) user behavior distributions
AIPSは多様なユーザー⾏動下で正確な推定を可能に AIPS (true) (灰⾊)：不偏性を満たしながら分散を下げるのでMSEが⼩さく August 2023 Adaptive OPE of Ranking Policies @ KDDʼ23 47 performance: a lower value is better (simple) (diverse) (complex) user behavior distributions
AIPSは多様なユーザー⾏動下で正確な推定を可能に AIPS (true) (灰⾊)：しかし、ユーザー⾏動が複雑になると徐々に分散が⼤きく August 2023 Adaptive OPE of Ranking Policies @ KDDʼ23 48 performance: a lower value is better (simple) (diverse) (complex) user behavior distributions
AIPSは多様なユーザー⾏動下で正確な推定を可能に AIPS (緑) ：偏りと分散を両⽅バランス良く⼩さくできる August 2023 Adaptive OPE of Ranking Policies @ KDDʼ23 49 performance: a lower value is better (simple) (diverse) (complex) user behavior distributions
AIPSは多様なユーザー⾏動下で正確な推定を可能に AIPS (緑) ：特に多様かつ複雑なユーザー⾏動のもとで正確に性能推定! August 2023 Adaptive OPE of Ranking Policies @ KDDʼ23 50 performance: a lower value is better (simple) (diverse) (complex) user behavior distributions
References (1/2) [Saito+,21] Yuta Saito, Shunsuke Aihara, Megumi Matsutani, and Yusuke Narita. “Open Bandit Dataset and Pipeline: Towards Realistic and Reproducible Off-Policy Evaluation.” NeurIPS dataset&benchmark, 2021. https://arxiv.org/abs/2008.07146 [Li+,18] Shuai Li, Yasin Abbasi-Yadkori, Branislav Kveton, S. Muthukrishnan, Vishwa Vinay, and Zheng Wen. “Offline Evaluation of Ranking Policies with Click Models.” KDD, 2018. https://arxiv.org/abs/1804.10488 [McInerney+,20] James McInerney, Brian Brost, Praveen Chandar, Rishabh Mehrotra, and Ben Carterette. “Counterfactual Evaluation of Slate Recommendations with Sequential Reward Interactions.” KDD, 2020. https://arxiv.org/abs/2007.12986 [Strehl+,10] Alex Strehl, John Langford, Sham Kakade, and Lihong Li. “Learning from Logged Implicit Exploration Data.” NeurIPS, 2010. https://arxiv.org/abs/1003.0120 [Athey&Imbens,16] Susan Athey and Guido Imbens. “Recursive Partitioning for Heterogeneous Causal Effects.” PNAS, 2016. https://arxiv.org/abs/1504.01132 August 2023 Adaptive OPE of Ranking Policies @ KDDʼ23 56
aiueola
haraduka
shuntaros
nttcom
matsui_528
gpeyre
yukizenimoto
yosukefk
kuehara
hide2kano
stktu
yuki_koyama
hpprc
productmarketing
wjessup
jcasabona
danielanewman
lemiorhan
caitiem20
hatefulcrawdad
bermonpainter
jacobian
aleyda
paigeccino
morganepeng