CFMLの概要と研究動向 / cfml #1 introduction

CFML勉強会#1の資料です。(https://connpass.com/event/128714/)

Transcript

1. CFMLͷ֓ཁͱݚڀಈ޲ Kazuki Taniguchi CFMLษڧձ#1
2. • ৬ྺ • 2014.4-2019.3 • גࣜձࣾαΠόʔΤʔδΣϯτ ΞυςΫຊ෦ AI Lab • 2019.4- • ౎಺ͷελʔτΞοϓ • ϑϦʔϥϯε(AI/MLͷݚڀ։ൃ) • ݚڀ෼໺ • Pattern Recognition / Image Super Resolution • Recommendation / Response Prediction • Counterfactual ML ࣗݾ঺հ ୩ޱ ࿨ً (@kazk1018)
3. Summary • Counterfactual Machine Learningͷ֓ཁ • ൓ࣄ࣮͕ੜ͡ΔσʔλΛ༻͍ͨػցֶशͰ͋Δ • Interactive LearningͱCausal Inference͕ڞมྔγϑτͷ෦ ෼໰୊Ͱ͋Δ͜ͱΛઆ໌͢Δ • ݚڀಈ޲ • ޿ࠂͷίϯϖʹΑͬͯσʔληοτ͕ެ։͞Ε͍ͯΔ • ֶ֤ձͷWorkshopΛ঺հ͢Δ
4. Introduction
5. ໰୊ઃఆ ঎඼" ঎඼# ঎඼$ ෳ਺ͷީิͷத͔Β୯ҰͷΞΠςϜΛϢʔβʹਪન͠ɺ Ϣʔβ͔ΒFeedbackΛಘΔ໰୊Λߟ͑Δ ঎඼" બ୒ ΠΠω!!
6. ໰୊ઃఆ ͋ΔϢʔβ͕ਪનͨ͠঎඼ΛΫϦοΫ(Feedback)͔ͨ͠ Ͳ͏͔ͷϩάΛऔಘ͍ͯ͠Δ ࣌ࠁ ঎඼ Ϣʔβ ΫϦοΫ 
    " 9 /P 
    # : /P 
    $ 9 :FT 
    # ; /P 
    $ ; /P աڈϩά
7. ໰୊ઃఆ Ϣʔβͷ৘ใ͔Β঎඼Λܾఆ͢Δํࡦ(Policy)ʹ͍ͭͯ
 ҎԼͷํ๏Λߟ͑Δ • طଘͷPolicyΛ ͱͨ͠ͱ͖ʹ৽͍͠Policy ΛධՁ͍ͨ͠ (Evaluation) • طଘͷϩάΛར༻ͯ͠৽͍͠ Λֶश͍ͨ͠ (Learning) π0 π π
8. Online Evaluation Randomized Controlled Experiment (A/B Testing) • ਖ਼֬ͳൺֱΛߦ͏͜ͱ͕Մೳ (Gold Standard) • ݁Ռ͕ग़Δ·Ͱʹ͕͔͔࣌ؒΔ • ৽͍͠ํࡦͷόά΍UX௿ԼͷϦεΫ͕͋Δ • ຊ൪ಋೖͷ։ൃίετ͕େ͖͍ 50% 50% π π0
9. Offline Evaluation • طଘͷϩάσʔλ͔Β৽͍͠PolicyΛධՁ͢Δ • ࣮ࡍʹ͸ݟ͍ͤͯͳ͍঎඼Λબ୒͢Δ৔߹͸ධՁ͕Ͱ ͖ͳ͍ ࣌ࠁ ঎඼ Ϣʔβ ΫϦοΫ 
    " 9 /P → BΛਪનͨ͠৔߹͸ʁ π(x) ΫϦοΫ͞ΕΔ͔Ͳ͏͔͸Θ͔Βͳ͍ (Counterfactual)
10. Learning • Learning͸ධՁ஋͕࠷େͱͳΔํࡦΛݟ͚ͭΔ • ධՁ͸ઌड़ͨ͠௨Γਖ਼͘͠ߦ͏͜ͱ͕Ͱ͖ͳ͍
11. Counterfactual Machine Learning ൓ࣄ࣮͕ੜ͡ΔσʔλΛ༻͍ͨػցֶश Causal Inference Interactive Learning Counterfactual Machine Learning ※͜ͷൃදͰ͸ڞมྔγϑτͷ؍఺͔Βݟ͍ͯ͘ (From A Machine Learning Perspective)
12. Related Works
13. ڞมྔγϑτ • ҎԼͷ৚݅ͷ໰୊Λѻ͏ p(x) ≠ p′(x) p(y|x) = p′(y|x) ͸ ʹಠཱʹै͍, ͸ ʹ ಠཱʹै͏ͱԾఆ͢Δ D = {(xi , yi )}n i=1 p(x, y) D′ = {x′ i }m i=1 ∫ p′(x, y)dy ͜ͷͱ͖, ࣍ͷΑ͏ͳ৚݅Λຬͨ͢ͱ͖Λڞมྔγϑτͱ͍͏
14. ڞมྔγϑτ • ྫ) Ի੠ೝࣝ, ը૾ೝࣝ, etc…
 ɹ : Ի੠σʔλ, : ࿩ऀ, : ࢠڙ, : େਓ
 ɹ : ը૾σʔλ, : ਓ෺, : ࣨ಺, : ԰֎ x y p(x) p′(x) x y p(x) p′(x) p(x, y) = p(y|x)p(x) p′(x, y) = p(y|x)p′(x) x y y′ x′
15. ڞมྔγϑτԼͷ༧ଌϞσϧ • ͱ Λ༻͍ͯ৽ͨͳೖྗ ʹର͢Δऔಘ Λ ༧ଌ͢ΔϞσϧ Λֶश͍ͨ͠ • ଛࣦؔ਺Λ ͱ͢Δͱ͖ڞมྔγϑτ͸ॏཁ౓ॏΈ෇ ͖Λ༻͍Δ͜ͱͰղܾ͢Δ͜ͱ͕஌ΒΕ͍ͯΔ {(xi , yi )}n i=0 {x′}m i=0 x′ y fθ (x) loss(y, fθ ) minθ n ∑ i=0 w(xi )loss(yi , fθ (xi ))) w(xi ) = p′(xi ) p(xi )
16. ڞมྔγϑτԼͷ༧ଌϞσϧ • ূ໌ Ep′[loss(y, fθ (x))] = ∫ ∫ loss(y, fθ (x))p′(x, y)dxdy = ∫ ∫ loss(y, fθ (x))p′(x)p′(y|x)dxdy = ∫ ∫ loss(y, fθ (x))p′(x)p′(y|x) p(x) p(x) dxdy = ∫ ∫ loss(y, fθ (x))p(x)p(y|x) p′(x) p(x) dxdy = ∫ ∫ loss(y, fθ (x))p(x, y)w(x)dxdy ≈ n ∑ i=0 loss(y, fθ (x))w(x)
17. Interactive Learning Context: x Policy: π(x) Action: a = π(x) Reward: δ(x, a) System a User $POUFYU "DUJPO 3FXBSE 5JNF
    
    
    6TFS
    9 " /P 5JNF
    
    
    6TFS
    : # /P 5JNF
    
    
    6TFS
    9 $ :FT 5JNF
    
    
    6TFS
    ; # /P 5JNF
    
    
    6TFS
    : $ /P Logging ᶃ ᶄ ᶅ ᶆ x ∼ P(x)
18. Interactive Learning 3FDPNNFOEBUJPO
    $POUFYUVBM
    CBOEJU 3FJOGPSDFNFOU
    -FBSOJOH $POUFYU 6TFS
    BOE
    *UFN
    *OGPSNBUJPO $POUFYU 4UBUF "DUJPO *UFN
    *% "SN "DUJPO 3FXBSE $MJDL
    1VSDIBTF 3FXBSE 3FXBSE (ৄ͘͠͸੪౻ͷൃදͰ)
19. Causal Inference • ͋ΔༀΛױऀʹ౤༩͢Δ͔Ͳ͏͔Ͱ3೔ޙʹපؾ͕࣏ͬ ͍ͯΔ͔Ͳ͏͔ͷҼՌޮՌΛଌΓ͍ͨ ☓ ױऀ: x Treatment: t පؾ͕࣏Δ͔Ͳ͏͔: y ༀΛ౤༩͍ͯ͠ͳ͍৔߹ ༀΛ౤༩ͨ͠৔߹
20. Causal Inference • ࣮ࡍʹ؍ଌ͢Δͷ͸ͲͪΒ͔Ұํ͚ͩͰ͋Δ ☓ ױऀ: x Treatment: t පؾ͕࣏Δ͔Ͳ͏͔: y ҼՌਪ࿦ͷࠜຊ໰୊ ൓ࣄ࣮ (ৄ͘͠͸҆ҪͷൃදͰ) ༀΛ౤༩͍ͯ͠ͳ͍৔߹ ༀΛ౤༩ͨ͠৔߹
21. ڞมྔγϑτͱͷؔ܎ • Interactive Learning • ͷੜ੒աఔʹ؍ଌ͕͋Δ͔Ͳ͏͔ͷҧ͍ • ϢʔβͷinteractionʹΑΔ؍ଌͷ༗ແͰੜ·ΕΔγϑτ D p(x) = q(o|x)p′(x) w(xi ) = 1 q(o|xi )
22. ڞมྔγϑτͱͷؔ܎ • Causal Inference • ͷੜ੒աఔʹtreatment͕͋Δ͔Ͳ͏͔ͷҧ͍ • ዞҙతͳtreatmentͷׂ౰ʹΑͬͯੜ·ΕΔγϑτ D p(x) = q(t|x)P(x) p′(x) = q(¬t|x)P(x) w(x) = q(¬t|x) q(t|x) (=Propensity Score)
23. ݚڀಈ޲
24. • CounterfactualΛѻͬͨػցֶशͷCompetition • ੈք࠷େͷDSPͰ͋ΔCriteo͕σʔληοτΛެ։ • ޿ࠂͷ഑ஔͷ༧ଌͰΫϦοΫͷ࠷େԽ • CounterfactualΛߟྀͨ͠ධՁࢦඪΛಋೖ Criteo Ad Placement Challenge https://www.crowdai.org/challenges/nips-17-workshop-criteo-ad-placement-challenge
25. Workshops • KDD • Causal Discovery, 2018/2019 • Offline and Online Evaluation of Interactive Systems, 2019 • NIPS • From ‘What if?’ To ‘What next?’, 2017 • Causal Learning, 2018 • RecSys • REVEAL, 2018/2019 • ICML • FAIM’18 Workshop(CausalML)
26. Summary
27. Summary • Counterfactual Machine Learningͷ֓ཁ • ൓ࣄ࣮͕ੜ͡ΔσʔλΛ༻͍ͨػցֶशͰ͋Δ • Interactive LearningͱCausal Inference͕ڞมྔγϑτͷ෦ ෼໰୊Ͱ͋Δ͜ͱΛઆ໌ͨ͠ • ݚڀಈ޲ • ޿ࠂͷίϯϖʹΑͬͯσʔληοτ͕ެ։͞Ε͍ͯΔ • ֶ֤ձͷWorkshopΛ঺հ͢Δ
28. References 1. Home Page of Thorsten Joachims
 http://www.cs.cornell.edu/people/tj/
 2. Counterfactual Reasoning and Learning Systems: The Example of Computational Advertising, Leon Bottou, et al…, JMLR 2013
 https://www.microsoft.com/en-us/research/wp-content/uploads/2013/11/ bottou13a.pdf
 3. ඇఆৗ؀ڥԼͰͷֶशɿڞมྔγϑτదԠɼ ΫϥεόϥϯεมԽదԠɼมԽݕ஌
 http://www.ms.k.u-tokyo.ac.jp/2014/NonstationarityReview-jp.pdf
 4. Offline Evaluation and Optimization for Interactive Systems
 https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/ tutorial.pdf Πϥετ by ͔Θ͍͍ϑϦʔૉࡐू ͍Β͢ͱ΍