Pro Yearly is on sale from $80 to $50! »

CFMLの概要と研究動向 / cfml #1 introduction

CFMLの概要と研究動向 / cfml #1 introduction

CFML勉強会#1の資料です。(https://connpass.com/event/128714/)

F2bd408c57c20505bd4202c9e28ecd59?s=128

Kazuki Taniguchi

July 10, 2019
Tweet

Transcript

  1. CFMLͷ֓ཁͱݚڀಈ޲ Kazuki Taniguchi CFMLษڧձ#1

  2. • ৬ྺ • 2014.4-2019.3 • גࣜձࣾαΠόʔΤʔδΣϯτ ΞυςΫຊ෦ AI Lab •

    2019.4- • ౎಺ͷελʔτΞοϓ • ϑϦʔϥϯε(AI/MLͷݚڀ։ൃ) • ݚڀ෼໺ • Pattern Recognition / Image Super Resolution • Recommendation / Response Prediction • Counterfactual ML ࣗݾ঺հ ୩ޱ ࿨ً (@kazk1018)
  3. Summary • Counterfactual Machine Learningͷ֓ཁ • ൓ࣄ࣮͕ੜ͡ΔσʔλΛ༻͍ͨػցֶशͰ͋Δ • Interactive LearningͱCausal

    Inference͕ڞมྔγϑτͷ෦ ෼໰୊Ͱ͋Δ͜ͱΛઆ໌͢Δ • ݚڀಈ޲ • ޿ࠂͷίϯϖʹΑͬͯσʔληοτ͕ެ։͞Ε͍ͯΔ • ֶ֤ձͷWorkshopΛ঺հ͢Δ
  4. Introduction

  5. ໰୊ઃఆ ঎඼" ঎඼# ঎඼$ ෳ਺ͷީิͷத͔Β୯ҰͷΞΠςϜΛϢʔβʹਪન͠ɺ Ϣʔβ͔ΒFeedbackΛಘΔ໰୊Λߟ͑Δ ঎඼" બ୒ ΠΠω!!

  6. ໰୊ઃఆ ͋ΔϢʔβ͕ਪનͨ͠঎඼ΛΫϦοΫ(Feedback)͔ͨ͠ Ͳ͏͔ͷϩάΛऔಘ͍ͯ͠Δ ࣌ࠁ ঎඼ Ϣʔβ ΫϦοΫ  " 9

    /P  # : /P  $ 9 :FT  # ; /P  $ ; /P աڈϩά
  7. ໰୊ઃఆ Ϣʔβͷ৘ใ͔Β঎඼Λܾఆ͢Δํࡦ(Policy)ʹ͍ͭͯ
 ҎԼͷํ๏Λߟ͑Δ • طଘͷPolicyΛ ͱͨ͠ͱ͖ʹ৽͍͠Policy ΛධՁ͍ͨ͠ (Evaluation) • طଘͷϩάΛར༻ͯ͠৽͍͠

    Λֶश͍ͨ͠ (Learning) π0 π π
  8. Online Evaluation Randomized Controlled Experiment (A/B Testing) • ਖ਼֬ͳൺֱΛߦ͏͜ͱ͕Մೳ (Gold

    Standard) • ݁Ռ͕ग़Δ·Ͱʹ͕͔͔࣌ؒΔ • ৽͍͠ํࡦͷόά΍UX௿ԼͷϦεΫ͕͋Δ • ຊ൪ಋೖͷ։ൃίετ͕େ͖͍ 50% 50% π π0
  9. Offline Evaluation • طଘͷϩάσʔλ͔Β৽͍͠PolicyΛධՁ͢Δ • ࣮ࡍʹ͸ݟ͍ͤͯͳ͍঎඼Λબ୒͢Δ৔߹͸ධՁ͕Ͱ ͖ͳ͍ ࣌ࠁ ঎඼ Ϣʔβ

    ΫϦοΫ  " 9 /P → BΛਪનͨ͠৔߹͸ʁ π(x) ΫϦοΫ͞ΕΔ͔Ͳ͏͔͸Θ͔Βͳ͍ (Counterfactual)
  10. Learning • Learning͸ධՁ஋͕࠷େͱͳΔํࡦΛݟ͚ͭΔ • ධՁ͸ઌड़ͨ͠௨Γਖ਼͘͠ߦ͏͜ͱ͕Ͱ͖ͳ͍

  11. Counterfactual Machine Learning ൓ࣄ࣮͕ੜ͡ΔσʔλΛ༻͍ͨػցֶश Causal Inference Interactive Learning Counterfactual Machine

    Learning ※͜ͷൃදͰ͸ڞมྔγϑτͷ؍఺͔Βݟ͍ͯ͘ (From A Machine Learning Perspective)
  12. Related Works

  13. ڞมྔγϑτ • ҎԼͷ৚݅ͷ໰୊Λѻ͏ p(x) ≠ p′(x) p(y|x) = p′(y|x) ͸

    ʹಠཱʹै͍, ͸ ʹ ಠཱʹै͏ͱԾఆ͢Δ D = {(xi , yi )}n i=1 p(x, y) D′ = {x′ i }m i=1 ∫ p′(x, y)dy ͜ͷͱ͖, ࣍ͷΑ͏ͳ৚݅Λຬͨ͢ͱ͖Λڞมྔγϑτͱ͍͏
  14. ڞมྔγϑτ • ྫ) Ի੠ೝࣝ, ը૾ೝࣝ, etc…
 ɹ : Ի੠σʔλ, :

    ࿩ऀ, : ࢠڙ, : େਓ
 ɹ : ը૾σʔλ, : ਓ෺, : ࣨ಺, : ԰֎ x y p(x) p′(x) x y p(x) p′(x) p(x, y) = p(y|x)p(x) p′(x, y) = p(y|x)p′(x) x y y′ x′
  15. ڞมྔγϑτԼͷ༧ଌϞσϧ • ͱ Λ༻͍ͯ৽ͨͳೖྗ ʹର͢Δऔಘ Λ ༧ଌ͢ΔϞσϧ Λֶश͍ͨ͠ • ଛࣦؔ਺Λ

    ͱ͢Δͱ͖ڞมྔγϑτ͸ॏཁ౓ॏΈ෇ ͖Λ༻͍Δ͜ͱͰղܾ͢Δ͜ͱ͕஌ΒΕ͍ͯΔ {(xi , yi )}n i=0 {x′}m i=0 x′ y fθ (x) loss(y, fθ ) minθ n ∑ i=0 w(xi )loss(yi , fθ (xi ))) w(xi ) = p′(xi ) p(xi )
  16. ڞมྔγϑτԼͷ༧ଌϞσϧ • ূ໌ Ep′[loss(y, fθ (x))] = ∫ ∫ loss(y,

    fθ (x))p′(x, y)dxdy = ∫ ∫ loss(y, fθ (x))p′(x)p′(y|x)dxdy = ∫ ∫ loss(y, fθ (x))p′(x)p′(y|x) p(x) p(x) dxdy = ∫ ∫ loss(y, fθ (x))p(x)p(y|x) p′(x) p(x) dxdy = ∫ ∫ loss(y, fθ (x))p(x, y)w(x)dxdy ≈ n ∑ i=0 loss(y, fθ (x))w(x)
  17. Interactive Learning Context: x Policy: π(x) Action: a = π(x)

    Reward: δ(x, a) System a User $POUFYU "DUJPO 3FXBSE 5JNF 6TFS9 " /P 5JNF 6TFS: # /P 5JNF 6TFS9 $ :FT 5JNF 6TFS; # /P 5JNF 6TFS: $ /P Logging ᶃ ᶄ ᶅ ᶆ x ∼ P(x)
  18. Interactive Learning 3FDPNNFOEBUJPO $POUFYUVBM CBOEJU 3FJOGPSDFNFOU -FBSOJOH $POUFYU 6TFSBOE*UFN *OGPSNBUJPO

    $POUFYU 4UBUF "DUJPO *UFN *% "SN "DUJPO 3FXBSE $MJDL 1VSDIBTF 3FXBSE 3FXBSE (ৄ͘͠͸੪౻ͷൃදͰ)
  19. Causal Inference • ͋ΔༀΛױऀʹ౤༩͢Δ͔Ͳ͏͔Ͱ3೔ޙʹපؾ͕࣏ͬ ͍ͯΔ͔Ͳ͏͔ͷҼՌޮՌΛଌΓ͍ͨ ☓ ױऀ: x Treatment: t

    පؾ͕࣏Δ͔Ͳ͏͔: y ༀΛ౤༩͍ͯ͠ͳ͍৔߹ ༀΛ౤༩ͨ͠৔߹
  20. Causal Inference • ࣮ࡍʹ؍ଌ͢Δͷ͸ͲͪΒ͔Ұํ͚ͩͰ͋Δ ☓ ױऀ: x Treatment: t පؾ͕࣏Δ͔Ͳ͏͔:

    y ҼՌਪ࿦ͷࠜຊ໰୊ ൓ࣄ࣮ (ৄ͘͠͸҆ҪͷൃදͰ) ༀΛ౤༩͍ͯ͠ͳ͍৔߹ ༀΛ౤༩ͨ͠৔߹
  21. ڞมྔγϑτͱͷؔ܎ • Interactive Learning • ͷੜ੒աఔʹ؍ଌ͕͋Δ͔Ͳ͏͔ͷҧ͍ • ϢʔβͷinteractionʹΑΔ؍ଌͷ༗ແͰੜ·ΕΔγϑτ D p(x)

    = q(o|x)p′(x) w(xi ) = 1 q(o|xi )
  22. ڞมྔγϑτͱͷؔ܎ • Causal Inference • ͷੜ੒աఔʹtreatment͕͋Δ͔Ͳ͏͔ͷҧ͍ • ዞҙతͳtreatmentͷׂ౰ʹΑͬͯੜ·ΕΔγϑτ D p(x)

    = q(t|x)P(x) p′(x) = q(¬t|x)P(x) w(x) = q(¬t|x) q(t|x) (=Propensity Score)
  23. ݚڀಈ޲

  24. • CounterfactualΛѻͬͨػցֶशͷCompetition • ੈք࠷େͷDSPͰ͋ΔCriteo͕σʔληοτΛެ։ • ޿ࠂͷ഑ஔͷ༧ଌͰΫϦοΫͷ࠷େԽ • CounterfactualΛߟྀͨ͠ධՁࢦඪΛಋೖ Criteo Ad

    Placement Challenge https://www.crowdai.org/challenges/nips-17-workshop-criteo-ad-placement-challenge
  25. Workshops • KDD • Causal Discovery, 2018/2019 • Offline and

    Online Evaluation of Interactive Systems, 2019 • NIPS • From ‘What if?’ To ‘What next?’, 2017 • Causal Learning, 2018 • RecSys • REVEAL, 2018/2019 • ICML • FAIM’18 Workshop(CausalML)
  26. Summary

  27. Summary • Counterfactual Machine Learningͷ֓ཁ • ൓ࣄ࣮͕ੜ͡ΔσʔλΛ༻͍ͨػցֶशͰ͋Δ • Interactive LearningͱCausal

    Inference͕ڞมྔγϑτͷ෦ ෼໰୊Ͱ͋Δ͜ͱΛઆ໌ͨ͠ • ݚڀಈ޲ • ޿ࠂͷίϯϖʹΑͬͯσʔληοτ͕ެ։͞Ε͍ͯΔ • ֶ֤ձͷWorkshopΛ঺հ͢Δ
  28. References 1. Home Page of Thorsten Joachims
 http://www.cs.cornell.edu/people/tj/
 2. Counterfactual

    Reasoning and Learning Systems: The Example of Computational Advertising, Leon Bottou, et al…, JMLR 2013
 https://www.microsoft.com/en-us/research/wp-content/uploads/2013/11/ bottou13a.pdf
 3. ඇఆৗ؀ڥԼͰͷֶशɿڞมྔγϑτదԠɼ ΫϥεόϥϯεมԽదԠɼมԽݕ஌
 http://www.ms.k.u-tokyo.ac.jp/2014/NonstationarityReview-jp.pdf
 4. Offline Evaluation and Optimization for Interactive Systems
 https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/ tutorial.pdf Πϥετ by ͔Θ͍͍ϑϦʔૉࡐू ͍Β͢ͱ΍