パターン認識と機械学習 〜指数型分布族とノンパラメトリック〜

株式会社サイバーエージェントのPRML輪読会で発表した内容です

Transcript

1. ʮύλʔϯೝࣝͱػցֶशʯ
   ྠಡษڧձ
   ʙࢦ਺ܕ෼෍଒ɾϊϯύϥϝτϦοΫ๏ʙ
   

   View Slide

2. ࣗݾ঺հ
   w ໊લ
   w খּݪޫو .JUTVLJ0("4")"3"
   
   w ೖࣾ೥౓
   w ೥౓
   w ॴଐ
   w ג
   $ZCFS;։ൃΤϯδχΞ
   w ֶੜ࣌୅ͷݚڀ෼໺
   w ࣗવݴޠॲཧɾػցֶश
   

   View Slide

3. ໨࣍
   w ࢦ਺ܕ෼෍଒
   w ࠷໬ਪఆͱे෼౷ܭྔ
   w ڞ໾ࣄલ෼෍
   w ແ৘ใࣄલ෼෍
   w ϊϯύϥϝτϦοΫ๏
   w Χʔωϧີ౓ਪఆ๏
   w ࠷ۙ๣๏
   

   View Slide

4. ࢦ਺ܕ෼෍଒ Q
   
   w ࣜ
   Ͱఆٛ͞ΕΔ෼෍ͷ଒ ू߹
   
   !
   w ʮΨ΢ε෼෍ʯʮଟ߲෼෍ʯͳͲɺ

   13.-ʹग़ͯ͘Δଟ͘ͷ෼෍͕ࢦ਺ܕ෼෍଒ʹؚ·ΕΔ

   ˠࣜ
   Ͱఆٛ͠௚͢͜ͱ͕Ͱ͖Δ
   w ˞Y͸εΧϥʔͰ΋ϕΫτϧͰ΋ྑ͍
   w ˞Y͸཭ࢄͰ΋࿈ଓͰ΋ྑ͍
   
   
   

   View Slide

5. ࢦ਺ܕ෼෍଒ Q
   
   !
   w Yʹؔ͢Δؔ਺
   w TDBMJOHDPOTUBOUͱ΋ݺ͹Ε .-B11ΑΓ
   ɺ

   ʮʯ͕ೖΔ͜ͱ΋͋Δ ϕϧψʔΠ෼෍ɺΨϯϚ෼෍
   
   
   
   h (
   x
   )
   

   View Slide

6. ࢦ਺ܕ෼෍଒ Q
   
   !
   w Бʹؔ͢Δؔ਺
   w ֬཰ີ౓ؔ਺ͷੵ෼஋͕ʹͳΔΑ͏ʹ

   ਖ਼نԽ͢ΔͨΊͷ΋ͷ
   
   
   g(⌘)
   g
   (
   ⌘
   )
   Z
   h
   (x) exp
   ⌘T
   u (x)
   d
   x = 1
   
   
   Z
   (
   ⌘
   ) =
   1
   g
   (
   ⌘
   )
   =
   Z
   h
   (x) exp
   ⌘T
   u (x)
   d
   x
   

   View Slide

7. ϕϧψʔΠ෼෍͸ࢦ਺ܕ෼෍଒͔ʁ
   !
   w ແཧ΍ΓFYQͷதʹೖΕͯΈΔ
   !
   !
   !
   w БΛࣜ
   ͷΑ͏ʹఆٛ͢Δ
   Bern
   (
   x
   |
   µ
   ) =
   µx(1
   µ
   )1
   x
   
   Bern(x
   |
   µ) = exp
   {
   ln µx
   (1 µ)
   1
   x}
   = exp
   {
   x ln µ + (1 x) ln 1 µ
   }
   = exp
   {
   x(ln µ ln 1 µ) + ln 1 µ
   }
   = (1 µ) exp
   {
   ln(
   µ
   1 µ
   )x
   }
   
   
   
   ⌘ = ln(
   µ
   1 µ
   )
   

   View Slide

8. ϕϧψʔΠ෼෍͸ࢦ਺ܕ෼෍଒͔ʁ
   !
   w ࠷ऴతʹ͸ɺ
   !
   w ͱͳΓɺࣜ
   ͱରԠͨ͠
   Bern
   (
   x
   |
   µ
   ) =
   µx(1
   µ
   )1
   x
   
   
   
   
   
   

   View Slide

9. ࢀߟɿࢦ਺ܕ෼෍଒ʹؚ·Εͳ͍΋ͷ
   w ࠞ߹ਖ਼ن෼෍

   

   

   

   FYQͷ࿨ʹͳͬͯ͠·͍ɺࣜ
   ʹ͸ͳΒͳ͍
   
   
   

   View Slide

10. ࠷໬ਪఆ
    w ࢦ਺ܕ෼෍଒ͷҰൠܗͷࣜ
    ͔Βɺ

    ࠷໬ਪఆྔБΛٻΊΔ
    w ಠཱʹಉ෼෍ʹै͏σʔλू߹9ʹ͍ͭͯߟ͑Δͱɺ

    ͜ͷ໬౓ؔ਺͸
    !
    w ର਺໬౓ؔ਺͸
    

    View Slide

11. ࠷໬ਪఆ
    w ର਺໬౓ؔ਺ͷ Бʹؔͯ͠ͷ
    ޯ഑͕ͱͳΔ஋Λݟͭ
    ͚͍ͨ
    
    
    

    View Slide

12. ࠷໬ਪఆ
    w ݪଇͱͯ͠ɺࣜ
    Λղ͘ͱБ͸ಘΒΕΔ
    !
    !
    w ·ͨɺ࠷໬ਪఆ஋͸ʹґଘ͢Δ े෼౷ܭྔ
    
    w ݴ͍׵͑Δͱɺ࠷໬ਪఆΛٻΊΔͨΊʹ͸ɺ

    ɹɹɹͷ૯࿨ ·ͨ͸ฏۉ
    ͷΈ͕͋Ε͹Α͍
    
    
    

    View Slide

13. ࠷໬ਪఆͱਅͷύϥϝʔλ
    w Бͷ࠷໬ਪఆ஋͸ࣜ
    Λղ͘ͱಘΒΕΔ
    !
    !
    w ͷఆٛʹجͮ͘ͱɺ
    !
    !
    w ͭ·Γɺ/ˠ㱣ͷۃݶͰ͸ɺ࠷໬ਪఆ஋ʹਅͷ஋
    
    
    g
    (
    ⌘
    )
    Z
    h
    (x) exp
    ⌘T
    u (x)
    d
    x = 1
    
    
    
    
    

    View Slide

14. ڞ໾ࣄલ෼෍
    w ࢦ਺ܕ෼෍଒ͷ೚ҙͷ෼෍ʹ͍ͭͯɺ

    ࣍ͷܗͰॻ͚Δڞ໾ࣄલ෼෍͕ଘࡏ͢Δ
    !
    w ಋग़͸ॻ͍ͯͳ͍͕ɺڞ໾Ͱ͋Δ͜ͱ͕͔֬ΊΒΕΔ

    ໬౓ؔ਺
    ͱࣄલ෼෍
    Λ͔͚ɺ

    ࣄޙ෼෍ΛٻΊΔ
    
    
    

    View Slide

15. ڞ໾ࣄલ෼෍
    w ಋग़͸ॻ͍ͯͳ͍͕ɺڞ໾Ͱ͋Δ͜ͱ͕͔֬ΊΒΕΔ

    ໬౓ؔ਺
    ͱࣄલ෼෍
    Λ͔͚ɺ

    ࣄޙ෼෍ΛٻΊΔ
    
    
    
    
    

    View Slide

16. ڞ໾ࣄલ෼෍
    w ࣄલ෼෍ͷύϥϝʔλΛɺ

    Ծ૝؍ଌ஋ͱͯ͠ղऍ͢Δ͜ͱ΋Ͱ͖Δ
    !
    !
    !
    !
    w DGQɹೋ߲෼෍ͷڞ໾ࣄલ෼෍ʮϕʔλ෼෍ʯͷ

    ɹɹɹɹɹύϥϝʔλΛɺԾ૝ͷ؍ଌͱͯ͠ղऍͨ͠
    
    
    Ծ૝ͷ؍ଌ਺

    /ʹ૬౰
    
    Ծ૝ͷ؍ଌ஋

    V Y
    ʹ૬౰
    
    

    View Slide

17. ແ৘ใࣄલ෼෍
    w ࣄલ෼෍Λஔ͖͍͕ͨɺ෼෍ ΍ύϥϝʔλ
    ʹ͍ͭͯͷ

    ஌͕ࣝͳ͍ͱ͖
    w Ұ༷෼෍Λஔ͚͹ྑ͍ʁ
    !
    w Е͕࿈ଓ͔ͭൣғ͕ܾ·ͬͯͳ͍ͱ͖ɺ

    Еʹ͍ͭͯͷੵ෼͕ൃࢄͯ͠͠·͍ɺਖ਼نԽͰ͖ͳ͍

    ˠมଇࣄલ෼෍
    

    View Slide

18. ແ৘ใࣄલ෼෍
    w ࣍ͷΑ͏ͳฏߦҠಈෆมੑΛ࣋ͬͨ෼෍Λߟ͑Δ

    ྫɿਖ਼ن෼෍
    

    w ˞ฏߦҠಈෆมੑ
    w YΛఆ਺෼Ҡಈͯ͠΋ɺҐஔύϥϝʔλЖΛಉ͚ͩ͡Ҡಈ͢Ε͹ɺ

    ֬཰ີ౓ͷܗ͸มΘΒͳ͍
    
    
    ͷͱ͖ ͱ͢Δͱɺ
    
    
    

    View Slide

19. ແ৘ใࣄલ෼෍
    w ฏߦҠಈෆมੑΛ࣋ͭࣄલ෼෍ʹ͍ͭͯߟ͑Δͱɺ

    ੵ෼͕۠ؒฏߦҠಈͯ͠΋ɺͦͷ֬཰͸มΘΒͳ͍
    !
    !
    w Αͬͯɺࣜ
    ΑΓఆ਺ͱͳΔ

    
    
    
    
    

    View Slide

20. ແ৘ใࣄલ෼෍
    w Ψ΢ε෼෍ͷЖͷ৔߹ɺ

    М@?ˠ㱣ͷۃݶͰແ৘ใࣄલ෼෍ͱͳΔ
    !
    !
    !
    w ࣄޙ෼෍ʹɺࣄલ෼෍ͷύϥϝʔλ͕Өڹ͠ͳ͘ͳΔ
    
    
    
    
    

    View Slide

21. ϊϯύϥϝτϦοΫ๏
    w ύϥϝτϦοΫ
    w ີ౓ؔ਺ Ϟσϧ
    ΛબΜͰɺύϥϝʔλΛσʔλ͔Βਪఆ͢Δ

    ˠϞσϧ͕σʔλΛද͢ͷʹශऑͩͱɺ༧ଌਫ਼౓͸ѱ͍
    w ྫ
    Ψ΢ε෼෍Λσʔλʹ౰ͯ͸ΊͯɺЖɾМ?Λਪఆͨ͠

    ˠσʔλ͕ଟๆੑͩͱɺΨ΢ε෼෍Ͱ͸ଊ͑ΒΕͳ͍
    w ϊϯύϥϝτϦοΫ
    w ෼෍ͷܗঢ়ʹஔ͘Ծఆ͕গͳ͍
    w ྫ
    ଟๆੑͩͱ͔୯ๆੑͳͲͷԾఆ͸ஔ͔ͳ͍
    

    View Slide

22. ώετάϥϜີ౓ਪఆ๏
    w ਅͷ֬཰ີ౓ؔ਺ ྘ઢ
    ͔Β

    ੜ੒͞Εͨͷσʔλ఺ΑΓ

    ਪఆ ੨ώετάϥϜ
    ͨ͠΋ͷ
    w YΛ෯϶ͷ۠ؒʹ۠੾Γɺ

    ͦͷ۠ؒʹೖͬͨYͷ؍ଌ਺Λ

    Χ΢ϯτ͢Δɻ

    ͜ΕΛɺࣜ
    Ͱਖ਼نԽͨ͠΋ͷ
    
    
    

    View Slide

23. ώετάϥϜີ౓ਪఆ๏
    w ࣍ݩɾ̎࣍ݩఔ౓ͷ؆୯ͳՄࢹԽʹ͸໾ཱͭɺ

    ؆ศͳํ๏
    w ͜ͷΞϓϩʔν͔Βɺ࣍ͷ͕̎ͭΘ͔Δ
    w ͋Δ஋ͷ֬཰ີ౓Λਪఆ͢Δʹ͸ɺۙ๣ͷ؍ଌ఺ͷ஋Λߟྀ͢Δ
    ඞཁ͕͋Δ
    w ۠ؒͷ෯͸େ͖͗ͯ͢΋

    খ͗ͯ͢͞΋͍͚ͳ͍
    w খɿσʔλʹӨڹ͗͢͠Δ
    w େɿݩͷ෼෍Λશ͘࠶ݱͰ͖ͳ͍
    w ˠϞσϧͷෳࡶ͞ͷબ୒ʹࣅ͍ͯΔ
    

    View Slide

24. ώετάϥϜີ౓ਪఆ๏ͷ໰୊఺
    w ਪఆͨ͠ີ౓͕ෆ࿈ଓͰ͋Δ ۠ؒͱ۠ؒͷؒ
    
    w ࣍ݩͷढ͍
    w Yͷ࣍ݩ਺Λ%ͱ͢Δͱɺ۠ؒͷ૯਺͸.?%ݸ
    

    View Slide

25. Χʔωϧີ౓ਪఆ๏
    w ະ஌ͷ֬཰ີ౓Q Y
    ͔ΒಘΒΕͨ؍ଌू߹Λ࢖ͬͯɺ

    Q Y
    ͷ஋Λਪఆ͍ͨ͠
    w YΛؚΉখ͞ͳྖҬ3ͷ֬཰Λ1ͱ͢Δ
    !
    w /ݸͷ؍ଌ஋͕ಘΒΕͨͱͯ͠ɺ,ݸͷ؍ଌ஋͕

    3ʹؚ·ΕΔ֬཰͸ɺೋ߲෼෍ʹै͏
    P =
    Z
    R
    p(
    x
    )d
    x
    p(K|N, P) = Bin(K|N, P)
    
    
    
    
    

    View Slide

26. Χʔωϧີ౓ਪఆ๏
    w ೋ߲෼෍ͷظ଴஋ɾ෼ࢄΑΓɺ࣍ͷؔ܎͕ࣜಘΒΕΔ

    

    

    w /͕େ͖͍ͱ͖ɺ෼ࢄ͸খ͘͞ͳΓɺظ଴஋ͷؔ܎͔Β
    w ·ͨɺ3͕খ͘͞ɺQ Y
    ͕3಺ͰҰఆͩͱۙࣅ͢Δͱ
    w Ҏ্ΑΓɺ࣍ͷີ౓ਪఆͷؔ܎͕ࣜಘΒΕΔ
    var
    
    K
    N
    =
    P(1 P)
    N
    E
    
    K
    N
    = P
    K ' NP
    P ' p(
    x
    )V
    p(
    x
    ) =
    K
    NV
    
    
    
    
    
    
    

    View Slide

27. Χʔωϧີ౓ਪఆ๏
    w Ҏ্ΑΓɺ࣍ͷີ౓ਪఆͷؔ܎͕ࣜಘΒΕΔ
    !
    w ֬཰ີ౓Q Y
    Λਪఆ͢ΔͨΊʹɺ,ͱ7Λਪఆ͢Δ
    w ,ΛݻఆͰ7Λਪఆ

    ˠ,ۙ๣ີ౓ਪఆ๏
    w 7ΛݻఆͰ,Λਪఆ

    ˠΧʔωϧີ౓ਪఆ๏
    p(
    x
    ) =
    K
    NV
    
    
    

    View Slide

28. Χʔωϧີ౓ਪఆ๏
    w 7Λݻఆ͠ɺ,Λਪఆ͍ͨ͠
    w ֬཰ີ౓Q Y
    ΛٻΊ͍ͨ఺ΛYɺ؍ଌ఺ΛY@Oͱ͢Δ
    w Ұล͕IͰɺYΛத৺ͱ͢Δখ͞ͳ௒ཱํମͷ

    தʹ͋Δ఺ͷ૯਺͸
    !
    w ҰลIͷ௒ཱํମͳͷͰɺ7͸I?%ͱͳΓɺ
    K =
    K
    X
    n=1
    k
    ✓
    x xn
    h
    ◆
    p(
    x
    ) =
    1
    N
    K
    X
    n=1
    1
    hD
    k
    ✓
    x xn
    h
    ◆
    
    
    
    
    

    View Slide

29. Χʔωϧີ౓ਪఆ๏
    w খ͞ͳ௒ཱํମͷҰลIͷେ͖͕͞

    ฏ׈ԽͷͨΊͷύϥϝʔλʹͳ͍ͬͯΔ
    w I͕ݻఆʹͳͬͯ͠·͏

    ˠσʔλີ౓͕ߴ͍ྖҬͱ௿͍ྖҬͰɺෆ౎߹͕͋Δ
    

    View Slide

30. ,ۙ๣ີ౓ਪఆ๏
    w ,Λݻఆ͠ɺ7Λਪఆ͍ͨ͠
    w ֬཰ີ౓Q Y
    ΛٻΊ͍ͨ఺ΛYɺ؍ଌ఺ΛY@Oͱ͢Δ
    w YΛத৺ͱͯ͠ɺ఺͕,ݸؚ·ΕΔΑ͏ͳ௒ٿΛ୳͢ͱ

    7͸Ұҙʹఆ·Γɺ֬཰ີ౓͸ਪఆ͞ΕΔ
    ਤ͸XXXPDXUJUFDIBDKQJOEFYQIQ NPEVMF(FOFSBMBDUJPO%PXO-PBEpMF QEGUZQFDBMΑΓ
    p(
    x
    ) =
    K
    NV
    

    View Slide

31. ,ۙ๣ີ౓ਪఆ๏
    w ,͕ฏ׈Խύϥϝʔλʔͱͳ͍ͬͯΔ
    

    View Slide

32. ·ͱΊΔͱʜ
    w Χʔωϧີ౓ਪఆ๏
    w ྖҬͷମੵΛݻఆ͢Δ
    w Ұลͷ௕͕͞Iͳ௒ཱํମʹɺ؍ଌ఺YO͕Կݸ͋Δ͔ΛٻΊͨ
    w I͕ฏ׈Խύϥϝʔλʔ
    w ,ۙ๣๏
    w ྖҬ಺ͷɺ؍ଌ఺YOͷݸ਺Λݻఆ͢Δ
    w ؍ଌ఺YO͕LݸʹͳΔΑ͏ʹɺྖҬΛ޿͛ͨ
    w L͕ฏ׈Խύϥϝʔλʔ
    

    View Slide

33. ,ۙ๣๏Λ࢖ͬͨΫϥε෼ྨ
    w ,ۙ๣๏ͱ."1ਪఆΛ࢖ͬͯɺΫϥε෼ྨΛߦ͏
    w YͷΫϥε$@Lͷࣄޙ֬཰ΛٻΊ͍ͨ
    

    View Slide

34. ,ۙ๣๏Λ࢖ͬͨΫϥε෼ྨ
    w ϕΠζͷఆཧΑΓɺ
    !
    w ֬཰ີ౓Q Y
    ͸ɺઌ΄ͲٻΊͨͱ͓Γ
    !
    w ࣄલ෼෍͸ɺશͯͷ؍ଌ఺ͷ͏ͪΫϥεʹଐ͢Δ؍ଌ఺
    !
    w ໬౓͸ɺͦͷΫϥεʹଐ͢Δ؍ଌ఺Ͱͷ֬཰ີ౓ΑΓɺ
    p(Ck
    |
    x
    ) =
    p(
    x
    |Ck)p(Ck)
    p(
    x
    )
    p(
    x
    ) =
    K
    NV
    p(Ck) =
    Nk
    N
    p(
    x
    |Ck) =
    Kk
    NkV
    

    View Slide

35. ,ۙ๣๏Λ࢖ͬͨΫϥε෼ྨ
    w ϕΠζͷఆཧʹ୅ೖ͢Δͱɺ
    !
    w Αͬͯɺ,ۙ๣ͷ͏ͪɺΫϥε$@Lʹଐ͢Δ఺ͷ਺Ͱ

    ଟ਺ܾΛऔΕ͹Α͍
    w ಛʹɺ,ͷͱ͖࠷ۙ๣๏ͱݺ͹ΕΔ
    p(Ck
    |
    x
    ) =
    p(
    x
    |Ck)p(Ck)
    p(
    x
    )
    =
    Kk
    K
    ˖ʹ͍ۙ̏ͭͷ఺Ͱଟ਺ܾΛऔ͍ͬͯΔ
    ࠷ۙ๣๏Ͱ͸ɺ
    ࠷ۙ๣๏Ͱ͸ɺΫϥεͷҟͳΔ఺ͷରͷ

    ਨ௚ೋ౳෼ઢʹͳ͍ͬͯΔ
    

    View Slide

36. ໰୊఺
    w ͋ΔYͷ֬཰ີ౓Q Y
    Λਪఆ͢Δʹ͋ͨͬͯɺ

    શͯͷσʔλ఺Λอ࣋͢Δඞཁ͕͋Δ
    w σʔλ఺͕૿͑Δͱɺۙ๣Λ୳ࡧ͍͕ͯ࣌ؒ͘͠๲େʹ
    ͳΔ

    ˠ୳ࡧ͢ΔͨΊͷ໦ߏ଄Λ࡞Δ
    ຊདྷ͸ɺ࠷΋͍ۙ఺Λશ୳ࡧ͢Δඞཁ͕͋Δ
    

    View Slide

37. ͓ΘΓ
    

    View Slide