Upgrade to Pro — share decks privately, control downloads, hide ads and more …

パターン認識と機械学習 〜指数型分布族とノンパラメトリック〜

パターン認識と機械学習 〜指数型分布族とノンパラメトリック〜

株式会社サイバーエージェントのPRML輪読会で発表した内容です

Mitsuki Ogasahara

July 11, 2014
Tweet

More Decks by Mitsuki Ogasahara

Other Decks in Science

Transcript

  1. ʮύλʔϯೝࣝͱػցֶशʯ
    ྠಡษڧձ
    ʙࢦ਺ܕ෼෍଒ɾϊϯύϥϝτϦοΫ๏ʙ

    View Slide

  2. ࣗݾ঺հ
    w ໊લ
    w খּݪޫو .JUTVLJ0("4")"3"

    w ೖࣾ೥౓
    w ೥౓
    w ॴଐ
    w ג
    $ZCFS;։ൃΤϯδχΞ
    w ֶੜ࣌୅ͷݚڀ෼໺
    w ࣗવݴޠॲཧɾػցֶश

    View Slide

  3. ໨࣍
    w ࢦ਺ܕ෼෍଒
    w ࠷໬ਪఆͱे෼౷ܭྔ
    w ڞ໾ࣄલ෼෍
    w ແ৘ใࣄલ෼෍
    w ϊϯύϥϝτϦοΫ๏
    w Χʔωϧີ౓ਪఆ๏
    w ࠷ۙ๣๏

    View Slide

  4. ࢦ਺ܕ෼෍଒ Q

    w ࣜ
    Ͱఆٛ͞ΕΔ෼෍ͷ଒ ू߹

    !
    w ʮΨ΢ε෼෍ʯʮଟ߲෼෍ʯͳͲɺ

    13.-ʹग़ͯ͘Δଟ͘ͷ෼෍͕ࢦ਺ܕ෼෍଒ʹؚ·ΕΔ

    ˠࣜ
    Ͱఆٛ͠௚͢͜ͱ͕Ͱ͖Δ
    w ˞Y͸εΧϥʔͰ΋ϕΫτϧͰ΋ྑ͍
    w ˞Y͸཭ࢄͰ΋࿈ଓͰ΋ྑ͍


    View Slide

  5. ࢦ਺ܕ෼෍଒ Q

    !
    w Yʹؔ͢Δؔ਺
    w TDBMJOHDPOTUBOUͱ΋ݺ͹Ε .-B11ΑΓ
    ɺ

    ʮʯ͕ೖΔ͜ͱ΋͋Δ ϕϧψʔΠ෼෍ɺΨϯϚ෼෍



    h (
    x
    )

    View Slide

  6. ࢦ਺ܕ෼෍଒ Q

    !
    w Бʹؔ͢Δؔ਺
    w ֬཰ີ౓ؔ਺ͷੵ෼஋͕ʹͳΔΑ͏ʹ

    ਖ਼نԽ͢ΔͨΊͷ΋ͷ


    g(⌘)
    g
    (

    )
    Z
    h
    (x) exp
    ⌘T
    u (x)
    d
    x = 1


    Z
    (

    ) =
    1
    g
    (

    )
    =
    Z
    h
    (x) exp
    ⌘T
    u (x)
    d
    x

    View Slide

  7. ϕϧψʔΠ෼෍͸ࢦ਺ܕ෼෍଒͔ʁ
    !
    w ແཧ΍ΓFYQͷதʹೖΕͯΈΔ
    !
    !
    !
    w БΛࣜ
    ͷΑ͏ʹఆٛ͢Δ
    Bern
    (
    x
    |
    µ
    ) =
    µx(1
    µ
    )1
    x

    Bern(x
    |
    µ) = exp
    {
    ln µx
    (1 µ)
    1
    x}
    = exp
    {
    x ln µ + (1 x) ln 1 µ
    }
    = exp
    {
    x(ln µ ln 1 µ) + ln 1 µ
    }
    = (1 µ) exp
    {
    ln(
    µ
    1 µ
    )x
    }



    ⌘ = ln(
    µ
    1 µ
    )

    View Slide

  8. ϕϧψʔΠ෼෍͸ࢦ਺ܕ෼෍଒͔ʁ
    !
    w ࠷ऴతʹ͸ɺ
    !
    w ͱͳΓɺࣜ
    ͱରԠͨ͠
    Bern
    (
    x
    |
    µ
    ) =
    µx(1
    µ
    )1
    x





    View Slide

  9. ࢀߟɿࢦ਺ܕ෼෍଒ʹؚ·Εͳ͍΋ͷ
    w ࠞ߹ਖ਼ن෼෍




    FYQͷ࿨ʹͳͬͯ͠·͍ɺࣜ
    ʹ͸ͳΒͳ͍


    View Slide

  10. ࠷໬ਪఆ
    w ࢦ਺ܕ෼෍଒ͷҰൠܗͷࣜ
    ͔Βɺ

    ࠷໬ਪఆྔБΛٻΊΔ
    w ಠཱʹಉ෼෍ʹै͏σʔλू߹9ʹ͍ͭͯߟ͑Δͱɺ

    ͜ͷ໬౓ؔ਺͸
    !
    w ର਺໬౓ؔ਺͸

    View Slide

  11. ࠷໬ਪఆ
    w ର਺໬౓ؔ਺ͷ Бʹؔͯ͠ͷ
    ޯ഑͕ͱͳΔ஋Λݟͭ
    ͚͍ͨ


    View Slide

  12. ࠷໬ਪఆ
    w ݪଇͱͯ͠ɺࣜ
    Λղ͘ͱБ͸ಘΒΕΔ
    !
    !
    w ·ͨɺ࠷໬ਪఆ஋͸ʹґଘ͢Δ े෼౷ܭྔ

    w ݴ͍׵͑Δͱɺ࠷໬ਪఆΛٻΊΔͨΊʹ͸ɺ

    ɹɹɹͷ૯࿨ ·ͨ͸ฏۉ
    ͷΈ͕͋Ε͹Α͍


    View Slide

  13. ࠷໬ਪఆͱਅͷύϥϝʔλ
    w Бͷ࠷໬ਪఆ஋͸ࣜ
    Λղ͘ͱಘΒΕΔ
    !
    !
    w ͷఆٛʹجͮ͘ͱɺ
    !
    !
    w ͭ·Γɺ/ˠ㱣ͷۃݶͰ͸ɺ࠷໬ਪఆ஋ʹਅͷ஋


    g
    (

    )
    Z
    h
    (x) exp
    ⌘T
    u (x)
    d
    x = 1




    View Slide

  14. ڞ໾ࣄલ෼෍
    w ࢦ਺ܕ෼෍଒ͷ೚ҙͷ෼෍ʹ͍ͭͯɺ

    ࣍ͷܗͰॻ͚Δڞ໾ࣄલ෼෍͕ଘࡏ͢Δ
    !
    w ಋग़͸ॻ͍ͯͳ͍͕ɺڞ໾Ͱ͋Δ͜ͱ͕͔֬ΊΒΕΔ

    ໬౓ؔ਺
    ͱࣄલ෼෍
    Λ͔͚ɺ

    ࣄޙ෼෍ΛٻΊΔ


    View Slide

  15. ڞ໾ࣄલ෼෍
    w ಋग़͸ॻ͍ͯͳ͍͕ɺڞ໾Ͱ͋Δ͜ͱ͕͔֬ΊΒΕΔ

    ໬౓ؔ਺
    ͱࣄલ෼෍
    Λ͔͚ɺ

    ࣄޙ෼෍ΛٻΊΔ




    View Slide

  16. ڞ໾ࣄલ෼෍
    w ࣄલ෼෍ͷύϥϝʔλΛɺ

    Ծ૝؍ଌ஋ͱͯ͠ղऍ͢Δ͜ͱ΋Ͱ͖Δ
    !
    !
    !
    !
    w DGQɹೋ߲෼෍ͷڞ໾ࣄલ෼෍ʮϕʔλ෼෍ʯͷ

    ɹɹɹɹɹύϥϝʔλΛɺԾ૝ͷ؍ଌͱͯ͠ղऍͨ͠


    Ծ૝ͷ؍ଌ਺

    /ʹ૬౰

    Ծ૝ͷ؍ଌ஋

    V Y
    ʹ૬౰

    View Slide

  17. ແ৘ใࣄલ෼෍
    w ࣄલ෼෍Λஔ͖͍͕ͨɺ෼෍ ΍ύϥϝʔλ
    ʹ͍ͭͯͷ

    ஌͕ࣝͳ͍ͱ͖
    w Ұ༷෼෍Λஔ͚͹ྑ͍ʁ
    !
    w Е͕࿈ଓ͔ͭൣғ͕ܾ·ͬͯͳ͍ͱ͖ɺ

    Еʹ͍ͭͯͷੵ෼͕ൃࢄͯ͠͠·͍ɺਖ਼نԽͰ͖ͳ͍

    ˠมଇࣄલ෼෍

    View Slide

  18. ແ৘ใࣄલ෼෍
    w ࣍ͷΑ͏ͳฏߦҠಈෆมੑΛ࣋ͬͨ෼෍Λߟ͑Δ

    ྫɿਖ਼ن෼෍

    w ˞ฏߦҠಈෆมੑ
    w YΛఆ਺෼Ҡಈͯ͠΋ɺҐஔύϥϝʔλЖΛಉ͚ͩ͡Ҡಈ͢Ε͹ɺ

    ֬཰ີ౓ͷܗ͸มΘΒͳ͍


    ͷͱ͖ ͱ͢Δͱɺ


    View Slide

  19. ແ৘ใࣄલ෼෍
    w ฏߦҠಈෆมੑΛ࣋ͭࣄલ෼෍ʹ͍ͭͯߟ͑Δͱɺ

    ੵ෼͕۠ؒฏߦҠಈͯ͠΋ɺͦͷ֬཰͸มΘΒͳ͍
    !
    !
    w Αͬͯɺࣜ
    ΑΓఆ਺ͱͳΔ





    View Slide

  20. ແ৘ใࣄલ෼෍
    w Ψ΢ε෼෍ͷЖͷ৔߹ɺ

    М@?ˠ㱣ͷۃݶͰແ৘ใࣄલ෼෍ͱͳΔ
    !
    !
    !
    w ࣄޙ෼෍ʹɺࣄલ෼෍ͷύϥϝʔλ͕Өڹ͠ͳ͘ͳΔ




    View Slide

  21. ϊϯύϥϝτϦοΫ๏
    w ύϥϝτϦοΫ
    w ີ౓ؔ਺ Ϟσϧ
    ΛબΜͰɺύϥϝʔλΛσʔλ͔Βਪఆ͢Δ

    ˠϞσϧ͕σʔλΛද͢ͷʹශऑͩͱɺ༧ଌਫ਼౓͸ѱ͍
    w ྫ
    Ψ΢ε෼෍Λσʔλʹ౰ͯ͸ΊͯɺЖɾМ?Λਪఆͨ͠

    ˠσʔλ͕ଟๆੑͩͱɺΨ΢ε෼෍Ͱ͸ଊ͑ΒΕͳ͍
    w ϊϯύϥϝτϦοΫ
    w ෼෍ͷܗঢ়ʹஔ͘Ծఆ͕গͳ͍
    w ྫ
    ଟๆੑͩͱ͔୯ๆੑͳͲͷԾఆ͸ஔ͔ͳ͍

    View Slide

  22. ώετάϥϜີ౓ਪఆ๏
    w ਅͷ֬཰ີ౓ؔ਺ ྘ઢ
    ͔Β

    ੜ੒͞Εͨͷσʔλ఺ΑΓ

    ਪఆ ੨ώετάϥϜ
    ͨ͠΋ͷ
    w YΛ෯϶ͷ۠ؒʹ۠੾Γɺ

    ͦͷ۠ؒʹೖͬͨYͷ؍ଌ਺Λ

    Χ΢ϯτ͢Δɻ

    ͜ΕΛɺࣜ
    Ͱਖ਼نԽͨ͠΋ͷ


    View Slide

  23. ώετάϥϜີ౓ਪఆ๏
    w ࣍ݩɾ̎࣍ݩఔ౓ͷ؆୯ͳՄࢹԽʹ͸໾ཱͭɺ

    ؆ศͳํ๏
    w ͜ͷΞϓϩʔν͔Βɺ࣍ͷ͕̎ͭΘ͔Δ
    w ͋Δ஋ͷ֬཰ີ౓Λਪఆ͢Δʹ͸ɺۙ๣ͷ؍ଌ఺ͷ஋Λߟྀ͢Δ
    ඞཁ͕͋Δ
    w ۠ؒͷ෯͸େ͖͗ͯ͢΋

    খ͗ͯ͢͞΋͍͚ͳ͍
    w খɿσʔλʹӨڹ͗͢͠Δ
    w େɿݩͷ෼෍Λશ͘࠶ݱͰ͖ͳ͍
    w ˠϞσϧͷෳࡶ͞ͷબ୒ʹࣅ͍ͯΔ

    View Slide

  24. ώετάϥϜີ౓ਪఆ๏ͷ໰୊఺
    w ਪఆͨ͠ີ౓͕ෆ࿈ଓͰ͋Δ ۠ؒͱ۠ؒͷؒ

    w ࣍ݩͷढ͍
    w Yͷ࣍ݩ਺Λ%ͱ͢Δͱɺ۠ؒͷ૯਺͸.?%ݸ

    View Slide

  25. Χʔωϧີ౓ਪఆ๏
    w ະ஌ͷ֬཰ີ౓Q Y
    ͔ΒಘΒΕͨ؍ଌू߹Λ࢖ͬͯɺ

    Q Y
    ͷ஋Λਪఆ͍ͨ͠
    w YΛؚΉখ͞ͳྖҬ3ͷ֬཰Λ1ͱ͢Δ
    !
    w /ݸͷ؍ଌ஋͕ಘΒΕͨͱͯ͠ɺ,ݸͷ؍ଌ஋͕

    3ʹؚ·ΕΔ֬཰͸ɺೋ߲෼෍ʹै͏
    P =
    Z
    R
    p(
    x
    )d
    x
    p(K|N, P) = Bin(K|N, P)




    View Slide

  26. Χʔωϧີ౓ਪఆ๏
    w ೋ߲෼෍ͷظ଴஋ɾ෼ࢄΑΓɺ࣍ͷؔ܎͕ࣜಘΒΕΔ



    w /͕େ͖͍ͱ͖ɺ෼ࢄ͸খ͘͞ͳΓɺظ଴஋ͷؔ܎͔Β
    w ·ͨɺ3͕খ͘͞ɺQ Y
    ͕3಺ͰҰఆͩͱۙࣅ͢Δͱ
    w Ҏ্ΑΓɺ࣍ͷີ౓ਪఆͷؔ܎͕ࣜಘΒΕΔ
    var

    K
    N
    =
    P(1 P)
    N
    E

    K
    N
    = P
    K ' NP
    P ' p(
    x
    )V
    p(
    x
    ) =
    K
    NV






    View Slide

  27. Χʔωϧີ౓ਪఆ๏
    w Ҏ্ΑΓɺ࣍ͷີ౓ਪఆͷؔ܎͕ࣜಘΒΕΔ
    !
    w ֬཰ີ౓Q Y
    Λਪఆ͢ΔͨΊʹɺ,ͱ7Λਪఆ͢Δ
    w ,ΛݻఆͰ7Λਪఆ

    ˠ,ۙ๣ີ౓ਪఆ๏
    w 7ΛݻఆͰ,Λਪఆ

    ˠΧʔωϧີ౓ਪఆ๏
    p(
    x
    ) =
    K
    NV


    View Slide

  28. Χʔωϧີ౓ਪఆ๏
    w 7Λݻఆ͠ɺ,Λਪఆ͍ͨ͠
    w ֬཰ີ౓Q Y
    ΛٻΊ͍ͨ఺ΛYɺ؍ଌ఺Λ[email protected]ͱ͢Δ
    w Ұล͕IͰɺYΛத৺ͱ͢Δখ͞ͳ௒ཱํମͷ

    தʹ͋Δ఺ͷ૯਺͸
    !
    w ҰลIͷ௒ཱํମͳͷͰɺ7͸I?%ͱͳΓɺ
    K =
    K
    X
    n=1
    k

    x xn
    h

    p(
    x
    ) =
    1
    N
    K
    X
    n=1
    1
    hD
    k

    x xn
    h





    View Slide

  29. Χʔωϧີ౓ਪఆ๏
    w খ͞ͳ௒ཱํମͷҰลIͷେ͖͕͞

    ฏ׈ԽͷͨΊͷύϥϝʔλʹͳ͍ͬͯΔ
    w I͕ݻఆʹͳͬͯ͠·͏

    ˠσʔλີ౓͕ߴ͍ྖҬͱ௿͍ྖҬͰɺෆ౎߹͕͋Δ

    View Slide

  30. ,ۙ๣ີ౓ਪఆ๏
    w ,Λݻఆ͠ɺ7Λਪఆ͍ͨ͠
    w ֬཰ີ౓Q Y
    ΛٻΊ͍ͨ఺ΛYɺ؍ଌ఺Λ[email protected]ͱ͢Δ
    w YΛத৺ͱͯ͠ɺ఺͕,ݸؚ·ΕΔΑ͏ͳ௒ٿΛ୳͢ͱ

    7͸Ұҙʹఆ·Γɺ֬཰ີ౓͸ਪఆ͞ΕΔ
    ਤ͸XXXPDXUJUFDIBDKQJOEFYQIQ NPEVMF(FOFSBMBDUJPO%PXO-PBEpMF QEGUZQFDBMΑΓ
    p(
    x
    ) =
    K
    NV

    View Slide

  31. ,ۙ๣ີ౓ਪఆ๏
    w ,͕ฏ׈Խύϥϝʔλʔͱͳ͍ͬͯΔ

    View Slide

  32. ·ͱΊΔͱʜ
    w Χʔωϧີ౓ਪఆ๏
    w ྖҬͷମੵΛݻఆ͢Δ
    w Ұลͷ௕͕͞Iͳ௒ཱํମʹɺ؍ଌ఺YO͕Կݸ͋Δ͔ΛٻΊͨ
    w I͕ฏ׈Խύϥϝʔλʔ
    w ,ۙ๣๏
    w ྖҬ಺ͷɺ؍ଌ఺YOͷݸ਺Λݻఆ͢Δ
    w ؍ଌ఺YO͕LݸʹͳΔΑ͏ʹɺྖҬΛ޿͛ͨ
    w L͕ฏ׈Խύϥϝʔλʔ

    View Slide

  33. ,ۙ๣๏Λ࢖ͬͨΫϥε෼ྨ
    w ,ۙ๣๏ͱ."1ਪఆΛ࢖ͬͯɺΫϥε෼ྨΛߦ͏
    w YͷΫϥε[email protected]ͷࣄޙ֬཰ΛٻΊ͍ͨ

    View Slide

  34. ,ۙ๣๏Λ࢖ͬͨΫϥε෼ྨ
    w ϕΠζͷఆཧΑΓɺ
    !
    w ֬཰ີ౓Q Y
    ͸ɺઌ΄ͲٻΊͨͱ͓Γ
    !
    w ࣄલ෼෍͸ɺશͯͷ؍ଌ఺ͷ͏ͪΫϥεʹଐ͢Δ؍ଌ఺
    !
    w ໬౓͸ɺͦͷΫϥεʹଐ͢Δ؍ଌ఺Ͱͷ֬཰ີ౓ΑΓɺ
    p(Ck
    |
    x
    ) =
    p(
    x
    |Ck)p(Ck)
    p(
    x
    )
    p(
    x
    ) =
    K
    NV
    p(Ck) =
    Nk
    N
    p(
    x
    |Ck) =
    Kk
    NkV

    View Slide

  35. ,ۙ๣๏Λ࢖ͬͨΫϥε෼ྨ
    w ϕΠζͷఆཧʹ୅ೖ͢Δͱɺ
    !
    w Αͬͯɺ,ۙ๣ͷ͏ͪɺΫϥε[email protected]ʹଐ͢Δ఺ͷ਺Ͱ

    ଟ਺ܾΛऔΕ͹Α͍
    w ಛʹɺ,ͷͱ͖࠷ۙ๣๏ͱݺ͹ΕΔ
    p(Ck
    |
    x
    ) =
    p(
    x
    |Ck)p(Ck)
    p(
    x
    )
    =
    Kk
    K
    ˖ʹ͍ۙ̏ͭͷ఺Ͱଟ਺ܾΛऔ͍ͬͯΔ
    ࠷ۙ๣๏Ͱ͸ɺ
    ࠷ۙ๣๏Ͱ͸ɺΫϥεͷҟͳΔ఺ͷରͷ

    ਨ௚ೋ౳෼ઢʹͳ͍ͬͯΔ

    View Slide

  36. ໰୊఺
    w ͋ΔYͷ֬཰ີ౓Q Y
    Λਪఆ͢Δʹ͋ͨͬͯɺ

    શͯͷσʔλ఺Λอ࣋͢Δඞཁ͕͋Δ
    w σʔλ఺͕૿͑Δͱɺۙ๣Λ୳ࡧ͍͕ͯ࣌ؒ͘͠๲େʹ
    ͳΔ

    ˠ୳ࡧ͢ΔͨΊͷ໦ߏ଄Λ࡞Δ
    ຊདྷ͸ɺ࠷΋͍ۙ఺Λશ୳ࡧ͢Δඞཁ͕͋Δ

    View Slide

  37. ͓ΘΓ

    View Slide