Mitsuki Ogasahara
July 11, 2014
280

# パターン認識と機械学習 〜指数型分布族とノンパラメトリック〜

July 11, 2014

## Transcript

2. ### ࣗݾ঺հ w ໊લ w খּݪޫو .JUTVLJ0("4")"3"  w ೖࣾ೥౓ w

೥౓ w ॴଐ w ג \$ZCFS;։ൃΤϯδχΞ w ֶੜ࣌୅ͷݚڀ෼໺ w ࣗવݴޠॲཧɾػցֶश
3. ### ໨࣍ w ࢦ਺ܕ෼෍଒ w ࠷໬ਪఆͱे෼౷ܭྔ w ڞ໾ࣄલ෼෍ w ແ৘ใࣄલ෼෍ w

ϊϯύϥϝτϦοΫ๏ w Χʔωϧີ౓ਪఆ๏ w ࠷ۙ๣๏
4. ### ࢦ਺ܕ෼෍଒ Q w ࣜ  Ͱఆٛ͞ΕΔ෼෍ͷ଒ ू߹  ! w

ʮΨ΢ε෼෍ʯʮଟ߲෼෍ʯͳͲɺ  13.-ʹग़ͯ͘Δଟ͘ͷ෼෍͕ࢦ਺ܕ෼෍଒ʹؚ·ΕΔ  ˠࣜ  Ͱఆٛ͠௚͢͜ͱ͕Ͱ͖Δ w ˞Y͸εΧϥʔͰ΋ϕΫτϧͰ΋ྑ͍ w ˞Y͸཭ࢄͰ΋࿈ଓͰ΋ྑ͍ 
5. ### ࢦ਺ܕ෼෍଒ Q ! w Yʹؔ͢Δؔ਺ w TDBMJOHDPOTUBOUͱ΋ݺ͹Ε .-B11ΑΓ ɺ  ʮʯ͕ೖΔ͜ͱ΋͋Δ

ϕϧψʔΠ෼෍ɺΨϯϚ෼෍  h ( x )
6. ### ࢦ਺ܕ෼෍଒ Q ! w Бʹؔ͢Δؔ਺ w ֬཰ີ౓ؔ਺ͷੵ෼஋͕ʹͳΔΑ͏ʹ  ਖ਼نԽ͢ΔͨΊͷ΋ͷ  g(⌘)

g ( ⌘ ) Z h (x) exp ⌘T u (x) d x = 1  Z ( ⌘ ) = 1 g ( ⌘ ) = Z h (x) exp ⌘T u (x) d x
7. ### ϕϧψʔΠ෼෍͸ࢦ਺ܕ෼෍଒͔ʁ ! w ແཧ΍ΓFYQͷதʹೖΕͯΈΔ ! ! ! w БΛࣜ 

ͷΑ͏ʹఆٛ͢Δ Bern ( x | µ ) = µx(1 µ )1 x  Bern(x | µ) = exp { ln µx (1 µ) 1 x} = exp { x ln µ + (1 x) ln 1 µ } = exp { x(ln µ ln 1 µ) + ln 1 µ } = (1 µ) exp { ln( µ 1 µ )x }   ⌘ = ln( µ 1 µ )
8. ### ϕϧψʔΠ෼෍͸ࢦ਺ܕ෼෍଒͔ʁ ! w ࠷ऴతʹ͸ɺ ! w ͱͳΓɺࣜ  ͱରԠͨ͠ Bern

( x | µ ) = µx(1 µ )1 x   

w ର਺໬౓ؔ਺͸

12. ### ࠷໬ਪఆ w ݪଇͱͯ͠ɺࣜ  Λղ͘ͱБ͸ಘΒΕΔ ! ! w ·ͨɺ࠷໬ਪఆ஋͸ʹґଘ͢Δ े෼౷ܭྔ

 w ݴ͍׵͑Δͱɺ࠷໬ਪఆΛٻΊΔͨΊʹ͸ɺ  ɹɹɹͷ૯࿨ ·ͨ͸ฏۉ ͷΈ͕͋Ε͹Α͍ 
13. ### ࠷໬ਪఆͱਅͷύϥϝʔλ w Бͷ࠷໬ਪఆ஋͸ࣜ  Λղ͘ͱಘΒΕΔ ! ! w ͷఆٛʹجͮ͘ͱɺ !

! w ͭ·Γɺ/ˠ㱣ͷۃݶͰ͸ɺ࠷໬ਪఆ஋ʹਅͷ஋  g ( ⌘ ) Z h (x) exp ⌘T u (x) d x = 1  
14. ### ڞ໾ࣄલ෼෍ w ࢦ਺ܕ෼෍଒ͷ೚ҙͷ෼෍ʹ͍ͭͯɺ  ࣍ͷܗͰॻ͚Δڞ໾ࣄલ෼෍͕ଘࡏ͢Δ ! w ಋग़͸ॻ͍ͯͳ͍͕ɺڞ໾Ͱ͋Δ͜ͱ͕͔֬ΊΒΕΔ  ໬౓ؔ਺  ͱࣄલ෼෍

 Λ͔͚ɺ  ࣄޙ෼෍ΛٻΊΔ 


16. ### ڞ໾ࣄલ෼෍ w ࣄલ෼෍ͷύϥϝʔλΛɺ  Ծ૝؍ଌ஋ͱͯ͠ղऍ͢Δ͜ͱ΋Ͱ͖Δ ! ! ! ! w DGQɹೋ߲෼෍ͷڞ໾ࣄલ෼෍ʮϕʔλ෼෍ʯͷ

ɹɹɹɹɹύϥϝʔλΛɺԾ૝ͷ؍ଌͱͯ͠ղऍͨ͠  Ծ૝ͷ؍ଌ਺  /ʹ૬౰ Ծ૝ͷ؍ଌ஋  V Y ʹ૬౰
17. ### ແ৘ใࣄલ෼෍ w ࣄલ෼෍Λஔ͖͍͕ͨɺ෼෍ ΍ύϥϝʔλ ʹ͍ͭͯͷ  ஌͕ࣝͳ͍ͱ͖ w Ұ༷෼෍Λஔ͚͹ྑ͍ʁ ! w

Е͕࿈ଓ͔ͭൣғ͕ܾ·ͬͯͳ͍ͱ͖ɺ  Еʹ͍ͭͯͷੵ෼͕ൃࢄͯ͠͠·͍ɺਖ਼نԽͰ͖ͳ͍  ˠมଇࣄલ෼෍
18. ### ແ৘ใࣄલ෼෍ w ࣍ͷΑ͏ͳฏߦҠಈෆมੑΛ࣋ͬͨ෼෍Λߟ͑Δ  ྫɿਖ਼ن෼෍   w ˞ฏߦҠಈෆมੑ w YΛఆ਺෼Ҡಈͯ͠΋ɺҐஔύϥϝʔλЖΛಉ͚ͩ͡Ҡಈ͢Ε͹ɺ  ֬཰ີ౓ͷܗ͸มΘΒͳ͍

 ͷͱ͖ ͱ͢Δͱɺ 

 


21. ### ϊϯύϥϝτϦοΫ๏ w ύϥϝτϦοΫ w ີ౓ؔ਺ Ϟσϧ ΛબΜͰɺύϥϝʔλΛσʔλ͔Βਪఆ͢Δ  ˠϞσϧ͕σʔλΛද͢ͷʹශऑͩͱɺ༧ଌਫ਼౓͸ѱ͍ w ྫ

Ψ΢ε෼෍Λσʔλʹ౰ͯ͸ΊͯɺЖɾМ?Λਪఆͨ͠  ˠσʔλ͕ଟๆੑͩͱɺΨ΢ε෼෍Ͱ͸ଊ͑ΒΕͳ͍ w ϊϯύϥϝτϦοΫ w ෼෍ͷܗঢ়ʹஔ͘Ծఆ͕গͳ͍ w ྫ ଟๆੑͩͱ͔୯ๆੑͳͲͷԾఆ͸ஔ͔ͳ͍
22. ### ώετάϥϜີ౓ਪఆ๏ w ਅͷ֬཰ີ౓ؔ਺ ྘ઢ ͔Β  ੜ੒͞Εͨͷσʔλ఺ΑΓ  ਪఆ ੨ώετάϥϜ ͨ͠΋ͷ w

YΛ෯϶ͷ۠ؒʹ۠੾Γɺ  ͦͷ۠ؒʹೖͬͨYͷ؍ଌ਺Λ  Χ΢ϯτ͢Δɻ  ͜ΕΛɺࣜ  Ͱਖ਼نԽͨ͠΋ͷ 
23. ### ώετάϥϜີ౓ਪఆ๏ w ࣍ݩɾ̎࣍ݩఔ౓ͷ؆୯ͳՄࢹԽʹ͸໾ཱͭɺ  ؆ศͳํ๏ w ͜ͷΞϓϩʔν͔Βɺ࣍ͷ͕̎ͭΘ͔Δ w ͋Δ஋ͷ֬཰ີ౓Λਪఆ͢Δʹ͸ɺۙ๣ͷ؍ଌ఺ͷ஋Λߟྀ͢Δ ඞཁ͕͋Δ w

۠ؒͷ෯͸େ͖͗ͯ͢΋  খ͗ͯ͢͞΋͍͚ͳ͍ w খɿσʔλʹӨڹ͗͢͠Δ w େɿݩͷ෼෍Λશ͘࠶ݱͰ͖ͳ͍ w ˠϞσϧͷෳࡶ͞ͷબ୒ʹࣅ͍ͯΔ

25. ### Χʔωϧີ౓ਪఆ๏ w ະ஌ͷ֬཰ີ౓Q Y ͔ΒಘΒΕͨ؍ଌू߹Λ࢖ͬͯɺ  Q Y ͷ஋Λਪఆ͍ͨ͠ w YΛؚΉখ͞ͳྖҬ3ͷ֬཰Λ1ͱ͢Δ

! w /ݸͷ؍ଌ஋͕ಘΒΕͨͱͯ͠ɺ,ݸͷ؍ଌ஋͕  3ʹؚ·ΕΔ֬཰͸ɺೋ߲෼෍ʹै͏ P = Z R p( x )d x p(K|N, P) = Bin(K|N, P)  
26. ### Χʔωϧີ౓ਪఆ๏ w ೋ߲෼෍ͷظ଴஋ɾ෼ࢄΑΓɺ࣍ͷؔ܎͕ࣜಘΒΕΔ      w /͕େ͖͍ͱ͖ɺ෼ࢄ͸খ͘͞ͳΓɺظ଴஋ͷؔ܎͔Β w ·ͨɺ3͕খ͘͞ɺQ Y

͕3಺ͰҰఆͩͱۙࣅ͢Δͱ w Ҏ্ΑΓɺ࣍ͷີ౓ਪఆͷؔ܎͕ࣜಘΒΕΔ var  K N = P(1 P) N E  K N = P K ' NP P ' p( x )V p( x ) = K NV   
27. ### Χʔωϧີ౓ਪఆ๏ w Ҏ্ΑΓɺ࣍ͷີ౓ਪఆͷؔ܎͕ࣜಘΒΕΔ ! w ֬཰ີ౓Q Y Λਪఆ͢ΔͨΊʹɺ,ͱ7Λਪఆ͢Δ w ,ΛݻఆͰ7Λਪఆ

ˠ,ۙ๣ີ౓ਪఆ๏ w 7ΛݻఆͰ,Λਪఆ  ˠΧʔωϧີ౓ਪఆ๏ p( x ) = K NV 
28. ### Χʔωϧີ౓ਪఆ๏ w 7Λݻఆ͠ɺ,Λਪఆ͍ͨ͠ w ֬཰ີ౓Q Y ΛٻΊ͍ͨ఺ΛYɺ؍ଌ఺ΛY@Oͱ͢Δ w Ұล͕IͰɺYΛத৺ͱ͢Δখ͞ͳ௒ཱํମͷ  தʹ͋Δ఺ͷ૯਺͸

! w ҰลIͷ௒ཱํମͳͷͰɺ7͸I?%ͱͳΓɺ K = K X n=1 k ✓ x xn h ◆ p( x ) = 1 N K X n=1 1 hD k ✓ x xn h ◆  

30. ### ,ۙ๣ີ౓ਪఆ๏ w ,Λݻఆ͠ɺ7Λਪఆ͍ͨ͠ w ֬཰ີ౓Q Y ΛٻΊ͍ͨ఺ΛYɺ؍ଌ఺ΛY@Oͱ͢Δ w YΛத৺ͱͯ͠ɺ఺͕,ݸؚ·ΕΔΑ͏ͳ௒ٿΛ୳͢ͱ  7͸Ұҙʹఆ·Γɺ֬཰ີ౓͸ਪఆ͞ΕΔ

ਤ͸XXXPDXUJUFDIBDKQJOEFYQIQ NPEVMF(FOFSBMBDUJPO%PXO-PBEpMF QEGUZQFDBMΑΓ p( x ) = K NV

32. ### ·ͱΊΔͱʜ w Χʔωϧີ౓ਪఆ๏ w ྖҬͷମੵΛݻఆ͢Δ w Ұลͷ௕͕͞Iͳ௒ཱํମʹɺ؍ଌ఺YO͕Կݸ͋Δ͔ΛٻΊͨ w I͕ฏ׈Խύϥϝʔλʔ w

,ۙ๣๏ w ྖҬ಺ͷɺ؍ଌ఺YOͷݸ਺Λݻఆ͢Δ w ؍ଌ఺YO͕LݸʹͳΔΑ͏ʹɺྖҬΛ޿͛ͨ w L͕ฏ׈Խύϥϝʔλʔ

34. ### ,ۙ๣๏Λ࢖ͬͨΫϥε෼ྨ w ϕΠζͷఆཧΑΓɺ ! w ֬཰ີ౓Q Y ͸ɺઌ΄ͲٻΊͨͱ͓Γ ! w

ࣄલ෼෍͸ɺશͯͷ؍ଌ఺ͷ͏ͪΫϥεʹଐ͢Δ؍ଌ఺ ! w ໬౓͸ɺͦͷΫϥεʹଐ͢Δ؍ଌ఺Ͱͷ֬཰ີ౓ΑΓɺ p(Ck | x ) = p( x |Ck)p(Ck) p( x ) p( x ) = K NV p(Ck) = Nk N p( x |Ck) = Kk NkV
35. ### ,ۙ๣๏Λ࢖ͬͨΫϥε෼ྨ w ϕΠζͷఆཧʹ୅ೖ͢Δͱɺ ! w Αͬͯɺ,ۙ๣ͷ͏ͪɺΫϥε\$@Lʹଐ͢Δ఺ͷ਺Ͱ  ଟ਺ܾΛऔΕ͹Α͍ w ಛʹɺ,ͷͱ͖࠷ۙ๣๏ͱݺ͹ΕΔ p(Ck

| x ) = p( x |Ck)p(Ck) p( x ) = Kk K ˖ʹ͍ۙ̏ͭͷ఺Ͱଟ਺ܾΛऔ͍ͬͯΔ ࠷ۙ๣๏Ͱ͸ɺ ࠷ۙ๣๏Ͱ͸ɺΫϥεͷҟͳΔ఺ͷରͷ  ਨ௚ೋ౳෼ઢʹͳ͍ͬͯΔ
36. ### ໰୊఺ w ͋ΔYͷ֬཰ີ౓Q Y Λਪఆ͢Δʹ͋ͨͬͯɺ  શͯͷσʔλ఺Λอ࣋͢Δඞཁ͕͋Δ w σʔλ఺͕૿͑Δͱɺۙ๣Λ୳ࡧ͍͕ͯ࣌ؒ͘͠๲େʹ ͳΔ  ˠ୳ࡧ͢ΔͨΊͷ໦ߏ଄Λ࡞Δ

ຊདྷ͸ɺ࠷΋͍ۙ఺Λશ୳ࡧ͢Δඞཁ͕͋Δ