Mitsuki Ogasahara
July 11, 2014
240

# パターン認識と機械学習 〜指数型分布族とノンパラメトリック〜

July 11, 2014

## Transcript

1. ʮύλʔϯೝࣝͱػցֶशʯ
ྠಡษڧձ
ʙࢦ਺ܕ෼෍଒ɾϊϯύϥϝτϦοΫ๏ʙ

2. ࣗݾ঺հ
w ໊લ
w খּݪޫو .JUTVLJ0("4")"3"

w ೖࣾ೥౓
w ೥౓
w ॴଐ
w ג
\$ZCFS;։ൃΤϯδχΞ
w ֶੜ࣌୅ͷݚڀ෼໺
w ࣗવݴޠॲཧɾػցֶश

3. ໨࣍
w ࢦ਺ܕ෼෍଒
w ࠷໬ਪఆͱे෼౷ܭྔ
w ڞ໾ࣄલ෼෍
w ແ৘ใࣄલ෼෍
w ϊϯύϥϝτϦοΫ๏
w Χʔωϧີ౓ਪఆ๏
w ࠷ۙ๣๏

4. ࢦ਺ܕ෼෍଒ Q

w ࣜ
Ͱఆٛ͞ΕΔ෼෍ͷ଒ ू߹

!
w ʮΨ΢ε෼෍ʯʮଟ߲෼෍ʯͳͲɺ
13.-ʹग़ͯ͘Δଟ͘ͷ෼෍͕ࢦ਺ܕ෼෍଒ʹؚ·ΕΔ
ˠࣜ
Ͱఆٛ͠௚͢͜ͱ͕Ͱ͖Δ
w ˞Y͸εΧϥʔͰ΋ϕΫτϧͰ΋ྑ͍
w ˞Y͸཭ࢄͰ΋࿈ଓͰ΋ྑ͍

5. ࢦ਺ܕ෼෍଒ Q

!
w Yʹؔ͢Δؔ਺
w TDBMJOHDPOTUBOUͱ΋ݺ͹Ε .-B11ΑΓ
ɺ
ʮʯ͕ೖΔ͜ͱ΋͋Δ ϕϧψʔΠ෼෍ɺΨϯϚ෼෍

h (
x
)

6. ࢦ਺ܕ෼෍଒ Q

!
w Бʹؔ͢Δؔ਺
w ֬཰ີ౓ؔ਺ͷੵ෼஋͕ʹͳΔΑ͏ʹ
ਖ਼نԽ͢ΔͨΊͷ΋ͷ

g(⌘)
g
(

)
Z
h
(x) exp
⌘T
u (x)
d
x = 1

Z
(

) =
1
g
(

)
=
Z
h
(x) exp
⌘T
u (x)
d
x

7. ϕϧψʔΠ෼෍͸ࢦ਺ܕ෼෍଒͔ʁ
!
w ແཧ΍ΓFYQͷதʹೖΕͯΈΔ
!
!
!
w БΛࣜ
ͷΑ͏ʹఆٛ͢Δ
Bern
(
x
|
µ
) =
µx(1
µ
)1
x

Bern(x
|
µ) = exp
{
ln µx
(1 µ)
1
x}
= exp
{
x ln µ + (1 x) ln 1 µ
}
= exp
{
x(ln µ ln 1 µ) + ln 1 µ
}
= (1 µ) exp
{
ln(
µ
1 µ
)x
}

⌘ = ln(
µ
1 µ
)

8. ϕϧψʔΠ෼෍͸ࢦ਺ܕ෼෍଒͔ʁ
!
w ࠷ऴతʹ͸ɺ
!
w ͱͳΓɺࣜ
ͱରԠͨ͠
Bern
(
x
|
µ
) =
µx(1
µ
)1
x

9. ࢀߟɿࢦ਺ܕ෼෍଒ʹؚ·Εͳ͍΋ͷ
w ࠞ߹ਖ਼ن෼෍

FYQͷ࿨ʹͳͬͯ͠·͍ɺࣜ
ʹ͸ͳΒͳ͍

10. ࠷໬ਪఆ
w ࢦ਺ܕ෼෍଒ͷҰൠܗͷࣜ
͔Βɺ
࠷໬ਪఆྔБΛٻΊΔ
w ಠཱʹಉ෼෍ʹै͏σʔλू߹9ʹ͍ͭͯߟ͑Δͱɺ
͜ͷ໬౓ؔ਺͸
!
w ର਺໬౓ؔ਺͸

11. ࠷໬ਪఆ
w ର਺໬౓ؔ਺ͷ Бʹؔͯ͠ͷ
ޯ഑͕ͱͳΔ஋Λݟͭ
͚͍ͨ

12. ࠷໬ਪఆ
w ݪଇͱͯ͠ɺࣜ
Λղ͘ͱБ͸ಘΒΕΔ
!
!
w ·ͨɺ࠷໬ਪఆ஋͸ʹґଘ͢Δ े෼౷ܭྔ

w ݴ͍׵͑Δͱɺ࠷໬ਪఆΛٻΊΔͨΊʹ͸ɺ
ɹɹɹͷ૯࿨ ·ͨ͸ฏۉ
ͷΈ͕͋Ε͹Α͍

13. ࠷໬ਪఆͱਅͷύϥϝʔλ
w Бͷ࠷໬ਪఆ஋͸ࣜ
Λղ͘ͱಘΒΕΔ
!
!
w ͷఆٛʹجͮ͘ͱɺ
!
!
w ͭ·Γɺ/ˠ㱣ͷۃݶͰ͸ɺ࠷໬ਪఆ஋ʹਅͷ஋

g
(

)
Z
h
(x) exp
⌘T
u (x)
d
x = 1

14. ڞ໾ࣄલ෼෍
w ࢦ਺ܕ෼෍଒ͷ೚ҙͷ෼෍ʹ͍ͭͯɺ
࣍ͷܗͰॻ͚Δڞ໾ࣄલ෼෍͕ଘࡏ͢Δ
!
w ಋग़͸ॻ͍ͯͳ͍͕ɺڞ໾Ͱ͋Δ͜ͱ͕͔֬ΊΒΕΔ
໬౓ؔ਺
ͱࣄલ෼෍
Λ͔͚ɺ
ࣄޙ෼෍ΛٻΊΔ

15. ڞ໾ࣄલ෼෍
w ಋग़͸ॻ͍ͯͳ͍͕ɺڞ໾Ͱ͋Δ͜ͱ͕͔֬ΊΒΕΔ
໬౓ؔ਺
ͱࣄલ෼෍
Λ͔͚ɺ
ࣄޙ෼෍ΛٻΊΔ

16. ڞ໾ࣄલ෼෍
w ࣄલ෼෍ͷύϥϝʔλΛɺ
Ծ૝؍ଌ஋ͱͯ͠ղऍ͢Δ͜ͱ΋Ͱ͖Δ
!
!
!
!
w DGQɹೋ߲෼෍ͷڞ໾ࣄલ෼෍ʮϕʔλ෼෍ʯͷ
ɹɹɹɹɹύϥϝʔλΛɺԾ૝ͷ؍ଌͱͯ͠ղऍͨ͠

Ծ૝ͷ؍ଌ਺
/ʹ૬౰

Ծ૝ͷ؍ଌ஋
V Y
ʹ૬౰

17. ແ৘ใࣄલ෼෍
w ࣄલ෼෍Λஔ͖͍͕ͨɺ෼෍ ΍ύϥϝʔλ
ʹ͍ͭͯͷ
஌͕ࣝͳ͍ͱ͖
w Ұ༷෼෍Λஔ͚͹ྑ͍ʁ
!
w Е͕࿈ଓ͔ͭൣғ͕ܾ·ͬͯͳ͍ͱ͖ɺ
Еʹ͍ͭͯͷੵ෼͕ൃࢄͯ͠͠·͍ɺਖ਼نԽͰ͖ͳ͍
ˠมଇࣄલ෼෍

18. ແ৘ใࣄલ෼෍
w ࣍ͷΑ͏ͳฏߦҠಈෆมੑΛ࣋ͬͨ෼෍Λߟ͑Δ
ྫɿਖ਼ن෼෍

w ˞ฏߦҠಈෆมੑ
w YΛఆ਺෼Ҡಈͯ͠΋ɺҐஔύϥϝʔλЖΛಉ͚ͩ͡Ҡಈ͢Ε͹ɺ
֬཰ີ౓ͷܗ͸มΘΒͳ͍

ͷͱ͖ ͱ͢Δͱɺ

19. ແ৘ใࣄલ෼෍
w ฏߦҠಈෆมੑΛ࣋ͭࣄલ෼෍ʹ͍ͭͯߟ͑Δͱɺ
ੵ෼͕۠ؒฏߦҠಈͯ͠΋ɺͦͷ֬཰͸มΘΒͳ͍
!
!
w Αͬͯɺࣜ
ΑΓఆ਺ͱͳΔ

20. ແ৘ใࣄલ෼෍
w Ψ΢ε෼෍ͷЖͷ৔߹ɺ
М@?ˠ㱣ͷۃݶͰແ৘ใࣄલ෼෍ͱͳΔ
!
!
!
w ࣄޙ෼෍ʹɺࣄલ෼෍ͷύϥϝʔλ͕Өڹ͠ͳ͘ͳΔ

21. ϊϯύϥϝτϦοΫ๏
w ύϥϝτϦοΫ
w ີ౓ؔ਺ Ϟσϧ
ΛબΜͰɺύϥϝʔλΛσʔλ͔Βਪఆ͢Δ
ˠϞσϧ͕σʔλΛද͢ͷʹශऑͩͱɺ༧ଌਫ਼౓͸ѱ͍
w ྫ
Ψ΢ε෼෍Λσʔλʹ౰ͯ͸ΊͯɺЖɾМ?Λਪఆͨ͠
ˠσʔλ͕ଟๆੑͩͱɺΨ΢ε෼෍Ͱ͸ଊ͑ΒΕͳ͍
w ϊϯύϥϝτϦοΫ
w ෼෍ͷܗঢ়ʹஔ͘Ծఆ͕গͳ͍
w ྫ
ଟๆੑͩͱ͔୯ๆੑͳͲͷԾఆ͸ஔ͔ͳ͍

22. ώετάϥϜີ౓ਪఆ๏
w ਅͷ֬཰ີ౓ؔ਺ ྘ઢ
͔Β
ੜ੒͞Εͨͷσʔλ఺ΑΓ
ਪఆ ੨ώετάϥϜ
ͨ͠΋ͷ
w YΛ෯϶ͷ۠ؒʹ۠੾Γɺ
ͦͷ۠ؒʹೖͬͨYͷ؍ଌ਺Λ
Χ΢ϯτ͢Δɻ
͜ΕΛɺࣜ
Ͱਖ਼نԽͨ͠΋ͷ

23. ώετάϥϜີ౓ਪఆ๏
w ࣍ݩɾ̎࣍ݩఔ౓ͷ؆୯ͳՄࢹԽʹ͸໾ཱͭɺ
؆ศͳํ๏
w ͜ͷΞϓϩʔν͔Βɺ࣍ͷ͕̎ͭΘ͔Δ
w ͋Δ஋ͷ֬཰ີ౓Λਪఆ͢Δʹ͸ɺۙ๣ͷ؍ଌ఺ͷ஋Λߟྀ͢Δ
ඞཁ͕͋Δ
w ۠ؒͷ෯͸େ͖͗ͯ͢΋
খ͗ͯ͢͞΋͍͚ͳ͍
w খɿσʔλʹӨڹ͗͢͠Δ
w େɿݩͷ෼෍Λશ͘࠶ݱͰ͖ͳ͍
w ˠϞσϧͷෳࡶ͞ͷબ୒ʹࣅ͍ͯΔ

24. ώετάϥϜີ౓ਪఆ๏ͷ໰୊఺
w ਪఆͨ͠ີ౓͕ෆ࿈ଓͰ͋Δ ۠ؒͱ۠ؒͷؒ

w ࣍ݩͷढ͍
w Yͷ࣍ݩ਺Λ%ͱ͢Δͱɺ۠ؒͷ૯਺͸.?%ݸ

25. Χʔωϧີ౓ਪఆ๏
w ະ஌ͷ֬཰ີ౓Q Y
͔ΒಘΒΕͨ؍ଌू߹Λ࢖ͬͯɺ
Q Y
ͷ஋Λਪఆ͍ͨ͠
w YΛؚΉখ͞ͳྖҬ3ͷ֬཰Λ1ͱ͢Δ
!
w /ݸͷ؍ଌ஋͕ಘΒΕͨͱͯ͠ɺ,ݸͷ؍ଌ஋͕
3ʹؚ·ΕΔ֬཰͸ɺೋ߲෼෍ʹै͏
P =
Z
R
p(
x
)d
x
p(K|N, P) = Bin(K|N, P)

26. Χʔωϧີ౓ਪఆ๏
w ೋ߲෼෍ͷظ଴஋ɾ෼ࢄΑΓɺ࣍ͷؔ܎͕ࣜಘΒΕΔ

w /͕େ͖͍ͱ͖ɺ෼ࢄ͸খ͘͞ͳΓɺظ଴஋ͷؔ܎͔Β
w ·ͨɺ3͕খ͘͞ɺQ Y
͕3಺ͰҰఆͩͱۙࣅ͢Δͱ
w Ҏ্ΑΓɺ࣍ͷີ౓ਪఆͷؔ܎͕ࣜಘΒΕΔ
var

K
N
=
P(1 P)
N
E

K
N
= P
K ' NP
P ' p(
x
)V
p(
x
) =
K
NV

27. Χʔωϧີ౓ਪఆ๏
w Ҏ্ΑΓɺ࣍ͷີ౓ਪఆͷؔ܎͕ࣜಘΒΕΔ
!
w ֬཰ີ౓Q Y
Λਪఆ͢ΔͨΊʹɺ,ͱ7Λਪఆ͢Δ
w ,ΛݻఆͰ7Λਪఆ
ˠ,ۙ๣ີ౓ਪఆ๏
w 7ΛݻఆͰ,Λਪఆ
ˠΧʔωϧີ౓ਪఆ๏
p(
x
) =
K
NV

28. Χʔωϧີ౓ਪఆ๏
w 7Λݻఆ͠ɺ,Λਪఆ͍ͨ͠
w ֬཰ີ౓Q Y
ΛٻΊ͍ͨ఺ΛYɺ؍ଌ఺Λ[email protected]ͱ͢Δ
w Ұล͕IͰɺYΛத৺ͱ͢Δখ͞ͳ௒ཱํମͷ
தʹ͋Δ఺ͷ૯਺͸
!
w ҰลIͷ௒ཱํମͳͷͰɺ7͸I?%ͱͳΓɺ
K =
K
X
n=1
k

x xn
h

p(
x
) =
1
N
K
X
n=1
1
hD
k

x xn
h

29. Χʔωϧີ౓ਪఆ๏
w খ͞ͳ௒ཱํମͷҰลIͷେ͖͕͞
ฏ׈ԽͷͨΊͷύϥϝʔλʹͳ͍ͬͯΔ
w I͕ݻఆʹͳͬͯ͠·͏
ˠσʔλີ౓͕ߴ͍ྖҬͱ௿͍ྖҬͰɺෆ౎߹͕͋Δ

30. ,ۙ๣ີ౓ਪఆ๏
w ,Λݻఆ͠ɺ7Λਪఆ͍ͨ͠
w ֬཰ີ౓Q Y
ΛٻΊ͍ͨ఺ΛYɺ؍ଌ఺Λ[email protected]ͱ͢Δ
w YΛத৺ͱͯ͠ɺ఺͕,ݸؚ·ΕΔΑ͏ͳ௒ٿΛ୳͢ͱ
7͸Ұҙʹఆ·Γɺ֬཰ີ౓͸ਪఆ͞ΕΔ
ਤ͸XXXPDXUJUFDIBDKQJOEFYQIQ NPEVMF(FOFSBMBDUJPO%PXO-PBEpMF QEGUZQFDBMΑΓ
p(
x
) =
K
NV

31. ,ۙ๣ີ౓ਪఆ๏
w ,͕ฏ׈Խύϥϝʔλʔͱͳ͍ͬͯΔ

32. ·ͱΊΔͱʜ
w Χʔωϧີ౓ਪఆ๏
w ྖҬͷମੵΛݻఆ͢Δ
w Ұลͷ௕͕͞Iͳ௒ཱํମʹɺ؍ଌ఺YO͕Կݸ͋Δ͔ΛٻΊͨ
w I͕ฏ׈Խύϥϝʔλʔ
w ,ۙ๣๏
w ྖҬ಺ͷɺ؍ଌ఺YOͷݸ਺Λݻఆ͢Δ
w ؍ଌ఺YO͕LݸʹͳΔΑ͏ʹɺྖҬΛ޿͛ͨ
w L͕ฏ׈Խύϥϝʔλʔ

33. ,ۙ๣๏Λ࢖ͬͨΫϥε෼ྨ
w ,ۙ๣๏ͱ."1ਪఆΛ࢖ͬͯɺΫϥε෼ྨΛߦ͏
w YͷΫϥε[email protected]ͷࣄޙ֬཰ΛٻΊ͍ͨ

34. ,ۙ๣๏Λ࢖ͬͨΫϥε෼ྨ
w ϕΠζͷఆཧΑΓɺ
!
w ֬཰ີ౓Q Y
͸ɺઌ΄ͲٻΊͨͱ͓Γ
!
w ࣄલ෼෍͸ɺશͯͷ؍ଌ఺ͷ͏ͪΫϥεʹଐ͢Δ؍ଌ఺
!
w ໬౓͸ɺͦͷΫϥεʹଐ͢Δ؍ଌ఺Ͱͷ֬཰ີ౓ΑΓɺ
p(Ck
|
x
) =
p(
x
|Ck)p(Ck)
p(
x
)
p(
x
) =
K
NV
p(Ck) =
Nk
N
p(
x
|Ck) =
Kk
NkV

35. ,ۙ๣๏Λ࢖ͬͨΫϥε෼ྨ
w ϕΠζͷఆཧʹ୅ೖ͢Δͱɺ
!
w Αͬͯɺ,ۙ๣ͷ͏ͪɺΫϥε[email protected]ʹଐ͢Δ఺ͷ਺Ͱ
ଟ਺ܾΛऔΕ͹Α͍
w ಛʹɺ,ͷͱ͖࠷ۙ๣๏ͱݺ͹ΕΔ
p(Ck
|
x
) =
p(
x
|Ck)p(Ck)
p(
x
)
=
Kk
K
˖ʹ͍ۙ̏ͭͷ఺Ͱଟ਺ܾΛऔ͍ͬͯΔ
࠷ۙ๣๏Ͱ͸ɺ
࠷ۙ๣๏Ͱ͸ɺΫϥεͷҟͳΔ఺ͷରͷ
ਨ௚ೋ౳෼ઢʹͳ͍ͬͯΔ

36. ໰୊఺
w ͋ΔYͷ֬཰ີ౓Q Y
Λਪఆ͢Δʹ͋ͨͬͯɺ
શͯͷσʔλ఺Λอ࣋͢Δඞཁ͕͋Δ
w σʔλ఺͕૿͑Δͱɺۙ๣Λ୳ࡧ͍͕ͯ࣌ؒ͘͠๲େʹ
ͳΔ
ˠ୳ࡧ͢ΔͨΊͷ໦ߏ଄Λ࡞Δ
ຊདྷ͸ɺ࠷΋͍ۙ఺Λશ୳ࡧ͢Δඞཁ͕͋Δ

37. ͓ΘΓ