「はじめてのパターン認識」読書会 第 4 章

Transcript

1. ʮ͸͡Ίͯͷύλʔϯೝࣝʯಡॻձ ୈ 4 ষ 2017 ೥ 11 ݄ 9 ೔
 horiem
2. Πϯτϩ: 100 ԁۄͷਅآ൑ఆ ॏ͞ [g] ௚ܘ [μm] ൒ܘ [μm] … ϥϕϧ 4.801 22601 11301 … ਅ 4.751 22599 11300 … آ 4.799 22602 11301 … ਅ … … … … …
3. Πϯτϩ: 100 ԁۄͷਅآ൑ఆ ॏ͞ [g] ௚ܘ [μm] ൒ܘ [μm] … ϥϕϧ 4.801 22601 11301 … ਅ 4.751 22599 11300 … آ 4.799 22602 11301 … ਅ … … … … … ૬ؔ͋ΔͷͰ͸ʁ
4. Πϯτϩ: 100 ԁۄͷਅآ൑ఆ ॏ͞ [g] ௚ܘ [μm] ൒ܘ [μm] … ϥϕϧ 4.801 22601 11301 … ਅ 4.751 22599 11300 … آ 4.799 22602 11301 … ਅ … … … … … ૬ؔ͋ΔͷͰ͸ʁ
5. ॏ͞ [g] ௚ܘ [μm] ൒ܘ [μm] … ϥϕϧ 4.801 22601 11301 … ਅ 4.751 22599 11300 … آ 4.799 22602 11301 … ਅ … … … … … Πϯτϩ: 100 ԁۄͷਅآ൑ఆ 100 ԁۄͷฏۉʢ4.8 gʣΑΓܰͦ͏͕ͩ
 ࠩ͸ 0.05 [g]
6. ॏ͞ [g] ௚ܘ [μm] ൒ܘ [μm] … ϥϕϧ 4.801 22601 11301 … ਅ 4.751 22599 11300 … آ 4.799 22602 11301 … ਅ … … … … … Πϯτϩ: 100 ԁۄͷਅآ൑ఆ 100 ԁۄͷฏۉʢ22600 μmʣͱಉ͡Α͏͕ͩ
 ࠩ͸ 2 [μm] >> 0.05
7. ॏ͞ [g] ௚ܘ [μm] ൒ܘ [μm] … ϥϕϧ 4.801 22601 11301 … ਅ 4.751 22599 11300 … آ 4.799 22602 11301 … ਅ … … … … … Πϯτϩ: 100 ԁۄͷਅآ൑ఆ • ಛ௃ؒͷ૬ؔΛͳ͍ͨ͘͠ • ୯Ґ͕ҧ͍ͬͯͯ΋౷ܭతʹൺֱ͍ͨ͠
8. ॏ͞ [g] ௚ܘ [μm] ൒ܘ [μm] … ϥϕϧ 4.801 22601 11301 … ਅ 4.751 22599 11300 … آ 4.799 22602 11301 … ਅ … … … … … Πϯτϩ: 100 ԁۄͷਅآ൑ఆ • ಛ௃ؒͷ૬ؔΛͳ͍ͨ͘͠ • ୯Ґ͕ҧ͍ͬͯͯ΋౷ܭతʹൺֱ͍ͨ͠ ➡ ؍ଌσʔλΛม׵͠Α͏ʂ
9. 4 ষͷ΋͘͡ 4. ֬཰Ϟσϧͱࣝผؔ਺ 1. ؍ଌσʔλͷઢܗม׵ 2. ֬཰Ϟσϧ 3. ֬཰Ϟσϧύϥϝʔλͷ࠷໬ਪఆ
10. 4 ষͷ΋͘͡ 4. ֬཰Ϟσϧͱࣝผؔ਺ 1. ؍ଌσʔλͷઢܗม׵ 2. ֬཰Ϟσϧ 3. ֬཰Ϟσϧύϥϝʔλͷ࠷໬ਪఆ
11. ฏۉϕΫτϧͱڞ෼ࢄߦྻ
12. ฏۉϕΫτϧ • ֤ಛ௃ྔʢશ෦Ͱ d ݸʣͷฏۉΛฒ΂ͨ΋ͷ • ྫ͑͹ɿ µ = ( µ1, µ2, . . . , µd)T = ( E { x1 } , E { x2 } , . . . , E { xd })T µ = (µweight, µdiameter, µradius)T = (4.80[g], 2260[µm], 1130[µm])T
13. ظ଴஋ • ֬཰ม਺͕࿈ଓͷͱ͖ʢ֬཰ີ౓ؔ਺ʣ µi = E { xi } = Z dxi xip ( xi) µi = E { xi } = X k x (k) i P ⇣ x (k) i ⌘ • ֬཰ม਺͕཭ࢄͷͱ͖ʢ֬཰࣭ྔؔ਺ʣ µ = ¯ x = 1 N N X i=1 xi • ؍ଌσʔλ͕ N ݸ༩͑ΒΕ͍ͯΔͱ͖
14. पล֬཰ • ֬཰ม਺͕཭ࢄͷͱ͖ʢ֬཰࣭ྔؔ਺ʣ • ஫໨͍ͯ͠Δಛ௃ྔͰͳ͍΋ͷ͸ͥΜͿ࿨ʢੵ෼ʣ
 ΛͱΔ ϋϯόʔά͕޷͖͔ʁ yes no sum ΤϏϑϥΠ ͕޷͖͔ʁ yes 60 40 100 no 30 20 50 sum 90 60 p ( xi) = Z dx1 Z dx2 · · · Z dxi 1 Z dxi+1 · · · Z dxd p ( x1, x2, . . . , xd)
15. ڞ෼ࢄߦྻ ⌃ = Var { x } = E ( x µ )( x µ )T = 0 B @ E {( x1 µ1)( x1 µ1)} . . . E {( x1 µ1)( xd µd)} . . . ... . . . E {( xd µd)( x1 µ1)} . . . E {( xd µd)( xd µd)} 1 C A = ( ij)
16. ڞ෼ࢄߦྻ • ؍ଌσʔλ͕ N ݸ༩͑ΒΕ͍ͯΔͱ͖ ij = 1 N N X n=1 ( xni µi)( xnj µj)
17. ڞ෼ࢄߦྻ • ຊདྷ͸ෆภ෼ࢄΛ࢖͏΂͖ • ظ଴஋Λͱͬͨͱ͖ʹਅͷ෼ࢄʹ
 ऩଋ͢ΔΑ͏ௐઅ͢Δ • ඪຊ਺͕ଟ͍৔߹͸େࠩͳ͍ͷͰ
 ͜͜Ͱ͸γϯϓϧʹ͍ͯ͠Δ sij = 1 N 1 N X n=1 ( xni µi)( xnj µj)
18. ෼ࢄͱඪ४ภࠩ • ෼ࢄ͸ฏۉ͔Βͷೋ৐ޡࠩͷظ଴஋ • େ͖͚Ε͹͹Β͍͍ͭͯΔ • ෼ࢄͷฏํ͕ࠜඪ४ภࠩ • ෼ࢄͩͱಛ௃ྔͷ୯Ґ͕มΘͬͯ͠·͏ͨΊ
 ΋ͱͷ୯Ґʹ໭͢ 2 i = ii = E ( xi µi)2 i = q 2 i
19. ڞ෼ࢄͱ૬ؔ܎਺ • ڞ෼ࢄ͸ҟͳΔಛ௃ྔؒͰ͹Β͖ͭํ͕
 ಉ͔͡Ͳ͏͔Λಛ௃͚ͮΔ • ਖ਼ͷ૬͕ؔ͋Ε͹ +ɺෛͷ૬͕ؔ͋Ε͹ - • ڞ෼ࢄ͸୯ҐΛ͍࣋ͬͯΔͨΊ
 ୯Ґ͕ҧ͏ڞ෼ࢄͲ͏͠ΛൺֱͰ͖ͳ͍ ij = E {( xi µi)( xj µj)}
20. ڞ෼ࢄͱ૬ؔ܎਺ ⇢ij = ij i j • ڞ෼ࢄΛແ࣍ݩԽͨ͠ͷ͕૬ؔ܎਺ • ਖ਼ͷ૬͕ؔ͋Ε͹ +ɺෛͷ૬͕ؔ͋Ε͹ - • ඞͣ [-1, 1] ΛͱΔ
21. ૬ؔ܎਺͸ઢܗ૬͔ؔ͠ΩϟονͰ͖ͳ͍ • x = [-2, -1, 0, 1, 2], y = x^2 ͷͱ͖
 ૬ؔ܎਺ ρ_xy ͸θϩ https://upload.wikimedia.org/wikipedia/commons/d/d4/Correlation_examples2.svg
22. ϕΫτϧతͳղऍ • N ݸͷଌఆ͕͋Δͱ͖ɺ ij = 1 N N X n=1 ( xni µi)( xnj µj) = 1 N N X n=1 dnidnj = 1 N di · dj di = ( x1i µi, x2i µi, . . . , xNi µi)T = ( d1i, d2i, . . . , dNi)T ͱ͓͘ͱɺڞ෼ࢄ͸಺ੵʢͷఆ਺ഒʣʹͳΔ ϕΫτϧۭؒͱͯ͠
 ѻ͏ͨΊʹ͸
 ֤ಛ௃ྔͰ୯Ґ͕
 ἧ͍ͬͯΔඞཁ͕͋Δ
23. 2 i = 1 N di · di = 1 N |di |2 i = = 1 p N |di | ϕΫτϧతͳղऍ ⇢ij = ij i j = (1 /N ) di · dj (1 / p N ) |di | (1 / p N ) |dj | = di · dj |di | |dj | = cos ✓ij
24. ϕΫτϧతͳղऍ • ׬શͳਖ਼ͷ૬͕ؔ͋Δͱ͖ ⇢ij = 1 cos ✓ij = 1 ✓ij = 0 di = cdj ( c > 0)
25. ϕΫτϧతͳղऍ di dj ⇢ij = 1 ⇢ij = 0 ⇢ij = 1 di dj di dj { } p N i p N j
26. ؍ଌσʔλͷඪ४Խ
27. • ಛ௃ؒͷ૬ؔΛͳ͍ͨ͘͠ • ୯Ґ͕ҧ͍ͬͯͯ΋౷ܭతʹൺֱ͍ͨ͠ ➡ ؍ଌσʔλΛม׵͠Α͏ʂ ॏ͞ [g] ௚ܘ [μm] ൒ܘ [μm] … ϥϕϧ 4.801 22601 11301 … ਅ 4.751 22599 11300 … آ 4.799 22602 11301 … ਅ … … … … … Πϯτϩʢ࠶๚ʣ
28. ฏۉɾ෼ࢄͱઢܗม׵ • ઢܗม׵Λߟ͑Δ y = ax + b E { y } = E { ax + b } = a E { x } + b = aµ + b • ฏۉͱ෼ࢄ͸ҎԼͷΑ͏ʹԠ౴ Var { y } = E ( y Ey )2 = E [ ax + b ( aµ + b )]2 = E a 2( x µ )2 = a 2E ( x µ )2 = a 2Var { x } = a 2 2
29. ඪ४Խ • ҎԼͷઢܗม׵Λ࢖͏ͱ
 ฏۉ 0 ɺ෼ࢄ 1 ͷಛ௃ྔ͕ಘΒΕΔ z = x µ E { z } = E ⇢ x µ = 1 (E { x } µ ) = 0 Var { z } = Var ⇢ x µ = 1 2 Var { x } = 1
30. ඪ४Խ x1 x2
31. ඪ४Խ x1 x2
32. ඪ४Խ x1 x2
33. ඪ४Խ x1 x2 µ1 µ2
34. ඪ४Խ x1 x2 µ1 µ2 1 2
35. ඪ४Խ x1 x2 µ1 µ2 1 2 1 z1 z2 1
36. ඪ४Խ x1 x2 µ1 µ2 1 2 1 z1 z2 1 • σʔλͷฏۉΛ 0 ɺ෼ࢄʢඪ४ภࠩʣΛ 1 ʹ • ແ࣍ݩԽ͞Ε͍ͯΔͷͰɺ୯Ґͷҧ͍΋ٵऩͰ͖Δ
37. ؍ଌσʔλͷແ૬ؔԽ
38. ݻ༗ϕΫτϧ • ڞ෼ࢄߦྻͷݻ༗஋໰୊Λղ͘ͱ
 d ຊͷϕΫτϧ͕ಘΒΕΔ • ڞ෼ࢄߦྻ͸ [d, d] ͷରশߦྻ ⌃si = isi
39. ݻ༗ϕΫτϧ • ରশߦྻʹରͯ͠ɿ • ݻ༗஋͸࣮਺ • ݻ༗ϕΫτϧ͸௚ަ ➡ ݻ༗ϕΫτϧ͸ਖ਼ن௚ަجఈ sT i sj = ij = ⇢ 1 (i = j) 0 (i 6= j)
40. ճసߦྻ • ݻ༗ϕΫτϧΛฒ΂ͯߦྻΛ࡞Δ • ਖ਼ن௚ަجఈΛฒ΂ͨߦྻ͸௚ަߦྻͱͳΔ S = (s1, s2, . . . , sd) (ST S)ij = sT i sj = ij ) ST S = I ) ST = S 1 • ͜ͷ৔߹͸ճసߦྻʢ㱬௚ަߦྻʣͱͳΔ
41. ແ૬ؔԽ y = ST x E { y } = E ST x = ST E { x } = ST µ Var { y } = E ( y E { y })( y E { y })T = E (ST x ST µ )(ST x ST µ )T = E ST ( x µ )[ST ( x µ )]T = E ST ( x µ )( x µ )T S = ST E ( x µ )( x µ )T S = ST ⌃S
42. ແ૬ؔԽ S 1⌃S = S 1⌃(s1, s2, . . . , sd) = S 1( 1s1, 2s2, . . . , dsd) = S 1S 0 B B B @ 1 0 . . . 0 0 2 . . . 0 . . . ... . . . 0 0 . . . d 1 C C C A = 0 B B B @ 1 0 . . . 0 0 2 . . . 0 . . . ... . . . 0 0 . . . d 1 C C C A = ⇤ • ͳͷͰɺແ૬ؔԽ͞Ε͍ͯΔ • ແ૬͕ؔͩɺඪ४Խ͸͞Ε͍ͯͳ͍ (Var {y})ij = 0 (i 6= j) (Var {y})ii = i
43. ന৭Խ
44. ന৭Խ • ඪ४Խʴແ૬ؔԽ u = ⇤ 1/2ST ( x µ ) (⇤ 1/2)ij = ⇢ 1/ p i (i = j) 0 (i 6= j)
45. ന৭Խ E { u } = E n⇤ 1/2ST ( x µ )o = ⇤ 1/2ST (E { x } µ ) = 0 Var { u } = E n⇤ 1/2ST ( x µ )( x µ )T S(⇤ 1/2)T o = ⇤ 1/2ST E ( x µ )( x µ )T S(⇤ 1/2)T = ⇤ 1/2ST ⌃S(⇤ 1/2)T = ⇤ 1/2⇤(⇤ 1/2)T = I • ඪ४Խ͞Εɺ͔ͭແ૬ؔԽ͞Εͨʂ
46. 4.1 ͷ·ͱΊ • ඪ४Խ • ୯ҐΛͦΖ͑ɺฏۉΛ 0 ɺ෼ࢄΛ 1 ʹ͢Δ • ແ૬ؔԽ • ૬͕ؔͳ͘ͳΔΑ͏ʹۭؒΛճసͤ͞Δ • ඪ४Խ͸͞Εͯͳ͍ • ന৭Խ • ඪ४Խ ʴ ແ૬ؔԽ • ୯ҐΛͦΖ͑ɺฏۉΛ 0 ɺ෼ࢄΛ 1 ʹ͠ɺ
 ૬͕ؔͳ͘ͳΔΑ͏ʹۭؒΛճసͤ͞Δ
47. 4 ষͷ΋͘͡ 4. ֬཰Ϟσϧͱࣝผؔ਺ 1. ؍ଌσʔλͷઢܗม׵ 2. ֬཰Ϟσϧ 3. ֬཰Ϟσϧύϥϝʔλͷ࠷໬ਪఆ
48. ֬཰Ϟσϧ • σʔλͷ෼෍ͷ਺ཧϞσϧ • ύϥϝτϦοΫϞσϧ • ෼෍ؔ਺ΛԾఆ͠ɺύϥϝʔλΛܾఆͯ͠
 ϞσϧԽ͢Δ • ϊϯύϥϝτϦοΫϞσϧ • ಛఆͷ෼෍ΛԾఆͤͣɺσʔλͦͷ΋ͷ͔Β
 ෼෍ͷදݱΛಘΔ
49. ֬཰Ϟσϧ͋Ε͜Ε • ύϥϝτϦοΫϞσϧ • ֬཰ม਺͕཭ࢄʢ֬཰࣭ྔؔ਺ʣ • ೋ߲෼෍ɺଟ߲෼෍ɺϙΞιϯ෼෍ͳͲ • ֬཰ม਺͕࿈ଓʢ֬཰ີ౓ؔ਺ʣ • ਖ਼ن෼෍ɺΧΠೋ৐෼෍ɺίʔγʔ෼෍ͳͲ • ϊϯύϥϝτϦοΫϞσϧ • ώετάϥϜ๏ɺkNN ๏ɺύϧπΣϯີ౓ਪఆ๏ ͳͲ
50. ਖ਼ن෼෍ؔ਺
51. ਖ਼ن෼෍ͷੑ࣭ʢൈਮʣ • ղੳతʹΑ͘ௐ΂ΒΕ͍ͯΔ • ඇਖ਼ن෼෍ʹ͕ͨ͠͏σʔλ΋
 ඪຊฏۉͷ෼෍͸ਖ਼ن෼෍ʹͳΔʢத৺ۃݶఆཧʣ • ਖ਼ن෼෍ʹ͕ͨ͠͏σʔλͷઢܗม׵͸
 ਖ਼ن෼෍ʹ͕ͨ͠͏ • ਖ਼ن෼෍ʹ͕ͨ͠͏ෳ਺ͷ֬཰ม਺ͷઢܗ݁߹͸
 ਖ਼ن෼෍ͱ͍͏ʢ࠶ੜੑʣ • ແ૬ؔͰ͋Δ͜ͱͱ౷ܭతʹಠཱͰ͋Δ͜ͱ͕౳Ձ
 ʢʮਖ਼ن෼෍ʹݶΓʯͷ෦෼ˠ ʮ਺ֶηϛφʔʯʹࡌͬͯΔ͔΋ʁʣ
52. ਖ਼ن෼෍ • 1 ࣍ݩਖ਼ن෼෍ N(x | µ, 2 ) = 1 p 2⇡ 2 exp  (x µ) 2 2 2 • ଟ࣍ݩਖ਼ن෼෍ N (x | µ , ⌃) = 1 (2 ⇡ ) d/2 | ⌃ |1/2 exp  1 2 (x µ) T ⌃ 1 (x µ)
53. ਖ਼ن෼෍ • ૬ؔͷ෼͚ͩճస͠ɺඪ४ภࠩͷ෼͚ͩҾ͖৳͹͞Ε͍ͯΔ • ന৭Խͷٯ ( x µ )T ⌃ 1( x µ ) = ( x µ )T [S⇤S 1] 1( x µ ) = ( x µ )T S⇤ 1S( x µ ) = [ST ( x µ )]T ⇤ 1[ST ( x µ )] = y T ⇤ 1 y (* y ⌘ ST ( x µ )) = y T (⇤1/2)T ⇤1/2 y = (⇤1/2 y )T (⇤1/2 y ) = z T z (* z ⌘ ⇤ 1/2 y )
54. ϚϋϥϊϏεڑ཭ • ෼෍ͷத৺͔ΒͲΕ͚ͩ཭Ε͍ͯΔ͔ͷࢦඪ • ന৭Խۭͨؒ͠Ͱͷڑ཭ d( x , µ ) = q ( x µ )T ⌃ 1( x µ ) = p z T z
55. ਖ਼ن෼෍͔Βಋ͔ΕΔ
 ࣝผؔ਺
56. Ϋϥε৚݅෇͖֬཰ • Ϋϥε৚݅෇͖֬཰͕ਖ਼ن෼෍Ͱ͋ΔͱԾఆ͢Δ ln P ( Ci | x) = p (x |Ci) P ( Ci) p (x) / p (x |Ci) P ( Ci) = P ( Ci) (2 ⇡ ) d/2 | ⌃i |1/2 exp  1 2 (x µi) T ⌃ 1 i (x µi) p (x |Ci) = 1 (2 ⇡ ) d/2 | ⌃i |1/2 exp  1 2 (x µi) T ⌃ 1 i (x µi)
57. Ϋϥε৚݅෇͖֬཰ • ؔ܎ͳ͍߲ΛΦϛοτɺ ×(-2) ln P(Ci | x ) = ln P(Ci) d 2 ln(2⇡) 1 2 ln |⌃i |1/2 1 2 ( x µi)T ⌃ 1 i ( x µi) gi( x ) = ( x µi)T ⌃ 1 i ( x µi) + ln |⌃i | 2 ln P(Ci) [Recognized class] = arg min i [ gi(x)]
58. ࣝผڥք • Ϋϥεؒͷڥ໨ʢࣝผڥքʣ͸ҎԼͷํఔࣜͰ
 ༩͑ΒΕΔ fij( x ) = gi( x ) gj( x ) = 0
59. ࣝผڥք fij( x ) = gi( x ) gj( x ) = ( x µi)T ⌃ 1 i ( x µi) + ln |⌃i | 2 ln P(Ci) ( x µj)T ⌃ 1 j ( x µj) ln |⌃j | + 2 ln P(Cj) = x ⌃ 1 i x x ⌃ 1 i µi µi⌃ 1 i x + µi⌃ 1 i µi x ⌃ 1 j x x ⌃ 1 j µj µj⌃ 1 j x + µj⌃ 1 j µj + ln |⌃i | ⌃j 2 ln P(Ci) P(Cj) = x (⌃ 1 i ⌃ 1 j ) x + 2( µ T j ⌃ 1 j µ T i ⌃ 1 i ) x + µT i ⌃ 1 i µiµT j ⌃ 1 j µj + ln |⌃i | ⌃j 2 ln P(Ci) P(Cj) ) x T S x + 2 c T x + F = 0 ʢ2 ࣍ࣝผؔ਺ʣ +µT i ⌃ 1 i µi µT j ⌃ 1 j µj + ln |⌃i | |⌃j | 2 ln P(Ci) P(Cj) fij( x ) = gi( x ) gj( x ) = ( x µi)T ⌃ 1 i ( x µi) + ln |⌃i | 2 ln P(Ci) ( x µj)T ⌃ 1 j ( x µj) ln |⌃j | + 2 ln P(Cj) = x ⌃ 1 i x x ⌃ 1 i µi µi⌃ 1 i x + µi⌃ 1 i µi x ⌃ 1 j x x ⌃ 1 j µj µj⌃ 1 j x + µj⌃ 1 j µj + ln |⌃i | ⌃j 2 ln P(Ci) P(Cj) = x (⌃ 1 i ⌃ 1 j ) x + 2( µ T j ⌃ 1 j µ T i ⌃ 1 i ) x + µT i ⌃ 1 i µiµT j ⌃ 1 j µj + ln |⌃i | ⌃j 2 ln P(Ci) P(Cj) x ⌃ 1 j x + x ⌃ 1 j µj + µj⌃ 1 j x µj⌃ 1 j µj
60. ࣝผڥքʢิ୊ʣ x T i ⌃ 1 i µi = (Scalar) = (x T i ⌃ 1 i µi) T = µ T i (⌃ 1 i ) T x = µ T i ⌃ 1 i x ( * ⌃ 1 i is a symmetric matrix) • ରশߦྻͷٯߦྻ͸ରশߦྻͰ͋Δ͜ͱʹ஫ҙ
61. ࣝผڥք • ;ͨͭͷΫϥεͷڞ෼ࢄߦྻ͕౳͍͠ͱ͖ fij( x ) = 2 c T x + F = 0 ʢઢܗࣝผؔ਺ʣ
62. ࣝผڥք ⌃i = ⌃j = I P(Ci) = P(Cj) • ͔ͭ ͷͱ͖ fij( x ) = 2( µ T j ⌃ 1 j µ T j ⌃ 1 i ) x + µ T i ⌃ 1 i µi µ T i ⌃ 1 i µi = 2 ( µ T j µ T i ) x + µ T i µi µ T j µj = 0 x T x + 2 µ T i x + µ T i µi x T x + 2 µ T j x µ T j µj = 0 ( x µi)T ( x µi) ( x µj)T ( x µj) = 0 ) ( x µi)T ( x µi) = ( x µj)T ( x µj) x T x 2 µ T i x + µ T i µi x T x + 2 µ T j x µ T j µj = 0 • ྆ล σ ͰׂΓɺ x^T x Λ଍͠Ҿ͖ ʢ࠷ۙ๣๏……ͱຊʹ͸ॻ͍ͯ͋Δ͕ઢܗ൑ผ෼ੳʢLDAʣͰ͸ʁ ʢฏۉ͔Βͷڑ཭Λൺ΂͍ͯΔͷͰʣʣ +µT i ⌃ 1 i µi µT j ⌃ 1 j µj fij( x ) = 2( µ T j ⌃ 1 j µ T j ⌃ 1 i ) x + µ T i ⌃ 1 i µi µ T i ⌃ 1 i µi = 2 ( µ T j µ T i ) x + µ T i µi µ T j µj = 0
63. ࣝผڥք µi µj
64. ࣝผڥք ܾఆڥք µi µj
65. ࣝผڥք ܾఆڥք µi µj µk
66. ࣝผڥք ܾఆڥք µi µj µk
67. ࣝผڥք ܾఆڥք cf. ϘϩϊΠਤ µi µj µk
68. 4.2 ͷ·ͱΊ • ਖ਼ن෼෍͸͍Ζ͍Ζੑ࣭͕͍͍ • ଟ࣍ݩਖ਼ن෼෍͸
 ૬ؔɾ෼ࢄʹΑΔճసɾऩॖ͕ߟྀ͞Ε͍ͯΔࣜ • 2 Ϋϥεؒͷࣄޙ֬཰͕౳͘͠ͳΔ఺ͷي੻͸
 ࣝผڥքΛ༩͑Δ
69. 4 ষͷ΋͘͡ 4. ֬཰Ϟσϧͱࣝผؔ਺ 1. ؍ଌσʔλͷઢܗม׵ 2. ֬཰Ϟσϧ 3. ֬཰Ϟσϧύϥϝʔλͷ࠷໬ਪఆ
70. ಠཱಉ෼෍ͱಉ࣌෼෍ • ಉҰͷʢਅͷʣ෼෍͔Βಠཱʹαϯϓϧ͞Εͨ΋ͷΛ
 i.i.d. ʢಠཱಉ෼෍: independently and identically distributedʣඪຊ
 ͱ͍͏ • i.i.d. ͷͱ͖ɺ N ݸͷαϯϓϧͷಉ࣌෼෍͸ҎԼɿ f( x1, x2, . . . , xN | ✓ ) = N Y i=1 f( xi | ✓ )
71. ࠷໬ਪఆ๏ • ࣮༻্͸σʔλ͕༩͑ΒΕ͍ͯͯύϥϝʔλ͕ະ஌ • ύϥϝʔλΛม਺ͱͯ͠ಉ࣌෼෍ΛͱΒ͑Δ L( ✓ ) = f( x1, x2, . . . , xN | ✓ ) ʢ໬౓ؔ਺ʣ • ໬౓ؔ਺Λ࠷େʹ͢ΔύϥϝʔλΛٻΊΔ
 ʢ࠷໬ਪఆ๏ʣ • ର਺Λͱͬͯ΋ۃ஋ͷҐஔ͸มΘΒͳ͍ͷͰ
 ໬౓ؔ਺ͷର਺ΛͱͬͯܭࢉΛ؆୯ʹͰ͖Δ͜ͱ͕͋Δ
72. 1 ม਺ਖ਼ن෼෍ͷ৔߹ L(µ, 2 ) = f(x1, x2, . . . , xN | µ, 2 ) = N Y i=1 1 p 2⇡ 2 exp  (xi µ) 2 2 2 = (2⇡ 2 ) N/2 exp " 1 2 2 N X i=1 (xi µ) 2 # Lln( µ, 2) = N 2 ln(2 ⇡ ) N 2 ln 2 1 2 2 N X i=1 ( xi µ )2
73. 1 ม਺ਖ਼ن෼෍ͷ৔߹ Lln( µ, 2) = N 2 ln(2 ⇡ ) N 2 ln 2 1 2 2 N X i=1 ( xi µ )2 • ର਺໬౓Λ֤ύϥϝʔλͰภඍ෼ͯ͠ۃ஋ΛٻΊΔ
74. 1 ม਺ਖ਼ن෼෍ͷ৔߹ @Lln(ˆ µ, 2) @µ = @ @µ " 1 2 2 N X i=1 ( xi µ )2 # µ=ˆ µ = 0 1 2 2 N X i=1 2( xi ˆ µ )( 1) = 0 N X i=1 xi N X i=1 ˆ µ = 0 ) ˆ µ = 1 N N X i=1 xi
75. 1 ม਺ਖ਼ن෼෍ͷ৔߹ @Lln( µ, ˆ2) @ 2 = @ @ 2 " N 2 ln 2 1 2 2 N X i=1 ( xi µ )2 # 2=ˆ2 = 0 N 2 1 ˆ2 1 2 1 (ˆ2)2 ( 1) N X i=1 ( xi µ )2 = 0 N ˆ2 + 1 (ˆ2)2 N X i=1 ( xi µ )2 = 0 ) ˆ2 = 1 N N X i=1 ( xi µ )2
76. 4.3 ͷ·ͱΊ • ໬౓ؔ਺ʢͷର਺ʣΛύϥϝʔλͰภඍ෼ͯ͠
 ໬౓͕࠷େͷͱ͜ΖΛٻΊΔ