Share
Embed
Transcript
76 slides
-
Πϯτϩ: 100 ԁۄͷਅآఆ ॏ͞ [g] ܘ [μm] ܘ [μm] …
ϥϕϧ 4.801 22601 11301 … ਅ 4.751 22599 11300 … آ 4.799 22602 11301 … ਅ … … … … … -
Πϯτϩ: 100 ԁۄͷਅآఆ ॏ͞ [g] ܘ [μm] ܘ [μm] …
ϥϕϧ 4.801 22601 11301 … ਅ 4.751 22599 11300 … آ 4.799 22602 11301 … ਅ … … … … … ૬ؔ͋ΔͷͰʁ -
Πϯτϩ: 100 ԁۄͷਅآఆ ॏ͞ [g] ܘ [μm] ܘ [μm] …
ϥϕϧ 4.801 22601 11301 … ਅ 4.751 22599 11300 … آ 4.799 22602 11301 … ਅ … … … … … ૬ؔ͋ΔͷͰʁ -
ॏ͞ [g] ܘ [μm] ܘ [μm] … ϥϕϧ 4.801 22601
11301 … ਅ 4.751 22599 11300 … آ 4.799 22602 11301 … ਅ … … … … … Πϯτϩ: 100 ԁۄͷਅآఆ 100 ԁۄͷฏۉʢ4.8 gʣΑΓܰͦ͏͕ͩ ࠩ 0.05 [g] -
ॏ͞ [g] ܘ [μm] ܘ [μm] … ϥϕϧ 4.801 22601
11301 … ਅ 4.751 22599 11300 … آ 4.799 22602 11301 … ਅ … … … … … Πϯτϩ: 100 ԁۄͷਅآఆ 100 ԁۄͷฏۉʢ22600 μmʣͱಉ͡Α͏͕ͩ ࠩ 2 [μm] >> 0.05 -
ॏ͞ [g] ܘ [μm] ܘ [μm] … ϥϕϧ 4.801 22601
11301 … ਅ 4.751 22599 11300 … آ 4.799 22602 11301 … ਅ … … … … … Πϯτϩ: 100 ԁۄͷਅآఆ • ಛؒͷ૬ؔΛͳ͍ͨ͘͠ • ୯Ґ͕ҧ͍ͬͯͯ౷ܭతʹൺֱ͍ͨ͠ -
ॏ͞ [g] ܘ [μm] ܘ [μm] … ϥϕϧ 4.801 22601
11301 … ਅ 4.751 22599 11300 … آ 4.799 22602 11301 … ਅ … … … … … Πϯτϩ: 100 ԁۄͷਅآఆ • ಛؒͷ૬ؔΛͳ͍ͨ͘͠ • ୯Ґ͕ҧ͍ͬͯͯ౷ܭతʹൺֱ͍ͨ͠ ➡ ؍ଌσʔλΛม͠Α͏ʂ -
ฏۉϕΫτϧ • ֤ಛྔʢશ෦Ͱ d ݸʣͷฏۉΛฒͨͷ • ྫ͑ɿ µ = (
µ1, µ2, . . . , µd)T = ( E { x1 } , E { x2 } , . . . , E { xd })T µ = (µweight, µdiameter, µradius)T = (4.80[g], 2260[µm], 1130[µm])T -
ظ • ֬ม͕࿈ଓͷͱ͖ʢ֬ີؔʣ µi = E { xi } =
Z dxi xip ( xi) µi = E { xi } = X k x (k) i P ⇣ x (k) i ⌘ • ֬ม͕ࢄͷͱ͖ʢ࣭֬ྔؔʣ µ = ¯ x = 1 N N X i=1 xi • ؍ଌσʔλ͕ N ݸ༩͑ΒΕ͍ͯΔͱ͖ -
पล֬ • ֬ม͕ࢄͷͱ͖ʢ࣭֬ྔؔʣ • ͍ͯ͠ΔಛྔͰͳ͍ͷͥΜͿʢੵʣ ΛͱΔ ϋϯόʔά͕͖͔ʁ yes no sum
ΤϏϑϥΠ ͕͖͔ʁ yes 60 40 100 no 30 20 50 sum 90 60 p ( xi) = Z dx1 Z dx2 · · · Z dxi 1 Z dxi+1 · · · Z dxd p ( x1, x2, . . . , xd) -
ڞࢄߦྻ ⌃ = Var { x } = E (
x µ )( x µ )T = 0 B @ E {( x1 µ1)( x1 µ1)} . . . E {( x1 µ1)( xd µd)} . . . ... . . . E {( xd µd)( x1 µ1)} . . . E {( xd µd)( xd µd)} 1 C A = ( ij) -
ڞࢄߦྻ • ؍ଌσʔλ͕ N ݸ༩͑ΒΕ͍ͯΔͱ͖ ij = 1 N N
X n=1 ( xni µi)( xnj µj) -
ڞࢄߦྻ • ຊདྷෆภࢄΛ͏͖ • ظΛͱͬͨͱ͖ʹਅͷࢄʹ ऩଋ͢ΔΑ͏ௐઅ͢Δ • ඪຊ͕ଟ͍߹େࠩͳ͍ͷͰ ͜͜Ͱγϯϓϧʹ͍ͯ͠Δ sij
= 1 N 1 N X n=1 ( xni µi)( xnj µj) -
ࢄͱඪ४ภࠩ • ࢄฏۉ͔Βͷೋޡࠩͷظ • େ͖͚ΕΒ͍͍ͭͯΔ • ࢄͷฏํ͕ࠜඪ४ภࠩ • ࢄͩͱಛྔͷ୯Ґ͕มΘͬͯ͠·͏ͨΊ ͱͷ୯Ґʹ͢
2 i = ii = E ( xi µi)2 i = q 2 i -
ڞࢄͱ૬ؔ • ڞࢄҟͳΔಛྔؒͰΒ͖ͭํ͕ ಉ͔͡Ͳ͏͔Λಛ͚ͮΔ • ਖ਼ͷ૬͕ؔ͋Ε +ɺෛͷ૬͕ؔ͋Ε - • ڞࢄ୯ҐΛ͍࣋ͬͯΔͨΊ
୯Ґ͕ҧ͏ڞࢄͲ͏͠ΛൺֱͰ͖ͳ͍ ij = E {( xi µi)( xj µj)} -
ڞࢄͱ૬ؔ ⇢ij = ij i j • ڞࢄΛແ࣍ݩԽͨ͠ͷ͕૬ؔ • ਖ਼ͷ૬͕ؔ͋Ε
+ɺෛͷ૬͕ؔ͋Ε - • ඞͣ [-1, 1] ΛͱΔ -
૬ؔઢܗ૬͔ؔ͠ΩϟονͰ͖ͳ͍ • x = [-2, -1, 0, 1, 2], y
= x^2 ͷͱ͖ ૬ؔ ρ_xy θϩ https://upload.wikimedia.org/wikipedia/commons/d/d4/Correlation_examples2.svg -
ϕΫτϧతͳղऍ • N ݸͷଌఆ͕͋Δͱ͖ɺ ij = 1 N N X
n=1 ( xni µi)( xnj µj) = 1 N N X n=1 dnidnj = 1 N di · dj di = ( x1i µi, x2i µi, . . . , xNi µi)T = ( d1i, d2i, . . . , dNi)T ͱ͓͘ͱɺڞࢄੵʢͷఆഒʣʹͳΔ ϕΫτϧۭؒͱͯ͠ ѻ͏ͨΊʹ ֤ಛྔͰ୯Ґ͕ ἧ͍ͬͯΔඞཁ͕͋Δ -
2 i = 1 N di · di = 1
N |di |2 i = = 1 p N |di | ϕΫτϧతͳղऍ ⇢ij = ij i j = (1 /N ) di · dj (1 / p N ) |di | (1 / p N ) |dj | = di · dj |di | |dj | = cos ✓ij -
ϕΫτϧతͳղऍ • શͳਖ਼ͷ૬͕ؔ͋Δͱ͖ ⇢ij = 1 cos ✓ij = 1
✓ij = 0 di = cdj ( c > 0) -
ϕΫτϧతͳղऍ di dj ⇢ij = 1 ⇢ij = 0 ⇢ij
= 1 di dj di dj { } p N i p N j -
• ಛؒͷ૬ؔΛͳ͍ͨ͘͠ • ୯Ґ͕ҧ͍ͬͯͯ౷ܭతʹൺֱ͍ͨ͠ ➡ ؍ଌσʔλΛม͠Α͏ʂ ॏ͞ [g] ܘ [μm]
ܘ [μm] … ϥϕϧ 4.801 22601 11301 … ਅ 4.751 22599 11300 … آ 4.799 22602 11301 … ਅ … … … … … Πϯτϩʢ࠶๚ʣ -
ฏۉɾࢄͱઢܗม • ઢܗมΛߟ͑Δ y = ax + b E {
y } = E { ax + b } = a E { x } + b = aµ + b • ฏۉͱࢄҎԼͷΑ͏ʹԠ Var { y } = E ( y Ey )2 = E [ ax + b ( aµ + b )]2 = E a 2( x µ )2 = a 2E ( x µ )2 = a 2Var { x } = a 2 2 -
ඪ४Խ • ҎԼͷઢܗมΛ͏ͱ ฏۉ 0 ɺࢄ 1 ͷಛྔ͕ಘΒΕΔ z =
x µ E { z } = E ⇢ x µ = 1 (E { x } µ ) = 0 Var { z } = Var ⇢ x µ = 1 2 Var { x } = 1 -
ඪ४Խ x1 x2 µ1 µ2 1 2 1 z1 z2
1 • σʔλͷฏۉΛ 0 ɺࢄʢඪ४ภࠩʣΛ 1 ʹ • ແ࣍ݩԽ͞Ε͍ͯΔͷͰɺ୯Ґͷҧ͍ٵऩͰ͖Δ -
ݻ༗ϕΫτϧ • ରশߦྻʹରͯ͠ɿ • ݻ༗࣮ • ݻ༗ϕΫτϧަ ➡ ݻ༗ϕΫτϧਖ਼نަجఈ sT
i sj = ij = ⇢ 1 (i = j) 0 (i 6= j) -
ճసߦྻ • ݻ༗ϕΫτϧΛฒͯߦྻΛ࡞Δ • ਖ਼نަجఈΛฒͨߦྻަߦྻͱͳΔ S = (s1, s2, .
. . , sd) (ST S)ij = sT i sj = ij ) ST S = I ) ST = S 1 • ͜ͷ߹ճసߦྻʢ㱬ަߦྻʣͱͳΔ -
ແ૬ؔԽ y = ST x E { y } =
E ST x = ST E { x } = ST µ Var { y } = E ( y E { y })( y E { y })T = E (ST x ST µ )(ST x ST µ )T = E ST ( x µ )[ST ( x µ )]T = E ST ( x µ )( x µ )T S = ST E ( x µ )( x µ )T S = ST ⌃S -
ແ૬ؔԽ S 1⌃S = S 1⌃(s1, s2, . . .
, sd) = S 1( 1s1, 2s2, . . . , dsd) = S 1S 0 B B B @ 1 0 . . . 0 0 2 . . . 0 . . . ... . . . 0 0 . . . d 1 C C C A = 0 B B B @ 1 0 . . . 0 0 2 . . . 0 . . . ... . . . 0 0 . . . d 1 C C C A = ⇤ • ͳͷͰɺແ૬ؔԽ͞Ε͍ͯΔ • ແ૬͕ؔͩɺඪ४Խ͞Ε͍ͯͳ͍ (Var {y})ij = 0 (i 6= j) (Var {y})ii = i -
ന৭Խ • ඪ४Խʴແ૬ؔԽ u = ⇤ 1/2ST ( x µ
) (⇤ 1/2)ij = ⇢ 1/ p i (i = j) 0 (i 6= j) -
ന৭Խ E { u } = E n⇤ 1/2ST (
x µ )o = ⇤ 1/2ST (E { x } µ ) = 0 Var { u } = E n⇤ 1/2ST ( x µ )( x µ )T S(⇤ 1/2)T o = ⇤ 1/2ST E ( x µ )( x µ )T S(⇤ 1/2)T = ⇤ 1/2ST ⌃S(⇤ 1/2)T = ⇤ 1/2⇤(⇤ 1/2)T = I • ඪ४Խ͞Εɺ͔ͭແ૬ؔԽ͞Εͨʂ -
4.1 ͷ·ͱΊ • ඪ४Խ • ୯ҐΛͦΖ͑ɺฏۉΛ 0 ɺࢄΛ 1 ʹ͢Δ
• ແ૬ؔԽ • ૬͕ؔͳ͘ͳΔΑ͏ʹۭؒΛճసͤ͞Δ • ඪ४Խ͞Εͯͳ͍ • ന৭Խ • ඪ४Խ ʴ ແ૬ؔԽ • ୯ҐΛͦΖ͑ɺฏۉΛ 0 ɺࢄΛ 1 ʹ͠ɺ ૬͕ؔͳ͘ͳΔΑ͏ʹۭؒΛճసͤ͞Δ -
֬Ϟσϧ • σʔλͷͷཧϞσϧ • ύϥϝτϦοΫϞσϧ • ؔΛԾఆ͠ɺύϥϝʔλΛܾఆͯ͠ ϞσϧԽ͢Δ • ϊϯύϥϝτϦοΫϞσϧ
• ಛఆͷΛԾఆͤͣɺσʔλͦͷͷ͔Β ͷදݱΛಘΔ -
֬Ϟσϧ͋Ε͜Ε • ύϥϝτϦοΫϞσϧ • ֬ม͕ࢄʢ࣭֬ྔؔʣ • ೋ߲ɺଟ߲ɺϙΞιϯͳͲ • ֬ม͕࿈ଓʢ֬ີؔʣ •
ਖ਼نɺΧΠೋɺίʔγʔͳͲ • ϊϯύϥϝτϦοΫϞσϧ • ώετάϥϜ๏ɺkNN ๏ɺύϧπΣϯີਪఆ๏ ͳͲ -
ਖ਼نͷੑ࣭ʢൈਮʣ • ղੳతʹΑ͘ௐΒΕ͍ͯΔ • ඇਖ਼نʹ͕ͨ͠͏σʔλ ඪຊฏۉͷਖ਼نʹͳΔʢத৺ۃݶఆཧʣ • ਖ਼نʹ͕ͨ͠͏σʔλͷઢܗม ਖ਼نʹ͕ͨ͠͏ •
ਖ਼نʹ͕ͨ͠͏ෳͷ֬มͷઢܗ݁߹ ਖ਼نͱ͍͏ʢ࠶ੜੑʣ • ແ૬ؔͰ͋Δ͜ͱͱ౷ܭతʹಠཱͰ͋Δ͜ͱ͕Ձ ʢʮਖ਼نʹݶΓʯͷ෦ˠ ʮֶηϛφʔʯʹࡌͬͯΔ͔ʁʣ -
ਖ਼ن • 1 ࣍ݩਖ਼ن N(x | µ, 2 ) =
1 p 2⇡ 2 exp (x µ) 2 2 2 • ଟ࣍ݩਖ਼ن N (x | µ , ⌃) = 1 (2 ⇡ ) d/2 | ⌃ |1/2 exp 1 2 (x µ) T ⌃ 1 (x µ) -
ਖ਼ن • ૬ؔͷ͚ͩճస͠ɺඪ४ภࠩͷ͚ͩҾ͖৳͞Ε͍ͯΔ • ന৭Խͷٯ ( x µ )T ⌃
1( x µ ) = ( x µ )T [S⇤S 1] 1( x µ ) = ( x µ )T S⇤ 1S( x µ ) = [ST ( x µ )]T ⇤ 1[ST ( x µ )] = y T ⇤ 1 y (* y ⌘ ST ( x µ )) = y T (⇤1/2)T ⇤1/2 y = (⇤1/2 y )T (⇤1/2 y ) = z T z (* z ⌘ ⇤ 1/2 y ) -
ϚϋϥϊϏεڑ • ͷத৺͔ΒͲΕ͚ͩΕ͍ͯΔ͔ͷࢦඪ • ന৭Խۭͨؒ͠Ͱͷڑ d( x , µ )
= q ( x µ )T ⌃ 1( x µ ) = p z T z -
Ϋϥε͖݅֬ • Ϋϥε͖͕݅֬ਖ਼نͰ͋ΔͱԾఆ͢Δ ln P ( Ci | x) =
p (x |Ci) P ( Ci) p (x) / p (x |Ci) P ( Ci) = P ( Ci) (2 ⇡ ) d/2 | ⌃i |1/2 exp 1 2 (x µi) T ⌃ 1 i (x µi) p (x |Ci) = 1 (2 ⇡ ) d/2 | ⌃i |1/2 exp 1 2 (x µi) T ⌃ 1 i (x µi) -
Ϋϥε͖݅֬ • ؔͳ͍߲ΛΦϛοτɺ ×(-2) ln P(Ci | x ) =
ln P(Ci) d 2 ln(2⇡) 1 2 ln |⌃i |1/2 1 2 ( x µi)T ⌃ 1 i ( x µi) gi( x ) = ( x µi)T ⌃ 1 i ( x µi) + ln |⌃i | 2 ln P(Ci) [Recognized class] = arg min i [ gi(x)] -
ࣝผڥք • ΫϥεؒͷڥʢࣝผڥքʣҎԼͷํఔࣜͰ ༩͑ΒΕΔ fij( x ) = gi( x
) gj( x ) = 0 -
ࣝผڥք fij( x ) = gi( x ) gj( x
) = ( x µi)T ⌃ 1 i ( x µi) + ln |⌃i | 2 ln P(Ci) ( x µj)T ⌃ 1 j ( x µj) ln |⌃j | + 2 ln P(Cj) = x ⌃ 1 i x x ⌃ 1 i µi µi⌃ 1 i x + µi⌃ 1 i µi x ⌃ 1 j x x ⌃ 1 j µj µj⌃ 1 j x + µj⌃ 1 j µj + ln |⌃i | ⌃j 2 ln P(Ci) P(Cj) = x (⌃ 1 i ⌃ 1 j ) x + 2( µ T j ⌃ 1 j µ T i ⌃ 1 i ) x + µT i ⌃ 1 i µiµT j ⌃ 1 j µj + ln |⌃i | ⌃j 2 ln P(Ci) P(Cj) ) x T S x + 2 c T x + F = 0 ʢ2 ࣍ࣝผؔʣ +µT i ⌃ 1 i µi µT j ⌃ 1 j µj + ln |⌃i | |⌃j | 2 ln P(Ci) P(Cj) fij( x ) = gi( x ) gj( x ) = ( x µi)T ⌃ 1 i ( x µi) + ln |⌃i | 2 ln P(Ci) ( x µj)T ⌃ 1 j ( x µj) ln |⌃j | + 2 ln P(Cj) = x ⌃ 1 i x x ⌃ 1 i µi µi⌃ 1 i x + µi⌃ 1 i µi x ⌃ 1 j x x ⌃ 1 j µj µj⌃ 1 j x + µj⌃ 1 j µj + ln |⌃i | ⌃j 2 ln P(Ci) P(Cj) = x (⌃ 1 i ⌃ 1 j ) x + 2( µ T j ⌃ 1 j µ T i ⌃ 1 i ) x + µT i ⌃ 1 i µiµT j ⌃ 1 j µj + ln |⌃i | ⌃j 2 ln P(Ci) P(Cj) x ⌃ 1 j x + x ⌃ 1 j µj + µj⌃ 1 j x µj⌃ 1 j µj -
ࣝผڥքʢิʣ x T i ⌃ 1 i µi = (Scalar)
= (x T i ⌃ 1 i µi) T = µ T i (⌃ 1 i ) T x = µ T i ⌃ 1 i x ( * ⌃ 1 i is a symmetric matrix) • ରশߦྻͷٯߦྻରশߦྻͰ͋Δ͜ͱʹҙ -
ࣝผڥք • ;ͨͭͷΫϥεͷڞࢄߦྻ͕͍͠ͱ͖ fij( x ) = 2 c T
x + F = 0 ʢઢܗࣝผؔʣ -
ࣝผڥք ⌃i = ⌃j = I P(Ci) = P(Cj) •
͔ͭ ͷͱ͖ fij( x ) = 2( µ T j ⌃ 1 j µ T j ⌃ 1 i ) x + µ T i ⌃ 1 i µi µ T i ⌃ 1 i µi = 2 ( µ T j µ T i ) x + µ T i µi µ T j µj = 0 x T x + 2 µ T i x + µ T i µi x T x + 2 µ T j x µ T j µj = 0 ( x µi)T ( x µi) ( x µj)T ( x µj) = 0 ) ( x µi)T ( x µi) = ( x µj)T ( x µj) x T x 2 µ T i x + µ T i µi x T x + 2 µ T j x µ T j µj = 0 • ྆ล σ ͰׂΓɺ x^T x Λ͠Ҿ͖ ʢ࠷ۙ๏……ͱຊʹॻ͍ͯ͋Δ͕ઢܗผੳʢLDAʣͰʁ ʢฏۉ͔ΒͷڑΛൺ͍ͯΔͷͰʣʣ +µT i ⌃ 1 i µi µT j ⌃ 1 j µj fij( x ) = 2( µ T j ⌃ 1 j µ T j ⌃ 1 i ) x + µ T i ⌃ 1 i µi µ T i ⌃ 1 i µi = 2 ( µ T j µ T i ) x + µ T i µi µ T j µj = 0 -
ಠཱಉͱಉ࣌ • ಉҰͷʢਅͷʣ͔Βಠཱʹαϯϓϧ͞ΕͨͷΛ i.i.d. ʢಠཱಉ: independently and identically distributedʣඪຊ ͱ͍͏
• i.i.d. ͷͱ͖ɺ N ݸͷαϯϓϧͷಉ࣌ҎԼɿ f( x1, x2, . . . , xN | ✓ ) = N Y i=1 f( xi | ✓ ) -
࠷ਪఆ๏ • ࣮༻্σʔλ͕༩͑ΒΕ͍ͯͯύϥϝʔλ͕ະ • ύϥϝʔλΛมͱͯ͠ಉ࣌ΛͱΒ͑Δ L( ✓ ) = f(
x1, x2, . . . , xN | ✓ ) ʢؔʣ • ؔΛ࠷େʹ͢ΔύϥϝʔλΛٻΊΔ ʢ࠷ਪఆ๏ʣ • ରΛͱͬͯۃͷҐஔมΘΒͳ͍ͷͰ ؔͷରΛͱͬͯܭࢉΛ؆୯ʹͰ͖Δ͜ͱ͕͋Δ -
1 มਖ਼نͷ߹ L(µ, 2 ) = f(x1, x2, . .
. , xN | µ, 2 ) = N Y i=1 1 p 2⇡ 2 exp (xi µ) 2 2 2 = (2⇡ 2 ) N/2 exp " 1 2 2 N X i=1 (xi µ) 2 # Lln( µ, 2) = N 2 ln(2 ⇡ ) N 2 ln 2 1 2 2 N X i=1 ( xi µ )2 -
1 มਖ਼نͷ߹ Lln( µ, 2) = N 2 ln(2 ⇡
) N 2 ln 2 1 2 2 N X i=1 ( xi µ )2 • ରΛ֤ύϥϝʔλͰภඍͯ͠ۃΛٻΊΔ -
1 มਖ਼نͷ߹ @Lln(ˆ µ, 2) @µ = @ @µ "
1 2 2 N X i=1 ( xi µ )2 # µ=ˆ µ = 0 1 2 2 N X i=1 2( xi ˆ µ )( 1) = 0 N X i=1 xi N X i=1 ˆ µ = 0 ) ˆ µ = 1 N N X i=1 xi -
1 มਖ਼نͷ߹ @Lln( µ, ˆ2) @ 2 = @ @
2 " N 2 ln 2 1 2 2 N X i=1 ( xi µ )2 # 2=ˆ2 = 0 N 2 1 ˆ2 1 2 1 (ˆ2)2 ( 1) N X i=1 ( xi µ )2 = 0 N ˆ2 + 1 (ˆ2)2 N X i=1 ( xi µ )2 = 0 ) ˆ2 = 1 N N X i=1 ( xi µ )2