Share
Embed
Transcript
21 slides
-
(x,y)ฏ໘্ͷσʔλΛྨ͢ΔઢΛࣜͰදݱ͢Δ ͜ͷ࣌ɺ(x,y)ฏ໘Λׂ͢Δઢ࣍ࣜͰද͞ΕΔ ͜ΕΛγάϞΠυؔМͰแΈɺ (x,y)ͰಘΒΕͨσʔλͷ ଐੑ͕t=1Ͱ͋Δ֬࣍ࣜͰද͞ΕΔ ରʹɺt=0Ͱ͋Δ֬࣍ࣜʹͳΔ 2࣍ݩೖྗ2Ϋϥεྨϩ δεςΟοΫճؼͷ෮श f(x, y)
= w0 + w1 x + w2 y f(x, y) = 0 P(x, y) = σ(w0 + w1 x + w2 y) 1 − P(x, y) -
χϡʔϥϧωοτϫʔΫ ෩ͷදݱʹͯ͠࠶ཧ x y ॏΈw1 ॏΈw2 f ग़ྗ ೖྗ ೖྗ
1 ͍ͭ1ͷ μϛʔೖྗ ॏΈw0 f(x, y) = w0 + w1 x + w2 y { P(x, y) = σ( f ) ⟶ t = 1 1 − P(x, y) ⟶ t = 0 ೖྗ૯ -
ࠓޙΛߟ͑ɺଟೖྗଟΫϥεΛѻ ͑ΔΑ͏ʹࣜͷදݱΛม͑Δ x1 x2 ॏΈw1 ॏΈw2 a ग़ྗ ೖྗ ೖྗ
x0 ͍ͭ1ͷ μϛʔೖྗ ॏΈw0 a = w0 x0 + w1 x1 + w2 x2 { y = σ(a) ⟶ P(t = 1|x) 1 − y ⟶ P(t = 0|x) ೖྗ૯ ͖݅֬ͷදݱ ʢx͕ϕΫτϧදهͳͷ ೖྗ͕ଟ࣍ݩͰ͋Δ ͜ͱʹ߹Θ͍ͤͯΔʣ ग़ྗΛyͱ͢Δ ೖྗ૯Λaͱ͢Δ ೖྗΛxͷఴࣈ Ͱදݱ͢Δ -
3Ϋϥεྨʹ֦ு͢Δ ͱ͜͏ͳΔ x1 x2 a0 ग़ྗ ೖྗ ೖྗ x0 ͍ͭ1ͷ
μϛʔೖྗ w00 ೖྗ૯ ग़ྗΛ3ͭʹ͢Δͷʹ߹Θͤͯɺ ೖྗ૯Λ3ͭʹ͠ɺॏΈ߹Θ ͤͯ૿͑Δ ॏΈͷఴࣈͷ͚ͭํ͕ٯ ͡Όͳ͍͔ͱࢥ͏͔͠Ε ͳ͍͕ɺ͜ͷํ͕৭ʑศར a1 a2 w01 w02 w20 w21 w22 y0 = exp(a0 ) ∑K−1 k=0 exp(ak ) → P(t = 0|x) y1 = exp(a1 ) ∑K−1 k=0 exp(ak ) → P(t = 1|x) y2 = exp(a2 ) ∑K−1 k=0 exp(ak ) → P(t = 2|x) Ҏ߱Ͱࡉ͔͘ݟ͍ͯ͘ ೖྗ૯ʹιϑτϚοΫεؔΛ௨ ͍ͯ͠ΔʢͦͷҙຯͰͦΕͧΕͷग़ ྗؔ͠߹͍ͬͯΔʣ a0 = ∑2 i=0 w0i xi -
ೖྗxͱతมt n ೖྗ ೖྗ ਖ਼ղσʔλ ʢΫϥεʣ 0 5.604765 -0.837603 0
1 5.093028 -1.098183 1 2 -2.595448 1.348614 1 3 -0.662749 -5.056531 0 4 15.573566 10.073330 2 5 7.084038 2.165339 2 6 6.204333 -2.945187 1 7 13.349965 12.577250 0 8 16.487809 6.629031 0 9 0.060047 -2.900301 2 x1 x2 t 1-of-Kූ߸Խ ʢone-hotදݱʣ [[1 0 0] [0 1 0] [0 1 0] [1 0 0] [0 0 1] [0 0 1] [0 1 0] [1 0 0] [1 0 0] [0 0 1]] NݸͷσʔλશମͰେจࣈͷX ೖྗ1ͭͰখจࣈͷϕΫτϧදهx = x0 x1 x2 NݸͷσʔλશମͰT x0ৗʹ1ͷ μϛʔೖྗ -
ॏΈwɺೖྗ૯aɺग़ ྗy ࠓ3ΫϥεྨͳͷͰɺೖྗ૯3ͭ͋ΔʢDೖྗ࣍ݩ ʣ Ϟσϧͷύϥϝʔλ·ͱΊͯߦྻͰද͢ͱ͜͏ͳΔ ೖྗ૯ΛιϑτϚοΫεؔʹೖྗͨ͠ͷΛग़ྗyͱ͢ΔʢK ྨ͢ΔΫϥεʣ ak = wk0
x0 + wk1 x1 + wk2 x2 = D ∑ i=0 wki xi (k = 0,1,2) yk = exp(ak ) ∑K−1 k=0 exp(ak ) (k = 0,1,2) w = w00 w01 w02 w10 w11 w12 w20 w21 w22 -
ࠓճͷ3Ϋϥεྨ ιϑτϚοΫεؔʹΑͬͯɺy0+y1+y2=1Ͱ͋Δ͜ͱ͕อূ͞Ε Δɻ͜ͷϞσϧͷग़ྗy0,y1,y2Λɺ֤Ϋϥεʹଐ͢Δ֬Λද͢Α͏ ʹֶश͢Δͷ͕ࠓճΓ͍ͨ͜ͱ ͪΖΜֶशͱ͍͍ײ͡ͷύϥϝʔλwΛݟ͚ͭΔ͜ͱͰ͋Δ x1 x2 a0 ग़ྗ x0
w00 ೖྗ૯ a1 a2 w01 w02 w20 w21 w22 y0 = softmax(a0 ) y1 = softmax(a1 ) y2 = softmax(a2 ) 5.6 -0.8 1 0.85 0.11 0.04 y1͕Ұ൪େ͖͍͔ΒΫϥε1ͬΆ͍ͳ -
తؔΛఆٛ͢ΔͨΊ ʹɺ·ͣؔ ɺશೖྗσʔλXʹରͯ͠શΫϥεσʔλT͕ੜ͞Εͨ֬ 1ͭͷೖྗσʔλxʹணͯ͠ɺͦͷΫϥε͕T=[1,0,0]ʢ͢ͳΘͪΫϥε0ʣ Ͱ͋ͬͨΒɺͦͷΫϥε͕ੜ͞Εͨ֬ Ϋϥε1Ͱ2Ͱಉ͡Α͏ʹදͤΔΑ͏ʹ͢Δͱ Nݸͷσʔλ͕ੜ͞Εͨ֬શ෦ֻ͚߹ΘͤΕ͍͍ͷͰ P(t = [1,0,0]|x)
= y0 P(t|x) = yt0 0 yt1 1 yt2 2 P(T|X) = N−1 ∏ n=0 P(tn |xn ) = N−1 ∏ n=0 ytn0 n0 ytn1 n1 ytn2 n2 = N−1 ∏ n=0 K−1 ∏ k=0 ytnk nk -
ฏۉަࠩΤϯτϩϐʔޡ ࠩ ฏۉަࠩΤϯτϩϐʔޡࠩؔɺͷෛͷ ରͷฏۉͳͷͰɺ ্ͷࣜʹͳͬͯ͠·͑͏ؔΛѻ Θͳ͍ͷͰɺ͋Δҙຯؔͷҙຯ͕Θ͔ͬ ͯͳͯ͘ͳ͍ͬͪΌͳ͍ E(w) = −
1 N log P(T|X) = − 1 N log N−1 ∏ n=0 K−1 ∏ k=0 ytnk nk = − 1 N N−1 ∑ n=0 K−1 ∑ k=0 tnk log ynk -
ฏۉަࠩΤϯτϩϐʔޡ ͕ࠩҙຯ͢Δͷ̍ ͋ΔҰͭͷσʔλ͚ͩʹ͢ΔͱɺɹɹɹɹͱͳΔ E(w) = − 1 N N−1 ∑
n=0 K−1 ∑ k=0 tnk log ynk tk log yk ͚ͩ͜͜ʹ ͢Δ a0 ग़ྗ ೖྗ૯ a1 a2 y0 = softmax(a0 ) y1 = softmax(a1 ) y2 = softmax(a2 ) 0.85 0.11 0.04 ਖ਼ղt 0 1 0 t0 log y0 = 0 t1 log y1 = − 0.1625 t2 log y2 = 0 ͜͜ͱ͜͜Λֻ ͚ͨʹͳΔ ͜͜ʢΛlogͱͬͨʣͱ͜͜ Λֻ͚ͨʹͳΔ -
ฏۉަࠩΤϯτϩϐʔޡ ͕ࠩҙຯ͢Δͷ̎ ͋ΔҰͭͷσʔλ͚ͩʹ͢ΔͱɺɹɹɹɹɹɹͱͳΔ tkone-hotදݱͷਖ਼ղσʔλͳͷͰɺtk=1Ͱ͋Δyk͚͕ͩEͷ߹ ࢉରʹͳΔʢ͋ͱશ෦θϩʣ yk0ʙ1ͷͰ͋Γɺ1ʹ͍ۙ΄ͲlogΛऔΔͱ0ʹۙ͘ͳΔɻ0ʹ ͍ۙ΄ͲlogΛऔΔͱϚΠφεํʹେ͖ͳͱͳΔ ͭ·Γɺྨͷ݁Ռ͕ਖ਼͍͠ͳΒɹɹɹɹɹɹɹ΄΅0ʹͳΔ ͠ɺؒҧ͍ͬͯΔ΄ͲϚΠφεํʹେ͖͘ͳΔɻ͜Εͭ·Γ ޡࠩΛݟ͍ͯΔ͜ͱʹͳΓɺ͜ΕΒΛσʔλͷ͚ͩ͠߹Θͤ
ͨE(W)ೋޡࠩͱରͯ͠ൃมΘΒͳ͍ʢೋޡࠩͷྨ൛ Έ͍ͨͳΠϝʔδʣɻ E(w) = − 1 N N−1 ∑ n=0 K−1 ∑ k=0 tnk log ynk tk log yk ͚ͩ͜͜ʹ ͢Δ tk log yk -
ޯ๏ʹΑΔղ ޯ๏ͰฏۉަࠩΤϯτϩϐʔޡࠩؔE(W)Λ࠷খԽ͢ΔWΛٻΊΔʹ ɺ͍ͭͷΑ͏ʹ֤wki ʹؔ͢ΔภඍΛ༻ҙ͢Εྑ͍ ಋग़লུ͢Δ͕ɺಋؔҎԼͷΑ͏ʹશͯͷkͱiʹରͯ͠ಉ͡ܗʹ ͳΔ ͪΖΜֶशଇ͜͏ ֶशଇʹैͬͯগͣͭ͠ύϥϝʔλwΛม͍͖͑ͯʢޯϕΫτϧͷ ରํͷࡔΛԼ͍͖ͬͯʣɺ࠷దͳύϥϝʔλwΛٻΊΔ ∂E
∂wki = 1 N N−1 ∑ n=0 (ynk − tnk )xni wki := wki − ∂E ∂wki -
ࢀߟจݙ தҪ ӻ࢘ʮITΤϯδχΞͷͨΊͷػցֶश ཧೖʯٕज़ධࣾ, 2015 ҏ౻ ਅʮPythonͰಈֶ͔ͯ͠Ϳʂ͋ͨΒ͠ ͍ػցֶशͷڭՊॻʯᠳӭࣾ, 2018 ཱੴݡޗʮֶ͘͞͠Ϳ
ػցֶशΛཧղ͢ ΔͨΊͷֶͷ͖΄Μ ʯϚΠφϏग़൛, 2017