機械学習勉強会08 2次元入力3クラス分類/MLStudy08

機械学習勉強会08 2次元入力3クラス分類

Transcript

1. 2࣍ݩೖྗ3Ϋϥε෼ ྨ χϡʔϥϧωοτϫʔΫ·Ͱ͋ͱҰา @hachiilcane
2. ͜Ε·Ͱݟ͖ͯͨ෼ྨ 2࣍ݩೖྗ2Ϋϥε෼ྨΛ௚ઢͰ෼͚Δ ͷ͕ύʔηϓτϩϯ 2࣍ݩೖྗ2Ϋϥε෼ྨΛʢ͋Δҙຯʣ ֬཰తͳ෯Λ࣋ͭ௚ઢͰ෼͚Δͷ͕ϩ δεςΟοΫճؼ
3. ࠓճ͸ ෼ྨͰ͖Δ਺Λ1ͭ૿΍ͯ͠ɺ2࣍ݩೖྗ3Ϋϥε ෼ྨΛϩδεςΟοΫճؼͰղ͘ t͕0·ͨ͸1ͩͱ̎୒ʹͳͬͪΌ͏͔ΒɺͦΕ ΛଟΫϥεʹ֦ு͢Δͱ͖ʹιϑτϚοΫεؔ ਺Λ࢖͏͚ͬͯͩͰͦΜͳ೉͍͠࿩͡Όͳ͍ αΫοͱֶΜͰɺ࣍ʹֶͿϑΟʔυϑΥϫʔυ χϡʔϥϧωοτ΁ͷ଍͕͔Γʹ͠Α͏
4. ೦ͷͨΊͷ֬ೝͰɺ3Ϋ ϥεͬͯʁ ͜Ε·Ͱ͸෼ྨͱͯ͠ʮපؾͰ͋ΔʯʮපؾͰͳ ͍ʯͷ2छྨʹ෼͚͍ͯͨʢˡ2Ϋϥεʣ ࠓ౓͸ʮʢපؾͷʣεςʔδ1ʯʮεςʔδ2ʯ ʮεςʔδ3ʯʹ෼ྨ͍ͨ͠ͱ͔ɺͦ͏͍͏ײ͡ ʢˡ3Ϋϥεʣ ྉཧΛʮ༸৯ʯʮ࿨৯ʯʮத՚ʯͷͲΕ͔ʹ ෼͚͍ͨͱ͔ Ϋϥε͸ɺଞʹ΋ΧςΰϦʔɺϥϕϧͱ͔ݴͬͨΓ ͢Δ ෼ྨʹ͓͚Δ໨తม਺tͷऔΓ͏Δ஋ͷ਺͕3ͭ͋ Δɺͱ΋ݴ͑Δ
5. (x,y)ฏ໘্ͷσʔλΛ෼ྨ͢Δ௚ઢΛ਺ࣜͰදݱ͢Δ ͜ͷ࣌ɺ(x,y)ฏ໘Λ෼ׂ͢Δ௚ઢ͸࣍ࣜͰද͞ΕΔ ͜ΕΛγάϞΠυؔ਺МͰแΈɺ఺ (x,y)ͰಘΒΕͨσʔλͷ ଐੑ͕t=1Ͱ͋Δ֬཰͸࣍ࣜͰද͞ΕΔ ൓ରʹɺt=0Ͱ͋Δ֬཰͸࣍ࣜʹͳΔ 2࣍ݩೖྗ2Ϋϥε෼ྨϩ δεςΟοΫճؼͷ෮श f(x, y) = w0 + w1 x + w2 y f(x, y) = 0 P(x, y) = σ(w0 + w1 x + w2 y) 1 − P(x, y)
6. χϡʔϥϧωοτϫʔΫ ෩ͷදݱʹͯ͠࠶੔ཧ x y ॏΈw1 ॏΈw2 f ग़ྗ ೖྗ ೖྗ 1 ͍ͭ΋1ͷ μϛʔೖྗ ॏΈw0 f(x, y) = w0 + w1 x + w2 y { P(x, y) = σ( f ) ⟶ t = 1 1 − P(x, y) ⟶ t = 0 ೖྗ૯࿨
7. ࠓޙΛߟ͑ɺଟೖྗଟΫϥεΛѻ ͑ΔΑ͏ʹ਺ࣜͷදݱΛม͑Δ x1 x2 ॏΈw1 ॏΈw2 a ग़ྗ ೖྗ ೖྗ x0 ͍ͭ΋1ͷ μϛʔೖྗ ॏΈw0 a = w0 x0 + w1 x1 + w2 x2 { y = σ(a) ⟶ P(t = 1|x) 1 − y ⟶ P(t = 0|x) ೖྗ૯࿨ ৚݅෇͖֬཰ͷදݱ ʢx͕ϕΫτϧදهͳͷ ͸ೖྗ͕ଟ࣍ݩͰ͋Δ ͜ͱʹ߹Θ͍ͤͯΔʣ ग़ྗΛyͱ͢Δ ೖྗ૯࿨Λaͱ͢Δ ೖྗΛxͷఴࣈ Ͱදݱ͢Δ
8. 2Ϋϥεͱ3Ϋϥεͷҧ͍ ͷ֓ཁ 2Ϋϥε෼ྨϩδεςΟοΫճؼ͸ɺग़ྗ ͕ҰͭͰɺγάϞΠυؔ਺Λ௨͢ɻग़ྗͷ େখ͕2ΫϥεΛද͢ 3Ϋϥε෼ྨϩδεςΟοΫճؼ͸ɺग़ྗ ͕3ͭͰɺιϑτϚοΫεؔ਺Λ௨͢ɻग़ྗ ͷͦΕͧΕͷ஋͕ͦΕͧΕͷΫϥεͷ֬཰ Λࣔ͢
9. 3Ϋϥε෼ྨʹ֦ு͢Δ ͱ͜͏ͳΔ x1 x2 a0 ग़ྗ ೖྗ ೖྗ x0 ͍ͭ΋1ͷ μϛʔೖྗ w00 ೖྗ૯࿨ ग़ྗΛ3ͭʹ͢Δͷʹ߹Θͤͯɺ ೖྗ૯࿨Λ3ͭʹ͠ɺॏΈ΋߹Θ ͤͯ૿͑Δ ॏΈͷఴࣈͷ͚ͭํ͕ٯ ͡Όͳ͍͔ͱࢥ͏͔΋͠Ε ͳ͍͕ɺ͜ͷํ͕৭ʑศར a1 a2 w01 w02 w20 w21 w22 y0 = exp(a0 ) ∑K−1 k=0 exp(ak ) → P(t = 0|x) y1 = exp(a1 ) ∑K−1 k=0 exp(ak ) → P(t = 1|x) y2 = exp(a2 ) ∑K−1 k=0 exp(ak ) → P(t = 2|x) Ҏ߱Ͱࡉ͔͘ݟ͍ͯ͘ ೖྗ૯࿨ʹιϑτϚοΫεؔ਺Λ௨ ͍ͯ͠ΔʢͦͷҙຯͰͦΕͧΕͷग़ ྗ͸ؔ܎͠߹͍ͬͯΔʣ a0 = ∑2 i=0 w0i xi
10. ೖྗxͱ໨తม਺t n ೖྗ ೖྗ ਖ਼ղσʔλ ʢΫϥεʣ 0 5.604765 -0.837603 0 1 5.093028 -1.098183 1 2 -2.595448 1.348614 1 3 -0.662749 -5.056531 0 4 15.573566 10.073330 2 5 7.084038 2.165339 2 6 6.204333 -2.945187 1 7 13.349965 12.577250 0 8 16.487809 6.629031 0 9 0.060047 -2.900301 2 x1 x2 t 1-of-Kූ߸Խ ʢone-hotදݱʣ [[1 0 0] [0 1 0] [0 1 0] [1 0 0] [0 0 1] [0 0 1] [0 1 0] [1 0 0] [1 0 0] [0 0 1]] NݸͷσʔλશମͰେจࣈͷX ೖྗ1ͭͰখจࣈͷϕΫτϧදهx = x0 x1 x2 NݸͷσʔλશମͰT x0͸ৗʹ1ͷ μϛʔೖྗ
11. ॏΈwɺೖྗ૯࿨aɺग़ ྗy ࠓ͸3Ϋϥε෼ྨ໰୊ͳͷͰɺೖྗ૯࿨͸3ͭ͋ΔʢD͸ೖྗ࣍ݩ ਺ʣ Ϟσϧͷύϥϝʔλ͸·ͱΊͯߦྻͰද͢ͱ͜͏ͳΔ ೖྗ૯࿨ΛιϑτϚοΫεؔ਺ʹೖྗͨ͠΋ͷΛग़ྗyͱ͢ΔʢK͸ ෼ྨ͢ΔΫϥε਺ʣ ak = wk0 x0 + wk1 x1 + wk2 x2 = D ∑ i=0 wki xi (k = 0,1,2) yk = exp(ak ) ∑K−1 k=0 exp(ak ) (k = 0,1,2) w = w00 w01 w02 w10 w11 w12 w20 w21 w22
12. ιϑτϚοΫεؔ਺ ෳ਺ͷ਺͕͋ͬͨͱ͖ɺͦΕΛιϑτϚοΫεؔ਺ʹೖྗ͢Δͱɺ ͦΕͧΕͷ਺ͷେখؔ܎Λอͬͨ··ɺҎԼͷ৚݅Λຬͨ͢਺஋ʹ ม׵͞ΕΔ 0͔Β̍·Ͱͷ஋ શͯΛ଍ͨ͠Β1 ͢ͳΘͪɺೖྗ͢Δ਺஋Λ֬཰Λද͢਺஋ʹม׵͢Δ͜ͱ͕Ͱ͖Δ ؔ਺ ෼ྨ͢Δχϡʔϥϧωοτͩͱग़ྗ૚ʹ͔·͢͜ͱ͕ଟ͍ͷͰ֮͑ ͓ͯ͘ͱྑ͍ yk = exp(ak ) ∑K−1 k=0 exp(ak )
13. ࠓճͷ3Ϋϥε෼ྨ͸ ιϑτϚοΫεؔ਺ʹΑͬͯɺy0+y1+y2=1Ͱ͋Δ͜ͱ͕อূ͞Ε Δɻ͜ͷϞσϧͷग़ྗy0,y1,y2Λɺ֤Ϋϥεʹଐ͢Δ֬཰Λද͢Α͏ ʹֶश͢Δͷ͕ࠓճ΍Γ͍ͨ͜ͱ ΋ͪΖΜֶशͱ͸͍͍ײ͡ͷύϥϝʔλwΛݟ͚ͭΔ͜ͱͰ͋Δ x1 x2 a0 ग़ྗ x0 w00 ೖྗ૯࿨ a1 a2 w01 w02 w20 w21 w22 y0 = softmax(a0 ) y1 = softmax(a1 ) y2 = softmax(a2 ) 5.6 -0.8 1 0.85 0.11 0.04 y1͕Ұ൪େ͖͍͔ΒΫϥε1ͬΆ͍ͳ
14. ໨తؔ਺Λఆٛ͢ΔͨΊ ʹɺ·ͣ͸໬౓ؔ਺ ໬౓͸ɺશೖྗσʔλXʹରͯ͠શΫϥεσʔλT͕ੜ੒͞Εͨ֬཰ 1ͭͷೖྗσʔλxʹண໨ͯ͠ɺͦͷΫϥε͕T=[1,0,0]ʢ͢ͳΘͪΫϥε0ʣ Ͱ͋ͬͨΒɺͦͷΫϥε͕ੜ੒͞Εͨ֬཰͸ Ϋϥε1Ͱ΋2Ͱ΋ಉ͡Α͏ʹදͤΔΑ͏ʹ͢Δͱ Nݸͷσʔλ͕ੜ੒͞Εͨ֬཰͸શ෦ֻ͚߹ΘͤΕ͹͍͍ͷͰ P(t = [1,0,0]|x) = y0 P(t|x) = yt0 0 yt1 1 yt2 2 P(T|X) = N−1 ∏ n=0 P(tn |xn ) = N−1 ∏ n=0 ytn0 n0 ytn1 n1 ytn2 n2 = N−1 ∏ n=0 K−1 ∏ k=0 ytnk nk
15. ฏۉަࠩΤϯτϩϐʔޡ ࠩ ฏۉަࠩΤϯτϩϐʔޡࠩؔ਺͸ɺ໬౓ͷෛͷ ର਺ͷฏۉͳͷͰɺ ্ͷࣜʹͳͬͯ͠·͑͹΋͏໬౓ؔ਺Λ௚઀ѻ Θͳ͍ͷͰɺ͋Δҙຯ໬౓ؔ਺ͷҙຯ͕Θ͔ͬ ͯͳͯ͘΋໰୊ͳ͍ͬͪΌ໰୊ͳ͍ E(w) = − 1 N log P(T|X) = − 1 N log N−1 ∏ n=0 K−1 ∏ k=0 ytnk nk = − 1 N N−1 ∑ n=0 K−1 ∑ k=0 tnk log ynk
16. ฏۉަࠩΤϯτϩϐʔޡ ͕ࠩҙຯ͢Δ΋ͷ͸̍ ͋ΔҰͭͷσʔλ͚ͩʹ஫໨͢ΔͱɺɹɹɹɹͱͳΔ E(w) = − 1 N N−1 ∑ n=0 K−1 ∑ k=0 tnk log ynk tk log yk ͚ͩ͜͜ʹ஫໨ ͢Δ a0 ग़ྗ ೖྗ૯࿨ a1 a2 y0 = softmax(a0 ) y1 = softmax(a1 ) y2 = softmax(a2 ) 0.85 0.11 0.04 ਖ਼ղt 0 1 0 t0 log y0 = 0 t1 log y1 = − 0.1625 t2 log y2 = 0 ͜͜ͱ͜͜Λֻ ͚ͨ஋ʹͳΔ ͜͜ʢΛlogͱͬͨ஋ʣͱ͜͜ Λֻ͚ͨ஋ʹͳΔ
17. ฏۉަࠩΤϯτϩϐʔޡ ͕ࠩҙຯ͢Δ΋ͷ͸̎ ͋ΔҰͭͷσʔλ͚ͩʹ஫໨͢ΔͱɺɹɹɹɹɹɹͱͳΔ tk͸one-hotදݱͷਖ਼ղσʔλͳͷͰɺtk=1Ͱ͋Δyk͚͕ͩEͷ߹ ࢉର৅ʹͳΔʢ͋ͱ͸શ෦θϩʣ yk͸0ʙ1ͷ஋Ͱ͋Γɺ1ʹ͍ۙ΄ͲlogΛऔΔͱ0ʹۙ͘ͳΔɻ0ʹ ͍ۙ΄ͲlogΛऔΔͱϚΠφεํ޲ʹେ͖ͳ஋ͱͳΔ ͭ·Γɺ෼ྨͷ݁Ռ͕ਖ਼͍͠ͳΒɹɹɹɹɹɹɹ͸΄΅0ʹͳΔ ͠ɺؒҧ͍ͬͯΔ΄ͲϚΠφεํ޲ʹେ͖͘ͳΔɻ͜Ε͸ͭ·Γ ޡࠩΛݟ͍ͯΔ͜ͱʹͳΓɺ͜ΕΒΛσʔλͷ਺͚ͩ଍͠߹Θͤ ͨE(W)͸ೋ৐ޡࠩͱରͯ͠ൃ૝͸มΘΒͳ͍ʢೋ৐ޡࠩͷ෼ྨ൛ Έ͍ͨͳΠϝʔδʣɻ E(w) = − 1 N N−1 ∑ n=0 K−1 ∑ k=0 tnk log ynk tk log yk ͚ͩ͜͜ʹ஫໨ ͢Δ tk log yk
18. ޯ഑๏ʹΑΔղ ޯ഑๏ͰฏۉަࠩΤϯτϩϐʔޡࠩؔ਺E(W)Λ࠷খԽ͢ΔWΛٻΊΔʹ ͸ɺ͍ͭ΋ͷΑ͏ʹ֤wki ʹؔ͢Δภඍ෼Λ༻ҙ͢Ε͹ྑ͍ ಋग़͸লུ͢Δ͕ɺಋؔ਺͸ҎԼͷΑ͏ʹશͯͷkͱiʹରͯ͠ಉ͡ܗʹ ͳΔ ΋ͪΖΜֶशଇ͸͜͏ ֶशଇʹैͬͯগͣͭ͠ύϥϝʔλwΛม͍͖͑ͯʢޯ഑ϕΫτϧͷ൓ ରํ޲ͷࡔΛԼ͍͖ͬͯʣɺ࠷దͳύϥϝʔλwΛٻΊΔ ∂E ∂wki = 1 N N−1 ∑ n=0 (ynk − tnk )xni wki := wki − ∂E ∂wki
19. ޯ഑๏͸લճͱಉָ͘͡ ͠Α͏ scipy.optimizeϥΠϒϥϦʹؚ·ΕΔ minimize()ͱ͍͏ؔ਺Ͱޯ഑๏͕ߦ͑Δ
20. ՝୊ one-hotදݱͷਖ਼ղσʔλΛ༻ҙ͢Δ scipy.optimizeͷminimize()Λ࢖ֶͬͯश͢Δ ΫϥεؒͷڥքઢΛάϥϑͰදࣔ͢Δɻ3Ϋϥε ͋ΔͷͰ୯७ʹ௚ઢͰදΘͤͳ͍ͷͰɺྫ͑͹ͦ ΕͧΕͷΫϥεͰ0.5Ҏ্ͷग़ྗ͕ಘΒΕΔྖҬ Λ౳ߴઢͳͲͰදࣔ͢Δͱྑ͍ʢmatplotlibͷ౳ ߴઢ͸contour()ͬͯ΍ͭʣ
21. ࢀߟจݙ தҪ ӻ࢘ʮITΤϯδχΞͷͨΊͷػցֶश ཧ࿦ೖ໳ʯٕज़ධ࿦ࣾ, 2015 ҏ౻ ਅʮPythonͰಈֶ͔ͯ͠Ϳʂ͋ͨΒ͠ ͍ػցֶशͷڭՊॻʯᠳӭࣾ, 2018 ཱੴݡޗʮ΍ֶ͘͞͠Ϳ ػցֶशΛཧղ͢ ΔͨΊͷ਺ֶͷ͖΄Μ ʯϚΠφϏग़൛, 2017