機械学習勉強会04 偏微分と連鎖律/MLStudy04

機械学習勉強会04 偏微分と連鎖律

Transcript

1. ภඍ෼ͱ࿈࠯཯ ػցֶशͷڧྗͳ෢ث @hachiilcane
2. લճ·Ͱ͸ ࠷ٸ߱Լ๏ʢޯ഑߱Լ๏ʣͱ͍͏ภඍ ෼Λ༻͍ͨ࠷దͳύϥϝʔλͷٻΊํ ΛֶΜͩ ภඍ෼ͷ࢓ํʹ͍ͭͯ͸આ໌͠ͳ͔ͬ ͨ
3. ࠓճֶͿ͜ͱ͸ ඍ෼ͱ͸ͳΜ͔ͩͬͨΛࢥ͍ग़͢ ภඍ෼ͱݴͬͯ΋େͨ͜͠ͱͳ͍͜ͱ ΛֶͿ ඍ෼͢Δ্Ͱͱͯ΋ศརͳ࿈࠯཯ʹͭ ֶ͍ͯͿ
4. ඍ෼ͱ͸ͳΜ͔ͩͬͨ มԽͷ౓߹͍Λࣔ͢ ʮॠؒͷมԽ཰ʯΛٻΊΔ͜ͱͱ΋ݴ ͑Δ
5. άϥϑͰݴ͏ͳΒ܏͖ w=1ͷͱ͖ ܏͖͸2 w=-1ͷͱ ͖܏͖͸-2 w=0ͷͱ͖ ܏͖͸0 ͢΂ͯͷwʹରͯ͠܏͖Λϓ ϩοτ͢Δͱɺf’(w)=2wͱͳΔ
6. ඍ෼ͷఆٛ ؔ਺f(x)ͷ఺xͰͷ܏͖͸ҎԼͷΑ͏ʹද ͞ΕΔ ͜Ε͕ඍ෼ͷఆٛ d dx f(x) = lim h!0 f(x + h) f(x) h
7. ඍ෼ͷެࣜ جຊதͷجຊ ઢܗੑ xʹؔ܎͠ͳ͍ఆ਺͸0 ͳͷͰɺ૯࿨ه߸ͱඍ෼ԋࢉࢠ͸ೖΕସ͑ΒΕΔ ͍͍ͩͨ͜ΕΒͷ૊Έ߹ΘͤͰ͍͚Δ f(x) = xn d dx f(x) = nxn 1 d dx a = 0 d dx n X i=0 xn = n X i=0 d dx xn d dx (f(x) + g(x)) = d dx f(x) + d dx g(x) d dx (af(x)) = a d dx f(x)
8. ඍ෼ͯ͠ಘΒΕͨಋؔ਺ ΋ؔ਺ ಋؔ਺f’(x)΋xͷؔ਺ͳͷͰɺx=0ͷͱ͖Ͱ ΋x=1ͷͱ͖Ͱ΋ɺ޷͖ͳxʹର͢Δ܏͖Λ ಘΔ͜ͱ͕Ͱ͖Δʢ͋Δݻఆͷ஋ͷͱ͖ͩ ͚ͷ࿩Ͱ͸ͳ͘ɺxͱ͍͏ม਺Λಋೖ͢Δ ͜ͱͰҰൠԽ͞Ε͍ͯΔʣ ݴ͍׵͑Δͱɺ۩ମతͳ܏͖Λಘ͍ͨͷͰ ͋Ε͹ɺxʹͳʹ͔୅ೖ͢Δ͜ͱʹͳΔ
9. ʮ਺ֶΨʔϧͷൿີϊʔτʯ ΋ͥͻࢀߟʹͯ͠΄͍͠ ඍ෼Λߟ͑Δͷ͸มԽΛͱΒ͑ΔͨΊ ༩͑ΒΕͨؔ਺Λඍ෼ͯ͠ಋؔ਺ΛಘΔɻͦ͏ ͢Ε͹ؔ਺ͷมԽͷ༷ࢠΛ஌Δ͜ͱ͕Ͱ͖Δ ಋؔ਺΋ؔ਺ɻඍ෼ͷܭࢉΛ͢Δͱ͖ɺؔ਺͔ Βผͷؔ਺Λ࡞ΔܭࢉΛ͍ͯ͠Δ͜ͱʹͳΔ ݁৓ߒʮ਺ֶΨʔϧͷൿີϊʔτɹඍ෼Λ௥͍͔͚ͯʯ
10. ͭ·Γ͜Μͳؔ܎ੑ f(x) f0(x) ਺஋ ؔ਺ ಋؔ਺ ॠؒͷมԽ཰ ʢ܏͖ʣ ඍ෼ ͋Δ۩ମతͳ xͷ஋Λ୅ೖ ʮͲΜͳxͰ΋ʯ ୅ೖͨ͠Βͦͷxʹ ରԠ͢Δ౴͕͑Θ͔ ΔΑʂ ʮͲΜͳxͰ΋ʯ ୅ೖͨ͠Βͦͷxʹ ରԠ͢ΔॠؒͷมԽ ཰͕Θ͔ΔΑʂ ʮ͋Δಛఆͷxͷͱ ͖ͷʯॠؒͷมԽ཰ ʢ܏͖ʣͩΑʂ
11. Ͱ͸ภඍ෼ͱ͸Կ͔ ม਺͕ҰͭͰ͸ͳ͍ɺͭ·Γଟม਺ؔ਺ͷͱ͖ͷඍ෼ ྫ͑͹w0ͱw1ͷؔ਺Ͱ͋ΔҎԼͷΑ͏ͳ΍ͭʢม਺͕x ͳͷ͔wͳͷ͔͸ຊ࣭తʹ͸ͲͬͪͰ΋͍͍͜ͱͳͷͰ ؾʹ͠ͳ͍͍ͯ͘ʣ ม਺͕ෳ਺͋Δ͔Β͠ΐ͏͕ͳ͍ͷͰҰͭͣͭඍ෼͢Δɻண ໨͢Δม਺Ҏ֎͸ఆ਺ͱΈͳͯ͠ඍ෼͢Δ ภඍ෼ͬͯͨͩ͜Ε͚ͩ f(w0, w1) = w2 0 + 2w0w1 + 3
12. ࣮͸લճखͰࢼΈΑ͏ͱ ͍ͯͨ͠࿩ͱಉ͡
13. ภඍ෼ͷٻΊํ ʮภඍ෼͢Δม਺͚ͩʹண໨ͯ͠ඍ෼͢ Δʯɻண໨͠ͳ͍ม਺͸ఆ਺ͱΈͳ͢ ภඍ෼͸ɺͦͷؔ਺ͷண໨ͨ͠ม਺ํ޲ ʹ͓͚Δʮ܏͖ʯΛද͍ͯ͠Δ f(w0, w1) = w2 0 + 2w0w1 + 3 @f @w0 = 2w0 + 2w1 @f @w1 = 2w0 ←w0Ͱภඍ෼ ←w1Ͱภඍ෼
14. ෳࡶͳؔ਺Λඍ෼·ͨ͸ ภඍ෼͍ͨ࣌͠ ྫͷ͜Μͳ΍ͭΛภඍ෼͠ΖͬͯݴΘ ΕΔͱɺͪΐͬͱ·͍ͭͪ͝Ό͏ ؔ਺͕ೖΕࢠʹͳ͍ͬͯΔ৔߹ʢ͜Ε Λ߹੒ؔ਺ͱݺͿʣɺͪΐͬͱָʹඍ ෼͢Δํ๏͕͋Δ f(x) = w0 + w1x ED = 1 2 N X n=1 (w0 + w1xn tn)2
15. ߹੒ؔ਺Λඍ෼͢Δʹ͸ ࿈࠯཯ f(w)͕f(g(w))ͷΑ͏ʹೖΕࢠʹͳ͍ͬͯͯɺw Ͱඍ෼͢Δ͜ͱΛߟ͑Δͱ͖ɺҎԼͷ࿈࠯཯ ͷެࣜΛ࢖ͬͯஈ֊తʹඍ෼͢Δ͜ͱ͕Ͱ͖ Δɻภඍ෼Ͱ΋ಉ͡ߟ͑Ͱߦ͚Δ ͜͏͍͏;͏ʹ΋දݱͰ͖Δ df dw = df dg · dg dw d dw f(g(w)) = df dg · dg dw dgͰ໿෼ͨ͠Βಉ͡ ࣜʹͳΔͶͱࢥ͏ͱ֮ ͑΍͍͢
16. ࿈࠯཯Λ࢖ͬͨ߹੒ؔ਺ ͷඍ෼ͷྫ ҎԼͷΑ͏ͳf(g(w))ΛwͰඍ෼͢Δ ࿈࠯཯Λ࢖͏ͱ͜͏ܭࢉ͢Δ f(g(w)) = g(w)2 g(w) = aw + b df dg = d dg g(w)2 = 2g(w) dg dw = d dw (aw + b) = a df dw = df dg · dg dw = 2ga = 2(aw + b)a = 2a2w + 2ab ͳͷͰɺ gΛల։ͯ͋͛͠Δ ͜ͱΛ๨Εͣʹ
17. Ͱ͸ɺޡࠩؔ਺Λ࿈࠯཯ Ͱภඍ෼ͯ͠ΈΔ̍ M=1ͱͨ͠ͱ͖ͷޡࠩؔ਺ ୯७ʹf(x)ΛೖΕࢠͷؔ਺ͱݟͯ΋͍͍ ͕ɺ͜͜Ͱ͸ɹɹɹɹɹΛೖΕࢠͷؔ ਺ͱݟͯ࿈࠯཯Λ࢖ͬͯΈΔ f(x) = w0 + w1x ED = 1 2 N X n=1 (f(xn) tn)2 f(xn) tn
18. Ͱ͸ɺޡࠩؔ਺Λ࿈࠯཯ Ͱภඍ෼ͯ͠ΈΔ̎ ·ͣw0Ͱภඍ෼͢ΔɻɹɹɹɹΛɹɹ ͱஔ͘ ED = 1 2 N X n=1 (g(w0))2 @ED @g(w0) = 1 2 N X n=1 2g(w0) = N X n=1 g(w0) @g(w0) @w0 = @ @w0 (w0 + w1xn tn) = 1 @ED @w0 = @ED @g(w0) · @g(w0) @w0 = N X n=1 g(w0) · 1 = N X n=1 (f(xn) tn) f(xn) tn g(w0)
19. Ͱ͸ɺޡࠩؔ਺Λ࿈࠯཯ Ͱภඍ෼ͯ͠ΈΔ̏ ࣍͸w1Ͱภඍ෼͢ΔɻɹɹɹɹΛɹɹ ͱஔ͘ f(xn) tn g(w1) ED = 1 2 N X n=1 (g(w1))2 @ED @g(w1) = 1 2 N X n=1 2g(w1) = N X n=1 g(w1) @g(w1) @w1 = @ @w1 (w0 + w1xn tn) = xn @ED @w1 = @ED @g(w1) · @g(w1) @w1 = N X n=1 g(w1) · xn = N X n=1 (f(xn) tn)xn
20. ภඍ෼ͱ͸ਤܗతʹ͸Կ Λද͍ͯ͠ΔͷͩΖ͏͔ 3DͷάϥϑΛͿͬͨ੾ͬͨஅ໘ΛݟΔΑ͏ͳΠϝʔδ w0ͷภඍ෼ͳΒɺw0ͷ࣠ʹฏߦʹͳΔΑ͏ʹfΛแஸ Ͱ੾ͬͨͱ͖ͷஅ໘ ੾Δ৔ॴ͸ແ਺ʹ͋Δ͕ɺw0ͷภඍ෼ʹw1͕ม਺ ͱؚͯ͠·Ε͍ͯΔͳΒɺ࠷ऴతʹ͸w1ΛԿ͔ͷ ஋Ͱݻఆʹ͢Δ͜ͱʹͳΔɻw1=-1Ͱ੾ͬͨஅ໘ ͸ɺw0ͷภඍ෼ͷಋؔ਺ͷw1ʹ-1Λ୅ೖ͕ͨࣜ͠ ͦͷ໘ͷ܏͖ͷࣜʹͳΔ
21. ۩ମྫ͸ ͢Έ·ͤΜɻྗਚ͖ͨͷͰলུ ఻͔͑ͨͬͨ͜ͱ͸ɺยํͷม਺Λݻఆ ͱߟ͑Δͱ͍͏͜ͱ͸ɺݻఆͨ͠஋Ͱ ʢ໘ͰʣάϥϑΛͿͬͨ੾͍ͬͯΔͱ͍ ͏͜ͱΛɺ࣮ࡍʹάϥϑͰ͔ࣔͨͬͨ͠ ֤ʑ͔֬Ίͯ͘Ε
22. ܁Γฦ͠ʹͳΔ͕ภඍ෼ ͱ͸ਤܗతʹߟ͑Δͱ w0ͱw1ʹؔ͢Δภඍ෼͸ɺͦΕͧΕ w0ํ޲ͷ܏͖ɺw1ํ޲ͷ܏͖Λ༩͑Δ ภඍ෼ʹΑͬͯ೚ҙͷҐஔ(w0, w1)Ͱͷ ̎ํ޲ͷ܏͖Λܭࢉ͢Δ͜ͱ͕Ͱ͖Δ
23. ภඍ෼͕Ͱ͖ΔͳΒɺ͍Ζ ͍ΖͳϞσϧʹରԠग़དྷΔ Ϟσϧͷಛ௃ม਺Λෳ਺ʹͨ͠Γɺଟ ߲ࣜ͡Όͳ͍ؔ਺ʹͨ͠Γͯ͠΋ɺภ ඍ෼ͯ͠໨తؔ਺ʢ͜Ε·Ͱޡࠩؔ਺ ͱݴ͍ͬͯͨ΋ͷͷҰൠతͳݺͼํʣ ͷ୩ఈΛݟ͚ͭΔ͜ͱͰ࠷దͳύϥ ϝʔλΛݟ͚ͭΔɺͱ͍͏΍Γํ͸൚ ༻తʹ࢖͑Δ
24. ා͕ΒͣʹνϟϨϯδ ͋ͱ͸࣮ࡍʹखΛಈ͔ͯ͠ཧղ͠Α͏
25. ࢀߟจݙ தҪ ӻ࢘ʮITΤϯδχΞͷͨΊͷػցֶशཧ࿦ೖ໳ʯٕज़ධ ࿦ࣾ, 2015 ҏ౻ ਅʮPythonͰಈֶ͔ͯ͠Ϳʂ͋ͨΒ͍͠ػցֶशͷڭՊ ॻʯᠳӭࣾ, 2018 ݁৓ߒʮ਺ֶΨʔϧͷൿີϊʔτ ඍ෼Λ௥͍͔͚ͯʯSBΫϦ ΤΠςΟϒ, 2015 ཱੴݡޗʮ΍ֶ͘͞͠Ϳ ػցֶशΛཧղ͢ΔͨΊͷ਺ֶͷ͖ ΄Μ ʯϚΠφϏग़൛, 2017