機械学習も筋肉が大事？意外と知らない数学

2019/9/11 みんなのPython勉強会でしゃべったときの資料です。
機械学習の話も筋肉の話もせず、ただひたすら数学の話をしました。

1. ػցֶश΋ے೑͕େࣄʁ ҙ֎ͱ஌Βͳ͍਺ֶ ΈΜͳͷPythonษڧձ @ΫϦʔΫɾΞϯυɾϦόʔ 2019/9/11 Ճ౻ެҰ
2. ͻͲ͍࿩
3. ࣗݾ঺հ ࢯ໊ɿՃ౻ެҰʢ͔ͱ͏͖Έ͔ͣʣ ॴଐɿιϑτόϯΫגࣜձࣾʢࠓ೥7݄Ҡ੶ʣ Twitterɿ@hamukazu ࢓ࣄɿػցֶशͷΞϧΰϦζϜΛߟ͑Δ͜ͱ झຯɿےτϨ
4. Ṗͷ҉߸ SQ: 120 BP: 100 DL: 90 ʢීஈͷτϨʔχϯάͰͷ਺஋ɺmax͸ࢼͨ͜͠ͱͳ͍ʣ
5. ຊͷ঺հ ॻ͖·ͨ͠ʂ ؂म͠·ͨ͠ʂ https://bit.ly/mlessence https://bit.ly/mlzukan
6. ۙگ ࣾ಺ͰʮػցֶशͷΤοηϯεʯΛಡΉษڧձΛ։࠵ͯ͠· ͢ɻ ಺༰͸ࠓͷͱ͜Ζ΄΅਺ֶͷߨٛɻ
7. ࠓ೔ͷ࿩ ਺ֶʹؔ͢Δ͜ͱͰɺ • ීஈ͔Β࣭໰Λड͚Δ͕ʮػցֶशͷΤοηϯεʯͰॻ͖ ͖Εͳ͔ͬͨ͜ͱ • ʮػցֶशͷΤοηϯεʯͷಡऀ͔Βड͚࣭ͨ໰
8. ॳڃฤ
9. Q: 0÷0͸Ͳ͏ͳΓ·͔͢ʁ A: ʮఆٛ͞Ε͍ͯͳ͍ʯͰ͢
10. ׂΓࢉͱ͸ͳΜͰ͔͋ͬͨ 6 ÷ 3 3 × ɹ= 6 ͱ͸ ͷ˘ʹ͸͍Δ΋ͷΛٻΊΑͷҙຯ ༩͑ΒΕͨa, bʹ͍ͭͯ b × x = a Λຬͨ͢x͕།Ұଘࡏ͢Δͱ͖ͦΕΛ a ÷ b ͱॻ͘ ͱͳΔx͸།ҰͰ͸ͳ͍ͷͰ0÷0͸ఆٛ͞Εͯͳ͍ʢundefinedʣ 0 × x = 0
11. Α͋͘Δؒҧ͍ https://detail.chiebukuro.yahoo.co.jp/qa/question_detail/q117470996 ͷղ͸ෆఆʢͳΜͰ΋͍͍ʣ 0 × x = 0 ํఔࣜ ͸ղͳ͠ʢෆೳʣ 0 × x = 1 ํఔࣜ 0÷0ͱ1÷0͸undefined ํఔࣜͷղΛग़͢͜ͱͱɺԋࢉͷఆٛ͸ผ
12. ڭ܇ɿ ఆٛʹ໭Δͷେࣄ
13. Q: ແݶʢ∞ʣ͸਺ͳͷͰ͔͢ʁ A: ʮ਺ʯͰ͸ͳ͍ͱΈΔͷ͕ҰൠతͰ͢ ∞ ∉ ℝ
14. ∞͕਺ͩͱࢥ͏ͱ͍Ζ͍Ζͱෆ౎߹͕ى͜Δ ྫ͑͹ ∞ − ∞ ͕ҰҙʹܾΊΒΕͳ͍ Ͱ͸ lim x→+0 1 x = ∞ ͬͯͲ͏͍͏͜ͱʁ lim x→+0 1 x ∞ Λܭࢉͨ͠ʮ݁Ռʯ͕͋ͬͯɺͦͷ݁Ռͱ ౳͍͠ͱ͍͏ҙຯͰ͸ͳ͍ʂ ͕ ͜ͷ౳߸͸஋͕౳͍͜͠ͱΛද͍ͯ͠ΔͷͰ͸ͳ͘ɺ ʮ=∞ʯ·ͰؚΊͯܗ༰ࢺͷΑ͏ͳ΋ͷͩͱࢥ͏ͱΑ͍ɻ
15. lim x→+0 f(x) = ∞ R ∈ ℝ δ ∈ ℝ 0 < x < δ f(x) > R ͷਖ਼֬ͳఆٛ͸ ʮ೚ҙͷ ʹ͍ͭͯ ͕ଘࡏͯ͠ ͳΒ͹ Ͱ͋Δʯ ҎԼɺԿݴͬͯΔ͔Θ͔Βͳ͍ਓͷͨΊͷ஫ऍ 2ਓʹΑΔήʔϜΛߟ͑Δ ϓϨΠϠAɿ࣮਺ R ΛҰͭબΜͰఏࣔ͢Δ ϓϨΠϠBɿϓϨΠϠAͷఏࣔͷ͋ͱʹ࣮਺ δ ΛҰͭબΜͰఏࣔ͢Δ 0 < x < δ f(x) > R ͳΒ͹ ʯ ͜ͷͱ໋͖୊ʮ ͕੒Γཱͯ͹ϓϨʔϠBͷউͪ lim x→+0 f(x) = ∞ Ͱ͋Δͱ͸ɺͭ·ΓϓϨΠϠB͕ඞউͰ͋Δ͜ͱ ʢϓϨΠϠA͕Ұੜݒ໋ҙ஍ѱͯ͠΋উͯͳ͍ʣ ϧʔϧɿ
16. ஫ҙ ਺ֶͱ࣮૷͸ผ >>> 0/0 Traceback (most recent call last): File "<stdin>", line 1, in <module> ZeroDivisionError: division by zero >>> import numpy as np >>> np.float64(0)/np.float64(0) nan >>> np.inf inf >>> np.inf+1 inf >>> np.inf-1 inf ͱ͘ʹແݶΛࡶʹѻ͏ͱ࿦ཧతໃ६ͷ΋ͱʹͳΓ͕ͪ
17. Q: ͳͥ a1 2 = a A: ࢦ਺͕ࣗવ਺ͷ৔߹ͷ๏ଇ͔Β ࣗવʹఆٛ͞Εͨ΋ͷͰ͢ a−1 = 1 a Ͱ ͳͷʁ
18. ax × ay = ax+y ࢦ਺๏ଇ ax ÷ ay = ax−y (ax)y = axy ͜Ε͕ɺx, y͕ࣗવ਺ͷͱ͖੒Γཱͭͷ͸Θ͔Δ 22 × 23 = (2 × 2) × (2 × 2 × 2) = 25 25 ÷ 23 = 2 × 2 × 2 × 2 × 2 2 × 2 × 2 = 22 (22)3 = (2 × 2) × (2 × 2) × (2 × 2) = 26 (1) (2) (3) ࢦ਺๏ଇ͕x, y͕ࣗવ਺Ҏ֎Ͱ΋੒ΓཱͭΑ͏ʹͯ͠ΈΔ a2 ÷ a2 = 1 a2 ÷ a2 = a2−2 = a0 ҰํͰ(2)ΑΓ Αͬͯ a0 = 1 1 a = 1 ÷ a = a0 ÷ a1 = a0−1 ʢ(2)ΑΓʣ = a−1 ྫɿ (a1 2)2 = a1 2 ×2 ʢ(3)ΑΓʣ = a1 = a Αͬͯ ͱ͸ɺ2৐ͯ͠ ʹͳΔ਺ a1 2 a a1 2 = a ͭ·Γ ʢx͕࣮਺ͷͱ͖ͷ ͸ɺ ax a > 0 ͷͱ͖ʹݶఆʣ
19. ͜͜ͰͷετʔϦʔɿ ΋ͱ΋ͱɹɹ͸x͕ࣗવ਺ͷͱ͖ͷΈΛߟ͍͑ͯͨ ࣗવ਺ͷͱ͖ʹ੒Γཱ͍ͬͯͨ๏ଇ͕੒ΓཱͭΑ͏ʹɺ ࣮਺ͷͱ͖ʹ֦ுͨ͠ ͜ͷΑ͏ʹɺݶఆతͳൣғͰߟ͑ΒΕ͍ͯͨ΋ͷΛɺ ͦΕ·Ͱͷ๏ଇ͕੒ΓཱͭΑ͏ʹ֦ு͢Δͱ͍͏͜ͱ ͕Α͋͘Δ ax ͜͏͍͏ͷ͸ɺ਺ֶͰ ʮʙͷ֓೦ͷࣗવͳ֦ுʯ ͱݴͬͨΓ͢Δɻ
20. ্ڃฤ
21. Q: ೋ࣍ܗࣜͷϔοηߦྻͷܭࢉ͕Θ͔Γ·ͤΜ ʢʮػցֶशͷΤοηϯεʯp168ʣ A: ͖ͪΜͱ੒෼͝ͱʹҙࣝͯ͠ܭࢉ͠·͠ΐ͏ ҎԼॻ੶ΑΓஸೡʹઆ໌͠·͢
22. f(x) = xT Ax ͷͱ͖ͷ ∇2f ΛٻΊ͍ͨ f(x) = n ∑ i=1 n ∑ j=1 aij xi xj ͳͷͰɺ͜ΕΛ Ͱภඍ෼͍ͨ͠ xk (k = 1,2,…, n) A͕ରশߦྻͱͯ͠ i ≠ k, j ≠ k ͷͱ͖ ∂ ∂xk (aij xi xj ) = 0 ͋ͱ͸ɺi, jͷҰํ͕kͷͱ͖ɺ྆ํ͕kͷͱ͖ʹ ෼͚ͯܭࢉ͢Ε͹Α͍
23. ∂f ∂xk = ∂ ∂xk akk x2 k + ∑ j≠k aik xi xk + ∑ i≠k akj xk xj = 2akk xk + ∑ j≠k aik xi + ∑ i≠k akj xj = 2akk xk + ∑ j≠k aki xi + ∑ i≠k akj xj = 2akk xk + 2∑ j≠k aki xi = 2 n ∑ i=1 aki xi ∇f = 2∑n i=1 a1i xi 2∑n i=1 a2i xi ⋮ 2∑n i=1 ani xi = 2Ax ↑͜͜ͰA͕ରশͰ͋Δ͜ͱΛ࢖ͬͨ ∇2f ͱ͸ɺ ∇f ͷ֤੒෼Λ xl (l = 1,2,…, n) Ͱภඍ෼ͨ͠΋ͷ
24. ∂ ∂xl ( 2 n ∑ i=1 aki xi) ∂ ∂xl (aki xi) = 0 i ≠ l ͷͱ͖ Λܭࢉ͍ͨ͠ɻ ͳͷͰ ͷͱ͖͚ͩΛߟྀ͢Ε͹Α͍ i = l ∂ ∂xl ( 2 n ∑ i=1 aki xi) = ∂ ∂xl (2akl xl) = 2akl ∇f ͜Ε͸ɺ ͷk൪໨ͷ੒෼Λ xl Ͱภඍ෼ͨ͠΋ͷͳͷͰ ͭ·Γ ∇2f ͷ ੒෼ (k, l) ∇2f ͷ ੒෼͕ (k, l) ͭ·Γ 2akl ͱ͍͏͜ͱ͸ ∇2f = 2A
25. Q: ࠷খೋ৐๏ͷܭࢉ A: ͖͞΄Ͳͷܭࢉ͕໾ʹཱͪ·͢ E(w) = ∥y − Xw∥2 ͷͱ͖ ∇E = − 2XTy + XT Xw ͕Θ͔Γ·ͤΜɻ ʢˡ࣮͸͜ͷεϥΠυͷ४උதʹޡ২͕ݟ͔ͭͬͨʣ
26. E(w) = ∥y − Xw∥2 = (y − Xw) T (y − Xw) = (yT − (Xw)T) (y − Xw) = (yT − wT XT) (y − Xw) = yTy − yT Xw − wT XTy + wT XT Xw ∇E = − 2XTy + 2XT Xw ∇(yT Xw) = XTy ∇(wT XTy) = XTy } ∇(wT XT Xw) = 2XT Xw ࣗ෼Ͱܭࢉͯ͠ΈΑ͏ ʢͦΜͳʹ೉͘͠ͳ͍͸ͣʣ ͖͞΄Ͳͷೋ࣍ܗࣜͷܭࢉͱಉ͡ Αͬͯ
27. ·ͱΊ • ఆٛʹ໭ͬͯߟ͑Δ͜ͱ͕༗ޮͳ͜ͱ΋͋Δ • ਺ֶͷཧ࿦ͱίϯϐϡʔλ্ͷ࣮૷͸ผ • ʮࣗવͳ֦ுʯͷߟ͑ํΛ஌͓ͬͯ͜͏ • ϔοηߦྻͷܭࢉ͸ɺҰͭҰͭ੒෼Λߟ͑ΔͱͦΕ΄Ͳ೉ ͘͠ͳ͍͔΋Αʢʁʣ