68

# Deep Learning 5章後半

ゼミの輪講資料．Goodfellow本5.5節〜5.11節

May 29, 2018

## Transcript

ঙ໺ݚڀࣨ B4
2. ### ໨࣍ 1. ࠷໬ਪఆ 2. ϕΠζ౷ܭ 3. ڭࢣ͋Γֶश 4. ڭࢣͳֶ͠श 5.

֬཰తޯ഑߱Լ๏ (SGD) 6. Deep Learning ΁ͷಈػ 2

8. ### ڭࢣ͋Γֶश ֬཰తڭࢣ͋Γֶश ෮श: ҰൠઢܗϞσϧ    y = θT

x + ϵ ϵ ∼ N(ϵ|0, σ2) ⇒ p(y|x; θ) = N(y; θT x, σ2) (5.80) ਖ਼ن෼෍ͷఆٛҬ͸ (−∞, ∞) ˠ {0, 1} ͷೋ஋෼ྨ໰୊ʹ͸࢖͑ͳ͍ 5

1 + e−x 6
10. ### ڭࢣ͋Γֶश ֬཰తڭࢣ͋Γֶश ϩδεςΟ οΫճؼ p(y = 1|x; θ) = 1

1 + e−θT x = 1 1 + e−(θ0+θ1x1+θ2x2+··· ) ˠ {0, 1} ͷೋ஋൑ผʹ࢖͑Δ 7

13. ### ڭࢣ͋Γֶश αϙʔτϕΫλʔϚγϯ (SVM) ฏ໘ͷํఔࣜ͸ ax + by + c =

0 ͳͷͰ͜ΕΛҰൠԽͯ͠ɼ෼ྨ௒ฏ໘ͷํఔࣜ͸αϙʔτϕΫτ ϧͷू߹ x∗ Λ༻͍ͯ w0 + wT x∗ = 0 ͱॻ͚Δɽֶश͢Δͷ͸͜ͷ܎਺ w Ͱ͋Δ 2 ͭͷࢧ࣋௒ฏ໘͸ɼ෼ྨ௒ฏ໘Λ ±k ͚ͩͣΒͯ͠    w0 + wT x∗ = k w0 + wT x∗ = −k ⇒ |w0 + wT x∗| = k Ͱ͋Δ 10
14. ### ڭࢣ͋Γֶश αϙʔτϕΫλʔϚγϯ (SVM) ௒ฏ໘ͷࣜ͸ఆ਺ഒͯ͠΋ಉ͡΋ͷΛࣔ͢ͷͰɼֶश݁Ռ͕Ұҙ ʹఆ·Βͳ͍ ˠҰҙੑΛ࣋ͭΑ͏ʹ੍໿Λ՝͢ ੍໿: |w0 + wT

x∗| = 1 ॏΈϕΫτϧʹ͍ͭͯඪ४Խ͢Δͱ |w0 + wT x∗| ∥w∥ = 1 ∥w∥ ͜ΕΛ࠷େԽ͢ΔΑ͏ʹֶश͢Δ 11

16. ### ڭࢣ͋Γֶश αϙʔτϕΫλʔϚγϯ (SVM) ྫ: ສ༗Ҿྗͷࣜ ಛ௃: ࣭ྔ m1, m2 ɼڑ཭

r f(m1, m2, r) = G m1m2 r2 ͜ΕΛ֤ಛ௃ʹؔͯ͠ઢܗʹ͍ͨ͠ ˠର਺ΛͱΔ logf(m1, m2, r) = logG + logm1 + logm2 − logr2 ઢܗʹͳͬͨ 13

21. ### ڭࢣͳֶ͠श k-means ๏ ΞϧΰϦζϜ 1. Ϋϥελத৺ͷॳظ஋ͱͯ͠ɼσʔλ఺͔Β k ݸͷηϯτϩ ΠυΛϥϯμϜʹબͿ (k

͸ط஌) 2. ֤αϯϓϧΛ࠷΋͍ۙηϯτϩΠυʹׂΓ౰ͯΔ 3. ֤ηϯτϩΠυΛࣗ਎ʹׂΓ౰ͯΒΕͨσʔλͷத৺ʹҠಈ ͢Δ 4. 2,3 Λ܁Γฦ͢ 17

25. ### ֬཰తޯ഑߱Լ๏ (SGD) ίετؔ਺͸ (ςετ) σʔλ͝ͱͷଛࣦؔ਺ͷ࿨ʹ෼ղͰ͖Δ͜ ͱ͕ଟ͍ ઢܗճؼͰ͸ɼର਺໬౓ L(x, y, θ)

Λ༻͍ͯ J(θ) = Ex,y∼ˆ pdata [L(x, y, θ)] = 1 m m ∑ i=1 L(x(i), y(i), θ) (5.96) L(x(i), y(i), θ) = −logp(y|x, θ) ͜ͷίετؔ਺ʹؔͯ͠ɼύϥϝʔλ θ ʹ͍ͭͯޯ഑๏Λద༻ 20
26. ### ֬཰తޯ഑߱Լ๏ (SGD) ∇θJ(θ) = ∇θ [ 1 m m ∑

i=1 L(x(i), y(i), θ) ] = 1 m m ∑ i=1 ∇θL(x(i), y(i), θ) (5.97) ͜ͷܭࢉྔ͸ O(m) Ͱɼσʔλ͕૿͑Δͱ͔ͳΓͭΒ͍ ˠ֬཰తޯ഑߱Լ๏ (SGD) 21
27. ### ֬཰తޯ഑߱Լ๏ (SGD) SGD ͸ޯ഑Λظ଴஋ͰදݱͰ͖Δͱߟ͑ɼαϯϓϧͷখ͍͞αϒ ηοτ (ϛχόον) ͷޯ഑๏Ͱۙࣅతʹٻ·Δͱ͢Δ B = {x(1),

. . . , x(m′)} ͷϛχόονΛҰ༷ϥϯμϜʹֶशσʔλ ηοτ͔Β΋ͬͯ͘Δ m′ ͸͍͍ͩͨ 100ʙ300 ͘Β͍Ͱɼm ͕ଟͯ͘΋ಉ༷ ޯ഑ͷਪఆྔ g ͸ g = 1 m′ ∇θ m′ ∑ i=1 L(x(i), y(i), θ) (5.98) ύϥϝʔλͷਪఆྔ͸ θ ← θ − ϵg 22

30. ### Deep Learning ΁ͷಈػ ࣍ݩͷढ͍ ྫͱͯ͠ k ۙ๣๏ (k-Nearest Neighbour) ͱ͍͏ֶशΞϧΰϦζϜ

Λߟ͑Δ k ۙ๣๏ ςετσʔλͷೖྗʹରͯ͠ɼಛ௃্ۭؒͰ΋ͬͱ΋͍ۙ k ݸ ͷֶशσʔλΛ୳͠ɼͦΕΒͷֶशσʔλͷଐ͢ΔΫϥεͷଟ ਺ܾͰςετσʔλʹׂΓৼΔΫϥεΛܾఆ͢Δ k = 3 ͷ৔߹ ೖྗ˔ͷϥϕϧ͸˙ 24

32. ### References I [1] ਢࢁರࢤ, ϕΠζਪ࿦ʹΑΔػցֶशೖ໳. ߨஊࣾ, 2017. [2] খ໺ాਸ, αϙʔτϕΫλʔϚγϯ.

ΦʔϜࣾ, 2007. [3] Sebastian Raschka ஶ, גࣜձࣾΫΠʔϓ༁, ୡਓσʔλαΠ ΤϯςΟετʹΑΔཧ࿦ͱ࣮ફ Python ػցֶशϓϩάϥϛ ϯά, ΠϯϓϨε, 2016.