Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Deep Learning 5章後半

Deep Learning 5章後半

ゼミの輪講資料.Goodfellow本5.5節〜5.11節

Takahiro Kawashima

May 29, 2018
Tweet

More Decks by Takahiro Kawashima

Other Decks in Science

Transcript

  1. ໨࣍ 1. ࠷໬ਪఆ 2. ϕΠζ౷ܭ 3. ڭࢣ͋Γֶश 4. ڭࢣͳֶ͠श 5.

    ֬཰తޯ഑߱Լ๏ (SGD) 6. Deep Learning ΁ͷಈػ 2
  2. ڭࢣ͋Γֶश ֬཰తڭࢣ͋Γֶश ෮श: ҰൠઢܗϞσϧ    y = θT

    x + ϵ ϵ ∼ N(ϵ|0, σ2) ⇒ p(y|x; θ) = N(y; θT x, σ2) (5.80) ਖ਼ن෼෍ͷఆٛҬ͸ (−∞, ∞) ˠ {0, 1} ͷೋ஋෼ྨ໰୊ʹ͸࢖͑ͳ͍ 5
  3. ڭࢣ͋Γֶश ֬཰తڭࢣ͋Γֶश ϩδεςΟ οΫճؼ p(y = 1|x; θ) = 1

    1 + e−θT x = 1 1 + e−(θ0+θ1x1+θ2x2+··· ) ˠ {0, 1} ͷೋ஋൑ผʹ࢖͑Δ 7
  4. ڭࢣ͋Γֶश αϙʔτϕΫλʔϚγϯ (SVM) ฏ໘ͷํఔࣜ͸ ax + by + c =

    0 ͳͷͰ͜ΕΛҰൠԽͯ͠ɼ෼ྨ௒ฏ໘ͷํఔࣜ͸αϙʔτϕΫτ ϧͷू߹ x∗ Λ༻͍ͯ w0 + wT x∗ = 0 ͱॻ͚Δɽֶश͢Δͷ͸͜ͷ܎਺ w Ͱ͋Δ 2 ͭͷࢧ࣋௒ฏ໘͸ɼ෼ྨ௒ฏ໘Λ ±k ͚ͩͣΒͯ͠    w0 + wT x∗ = k w0 + wT x∗ = −k ⇒ |w0 + wT x∗| = k Ͱ͋Δ 10
  5. ڭࢣ͋Γֶश αϙʔτϕΫλʔϚγϯ (SVM) ௒ฏ໘ͷࣜ͸ఆ਺ഒͯ͠΋ಉ͡΋ͷΛࣔ͢ͷͰɼֶश݁Ռ͕Ұҙ ʹఆ·Βͳ͍ ˠҰҙੑΛ࣋ͭΑ͏ʹ੍໿Λ՝͢ ੍໿: |w0 + wT

    x∗| = 1 ॏΈϕΫτϧʹ͍ͭͯඪ४Խ͢Δͱ |w0 + wT x∗| ∥w∥ = 1 ∥w∥ ͜ΕΛ࠷େԽ͢ΔΑ͏ʹֶश͢Δ 11
  6. ڭࢣ͋Γֶश αϙʔτϕΫλʔϚγϯ (SVM) ྫ: ສ༗Ҿྗͷࣜ ಛ௃: ࣭ྔ m1, m2 ɼڑ཭

    r f(m1, m2, r) = G m1m2 r2 ͜ΕΛ֤ಛ௃ʹؔͯ͠ઢܗʹ͍ͨ͠ ˠର਺ΛͱΔ logf(m1, m2, r) = logG + logm1 + logm2 − logr2 ઢܗʹͳͬͨ 13
  7. ڭࢣͳֶ͠श k-means ๏ ΞϧΰϦζϜ 1. Ϋϥελத৺ͷॳظ஋ͱͯ͠ɼσʔλ఺͔Β k ݸͷηϯτϩ ΠυΛϥϯμϜʹબͿ (k

    ͸ط஌) 2. ֤αϯϓϧΛ࠷΋͍ۙηϯτϩΠυʹׂΓ౰ͯΔ 3. ֤ηϯτϩΠυΛࣗ਎ʹׂΓ౰ͯΒΕͨσʔλͷத৺ʹҠಈ ͢Δ 4. 2,3 Λ܁Γฦ͢ 17
  8. ֬཰తޯ഑߱Լ๏ (SGD) ίετؔ਺͸ (ςετ) σʔλ͝ͱͷଛࣦؔ਺ͷ࿨ʹ෼ղͰ͖Δ͜ ͱ͕ଟ͍ ઢܗճؼͰ͸ɼର਺໬౓ L(x, y, θ)

    Λ༻͍ͯ J(θ) = Ex,y∼ˆ pdata [L(x, y, θ)] = 1 m m ∑ i=1 L(x(i), y(i), θ) (5.96) L(x(i), y(i), θ) = −logp(y|x, θ) ͜ͷίετؔ਺ʹؔͯ͠ɼύϥϝʔλ θ ʹ͍ͭͯޯ഑๏Λద༻ 20
  9. ֬཰తޯ഑߱Լ๏ (SGD) ∇θJ(θ) = ∇θ [ 1 m m ∑

    i=1 L(x(i), y(i), θ) ] = 1 m m ∑ i=1 ∇θL(x(i), y(i), θ) (5.97) ͜ͷܭࢉྔ͸ O(m) Ͱɼσʔλ͕૿͑Δͱ͔ͳΓͭΒ͍ ˠ֬཰తޯ഑߱Լ๏ (SGD) 21
  10. ֬཰తޯ഑߱Լ๏ (SGD) SGD ͸ޯ഑Λظ଴஋ͰදݱͰ͖Δͱߟ͑ɼαϯϓϧͷখ͍͞αϒ ηοτ (ϛχόον) ͷޯ഑๏Ͱۙࣅతʹٻ·Δͱ͢Δ B = {x(1),

    . . . , x(m′)} ͷϛχόονΛҰ༷ϥϯμϜʹֶशσʔλ ηοτ͔Β΋ͬͯ͘Δ m′ ͸͍͍ͩͨ 100ʙ300 ͘Β͍Ͱɼm ͕ଟͯ͘΋ಉ༷ ޯ഑ͷਪఆྔ g ͸ g = 1 m′ ∇θ m′ ∑ i=1 L(x(i), y(i), θ) (5.98) ύϥϝʔλͷਪఆྔ͸ θ ← θ − ϵg 22
  11. Deep Learning ΁ͷಈػ ࣍ݩͷढ͍ ྫͱͯ͠ k ۙ๣๏ (k-Nearest Neighbour) ͱ͍͏ֶशΞϧΰϦζϜ

    Λߟ͑Δ k ۙ๣๏ ςετσʔλͷೖྗʹରͯ͠ɼಛ௃্ۭؒͰ΋ͬͱ΋͍ۙ k ݸ ͷֶशσʔλΛ୳͠ɼͦΕΒͷֶशσʔλͷଐ͢ΔΫϥεͷଟ ਺ܾͰςετσʔλʹׂΓৼΔΫϥεΛܾఆ͢Δ k = 3 ͷ৔߹ ೖྗ˔ͷϥϕϧ͸˙ 24
  12. References I [1] ਢࢁರࢤ, ϕΠζਪ࿦ʹΑΔػցֶशೖ໳. ߨஊࣾ, 2017. [2] খ໺ాਸ, αϙʔτϕΫλʔϚγϯ.

    ΦʔϜࣾ, 2007. [3] Sebastian Raschka ஶ, גࣜձࣾΫΠʔϓ༁, ୡਓσʔλαΠ ΤϯςΟετʹΑΔཧ࿦ͱ࣮ફ Python ػցֶशϓϩάϥϛ ϯά, ΠϯϓϨε, 2016.