Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Deep Learning 5章後半

Sponsored · Your Podcast. Everywhere. Effortlessly. Share. Educate. Inspire. Entertain. You do you. We'll handle the rest.

Deep Learning 5章後半

ゼミの輪講資料.Goodfellow本5.5節〜5.11節

Avatar for Takahiro Kawashima

Takahiro Kawashima

May 29, 2018
Tweet

More Decks by Takahiro Kawashima

Other Decks in Science

Transcript

  1. ໨࣍ 1. ࠷໬ਪఆ 2. ϕΠζ౷ܭ 3. ڭࢣ͋Γֶश 4. ڭࢣͳֶ͠श 5.

    ֬཰తޯ഑߱Լ๏ (SGD) 6. Deep Learning ΁ͷಈػ 2
  2. ڭࢣ͋Γֶश ֬཰తڭࢣ͋Γֶश ෮श: ҰൠઢܗϞσϧ    y = θT

    x + ϵ ϵ ∼ N(ϵ|0, σ2) ⇒ p(y|x; θ) = N(y; θT x, σ2) (5.80) ਖ਼ن෼෍ͷఆٛҬ͸ (−∞, ∞) ˠ {0, 1} ͷೋ஋෼ྨ໰୊ʹ͸࢖͑ͳ͍ 5
  3. ڭࢣ͋Γֶश ֬཰తڭࢣ͋Γֶश ϩδεςΟ οΫճؼ p(y = 1|x; θ) = 1

    1 + e−θT x = 1 1 + e−(θ0+θ1x1+θ2x2+··· ) ˠ {0, 1} ͷೋ஋൑ผʹ࢖͑Δ 7
  4. ڭࢣ͋Γֶश αϙʔτϕΫλʔϚγϯ (SVM) ฏ໘ͷํఔࣜ͸ ax + by + c =

    0 ͳͷͰ͜ΕΛҰൠԽͯ͠ɼ෼ྨ௒ฏ໘ͷํఔࣜ͸αϙʔτϕΫτ ϧͷू߹ x∗ Λ༻͍ͯ w0 + wT x∗ = 0 ͱॻ͚Δɽֶश͢Δͷ͸͜ͷ܎਺ w Ͱ͋Δ 2 ͭͷࢧ࣋௒ฏ໘͸ɼ෼ྨ௒ฏ໘Λ ±k ͚ͩͣΒͯ͠    w0 + wT x∗ = k w0 + wT x∗ = −k ⇒ |w0 + wT x∗| = k Ͱ͋Δ 10
  5. ڭࢣ͋Γֶश αϙʔτϕΫλʔϚγϯ (SVM) ௒ฏ໘ͷࣜ͸ఆ਺ഒͯ͠΋ಉ͡΋ͷΛࣔ͢ͷͰɼֶश݁Ռ͕Ұҙ ʹఆ·Βͳ͍ ˠҰҙੑΛ࣋ͭΑ͏ʹ੍໿Λ՝͢ ੍໿: |w0 + wT

    x∗| = 1 ॏΈϕΫτϧʹ͍ͭͯඪ४Խ͢Δͱ |w0 + wT x∗| ∥w∥ = 1 ∥w∥ ͜ΕΛ࠷େԽ͢ΔΑ͏ʹֶश͢Δ 11
  6. ڭࢣ͋Γֶश αϙʔτϕΫλʔϚγϯ (SVM) ྫ: ສ༗Ҿྗͷࣜ ಛ௃: ࣭ྔ m1, m2 ɼڑ཭

    r f(m1, m2, r) = G m1m2 r2 ͜ΕΛ֤ಛ௃ʹؔͯ͠ઢܗʹ͍ͨ͠ ˠର਺ΛͱΔ logf(m1, m2, r) = logG + logm1 + logm2 − logr2 ઢܗʹͳͬͨ 13
  7. ڭࢣͳֶ͠श k-means ๏ ΞϧΰϦζϜ 1. Ϋϥελத৺ͷॳظ஋ͱͯ͠ɼσʔλ఺͔Β k ݸͷηϯτϩ ΠυΛϥϯμϜʹબͿ (k

    ͸ط஌) 2. ֤αϯϓϧΛ࠷΋͍ۙηϯτϩΠυʹׂΓ౰ͯΔ 3. ֤ηϯτϩΠυΛࣗ਎ʹׂΓ౰ͯΒΕͨσʔλͷத৺ʹҠಈ ͢Δ 4. 2,3 Λ܁Γฦ͢ 17
  8. ֬཰తޯ഑߱Լ๏ (SGD) ίετؔ਺͸ (ςετ) σʔλ͝ͱͷଛࣦؔ਺ͷ࿨ʹ෼ղͰ͖Δ͜ ͱ͕ଟ͍ ઢܗճؼͰ͸ɼର਺໬౓ L(x, y, θ)

    Λ༻͍ͯ J(θ) = Ex,y∼ˆ pdata [L(x, y, θ)] = 1 m m ∑ i=1 L(x(i), y(i), θ) (5.96) L(x(i), y(i), θ) = −logp(y|x, θ) ͜ͷίετؔ਺ʹؔͯ͠ɼύϥϝʔλ θ ʹ͍ͭͯޯ഑๏Λద༻ 20
  9. ֬཰తޯ഑߱Լ๏ (SGD) ∇θJ(θ) = ∇θ [ 1 m m ∑

    i=1 L(x(i), y(i), θ) ] = 1 m m ∑ i=1 ∇θL(x(i), y(i), θ) (5.97) ͜ͷܭࢉྔ͸ O(m) Ͱɼσʔλ͕૿͑Δͱ͔ͳΓͭΒ͍ ˠ֬཰తޯ഑߱Լ๏ (SGD) 21
  10. ֬཰తޯ഑߱Լ๏ (SGD) SGD ͸ޯ഑Λظ଴஋ͰදݱͰ͖Δͱߟ͑ɼαϯϓϧͷখ͍͞αϒ ηοτ (ϛχόον) ͷޯ഑๏Ͱۙࣅతʹٻ·Δͱ͢Δ B = {x(1),

    . . . , x(m′)} ͷϛχόονΛҰ༷ϥϯμϜʹֶशσʔλ ηοτ͔Β΋ͬͯ͘Δ m′ ͸͍͍ͩͨ 100ʙ300 ͘Β͍Ͱɼm ͕ଟͯ͘΋ಉ༷ ޯ഑ͷਪఆྔ g ͸ g = 1 m′ ∇θ m′ ∑ i=1 L(x(i), y(i), θ) (5.98) ύϥϝʔλͷਪఆྔ͸ θ ← θ − ϵg 22
  11. Deep Learning ΁ͷಈػ ࣍ݩͷढ͍ ྫͱͯ͠ k ۙ๣๏ (k-Nearest Neighbour) ͱ͍͏ֶशΞϧΰϦζϜ

    Λߟ͑Δ k ۙ๣๏ ςετσʔλͷೖྗʹରͯ͠ɼಛ௃্ۭؒͰ΋ͬͱ΋͍ۙ k ݸ ͷֶशσʔλΛ୳͠ɼͦΕΒͷֶशσʔλͷଐ͢ΔΫϥεͷଟ ਺ܾͰςετσʔλʹׂΓৼΔΫϥεΛܾఆ͢Δ k = 3 ͷ৔߹ ೖྗ˔ͷϥϕϧ͸˙ 24
  12. References I [1] ਢࢁರࢤ, ϕΠζਪ࿦ʹΑΔػցֶशೖ໳. ߨஊࣾ, 2017. [2] খ໺ాਸ, αϙʔτϕΫλʔϚγϯ.

    ΦʔϜࣾ, 2007. [3] Sebastian Raschka ஶ, גࣜձࣾΫΠʔϓ༁, ୡਓσʔλαΠ ΤϯςΟετʹΑΔཧ࿦ͱ࣮ફ Python ػցֶशϓϩάϥϛ ϯά, ΠϯϓϨε, 2016.