Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Deep Learning輪読 7.14

himkt
October 25, 2017
99

Deep Learning輪読 7.14

himkt

October 25, 2017
Tweet

Transcript

  1. ࣍ݩͷढ͍ (1) • ۭؒͷ࣍ݩ͕૿͑Δ => ܭࢉྔ͕૿͑Δ 3 y(x, w) =

    w0 + N X i=1 wixi + N X i=1 N X j=1 wi,jxixj + N X i=1 N X j=1 N X k=1 wi,j,kxixjxk y(x, w) = w0 + N X i=1 wixi + N X i=1 N X j=1 wi,jxixj y(x, w) = w0 + N X i=1 wixi
  2. Manifold hypothesis • ࣍ݩͷढ͍ʹཱͪ޲͔͏ => ଟ༷ମԾઆ • σʔλۭؒ͸ߴ࣍ݩͰ͋ͬͯ΋ɼ
 ΧςΰϦ৘ใ͸௿࣍ݩͷଟ༷ମͷ্Ͱදݱ͞ΕΔ •

    => ߴ࣍ݩͷσʔλΛߴ࣍ݩͷ··ѻΘͳͯ͘΋
 ෼ྨ໰୊͸ղ͚Δ? • MNISTσʔλ • 20x20 (400࣍ݩͷϕΫτϧ) • t-SNEͰ2࣍ݩʹม׵ => ࣝผͰ͖ͦ͏ 5 http://colah.github.io/posts/2014-10-Visualizing-MNIST
  3. Tangent distance algorithm • ϢʔΫϦουڑ཭Λߟ͑Δʁ • ϢʔΫϦουڑ཭͸͍͚ۙͲΧςΰϦ͸ҧ͏σʔλ • σʔλ͕ଐ͢Δଟ༷ମಉ࢜ͷڑ཭Λߟ͑Δʂ •

    ಉ͡ଟ༷ମ্ʹଘࡏ͢Δσʔλ -> ಉ͡ΧςΰϦ • ଟ༷ମಉ࢜ͷڑ཭͸ܭࢉ͕େม (?) • ଟ༷ମΛσʔλ఺ͷ઀ฏ໘Ͱۙࣅ • ଟ༷ମ͸͋Β͔͡Ί༻ҙ͢Δ ? 6 http://colah.github.io/posts/2014-10-Visualizing-MNIST Simard, Patrice, Yann LeCun, and John S. Denker.
 "Efficient pattern recognition using a new transformation distance."
 Advances in neural information processing systems. 1993.
  4. Tangent Prop • ଟ༷ମ্ͷมԽʹର͠ωοτϫʔΫͷग़ྗ͸ෆม
 ʹͳͬͯ΄͍͠ؾ࣋ͪΛਖ਼ଇԽ߲ͱͯ͠ೖΕΔ • v^i ͸઀ฏ໘ͷiຊ໨ͷϕΫτϧ • ଟ༷ମ͸͋Β͔͡Ί༻ҙ͢Δ

    ? • ޯ഑ͱ઀ϕΫτϧͷ಺ੵ • ಉ͡ํ޲ => େ͖͘ͳΔ ௚ަ͢Δ => খ͘͞ͳΔ • ಉ͡ํ޲ͷมԽʹݫ͘͠ͳΔ 7 ⌦(f) = X i (rf(x)T v(i) 2 Simard, Patrice, et al. "Tangent prop-a formalism for specifying selected invariances in an adaptive network." Advances in neural information processing systems. 1992.
  5. Manifold Tangent Classifier • CAE (contractive autoencoder) Λ࢖ͬͨख๏ʁ • Tangent

    prop͸઀ϕΫτϧΛܭࢉ͢ΔͨΊʹ
 ؔ਺Λࣗ෼Ͱࢦఆ͢Δඞཁ͕͋Δ • Manifold Tangent Classifierʹ͓͍ͯ͸ɼ
 ΦʔτΤϯίʔμʔ͕σʔλʹԠͯ͡઀ઢํ޲Λ
 ਪఆͯ͘͠ΕΔͨΊɼؔ਺Λࢦఆ͢Δඞཁ͕ͳ͍ 8 Rifai, Salah, et al. "The manifold tangent classifier.” Advances in Neural Information Processing Systems. 2011.