%FFQ-FBSOJOH#PPL %FFQ-FBSOJOH#PPLಡΈձ!य़೔ΤϦΞ Tangent Distance, Tangent Prop and Manifold Tangent Classifier

࣍ݩͷढ͍ • ࣍ݩͷढ͍: ߴ࣍ݩʹ൐͏༷ʑͳࠔ೉ • ػցֶशʹ͓͍ͯɼ࣍ݩ͕ߴ͘ͳΔͱͭΒ͍ 2 https://twitter.com/neubig/status/590026414180544514 http://windfall.hatenablog.com/entry/2015/07/02/084623

࣍ݩͷढ͍ (1) • ۭؒͷ࣍ݩ͕૿͑Δ => ܭࢉྔ͕૿͑Δ 3 y(x, w) = w0 + N X i=1 wixi + N X i=1 N X j=1 wi,jxixj + N X i=1 N X j=1 N X k=1 wi,j,kxixjxk y(x, w) = w0 + N X i=1 wixi + N X i=1 N X j=1 wi,jxixj y(x, w) = w0 + N X i=1 wixi

࣍ݩͷढ͍ (2) • ۭؒͷ࣍ݩ͕૿͑Δ => ඞཁͳֶशσʔλͷྔ͕૿͑Δ 4

Manifold hypothesis • ࣍ݩͷढ͍ʹཱͪ޲͔͏ => ଟ༷ମԾઆ • σʔλۭؒ͸ߴ࣍ݩͰ͋ͬͯ΋ɼ
 ΧςΰϦ৘ใ͸௿࣍ݩͷଟ༷ମͷ্Ͱදݱ͞ΕΔ • => ߴ࣍ݩͷσʔλΛߴ࣍ݩͷ··ѻΘͳͯ͘΋
 ෼ྨ໰୊͸ղ͚Δ? • MNISTσʔλ • 20x20 (400࣍ݩͷϕΫτϧ) • t-SNEͰ2࣍ݩʹม׵ => ࣝผͰ͖ͦ͏ 5 http://colah.github.io/posts/2014-10-Visualizing-MNIST

Tangent distance algorithm • ϢʔΫϦουڑ཭Λߟ͑Δʁ • ϢʔΫϦουڑ཭͸͍͚ۙͲΧςΰϦ͸ҧ͏σʔλ • σʔλ͕ଐ͢Δଟ༷ମಉ࢜ͷڑ཭Λߟ͑Δʂ • ಉ͡ଟ༷ମ্ʹଘࡏ͢Δσʔλ -> ಉ͡ΧςΰϦ • ଟ༷ମಉ࢜ͷڑ཭͸ܭࢉ͕େม (?) • ଟ༷ମΛσʔλ఺ͷ઀ฏ໘Ͱۙࣅ • ଟ༷ମ͸͋Β͔͡Ί༻ҙ͢Δ ? 6 http://colah.github.io/posts/2014-10-Visualizing-MNIST Simard, Patrice, Yann LeCun, and John S. Denker.
 "Efficient pattern recognition using a new transformation distance."
 Advances in neural information processing systems. 1993.

Tangent Prop • ଟ༷ମ্ͷมԽʹର͠ωοτϫʔΫͷग़ྗ͸ෆม
 ʹͳͬͯ΄͍͠ؾ࣋ͪΛਖ਼ଇԽ߲ͱͯ͠ೖΕΔ • v^i ͸઀ฏ໘ͷiຊ໨ͷϕΫτϧ • ଟ༷ମ͸͋Β͔͡Ί༻ҙ͢Δ ? • ޯ഑ͱ઀ϕΫτϧͷ಺ੵ • ಉ͡ํ޲ => େ͖͘ͳΔ ௚ަ͢Δ => খ͘͞ͳΔ • ಉ͡ํ޲ͷมԽʹݫ͘͠ͳΔ 7 ⌦(f) = X i (rf(x)T v(i) 2 Simard, Patrice, et al. "Tangent prop-a formalism for specifying selected invariances in an adaptive network." Advances in neural information processing systems. 1992.

Manifold Tangent Classifier • CAE (contractive autoencoder) Λ࢖ͬͨख๏ʁ • Tangent prop͸઀ϕΫτϧΛܭࢉ͢ΔͨΊʹ
 ؔ਺Λࣗ෼Ͱࢦఆ͢Δඞཁ͕͋Δ • Manifold Tangent Classifierʹ͓͍ͯ͸ɼ
 ΦʔτΤϯίʔμʔ͕σʔλʹԠͯ͡઀ઢํ޲Λ
 ਪఆͯ͘͠ΕΔͨΊɼؔ਺Λࢦఆ͢Δඞཁ͕ͳ͍ 8 Rifai, Salah, et al. "The manifold tangent classifier.” Advances in Neural Information Processing Systems. 2011.