Link
Embed
Share
Beginning
This slide
Copy link URL
Copy link URL
Copy iframe embed code
Copy iframe embed code
Copy javascript embed code
Copy javascript embed code
Share
Tweet
Share
Tweet
Slide 1
Slide 1 text
%FFQ-FBSOJOH#PPL %FFQ-FBSOJOH#PPLಡΈձ!य़ΤϦΞ Tangent Distance, Tangent Prop and Manifold Tangent Classifier
Slide 2
Slide 2 text
࣍ݩͷढ͍ • ࣍ݩͷढ͍: ߴ࣍ݩʹ͏༷ʑͳࠔ • ػցֶशʹ͓͍ͯɼ࣍ݩ͕ߴ͘ͳΔͱͭΒ͍ 2 https://twitter.com/neubig/status/590026414180544514 http://windfall.hatenablog.com/entry/2015/07/02/084623
Slide 3
Slide 3 text
࣍ݩͷढ͍ (1) • ۭؒͷ࣍ݩ͕૿͑Δ => ܭࢉྔ͕૿͑Δ 3 y(x, w) = w0 + N X i=1 wixi + N X i=1 N X j=1 wi,jxixj + N X i=1 N X j=1 N X k=1 wi,j,kxixjxk y(x, w) = w0 + N X i=1 wixi + N X i=1 N X j=1 wi,jxixj y(x, w) = w0 + N X i=1 wixi
Slide 4
Slide 4 text
࣍ݩͷढ͍ (2) • ۭؒͷ࣍ݩ͕૿͑Δ => ඞཁͳֶशσʔλͷྔ͕૿͑Δ 4
Slide 5
Slide 5 text
Manifold hypothesis • ࣍ݩͷढ͍ʹཱ͔ͪ͏ => ଟ༷ମԾઆ • σʔλۭؒߴ࣍ݩͰ͋ͬͯɼ ΧςΰϦใ࣍ݩͷଟ༷ମͷ্Ͱදݱ͞ΕΔ • => ߴ࣍ݩͷσʔλΛߴ࣍ݩͷ··ѻΘͳͯ͘ ྨղ͚Δ? • MNISTσʔλ • 20x20 (400࣍ݩͷϕΫτϧ) • t-SNEͰ2࣍ݩʹม => ࣝผͰ͖ͦ͏ 5 http://colah.github.io/posts/2014-10-Visualizing-MNIST
Slide 6
Slide 6 text
Tangent distance algorithm • ϢʔΫϦουڑΛߟ͑Δʁ • ϢʔΫϦουڑ͍͚ۙͲΧςΰϦҧ͏σʔλ • σʔλ͕ଐ͢Δଟ༷ମಉ࢜ͷڑΛߟ͑Δʂ • ಉ͡ଟ༷ମ্ʹଘࡏ͢Δσʔλ -> ಉ͡ΧςΰϦ • ଟ༷ମಉ࢜ͷڑܭࢉ͕େม (?) • ଟ༷ମΛσʔλͷฏ໘Ͱۙࣅ • ଟ༷ମ͋Β͔͡Ί༻ҙ͢Δ ? 6 http://colah.github.io/posts/2014-10-Visualizing-MNIST Simard, Patrice, Yann LeCun, and John S. Denker. "Efficient pattern recognition using a new transformation distance." Advances in neural information processing systems. 1993.
Slide 7
Slide 7 text
Tangent Prop • ଟ༷ମ্ͷมԽʹର͠ωοτϫʔΫͷग़ྗෆม ʹͳͬͯ΄͍͠ؾ࣋ͪΛਖ਼ଇԽ߲ͱͯ͠ೖΕΔ • v^i ฏ໘ͷiຊͷϕΫτϧ • ଟ༷ମ͋Β͔͡Ί༻ҙ͢Δ ? • ޯͱϕΫτϧͷੵ • ಉ͡ํ => େ͖͘ͳΔ ަ͢Δ => খ͘͞ͳΔ • ಉ͡ํͷมԽʹݫ͘͠ͳΔ 7 ⌦(f) = X i (rf(x)T v(i) 2 Simard, Patrice, et al. "Tangent prop-a formalism for specifying selected invariances in an adaptive network." Advances in neural information processing systems. 1992.
Slide 8
Slide 8 text
Manifold Tangent Classifier • CAE (contractive autoencoder) Λͬͨख๏ʁ • Tangent propϕΫτϧΛܭࢉ͢ΔͨΊʹ ؔΛࣗͰࢦఆ͢Δඞཁ͕͋Δ • Manifold Tangent Classifierʹ͓͍ͯɼ ΦʔτΤϯίʔμʔ͕σʔλʹԠͯ͡ઢํΛ ਪఆͯ͘͠ΕΔͨΊɼؔΛࢦఆ͢Δඞཁ͕ͳ͍ 8 Rifai, Salah, et al. "The manifold tangent classifier.” Advances in Neural Information Processing Systems. 2011.