Share
Embed
CompML: 次元削減によるDNN特徴量空間の可視化
CompMLでの発表資料です。次元削減の概要とDNNが生成する特徴量空間の可視化について説明しています。
ryuji0123
April 02, 2021
Transcript
26 slides
-
CompML ࣍ݩݮͷઃఆ ΦϒδΣΫτ ʹߴ࣍ݩ࠲ඪ ͕༩͑ΒΕͨσʔλू߹ ͔Β ΦϒδΣΫτ ʹରԠ͢Δ࣍ݩ࠲ඪ ΛٻΊΔɻͨͩ͠ɺ ྲྀΕओʹҎԼ̎ͭ
1. ࣍ݩ࠲ඪΛ༩͑ͨσʔλू߹ Λܭࢉ 2. ͱ͍͏ؔΛ༩͑ΔࣹӨΛٻΊɺ࣍ݩ࠲ඪΛܭࢉ oi xi D = {(oi , xi ) |(1 ≤ i ≤ N)} oi yi xi ∈ RD, yi ∈ Rd, D > > d D′ = {(oi , yi ) |(1 ≤ i ≤ n)} yi = f(xi ) 5 -
CompML ࣍ݩݮͷྫ: Locally Linear Embedding [Sam et al., 2000[1]] ΦϒδΣΫτ
ͷ࠲ඪ ΛۙͷΦϒδΣΫτ ͷ࠲ඪ ͷઢܗͰۙࣅ Λ༻͍ͯ Λֶशͨ͠ޙʹɺ Λ༻͍ͯ Λֶश ߴ࣍ݩ࠲ඪ: ࣍ݩ࠲ඪ: oi xi oij xij xi wij wij yi ε(W ) = ∑ i |xi − ∑ j wij xij |2 ϕ(y) = ∑ i |yi − ∑ j wij yij |2 6 -
CompML ࣍ݩݮͷྫ: SNE [Hinton et al., 2002[2]] ΦϒδΣΫτ ͱ ͷۙ
(͖݅) ֬Λ࠲ඪʹج͖ͮܭࢉ͠ɺ ߴ࣍ݩۭؒͰͷ֬ͱ࣍ݩۭؒͰͷ֬ͱͷڑΛখ͘͢͞Δ ߴ࣍ݩ࠲ඪ: ࣍ݩ࠲ඪ: oi oj pj|i = exp( − ||xi − xj ||2 /2σ2 i ) ∑ k≠i exp( − ||xi − xk ||2 /2σ2 i ) qj|i = exp( − ||yi − yj ||2 /2σ2 i ) ∑ k≠i exp( − ||yi − yk ||2 /2σ2 i ) 7 MNISTͷՄࢹԽ -
CompML ࣍ݩݮͷྫ: SNE [Hinton et al., 2002[2]] ΦϒδΣΫτ ͱ ͷۙ
(͖݅) ֬Λ࠲ඪʹج͖ͮܭࢉ͠ɺ ߴ࣍ݩۭؒͰͷ֬ͱ࣍ݩۭؒͰͷ֬ͱͷڑΛখ͘͢͞Δ ڑ: ߋ৽ࣜ: oi oj C = ∑ i KL(Pi ||Qi ) = ∑ i,j pj|i log pj|i qj|i δC δyi = 2∑ j (pj|i − qj|i + pi|j − qi|j )(yi − yj ) 8 MNISTͷՄࢹԽ -
CompML ࣍ݩݮͷྫ: SNE [Hinton et al., 2002[2]] . 1 ߋ৽͕ࣜෳࡶͰܭࢉ͕͍
δC δyi = 2∑ j (pj|i − qj|i + pi|j − qi|j )(yi − yj ) 9 -
CompML . 2 Crowding Problem • ࣍ݩ + 1 ·Ͱ͔͠ΦϒδΣΫτΛۭؒʹִؒʹஔͰ͖ͳ͍
• ࣍ݩۭؒͰখ͞ͳڑΛਖ਼֬ʹදݱ͠Α͏ͱ͢Δͱɺߴ࣍ݩۭؒͰҰఆҎ্ͷڑ͕ ͋ͬͨΦϒδΣΫτಉ࢜࣍ݩۭؒͰඇৗʹԕ͘ʹΕΔ • ԕํʹ͋ΔଟͷΦϒδΣΫτ͕ۭؒͷத৺ͰͷΫϥελܗΛ્͢Δ 10 ࣍ݩݮͷྫ: SNE [Hinton et al., 2002[2]] -
CompML ࣍ݩݮͷྫ: t-SNE [Laurens et al., 2008[3]] ख๏. 1 Symmetric
SNE ͖݅ͷΘΓʹಉ࣌Λ༻͍ͯଛࣦؔΛରԽͤ͞ɺෳࡶ͞Λղফ ߴ࣍ݩ࠲ඪ: pj|i = exp( − ||xi − xj ||2 /2σ2 i ) ∑ k≠i exp( − ||xi − xk ||2 /2σ2 i ) pij = pi|j + pj|i 2N 11 MNISTͷՄࢹԽ -
CompML ࣍ݩݮͷྫ: t-SNE [Laurens et al., 2008[3]] ख๏. 1 Symmetric
SNE ͖݅ͷΘΓʹಉ࣌Λ༻͍ͯଛࣦؔΛରԽͤ͞ɺෳࡶ͞Λղফ ڑ: ߋ৽ࣜ: C = ∑ i KL(Pi ||Qi ) = ∑ i,j pij log pij qij δC δyi = 4∑ j (pij − qij )(yi − yj ) 12 MNISTͷՄࢹԽ -
CompML ࣍ݩݮͷྫ: t-SNE [Laurens et al., 2008[3]] ख๏. 2 t-Distributed
SNE ࣍ݩۭؒͰͷͷܭࢉʹɺΨεΑΓ͕ߴ͍ t Λ༻͍ͯ Crowding ProblemΛܰݮ ࣍ݩ࠲ඪ: qij = (1 + ||yi − yj ||2 )−1 ∑ k≠l (1 + ||yk − yl ||2 )−1 13 MNISTͷՄࢹԽ -
CompML ࣍ݩݮͷྫ: t-SNE [Laurens et al., 2008[3]] ख๏. 2 t-Distributed
SNE ࣍ݩۭؒͰͷͷܭࢉʹɺΨεΑΓ͕ߴ͍ t Λ༻͍ͯ Crowding ProblemΛܰݮ ڑ: ߋ৽ࣜ: C = ∑ i KL(Pi ||Qi ) = ∑ i,j pij log pij qij δC δyi = 4∑ j (pij − qij )(yi − yj )(1 + ||yi − yj ||2 )−1 14 MNISTͷՄࢹԽ -
CompML ࣍ݩݮ͕ѻ͏σʔλ ࠓճҎԼೋͭΛհ 1. (લड़) ߴ࣍ݩͷ࠲ඪΦϒδΣΫτಉ࢜ͷྨࣅ͕༩͑ΒΕ͍ͯΔσʔλ 2. (ࠓ͔Βհ) 1ͷ࠲ඪྨࣅ͕࣌ࠁຖʹ༩͑ΒΕ͍ͯΔσʔλ 1
͋Δ࣌ࠁͰͷεφοϓγϣοτͰ 2 ࣌ܥྻσʔλͷΑ͏ͳͷ 15 -
CompML ࣌ܥྻΛѻ͏࣍ݩݮ Ϟνϕʔγϣϯ • ຖ࣌ࠁͷσʔλʹผʑʹ࣍ݩݮΛద༻͢Δͱ ࣍ݩ࠲ඪʹҰ؏ੑ͕ͳ͍ • ߴ࣍ݩͷσʔλ͕େ͖͘มԽͯ͠ͳͯ͘ ࣍ݩ࠲ඪͷ͕มԽ͢Δ 16
Ϋϥεؒڑ͕େ͖͘ͳΔ ଟ࣍ݩΨεͷՄࢹԽ -
CompML Dynamic t-SNE [Paulo et al., 2016[4]] ख๏ ࣍ݩۭؒͰ࣌ࠁؒͷؒڑʹਖ਼ଇԽ߲ΛՃ͑Δ C(t)
= N ∑ i=1 KL(Pi (t)||Qi (t)) = N ∑ i,j=1 pij (t)log pij (t) qij (t) C = T ∑ t=1 C(t) + λ 2N N−1 ∑ i=1 T−1 ∑ t=1 ||Qi (t) − Qi (t + 1)||2 17 Ϋϥεؒڑ͕େ͖͘ͳΔ ଟ࣍ݩΨεͷՄࢹԽ -
CompML Dynamic t-SNE [Paulo et al., 2016[4]] DNNಛྔۭؒͷՄࢹԽ SVHN Λ
CNN Ͱֶशͤ͞ɺCNN ͷ࠷ޙͷӅΕ ͕ܗ͢Δಛྔۭؒ ( ) ΛՄࢹԽ D = 512 18 -
CompML Dynamic t-SNEͷॴײ numpyͷ࣮ͩͱɺ࣮ߦ͕͍ • ఔͰॲཧ͕ऴΘΒͳ͍ͱ࣮ݧαΠΫϧʹΈࠐΊͳ͍ • ͦͦ t-SNE ্͕͍ʹɺ࣌ܥྻσʔλͷ͕ଟ͍ͷͰ͔ͳΓ͍
• Dynamic t-SNE ͷ CUDA Խ t-SNE ΑΓߴͳख๏ͷ࣌ܥྻσʔλͷ֦ு͕ඞཁ 19 -
CompML DNNಛྔۭؒʹ͍ͭͯ ݱঢ়ͷઃఆ ՄࢹԽۭͯؒ͠ͷܗ্͕ख͍͍ͬͯ͘ΔͷΛ֬ೝ (ݸਓతʹΓͳ͍) • t-SNE ʹΑͬͯυϝΠϯؒͷΫϥελߏΛՄࢹԽ • Dynamic
t-SNEʹΑͬͯผੳʹదۭ͕ͨؒ͠ܗ͞ΕΔաఔΛՄࢹԽ 21 -
CompML ՄࢹԽͷϞνϕʔγϣϯ Ϟσϧ։ൃ͕దͳํʹ͔͍ͬͯΔ͔ௐΔख๏Λ࡞Γ͍ͨ • learning curveͷεΧϥʔͷมԽΛݟΔҎ֎ʹɺϞσϧͷσόοάํ๏͕ཉ͍͠ (ֶश͕ࣦഊ͢ΔݪҼΛՄࢹԽͰ͍ࣔࠦͨ͠) • ӅΕͷಛྔۭؒͰͷσʔλͷՄࢹԽ t-SNE
͕༻͍ΒΕ͍͕͢ɺผͷख๏ ʹΑͬͯΫϥελߏҎ֎ʹநग़Ͱ͖Δใ͕͋Δ͔͠Εͳ͍ 22 -
CompML ༧ଌ͞ΕΔ 1. ࣮ߦ࣌ؒ: શ࣌ࠁͷใΛࢀরͯ͠࠲ඪΛܭࢉ͢Δͱ͘ɺ࣮ݧαΠΫϧʹΈࠐΊͳ͍ 2. σʔλ: DNNͰ༻͍Δσʔλଟ͘ɺదͳ෦ू߹Λબͯ͠ՄࢹԽ͢Δඞཁ༗Γ C(t)
= N ∑ i=1 KL(Pi (t)||Qi (t)) = N ∑ i,j=1 pij (t)log pij (t) qij (t) C = T ∑ t=1 C(t) + λ 2N N−1 ∑ i=1 T−1 ∑ t=1 ||Qi (t) − Qi (t + 1)||2 23 -
CompML ࣮ݧ (ਐߦத) ҎԼͷ݅Ͱ࣮ݧ༧ఆ • ֶश্͕ख͍͘͘߹ • ֶश͕ࣦഊ or ेͳਫ਼͕ग़ͳ͍߹
• mixupͷಛྔۭؒʹհࡏ͢Δख๏Λ༻͍ͨ߹ • VAEͷۭؒʹಛఆͷߏΛԾఆͰ͖Δ߹ • Domain Adaptationͷෳͷσʔλ͕ଘࡏ͢Δ߹ 24 -
CompML ࢀߟจݙ • [1] Roweis, Sam T and Saul, Lawrence
K. Nonlinear dimensionality reduction by locally linear embedding. Science, 290(5500):2323–2326, 2000. • [2] G.E. Hinton and S.T. Roweis. Stochastic Neighbor Embedding. In Advances in Neural Information Processing Systems, volume 15, pages 833–840, Cambridge, MA, USA, 2002. The MIT Press. • [3] Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-SNE. Journal of Machine Learning Research, Vol. 9, pp. 2579–2605, 2008. • [4] Paulo E. Rauber, Alexandre X. Falcão, and Alexandru C. Telea. Visualizing time- dependent data using dynamic t-SNE. Proc. EuroVis Short Papers, 2(5), 2016. 26