Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Sign up for free
Menu
Search
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Pricing
Search
Sign in
Sign up for free
【ICML読み会】Unsupervised Deep Embedding for Cluste...
Search
Sponsored
·
Your Podcast. Everywhere. Effortlessly.
Share. Educate. Inspire. Entertain. You do you. We'll handle the rest.
→
Hayato Maki
July 16, 2016
Technology
1.4k
0
Share
Embed
Copy iframe code
Copy JS code
Copy link
Start on current slide
【ICML読み会】Unsupervised Deep Embedding for Clustering Analysis
Hayato Maki
July 16, 2016
Transcript
22 slides
ICML2016จհ Unsupervised Deep Embedding for Clustering Analysis Ross Girshick Jungian
Xie Ali Farhadi University of Washington Facebook AI Research University of Washington ൃදऀ ਅ༐ਓ ಸྑઌՊֶٕज़େֶӃେֶ ใՊֶݚڀՊ ത࢜ޙظ՝ఔ ೳίϛϡχέʔγϣϯݚڀࣨ 2016/07/16 @NAIST
3ߦͰཁ • ରɿݹయతͳΫϥελϦϯά • ख๏ɿਂֶशΛར༻ͨ࣍͠ݩݮύ ϥϝλͱΫϥελϦϯάͷಉ࣌ ࠷దԽ • ݁Ռɿैདྷख๏ΑΓߴ͍ਫ਼ɼ͍ ܭࢉ࣌ؒΛ࣮ݱ
ΫϥελϦϯάͷؔ࿈ݚڀ • k-means ٴͼ ࠞ߹ਖ਼نϞσϧ(GMM) • ೖྗͷ࣍ݩ͕ߴ͍ͱࣦഊ͍͢͠ • ࣍ݩݮͱΫϥελϦϯάΛಉ࣌ʹߦ͏ख๏ •
࣍ݩۭؒʹࣸ૾ɼࣸ૾ͨ͠ઌͰΫϥελϦϯά • ैདྷख๏ઢܗࣸ૾ͷΈ • εϖΫτϥϧɾΫϥελϦϯά • σʔλͷάϥϑߏΛར༻͢Δख๏ • k-meansΑΓྑ͍݁ՌʹͳΔ͜ͱ͕ଟ͍ • ܭࢉྔ͕αϯϓϧͷ̎·ͨ̐ʹൺྫ
ه߸ • σʔλɿ • σʔλɿ • Ϋϥελͷʢࣄલʹܾఆʣɿ • ࣸ૾ɿ •
ɹ ͷ࣍ݩ <<< ͷ࣍ݩ • ࣸ૾ͷύϥϝλ ΛDNNͰֶश • ࣸ૾ઌͷσʔλɿ • ηϯτϩΠυʢΫϥελΛද͢Δʣɿ n { xi 2 X }n i=1 k zi = f✓( xi) ✓ {zi 2 Z}n i=1 {µj 2 Z}k i=1 zi = f✓( xi) zi = f✓( xi)
ఏҊ๏ͷྲྀΕ ॳظԽ ࣍ݩݮ ΫϥελׂΓͯ KL divergenceܭࢉ ύϥϝλߋ৽
࣍ݩݮ • ਂֶशΛར༻ͨ͠ඇઢܗͳ࣍ݩԽࣸ૾ f✓ : X ! Z { xi
2 X }n i=1 zi = • ڭࢣͳֶ͠शͷͨΊɼަ ࠩݕূ๏ʹΑΔϋΠύʔύ ϥϝλͷௐͰ͖ͳ͍ • ͦͷͨΊɼΑ͘ΘΕΔ ωοτϫʔΫߏΛ༻ • ֤ͷ࣍ݩ (input)-500-500-2000-10 • શ݁߹ [van der Maaten, 09]
ΫϥελׂΓͯ Soft Asignment • ࣸ૾͞Εͨσʔλ ͱηϯτϩΠυ ͷྨࣅ ई (soft assignment)
ɼ ͕̹൪ͷΫϥελʹೖΔ֬ͱͯ͠ ղऍͰ͖Δɽ qij = 1 + kzi µj0 k2/↵ (↵+1)/2 P j0 (1 + kzi µj0 k2/↵) (↵+1)/2 ↵ = 1 {zi 2 Z}n i=1 µj [van der Maaten & Hinton, 08] qij = 1 + kzi µj0 k2/↵ (↵+1)/2 P j0 (1 + kzi µj0 k2/↵) (↵+1)/2 {zi 2 Z}n i=1 • ڭࢣͳֶ͠शʹ͓͍ͯɼަࠩݕূ๏͑ͳ͍ ͨΊɼ ʹݻఆɽ
KLμΠόʔδΣϯεʹΑΔDNNֶश • ఆతͳׂΓͯ • ඪʢཧతͳΫϥελϦϯάΛߦ͏ͱߟ͑Β ΕΔʣ • PͱQͷKLμΠόʔδΣϯεΛ࠷খԽ͢ΔΑ͏ʹDNN Λֶश •
Pͷઃఆ͕ຊख๏ͷΩϞ
ύϥϝλߋ৽ • DNNͷύϥϝλθ ͱ ηϯτϩΠυ μj Λߋ৽ • SGDͰߋ৽ (θόοΫϓϩύήʔγϣϯ)
ॳظԽ • DNNͷॳظԽɿ Stacked Auto Encoder Λར༻ • ηϯτϩΠυͷॳظԽɿॳظԽDNNΛར༻ͯ࣍͠ݩ ݮ͠ɼࣸ૾ઌͰk-means
࣮ݧ • σʔληοτ • ൺֱख๏ • k-means • LDGMI (εϖΫτϥϧɾΫϥελϦϯά)
• SEC (εϖΫτϥϧɾΫϥελϦϯά) • Without back propagation
࣮ • Stacked Auto EncoderͷॳظԽ • ฏۉ0ɼඪ४ภࠩ0.01ͷਖ਼نΛͬͨ ཚͰॏΈΛॳظԽ • ֤͝ͱʹ50000ճ෮ʢ20%Dropoutʣ
• Auto EncoderશମͰ100000ճ෮ͯ͠ fine tuning (Dropoutແ͠) • ϛχόοναΠζ=256 • ֶश=0.1 ←20000෮ຖʹ1/10
࣮ • ηϯτϩΠυͷॳظԽ • ҟͳΔॳظͰ20ճ࣮ߦͯ͠ϕετͳ ͷΛબ • KLμΠόʔδΣϯεͷ࠷খԽ • ֶश=0.01
(ݻఆ) • ऩଋఆ • ΫϥελͷׂΓ͕ͯมԽ͢Δσʔλ͕ 0.1%ҎԼʹͳΔ·Ͱ
ධՁج४ • Unsupervised Clustering Accuracy (ACC) • pi ɿਅͷϥϕϧ •
qi ɿΞϧΰϦζϜ͕ग़ྗͨ͠ϥϕϧ • map()ɿϥϕϧ͔ΒΫϥελͷ࠷దͳϚοϐϯά
݁Ռ • ఏҊ๏͕ϕετͷੑೳ • REUTERSʹ͍ͭͯ • ఏҊ๏ͷֶश࣌ؒ30ఔ • LDMGIͱSECϲ݄Ҏ্ͷܭࢉ࣌ؒͱςϥ ୯ҐͷϝϞϦ͕ඞཁ
ϋΠύʔύϥϝλʹର͢Δؤ݈ੑ • ҟͳΔ9ͭͷϋΠύʔύϥϝλʢΞχʔϦϯάʣ ͰੑೳΛൺֱ • ఏҊ๏ϋΠύʔύϥϝλͷมಈʹରͯ͠ؤ݈ɼ ͔ͭσʔληοτʹඇґଘ • ڭࢣͳֶ͠शʹ͓͍ͯॏཁͳੑ࣭
ඪPͷੑ࣭ • qij ͕େ͖΄Ͳ(֬৴͕େ͖͍΄Ͳ)ɼޯ͕ େ͖͘ͳΔ ˠP·͍͠ੑ࣭Λ͍࣋ͬͯΔ
࣍ݩݮͷ෮࠷దԽʹΑΔޮՌ • t-SNEΛར༻ͨ͠ՄࢹԽ [van der Maaten & Hinton, 08] •
ߋ৽͕ਐΉ΄ͲΫϥελʔͷ͕໌֬ʹ
Auto EncoderʹΑΔಛநग़ͷޮՌ • Auto EncoderͰಛநग़ˠ֤ΞϧΰϦζϜͰॲཧ • Auto EncoderʹΑΔߩݙ͕େ͖͍
ෆۉҰͳσʔληοτʹର͢Δؤ݈ੑ • αϯϓϧ͕࠷খͷΫϥεͷαϯϓϧΛɼ ࠷େͷαϯϓϧͷΫϥεͷrmin ഒʹઃఆɼ ͦͷଞͷΫϥε0.1ͣͭ૿͍ͯ͘͠ • ఏҊ๏αϯϓϧͷෆۉҰੑʹରͯ͠ؤ݈
݁ • ఏҊ๏ɼਂֶशΛར༻ͨ࣍͠ݩݮʹΑΓΫϥ ελϦϯάͷੑೳΛ্ • ࣍ݩݮͷύϥϝλͱΫϥελϦϯάͷ݁ՌΛಉ ࣌ʹ࠷దԽ • ΫϥελϦϯάఆతͳग़ྗͱඪͱͷKL μΠόʔδΣϯεΛଛࣦؔͱͯ͠όοΫϓϩύ
ήʔγϣϯ • ैདྷ๏ΑΓߴਫ਼͔ͭߴʢܭࢉ࣌ؒαϯϓϧ ʹରͯ͠ઢܗʹൺྫʣɼσʔληοτඇґଘɼϋΠύʔ ύϥϝλඇґଘɼαϯϓϧͷෆۉҰੑʹରͯ͠ؤ݈
Ҏ্
More Decks by Hayato Maki
See All by Hayato Maki
Billion-scale Embedding for E-commerce Recommendation in Alibaba
hamaki
0
140
Today was a Good Day: The Daily Life of Software Developers
hamaki
0
140
論文紹介:Relaxed Softmax for PU Learning
hamaki
3
1.1k
MIRU 2019 Lunch on Seminar
hamaki
1
300
コーディネート整合性を考慮したカテゴリ間推薦
hamaki
0
1.2k
Regularization_The Element of Statical Learning
hamaki
0
230
Neural Activity During Sentence Processing as Reflected in Theta, Alpha, Beta, and Gamma Oscillations
hamaki
0
280
Other Decks in Technology
See All in Technology
SSFの現在地 ― CAEP/RISCからAI Identityへ
oidfj
0
470
_マネジメントする人される人ナイト_2026__良いチームを作っていきたい_お気持ち_.pdf
xkxaxkx
0
790
市販IoT機器や自作IoT機器とHomeAssistantを駆使して家の電気代をできる限り見える化する方法の考案
harutiro
0
380
AIネイティブ組織の作り方。1人のPOとAIから始める、自社での実験
ama_ch
2
1.2k
会社説明資料_最新版
demaecan
0
32k
Context parameters で作る!はじめての型クラス
kworkdev
PRO
1
150
SORACOM × AWS IoT Core でSREする
butadora
0
110
お風呂場をサーバルームにしよう
yusuke427
0
340
異夢同船〜異なる夢でも同じ船でなんとかする〜
fortegp05
1
520
高品質と高スピードを支える品質のシフトレフトのアプローチ
goyoki
4
1.6k
Claude Codeで新規事業を伸ばせるか? Zennの試行錯誤の記録
wadayusuke
2
240
AIで実装が加速したのに、 なぜ私は仕事の大半で人と話し続けていたのか?
say_no_w
0
190
Featured
See All Featured
Optimizing for Happiness
mojombo
378
72k
Information Architects: The Missing Link in Design Systems
soysaucechin
1
1.3k
Leveraging Curiosity to Care for An Aging Population
cassininazir
1
570
16th Malabo Montpellier Forum Presentation
akademiya2063
PRO
0
470
4 Signs Your Business is Dying
shpigford
187
23k
brightonSEO & MeasureFest 2025 - Winning Strategies for Black Friday CRO & PPC - Christian Goodrich
cargoodrich
3
900
Are puppies a ranking factor?
jonoalderson
2
4.2k
Everyday Curiosity
cassininazir
0
400
Building a Modern Day E-commerce SEO Strategy
aleyda
45
9.3k
<Decoding/> the Language of Devs - We Love SEO 2024
nikkihalliwell
1
380
Amusing Abliteration
ianozsvald
1
390
Winning Ecommerce Organic Search in an AI Era - #searchnstuff2025
aleyda
2
2.2k