Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Sign up for free
Menu
Search
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Pricing
Search
Sign in
Sign up for free
文献紹介 8月10日
Search
Sponsored
·
Your Podcast. Everywhere. Effortlessly.
Share. Educate. Inspire. Entertain. You do you. We'll handle the rest.
→
gumigumi7
August 10, 2018
140
0
Share
Embed
Copy iframe code
Copy JS code
Copy link
Start on current slide
文献紹介 8月10日
Improving Distributional Similarity
with Lessons Learned from Word Embeddings
gumigumi7
August 10, 2018
Transcript
18 slides
( Improving Distributional Similarity with Lessons Learned from Word Embeddings
▪ ▪ Omer Levy, Yoav Goldberg, Ido Dagan.
▪ Improving Distributional Similarity with Lessons Learned from Word Embeddings. ▪ Transactions of the Association for Computational Linguistics. 2015. ▪ ▪ , Skip-gram, LSA 2
▪ , pi ur nu p i S a
clulroug a S ▪ ) ▪ pi u tea , pi pi a a 3
▪ V ▪ 2 count predict ▪ Don’t count,
predict! (Baroni et al., 2014) ▪ Word2Vec Skip-gram Shifted-PMI ▪ Neural word embeddings as implicit matrix factorization (Levy and Goldberg, 2014) 4
▪ 2 ▪ hyperparameter ▪ Word2Vec 5
▪ ))( (- 2 ▪ )( D L AW
P ▪ 2 - - 2 ▪ - - 2 - 2 VI M W ▪ GW))( (- 2 V M P ▪ ▪ 6
▪ 2 ▪ C D 7 The quick brown
fox jumps over the lazy dog 4 4 3 4 2 4 1 4 4 4 3 4 2 4 1 4 Word2Vec : 1 1 1 2 1 3 1 4 1 1 1 2 1 3 1 4 Glove :
▪ ▪ '()&$,.-/+ ▪ #
# % # ▪ !"$) 8 ! = 1 − % & &: , %:
▪ ▪ R G MR ▪ - - S
▪ Shifted PMI ▪ - 2 PID ▪ −log(&) W S NV 9 ( ) *( + = -./ 0, 2 − log(&) 3--./ 0, 2 = max(-./ 0, 2 − log(&), 0)
▪ 7 0 7 577 2 ▪ 7 0.SW
0 0 5 SV I ca db S W M b e P ▪ V S ▪ α WD C 10
▪ ▪ 2 G SV SV D ▪ G
D AC 11
▪ ▪ E D ▪ E D C D
12
▪ ▪ ▪ 2 13
14
▪ ▪ PPMI-Matrix ▪ SVD ▪ SGNS (Word2Vec)
▪ Glove ▪ ▪ 6$-"!#% ▪ &,-" 2$-"!#% ▪ '*(-")$+ ▪ 672)$+
16 ▪ # "!# ▪ SVD
17 ▪ )%+ '*(-$ ▪ ▪
&,"-$# SGNS
▪ - Se N ▪ - - - n
y bc ▪ N smp n y i ▪ G tu d H uo a 18
More Decks by gumigumi7
See All by gumigumi7
文献紹介 1月24日
gumigumi7
0
260
文献紹介 11月7日
gumigumi7
0
140
文献紹介 10月3日
gumigumi7
0
340
文献紹介 9月3日
gumigumi7
0
270
文献紹介 7月16日
gumigumi7
0
280
文献紹介 6月12日
gumigumi7
0
350
文献紹介 5月16日
gumigumi7
0
200
文献紹介 4月18日
gumigumi7
0
150
文献紹介 12月15日
gumigumi7
0
130
Featured
See All Featured
How to Talk to Developers About Accessibility
jct
2
620
BBQ
matthewcrist
90
10k
Unlocking the hidden potential of vector embeddings in international SEO
frankvandijk
0
1k
Organizational Design Perspectives: An Ontology of Organizational Design Elements
kimpetersen
PRO
1
880
Building Adaptive Systems
keathley
44
3.3k
The browser strikes back
jonoalderson
0
2k
The Curious Case for Waylosing
cassininazir
1
620
Designing Experiences People Love
moore
143
24k
How to audit for AI Accessibility on your Front & Back End
davetheseo
0
640
Imperfection Machines: The Place of Print at Facebook
scottboms
270
14k
The SEO identity crisis: Don't let AI make you average
varn
0
630
Breaking role norms: Why Content Design is so much more than writing copy - Taylor Woolridge
uxyall
2
530