Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
文献紹介 11月7日
Search
gumigumi7
October 31, 2018
0
120
文献紹介 11月7日
Explicit Retrofitting of Distributional Word Vectors
gumigumi7
October 31, 2018
Tweet
Share
More Decks by gumigumi7
See All by gumigumi7
文献紹介 1月24日
gumigumi7
0
230
文献紹介 10月3日
gumigumi7
0
320
文献紹介 9月3日
gumigumi7
0
250
文献紹介 8月10日
gumigumi7
0
120
文献紹介 7月16日
gumigumi7
0
260
文献紹介 6月12日
gumigumi7
0
330
文献紹介 5月16日
gumigumi7
0
180
文献紹介 4月18日
gumigumi7
0
140
文献紹介 12月15日
gumigumi7
0
110
Featured
See All Featured
The Art of Programming - Codeland 2020
erikaheidi
43
12k
個人開発の失敗を避けるイケてる考え方 / tips for indie hackers
panda_program
63
14k
Cheating the UX When There Is Nothing More to Optimize - PixelPioneers
stephaniewalter
275
13k
A Tale of Four Properties
chriscoyier
152
22k
Put a Button on it: Removing Barriers to Going Fast.
kastner
58
3.1k
Building Effective Engineering Teams - LeadDev
addyosmani
30
1.9k
jQuery: Nuts, Bolts and Bling
dougneiner
59
7.1k
How to name files
jennybc
65
93k
Building Applications with DynamoDB
mza
88
5.6k
Git: the NoSQL Database
bkeepers
PRO
423
63k
Atom: Resistance is Futile
akmur
260
25k
Fight the Zombie Pattern Library - RWD Summit 2016
marcelosomers
228
16k
Transcript
Goran Glavas, Ivan Vulic Proceedings of the 56th Annual Meeting
of the Association for Computational Linguistics (Long Papers), pages 34–45
)4 2 n 8"* %Retrofitting 51' ,+:&(Word vector space specialization
0 38#-/ n 96 96 ! $2.7
.& 3 n *4 #/NLP ,60 n
*4 #12,)-5*4+7*4 8%!"$ -34(' 9
5' 4 n Word vector space specialization l !$3LHGV=4 *JI:+
3L!$M9C l $%$@? 1PB(3L8O n Retrofitting l 3LD!$4 0TSE R>N9C l 0TSE2.!$@? ,QUV&6 n 2 9CK- explicit retrofitting (ER) ;A l 7)F!$/<"%#$ 3L
5 n ! n !
"$ 6 n 3)+-* !%& 1/+- l Ex) car
– automobile - car – drive - n Negative Sampling l '0 #-K+,2 (.
") 7 n .!" #" 2,. !# #"
′ %1&%(#" ; ()- l (: 0 n /' 2 +*1&$(
8 n 2"#%$ Mean Square Distance Objective (ER-MSD) n
!": #% (&!" 0 ) n !($% , $' ): #%$ ( &, #% )
9 n )$%$% '! ' Contrastive Objective (ER-CNT) n
!": &( ()$% 0 ") n !′: &( ' !(%& , %( ) # n ER-MSD )$%$%
'+ 10 n .1 %#&",2$ *!3 Topological Regularization
n !: ER-MSD ER-CNT -,2( n ": *0! *! )/, -,2( !# = ! + "!&'(
3G 11 n &+2> $ Word2Vec, GloVe, FastText ': n
F=@4=@5( WordNet, Roget’s Thesaurus ': l 1,023,082F=@ 380,8734=@5( l !%.@7 57,320@1 → Retrofitting /E )<.@06 ,8;? n ACB7!(#$ , #& )!%#AC': 9*-D" %(0.3:
12 n n
l Word Similarity (SimLex-999, SimVerb-3500) l Language Transfer n l Lexical Text Simplification (LIGHT-LS) l Dialog State Tracking
7- (Word Similarity) 13 • SimLex 4 %#< !6>'&$ +.095(;=
• :23 Attract-Repel !6&$," 8*&$)/ 1 ,"
/* (Lexical Text Simplification) 14 • Retrofitting (, Attract-Repel
% 2'135 • #. 59.6% Attract-Repel %"!' +$'1 7 A: Accuracy (-&), C: Changes ( 640))
-* (Lexical Text Simplification) 15 • 1+ $."!2 3 )
%04& (,villain(%#) Attract-Repel protagonist(#)/' • ER-CNT demon(5)/'
16 n Retrofitting Word vector space specialization =*
3 #"%$4916 n WordNet (<.+ ,* :/-2 8)0>'!#7; n Word Similarity Lexical Simplification A& 5.!#?0>@B