文献紹介 7月16日

Probabilistic FastText for Multi-Sense Word Embeddings

Transcript

1. ) ( Probabilistic FastText for Multi-Sense Word Embeddings
2. 
   ▪  ▪ Mikhail Khodak, Nikunj Saunshi, Yingyu Liang, Tengyu Ma, Brandon Stewart, Sanjeev Arora. ▪ Probabilistic FastText for Multi-Sense Word Embeddings. ▪ Proceedings of the Association of Computational Linguistics. 2018. ▪   ▪  GaussianMixtureSense Embedding 2
3. 
    ▪ n o V e dT WG oe2 W W ▪ V e a i M V a c ▪ V a n o 3
4. 
    ▪ e c V ▪ ▪ c V c W ▪ e c ” ” ▪ fastText ▪ Word2Vec subwordc V c 2 ▪ Word2Vec e c ” ” 4
5. 
    ▪ emt a ▪ ( ) ▪ G ul m emt er ksu a a u l m a emt ▪ emt M a S ▪ MUSE (Modularizing Unsupervised Sense Embeddings) ▪ api t api t a ▪ Contextual Word Similarity k e SoTA 5
6. 
    ▪ fa E u wn E g ue bd ▪ , B A A C A ▪ a g e bd G ▪ h mo ] V x ptk ▪ A , A C A C , ▪ ] V bdxMpt i pkLR D r ov cT W[ps 6
7. ( ) ( ▪      7
8. ( ) ( ▪  
      8 ! :   "#,% :  &  ' 
    (# :   1  
9. ( ) ( ▪ , ▪ 9
10. ( ) ( ▪  ▪
       10 Negative Sampling Margin
11. 
     ▪  
      11 
      Subword 
12. 
     12 ▪ 
    
13. 
    ▪ RockBank  
     
14. 
    ▪ SCWS-68 ▪ “... east bank of the Des Moines River ...” ++ “... basis of all money laundering ...” +. bank + money . :*$'5.  ()8 ▪ - bank + money 79)8. ",8   79)8'4.*/#97. //& ▪  -3'+!.1:  -6()
15. 
    15 ▪ 
       
16. 
    16 ▪ $ Subwords+ ( %%'  0-.1 &)(% ▪ Abnormal  Abnormality ! Subwords %"#+  ( *(
17. 
     ▪ a i rtx n e M sx ▪ e T T G ▪ F ea e e 17