EMNLP2014読み会 "Efficient Non-parametric Estimation of Multiple Embeddings per Word in Vector Space"

Transcript

1. Efficient Non-parametric Estimation of Multiple Embeddings per Word in Vector Space Arvind Neelakantan, Jeevan Shankar, Alexandre Passos, Andrew McCallum
   ®Preferred Infrastructure š~ôø (@unnonouno)
   EMNLP2014 @PFI
   
2. nìûÏ
   š~ôø (@unnonouno) !  n՞ǽ‹6ODKPYARbF6ÿå !  ´ò½‹5&7'+6:2" !  NLP¾q/fµt©†k2014- !  rc§‹¨Û¹IBM¢ýÊPFI 
   
3. îÓ NLP¾q/fYANS
   !  YANSIbXJBZ9o !  ¾qÊÒl40ÈãØ9/¾qÊÒln Ã/ÊÒ>˜4;$4/¸2: !  s6¤jCbJR@/»œ5üç !  ƒd0mê* $ !  YANSĀ3o !  žÇ½‹fŒecf’yg.ĀÑf> 2" !  à-;õ3f*"" 
   
4. “þ
   !  word2vec>ÙÉ )1(/àÇ.í™/”× WEPa>Á; !  àÇ/—å+ÇÂĄă•¥¿>tx.¥ !  ”×WEPa/™>nŽ*4;w„5«£ );
   
   
5. ù*=;Skip-gram [Mikolov+13]
   
   éā
   0
   Ă
   *
   ;
   $(&#%'
   
   
6. ‰{€™*=;Skip-gram
   !  àÇ/WEPa v(wt ) +GbODKP/WEP a v(c) /|Ð>IF\AQ.<;
   
   Ă
   *
   ;
   
7. Multi Sense Skip-gram (MSSG) model
   1.  ÞÚàÇ/GbODKPWEPa9›æG bODKP>±h 2.  ”×âÀ/g9iËGbODKP/ßú $ ”×>ˆó 3.  #/”×.v¼";”×WEPa>ˆó 4.  ˆó<$”×WEPa.Ý')ÞÚGbOD KPz );+.";Skip-gram+ t! 
   
8. ù*=;MSSG
   
   $(&#%'"!
     "
   Skip-gram
   
9. ‰{€™*=;MSSG
   !  ˆó $”×WEPa>¯%
   !  àÇ/WEPa0”×ï.ª:GbODKP /WEPa01(%ª; 
   
10. @aH`LZ*=;MSSG
    !  ÞÚGbO DKP> ¯')”× >ˆó"; G Qu ;%
    
    
11. Non-Parametric MSSG (NP-MSSG) model
    !  MSSG*0”×WEPa/™04¬&%'$ !  NP-MSSG*0<>nŽ*4; !  4w0àö.ƒ2*+GbODKPWEP a/›æƶ$9} ”×WEPa> ͜";%
    
    
12. NP-MSSG.;”×/ˆó/Ôw
    !  ƒ2*+ÆGbODKPh— $9}9 ”×Ëä>²:‘)8 !  k(wt )0ƒ2*wt .²:‘)$”×/™ !  vcontext 0wt /›æGbODKPWEPa !  (wt , k)0kˉ/”×/GbODKPg¦
    
    
13. q„/2+4
    !  MSSG !  àÇ.v )kÅ/”×WEPa>²:‘);S_ [ Mkè !    "  ! !  ˆó<$”×WEPa>¯')ñ‚.Skip-gram !  NP-MSSG !  kÅ/”×WEPa>²:‘);/0t! !     "#
       !  ˆó<$”×WEPa*Skip-gram";/0t!
    
    
14. ŠÌ¡°Apple
    !  ° +J^VK<.p<;
    
    
15. ŠÌ¡°Run)
    !  ”×/Ąă-àÇ.0º/”×WEPa³Á<;
    
    
16. ”×/ßú…>­—*;/ŠÌ
    localSimiËú$”×t¹/÷ÖcŸÜ
    
    
17. ·Î
    !  Š01+?,t!ŠÌ>6')$(^o^) !  E_K™>nŽ*z";w„.áð )$ /*?-/*/ +ëÄ !  ÞÚàÇ*4;8:QD][bPŒ‡/P TNE*4$1nÕ-–"; !  S_F_UWEPa>¯