論文紹介: Word-node2vec

- 首都大 小町研 NAACL 2019 読み会
- Word-Node2Vec: Improving Word Embedding with Document-Level Non-Local Word Co-occurrences. Sen et al., NAACL2019 を紹介
- paper URL: https://www.aclweb.org/anthology/N19-1109.pdf

Transcript

1. 8PSE/PEF7FD
 *NQSPWJOH
   8PSE
   &NCFEEJOH
   XJUI
   %PDVNFOU-FWFM
   /PO-PDBM
   8PSE
   $PPDDVSSFODFT 1SPDIFUB
   4FO
   %FCBTJT
   (BOHVMZ
   (BSFUI
   +'
   +POFT
 /""$- ঺հ
   খொݚ
   .
   উຢ
   ஐ
2. ֓ཁ w XPSEWFD
   (MPWF
   ͳͲ͸͋Δ୯ޠʹରͯ͠ɺ
 ૭෯಺ͷ୯ޠΛڞىͨ͠୯ޠͱͯ͠Δ
   w จॻ಺ͷ૭෯֎ͷ୯ޠʹରͯ͠΋ڞىΛݟͨϞσϧΛఏҊ
 ˠ
   XPSEOPEFWFD
 
   MPDBM
   ͱ
   OPOMPDBM
   ͷڞىɺͲͪΒ΋ݟΔ
 
   ۩ମతʹ͸ɺFEHF
   ͷॏΈ࡞੒࣌ʹ
   

   
   MPDBMɺOPOMPDBM
   Λߟྀ͍ͯ͠Δ
   w ධՁλεΫ
   XPSE
   TJNJMBSJUZ
   XPSE
   BOBMPHZ
   
 DPODFQU
   DBUFHPSJ[BUJPO
 bXPSEOPEFWFD
   PVUQFSGPSNT
   XPSEWFD
   BOE
   HMPWF
   PO
   B
   SBOHF
   PG
   EJ⒎FSFOU
   UBTLT` !2
3. άϥϑͷઃఆ
 ʢஶऀͷ࣮૷Ͱ֬ೝʣ w &NCFEEJOH
   ͷֶशσʔλ
 ˠ
   ෳ਺ͷ
   EPDVNFOUTʢEPDVNFOU
   ୯Ґͷ۠੾Γ͕ଘࡏʣ
   w OPEF
   ୯ޠɺFEHF
   ୯ޠͷڞىɺແ޲άϥϑ
   w ڞىͷఆٛ
   υΩϡϝϯτ಺Ͱग़ͯདྷͨ΋ͷશͯ
 ˠ
   EPDVNFOU
   ಺ͷग़ݱ୯ޠશ͕ͯ͋Δ୯ޠͷۙ๣
   OPEF
   ͱͳΔ
   w 4UPQXPSE
   Ͱ
   bUIF`
   ͱ͔͸֎ͯ͠Δ !3 w1 i w1 1 w1 N ʜ EPD
   
   MFOHUI
   / ؆୯ͷͨΊ
   JUI
   XPSE
   ʹؔ͢Δ
   FEHF
   ͷΈهࡌ w2 j w2 1 w2 O ʜ EPD
   
   MFOHUI
   0 if(w1 i = w2 j ) w1 i w1 1 w1 N ʜ w2 1 w2 O ʜ DPNNJU
   EFGDFFCFGGEEB
4. 4LJQHSBN
   
 XJUI
   /FHBUJWF
   4BNQMJOH w ఏҊख๏͸
   4LJQHSBN
   XJUI
   /FHBUJWF
   4BNQMJOH
   4(/4
   Λجʹͨ͠΋ͷ
 DPOUFYU
   XPSE
   ͷܾΊํʹάϥϑཁૉ͕ࠞೖ͚ͨͩ͠
   w ͱΓ͋͑ͣ
   4(/4
   ͷ͓͞Β͍
 ʢX
   ʹର͢Δ
   4(/4ʣ !4 Λ(w) = {i : ti = w} c(w) = ∪i∈Λ(w) ∪k j = − k j ≠ 0 ti+j Ω+ = ∪w∈V {w, c(w)} Ω− = ∪w∈V {w, ∪ {v : v ∼ (V − c(w))}} /PUBUJPO ֶशσʔλશମ
 ʢ5
   ͸૯
   UPLFO
   ਺ʣ ˡ
   Xʹର͢Δ
   JOEFY
   ू߹ ˡ
   Xʹର͢Δपลޠू߹ ਖ਼ྫ ෛྫ 8JOEPX
   TJ[F
   L
   7PDBC
   TFU
   7 ໨తؔ਺     
5. 4(/4
   (SBQI
   ͷ৔߹ !5 ֶश࣌ɺೖྗ͸άϥϑ
   (ʢ1
   ͷҰ൪ӈͷ΍ͭʣ OPEF FEHF c(w) = ∪i∈Λ(w) ∪k j = − k j ≠ 0 ti+j ˢ Дۙ๣ /PUBUJPO IPQ
   ਺ؔ਺ 8JOEPX
   TJ[F
   L
   ʢDPOUFYU
   XPSET
   ͕
   ࠷େ
   L
   ݸ·Ͱʣ h(u, v) ਖ਼ྫ
   ෛྫ ໨తؔ਺       
6. /POMPDBM
   ͳ
   XFJHIU w ֤
   %PDVNFOU
   ಺ͷ୯ޠͷڞىͰॏΈΛ࡞Δʢ૯
   %PD
   ਺
   .ʣ
   w ઢܗิؒΛ༻͍ͨฏ׈ԽΛߦ͏
 ʢϋΠύʔύϥϝʔλЋొ৔ʣ !6  
7. -PDBM
   XFJHIU
   ͱ૊Έ߹Θͤ w ࠓ౓͸
   MPDBM
   ͳڞىΛڧ͘ΈͨॏΈΛߏங
   w 8JOEPX
   TJ[F
   L
   ಺ͷ୯ޠͷΈΛڞى͍ͯ͠Δͱͯ͠ܭࢉ
   w ग़དྷ্͕ͬͨ
   MPDBM
   XFJHIU
   Λ
   OPOMPDBM
   XFJHIU
   ͱ૊Έ߹ΘͤΔ !7  
8. ॏΈΛ࢖ͬͯ
   DPOUFYU
   XPSET
   Λܾఆ͢Δʂ w $POUFYU
   XPSE
   ͸ۙ๣୯ޠ͔Β࠷େͰ
   XJOEPX
   TJ[F
   Lʢl
   
   Lʣ
   ʹͳΔ·Ͱ༻ҙ͢Δ
   w ۩ମతʹ͸֤ۙ๣఺ू߹ʢ//
   //ʜД//ʣ͔Β
   TUSBUJpFE
   ʹ
   TBNQMJOH
   ͢Δ
   w ͜ͷ
   TBNQMJOH
   ࣌ʹॏΈΛ࢖༻͢Δ
   w ࢓૊Έͱͯ͠͸
   K//
   ·ͰݟΔ͜ͱ͕Ͱ͖Δ͕ɺ//
   ·ͰͰे෼Ͱͨ͠ !8 w1 w2 w3 w4 w5 //
   ) 
   
   
   //
   )
   ˠ
   / 
   
   w1 w2 w3 w4 w5 w2 w3 ͷۙ๣୯ޠ TBNQMJOH
   ͷ֬཰ ෼฼͸ॏΈͷ࿨
   Ќ͸ϋΠύʔύϥϝʔλ
   ੍໿
   β2 = 1 − β1 β1 = β;   
9. ࣮ݧઃఆ w %BUB
   %#1FEJB
 ස౓ҎԼͷ୯ޠ͸ແࢹ
   TUPQXPSET
   औΓআ͖
   w ख๏
 MPDBM
   4(/4
   (MPWF
   'BTUUFYU
   &MNP
 %PDVNFOUMFWFM
   DPPDDVS
   -%"
 DPNCJOBUJPO
   MPDBM
   BOE
   OPOMPDBM
   4(/4
   ͱ
   -%"
   ͷઢܗิؒ
 OPEFWFD
   ΋
   CBTFMJOF
   w ධՁͱλεΫ
 XPSE
   TJNJMBSJUZ
   .&/
   84*.
 XPSE
   BOBMPHZ
   (PPHMF
   .43
   4FN&WBM
 DPODFQU
   DBUFHPSJ[BUJPO
   "1
   #-&44
   &44-@C
   w .FUSJD
 TJNJMBSJUZ
   4QFBSNBO`T
   Л
 ͦΕҎ֎
   BDDVSBDZ
   w ύϥϝʔλ
   TJNJMBSJUZ
   ͷ
   .&/
   ͕࠷େԽ͢ΔΑ͏ʹ
 ଞͷλεΫ͸ͦͷύϥϝʔλΛ࢖͍·Θ͠
   !9 DPODFQU
   DBUPHPSJ[BUJPO
    OPVO
   ʹରͯ͠
   DPODFQU
   Λ౰ͯΔ
   bDFSFNPOZ`
   bHSBEVBUJPO`
   
 ˠ
   b4PDJBM
   0DDBTJPO`
 ʢ෼ྨʹ࢖༻ͨ͠Ϟσϧ͸ෆ໌ʣ 
10. ࣮ݧ݁Ռ !10
11. ෼ੳ w TJNJMBSJUZ
    ͱ
    BOBMPHZ
    Ͱ
    XPSEOPEFWFD
    ͷϋΠύʔύϥϝʔλʢЋ
    ЌʣΛ͍Ζ͍Ζࢼͨ͠
    w Ќ͕େ͖͍
    ˠ
    //
    Λڧ͘
    TBNQMJOH
    ͢Δ !11 84*.
 ʢTJNJMBSJUZʣ .43
 ʢBOBMPHZʣ
12. ·ͱΊ w %PDVNFOUMFWFM
    ͷ
    OPOMPDBM
    ͳڞىΛߟྀͨ͠
    FNCFEEJOH
    ΛఏҊ
    w -PDBM
    ͱ
    OPOMPDBM
    Λ͍͍ײ͡ʹݟΕͨ !12
13. ࢀߟࢿྉ w ஶऀͷ࣮૷
 IUUQTHJUIVCDPNQSPDIFUB8PSE/PEF7FD !13