論文紹介: Incorporating Syntactic and Semantic Information in Word Embeddings using Graph Convolutional Networks
- TMU Komachi lab
- paper reading (ACL2019)
- Incorporating Syntactic and Semantic Information in
Word Embeddings using Graph Convolutional Networks. Vashishth et al., ACL2019
- paper URL: https://www.aclweb.org/anthology/P19-1320.pdf
w 5IFZBMTPQSFTFOU4FN($/ BGSBNFXPSLGPSJODPSQPSBUJOH EJWFSTFTFNBOUJDLOPXMFEHFJOMFBSOFEXPSEFNCFEEJOHT w 5IFZEFNPOTUSBUFUIBUPVSQSPQPTFENFUIPETPCUBJO TVCTUBOUJBMJNQSPWFNFOUPWFSTUBUFPGUIFBSUBQQSPBDIFT !2
w &EHF-BCFM(BUJOH.FDIBOJTN !3 v OPEF XPSE FEHF JOQVUOPEFGFBUVSFT d EJNFOTJPO n = |v| f = ReLU hk+1 v = f ∑ u∈𝒩+ (v) (Wk luv hk u + bk luv ) 𝒩+ (v) = 𝒩(v) ∪ {v} G = (V, E, X) E X 2 Rn⇥d hk+1 v = f ∑ u∈𝒩+ (v) gk luu × (Wk luv hk u + bk luv ) gk luv = σ ( ̂ Wk luv hk u + ̂ bk luv )
w &EHF-BCFM(BUJOH.FDIBOJTN !4 v OPEF XPSE FEHF JOQVUOPEFGFBUVSFT d EJNFOTJPO n = |v| GPS%JSFDUFE(SBQI ˠ6TFUIFPQQPTJUFEJSFDUJPO GSPNEFQFOEFOUTUPIFBET GPS-BCFMFE(SBQI ˠBOEMBCFMTQFDJpDNPEFMQBSBNFUFST Wk luv bk luv EJTDPWFS XBUFS PO W0 self W0 self W0 self W0 self W0 self h0 W0 subj W0 subj−1 W0 obj−1 W0 obj W0 case W0 nmod W0 nmod−1 4DJFOUJTUT .BST DBTF TVCK PCK ONPE -PPLVQUBCMF G Є G Є G Є G Є G Є h0 h0 h0 h0 W0 case−1 h1 h1 h1 h1 h1 h0 hk+1 v = f ∑ u∈𝒩+ (v) (Wk luv hk u + bk luv ) 𝒩+ (v) = 𝒩(v) ∪ {v}
USBJOJOH/FHBUJWF4BNQMJOH !6 4DJFOUJTUT .BST DBTF TVCK PCK ONPE -PPLVQUBCMF G Є G Є G Є G Є G Є h0 h0 h0 h0 W0 case−1 h1 h1 h1 h1 h1 EJTDPWFS XBUFS PO h0 W0 case W0 subj W0 subj−1 W0 obj−1 W0 obj W0 nmod W0 nmod−1 UBSHFUXBUFS $POUFYU FNCFEEJOH 0VUQVUMBZFS E = |V| ∑ t=1 log P (wt |wt 1 , wt 2 …wt Nt ) P (wt |wt 1 , wt 2 …wt Nt ) = exp (vT wt ht) ∑ |V| i=1 exp (vT wi ht) E = |V| ∑ t=1 ( vT wt ht − log |V| ∑ i=1 exp (vT wi ht)) hk+1 i = f ∑ j∈N(i) gk lij × (Wk lij hk j + bk lij )
ˠ5IJTNFUIPEDBOJODPSQPSBUFTZNNFUSJDBOEBTZNNFUSJD JOGPSNBUJPOKPJOUMZ w 1SFTFSWFUIFTFNBOUJDTFODPEFEJOQSFUSBJOFEFNCFEEJOHT ˠJOJUJBMJ[JOHCPUIUBSHFUBOEDPOUFYUFNCFEEJOHT ˠLFFQJOHUBSHFUFNCFEEJOHTpYFEEVSJOHUSBJOJOH w 4FN($/JODMVEFTTFMGMPPQTUPQSFTFSWFUIFJOJUJBMMFBSOFESFQSFTFOUBUJPO w USBJOJOH/FHBUJWF4BNQMJOH !7
ˠ5IJTNFUIPEDBOJODPSQPSBUFTZNNFUSJDBOEBTZNNFUSJD JOGPSNBUJPOKPJOUMZ w 1SFTFSWFUIFTFNBOUJDTFODPEFEJOQSFUSBJOFEFNCFEEJOHT ˠJOJUJBMJ[JOHCPUIUBSHFUBOEDPOUFYUFNCFEEJOHT ˠLFFQJOHUBSHFUFNCFEEJOHTpYFEEVSJOHUSBJOJOH w 4FN($/JODMVEFTTFMGMPPQTUPQSFTFSWFUIFJOJUJBMMFBSOFESFQSFTFOUBUJPO w USBJOJOH/FHBUJWF4BNQMJOH !8
w 5IJTQBQFSBMTPQSPQPTFT4FN($/ w KPJOUMZJODPSQPSBUJOHEJWFSTFTFNBOUJDJOGPSNBUJPOJOQSFUSBJOFEXPSE FNCFEEJOHT w 5IFDPNCJOBUJPOPG4ZO($/BOE4FN($/HJWFTUIFCFTUPWFSBMM QFSGPSNBODF !14
katsumata420
a1da4
yuukit
dmattsun
monochromegane
ryotoitoi
hponka
datascientistsociety
joisino
shunk031
masatoto
onely7
shpigford
chrisshort
holman
garrettdimon
soudai
akosma
honnibal
chriscoyier
productmarketing
mraible
malarkey
keathley
