Tweet
Tweet

Slide 1

Slide 1 text

*ODPSQPSBUJOH4ZOUBDUJDBOE 4FNBOUJD*OGPSNBUJPO
 JO8PSE&NCFEEJOHT
 VTJOH(SBQI$POWPMVUJPOBM/FUXPSLT 4IJLIBS7BTIJTIUI .BOJL#IBOEBSJ 1SBUFFL:BEBW 1JZVTI3BJ  $IJSBOKJC#IBUUBDIBSZZB 1BSUIB5BMVLEBS
 "$- 5.6,PNBDIJMBC.4BUPSV,BUTVNBUB

Slide 2

Slide 2 text

"CTUSBDU w 5IJTQBQFSQSPQPTFT4ZO($/ B(SBQI$POWPMVUJPOCBTFE NFUIPEGPSMFBSOJOHXPSEFNCFEEJOHT w 6OMJLFQSFWJPVTNFUIPET 4ZO($/VUJMJ[FTTZOUBDUJD DPOUFYUGPSMFBSOJOHXPSESFQSFTFOUBUJPOTXJUIPVU JODSFBTJOHWPDBCVMBSZTJ[F w 5IFZBMTPQSFTFOU4FN($/ BGSBNFXPSLGPSJODPSQPSBUJOH EJWFSTFTFNBOUJDLOPXMFEHFJOMFBSOFEXPSEFNCFEEJOHT w 5IFZEFNPOTUSBUFUIBUPVSQSPQPTFENFUIPETPCUBJO TVCTUBOUJBMJNQSPWFNFOUPWFSTUBUFPGUIFBSUBQQSPBDIFT !2

Slide 3

Slide 3 text

(SBQI$POWPMVUJPOBM /FUXPSLTGPS/-1 w %F⒎FSSBSEFUBM /*14,JQGBOE8FMMJOH  *$-3.BSDIFHHJBOJBOE5JUPW &./-1 w ($/TPO%JSFDUFE-BCFMFE(SBQIT
 
 w &EHF-BCFM(BUJOH.FDIBOJTN
 
 !3 v OPEF XPSE FEHF JOQVUOPEFGFBUVSFT d EJNFOTJPO n = |v| f = ReLU hk+1 v = f ∑ u∈𝒩+ (v) (Wk luv hk u + bk luv ) 𝒩+ (v) = 𝒩(v) ∪ {v} G = (V, E, X) E X 2 Rn⇥d hk+1 v = f ∑ u∈𝒩+ (v) gk luu × (Wk luv hk u + bk luv ) gk luv = σ ( ̂ Wk luv hk u + ̂ bk luv )

Slide 4

Slide 4 text

(SBQI$POWPMVUJPOBM /FUXPSLTGPS/-1 w %F⒎FSSBSEFUBM /*14,JQGBOE8FMMJOH  *$-3.BSDIFHHJBOJBOE5JUPW &./-1 w ($/TPO%JSFDUFE-BCFMFE(SBQIT
 
 w &EHF-BCFM(BUJOH.FDIBOJTN
 
 !4 v OPEF XPSE FEHF JOQVUOPEFGFBUVSFT d EJNFOTJPO n = |v|  GPS%JSFDUFE(SBQI
 ˠ6TFUIFPQQPTJUFEJSFDUJPO GSPNEFQFOEFOUTUPIFBET   GPS-BCFMFE(SBQI
 ˠBOEMBCFMTQFDJpDNPEFMQBSBNFUFST Wk luv bk luv EJTDPWFS XBUFS PO W0 self W0 self W0 self W0 self W0 self h0 W0 subj W0 subj−1 W0 obj−1 W0 obj W0 case W0 nmod W0 nmod−1 4DJFOUJTUT .BST DBTF TVCK PCK ONPE -PPLVQUBCMF G Є G Є G Є G Є G Є h0 h0 h0 h0 W0 case−1 h1 h1 h1 h1 h1 h0 hk+1 v = f ∑ u∈𝒩+ (v) (Wk luv hk u + bk luv ) 𝒩+ (v) = 𝒩(v) ∪ {v}

Slide 5

Slide 5 text

.PUJWBUJPO w 4LJQHSBN (MP7F FUDʜ
 ˠ5IFTFNFUIPETIBWFBSFTUSJDUJPOPGUIFXJOEPXTJ[F
 ˠ6TJOH%FQFOEFODZCBTFEDPOUFYUIFMQTUPBMMFWJBUFUIJTQSPCMFN w "MMFYJTUJOHTZOUBDUJDDPOUFYUCBTFENFUIPETFYQBOEWPDBCTJ[F
 ʢlTDJFOUJTUT@TVCKz lXBUFS@PCKz lNBST@ONPEzOFFETUPCFJODMVEFEBTBQBSUPG WPDBCGPSlEJTDPWFSzʣ
 ˠ5IJTQBQFSQSPQPTFT4ZO($/ XIJDIFNQMPZT($/T
 5IJTNFUIPEEPFTOPUFYQBOEWPDBCTJ[F w &OSJDIJOHXPSEFNCFEEJOHTXJUITFNBOUJDLOPXMFEHFIFMQTUPJNQSPWFUIFJS RVBMJUZGPSTFWFSBM/-1UBTLT
 ˠ&YJTUJOHBQQSPBDIFTBSFFJUIFSJODBQBCMFPGVUJMJ[JOHUIFTFEJWFSTFSFMBUJPOTPS OFFEUPCFFYQMJDJUMZNPEFMFEGPSFYQMPJUJOHUIFN
 ˠ5IJTQBQFSQSPQPTFT4FN($/ XIJDIMFBSOTUPVUJMJ[FNVMUJQMFTFNBOUJD DPOTUSBJOTCZNPEFMJOHUIFNBTEJ⒎FSFOUFEHFUZQFT !5  3FUSPpU FUDʜ

Slide 6

Slide 6 text

4ZO($/ w JOQVUEFQFOEFODZQBSTFETFOUFODF w -JLFB$#08NPEFM w 4ZO($/EPFTOPUJODMVEFTFMGMPPQT w ($/TMBZFSJTPOF w USBJOJOH/FHBUJWF4BNQMJOH !6 4DJFOUJTUT .BST DBTF TVCK PCK ONPE -PPLVQUBCMF G Є G Є G Є G Є G Є h0 h0 h0 h0 W0 case−1 h1 h1 h1 h1 h1 EJTDPWFS XBUFS PO h0 W0 case W0 subj W0 subj−1 W0 obj−1 W0 obj W0 nmod W0 nmod−1 UBSHFUXBUFS $POUFYU
 FNCFEEJOH 0VUQVUMBZFS E = |V| ∑ t=1 log P (wt |wt 1 , wt 2 …wt Nt ) P (wt |wt 1 , wt 2 …wt Nt ) = exp (vT wt ht) ∑ |V| i=1 exp (vT wi ht) E = |V| ∑ t=1 ( vT wt ht − log |V| ∑ i=1 exp (vT wi ht)) hk+1 i = f ∑ j∈N(i) gk lij × (Wk lij hk j + bk lij )

Slide 7

Slide 7 text

4FN($/ w JOQVUDPSQVTBOETFNBOUJDLOPXMFEHF GSPN8PSE/FUPS11%#  w *ODPSQPSBUJOHTFNBOUJDLOPXMFEHFJOQSFUSBJOFEXPSEFNCFEEJOHT w &YJTUJOHBQQSPBDIFTTZOPOZNZBOEBOUPOZNZ TZNNFUSJDJOGPSNBUJPO 
 ˠ5IJTNFUIPEDBOJODPSQPSBUFTZNNFUSJDBOEBTZNNFUSJD JOGPSNBUJPOKPJOUMZ w 1SFTFSWFUIFTFNBOUJDTFODPEFEJOQSFUSBJOFEFNCFEEJOHT
 ˠJOJUJBMJ[JOHCPUIUBSHFUBOEDPOUFYUFNCFEEJOHT
 ˠLFFQJOHUBSHFUFNCFEEJOHTpYFEEVSJOHUSBJOJOH w 4FN($/JODMVEFTTFMGMPPQTUPQSFTFSWFUIFJOJUJBMMFBSOFESFQSFTFOUBUJPO w USBJOJOH/FHBUJWF4BNQMJOH !7

Slide 8

Slide 8 text

4FN($/ w JOQVUDPSQVTBOETFNBOUJDLOPXMFEHF GSPN8PSE/FUPS11%#  w *ODPSQPSBUJOHTFNBOUJDLOPXMFEHFJOQSFUSBJOFEXPSEFNCFEEJOHT w &YJTUJOHBQQSPBDIFTTZOPOZNZBOEBOUPOZNZ TZNNFUSJDJOGPSNBUJPO 
 ˠ5IJTNFUIPEDBOJODPSQPSBUFTZNNFUSJDBOEBTZNNFUSJD JOGPSNBUJPOKPJOUMZ w 1SFTFSWFUIFTFNBOUJDTFODPEFEJOQSFUSBJOFEFNCFEEJOHT
 ˠJOJUJBMJ[JOHCPUIUBSHFUBOEDPOUFYUFNCFEEJOHT
 ˠLFFQJOHUBSHFUFNCFEEJOHTpYFEEVSJOHUSBJOJOH w 4FN($/JODMVEFTTFMGMPPQTUPQSFTFSWFUIFJOJUJBMMFBSOFESFQSFTFOUBUJPO w USBJOJOH/FHBUJWF4BNQMJOH !8

Slide 9

Slide 9 text

&YQFSJNFOUTFUUJOHT w %BUB8JLJQFEJB
 NJMMJPOTFOUT CJMMJPOUPLFOTCJMMJPOTZOUBDUJDEFQFOEFODJFT  w #BTFMJOFT w WT4ZO($/
 8PSEWFD $#08 (MP7F%FQT EFQFOEFODZ &95 TFDPOEPSEFS EFQFOEFODZ  w WT4FN($/
 3FUSPpU$PVOUFSpU BOUPOZN TZOPOZN +PJOU3FQT KPJOUMFBSOJOH  w &WBMVBUJPO w *OUSJOTJD5BTLT8PSE4JNJMBSJUZ$PODFQU$BUFHPSJ[BUJPO8PSE"OBMPHZ w &YUSJOTJD5BTLT/&32"104UBHHJOH$PSFGFSFODF3FTPMVUJPO !9

Slide 10

Slide 10 text

4ZO($/3FTVMUT !10  *OUSJOTJD
 4ZO($/NPSFF⒎FDUJWFMZDBQUVSFTFNBOUJDBOE TZOUBDUJDQSPQFSUJFTPGXPSET
 843UIFTZOUBDUJDDPOUFYUDBQUVSF GVODUJPOBMTJNJMBSJUZSBUIFSUIBOUPQJDBMTJNJMBSJUZ

Slide 11

Slide 11 text

4FN($/SFTVMUT
 VTJOHWBSJPVTSFMBUJPOT !11  /PUBUJPO
 FH4FN($/ 4ZO($/  
 ˠJOJU4ZO($/QPTUQSPDFTTJOH4FN($/
 OVNPGSFMBUJPOT  3FMBUJPO
 3ˠTZOPOZN
 3ˠTZOPOZNBOEBOUPOZN
 3ˠTZOPOZN BOUPOZN IZQFSOZNBOEIZQPOZN  4FN($/XJUI4ZO($/HJWFT
 UIFCFTUQFSGPSNBODF

Slide 12

Slide 12 text

4FN($/SFTVMUT
 VTJOHTBNFSFMBUJPO $PNQBSF4FN($/BHBJOTUPUIFSCBTFMJOFTXJUIUIFTBNFTFNBOUJDJOGPSNBUJPO
 ˠ4ZOPOZNTGSPN11%# !12

Slide 13

Slide 13 text

XJUI&-.P !13 5IFSFTVMUTTIPXUIBUEFQFOEFODZDPOUFYUCBTFEFNCFEEJOHTFODPEF DPNQMFNFOUBSZJOGPSNBUJPOXIJDIJTOPUDBQUVSFECZ&-.PBTJUPOMZSFMJFT POTFRVFOUJBMDPOUFYU

Slide 14

Slide 14 text

$PODMVTJPO w 5IJTQBQFSIBTQSPQPTFE4ZO($/ w (SBQIDPOWPMVUJPOCBTFEBQQSPBDIVTJOHTZOUBDUJDDPOUFYUGPSMFBSOJOHXPSE SFQSFTFOUBUJPOT w 4ZO($/PWFSDPNFTUIFQSPCMFNPGWPDBCVMBSZFYQMPTJPO
 5IJTNFUIPEPVUQFSGPSNTTUBUFPGUIFBSUXPSEFNCFEEJOHBQQSPBDIFTPO TFWFSBMJOUSJOTJDBOEFYUSJOTJDUBTLT w 5IJTQBQFSBMTPQSPQPTFT4FN($/ w KPJOUMZJODPSQPSBUJOHEJWFSTFTFNBOUJDJOGPSNBUJPOJOQSFUSBJOFEXPSE FNCFEEJOHT w 5IFDPNCJOBUJPOPG4ZO($/BOE4FN($/HJWFTUIFCFTUPWFSBMM QFSGPSNBODF !14

Slide 15

Slide 15 text

3FMBUFEXPSLT w $POWPMVUJPOBMOFVSBMOFUXPSLTPOHSBQITXJUIGBTU MPDBMJ[FETQFDUSBMpMUFSJOH%F⒎FSSBSEFUBM /*14 w 4FNJTVQFSWJTFEDMBTTJpDBUJPOXJUIHSBQIDPOWPMVUJPOBM OFUXPSLT,JQGBOE8FMMJOH *$-3 w &ODPEJOHTFOUFODFTXJUIHSBQIDPOWPMVUJPOBMOFUXPSLT GPSTFNBOUJDSPMFMBCFMJOH.BSDIFHHJBOJBOE5JUPW  &./-1 w 4PVSDFDPEFIUUQTHJUIVCDPNNBMMMBCJJTD8PSE($/ !15