Neural Semantic Encoders

長岡技術科学大学
自然言語処理研究室
文献紹介（2019-03-05）
Neural Semantic Encoders
http://www.aclweb.org/anthology/E17-1038

Transcript

1. Neural Semantic Encoders 29015 -3 Tsendsuren Munkhdalai and Hong Yu Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, pages 397–407, 2017
2. "CTUSBDU • .FNPSZ
   /FUXPSLΛ֦ுͨ͠/FVSBM
   4FNBOUJD
   &ODPEFST
   /4&
   ΛఏҊ Ø ͋͘·Ͱ୯ޠදݱ &ODPEFS ͳͷͰ༷ʑͳλεΫʹԠ༻Մೳ Ø NFNPSZ
   VQEBUF
   ػೳͷ௥Ճ Ø .VMUJQMF
   NFNPSZ
   BDDFTT
   
   4IBSFE
   NFNPSZ
   BDDFTT • ͭͷ/-1λεΫʹ͓͍ͯ༗ޮੑΛࣔͨ͠
   
3. .FNPSZ
   /FUXPLT • .FNPSZ
   /FUXPSLT
   +BTPO
   8FTUPO
   FU
   BM
   
   ͸ʮڊେͳϝϞϦʯͱ ʮϝϞϦ΁ͷಡΈࠐΈॻ͖ࠐΈͳͲ͕Ͱ͖Δػೳʯ͔Βߏ੒͞ΕΔϝ ϞϦ෇͖χϡʔϥϧωοτϫʔΫͷ͜ͱ
   ( ( ) ) (   
    
4. &OEUP&OE
   .FNPSZ
   /FUXPSLT
   4BJOCBZBS
   4VLICBBUBS
   FU
   BM
   
   
5. /FVSBM
   4FNBOUJD
   &ODPEFST
    
        !"#$: &' ℎ)$$*+ ,-.-*: ℎ'
6. /FVSBM
   4FNBOUJD
   &ODPEFST
    
        !"#$: &' ℎ)$$*+ ,-.-*: ℎ' ϝϞϦʹ͸͋Β͔͡Ίશ୯ޠΛ֨ೲͯ͠ ͓͘
7. /FVSBM
   4FNBOUJD
   &ODPEFST
    
        !"#$: &' ℎ)$$*+ ,-.-*: ℎ' ୯ޠ &'
   Λೖྗͯ͠ɼ ͭͷϕΫτϧ ℎ'
   Λग़ྗ͢Δ ˠ ޙஈͷϞσϧʹ౉͢
8. /FVSBM
   4FNBOUJD
   &ODPEFST
    
        !"#$: &' ℎ)$$*+ ,-.-*: ℎ' ϝϞϦͷத͔Βண໨͢Δ औΓग़͢ εϩο τΛࢦࣔ͢͠ϕΫτϧΛ࡞Δ ˠ /'
9. /FVSBM
   4FNBOUJD
   &ODPEFST
    
        !"#$: &' ℎ)$$*+ ,-.-*: ℎ' औΓग़ͨ͠ϝϞϦεϩοτͱೖྗ୯ޠΛ ૊Έ߹ΘͤͨϕΫτϧΛ࡞Δ ˠ /'
10. /FVSBM
    4FNBOUJD
    &ODPEFST
     
         !"#$: &' ℎ)$$*+ ,-.-*: ℎ' /'͔Βग़ྗϕΫτϧΛ࡞Δ ˠ ℎ' ಡΈࠐΜͩεϩοτͷҐஔʹ৽͍͠ঢ়ଶ Λ্ॻ͖͢Δ NFNPSZ
    VPEBUF
11. /FVSBM
    4FNBOUJD
    &ODPEFST 0 0 M = #$ , #& , … , #( 
         0 )* 0 )* +* 1 0 0 -0 - ,* , )* #-,* .* ℎ* 0*1$ 0* 0 0
12. /FVSBM
    4FNBOUJD
    &ODPEFST 0 0 !" = $% , $' , … , $) 
         0 *" 0 *" +, 1 0 0 -0 - -, , *, $.,, /, ℎ, !,1% !, 0 0
13. .VMUJQMF
    NFNPSZ
    BDDFTT
     
         !"#$: &' .FNPSZ
    . ℎ)$$*+ ,-.-*: ℎ' .FNPSZ
    . ü ෳ਺ͷϝϞϦΛࢀরͰ͖Δ
14. 4IBSFE
    NFNPSZ
    BDDFTT
     
         !"#$: &',) ℎ+$$,- ./0/,: ℎ',) .FNPSZ
    .O ü ڞ௨ͷϝϞϦΛࢀরͰ͖Δ 
         !"#$: &1,) ℎ+$$,- ./0/,: ℎ1,)
15. &YQFSJNFOUT
     ͭͷ/-1λεΫͰ࣮ݧΛߦͳͬͨ • 4FOUFODF
    DMBTTJGJDBUJPO • "OTXFS
    TFOUFODF
    TFMFDUJPO • /BUVSBM
    MBOHIVBHF
    JOGFSFODF • %PDVNFOU
    NPEFMMJOH • /FVSBM
    NBDIJOF
    USBOTMBUJPO
16. 4FOUFODF
    DMBTTJGJDBUJPO
     %BUBTFU
    4UBOGPSE
    4FOUJNFOU
    5SFFCBOL
    ஋෼ྨ
    ϥϕϧ෼ྨ /4& .-1 4FOUFODF 4FOUJNFOU
    MBCFM
    
17. "OTXFS
    TFOUFODF
    TFMFDUJPO
     ࣭໰จʹରͯ͠ީิจͷத͔Βਖ਼͍͠ճ౴จΛબ୒͢ΔλεΫ %BUBTFU
    8JLJ2" /4&B /4&R
    
     .-1 "OTXFS 2VFTUJPO :FT/P TIBSFE
18. /BUVSBM
    MBOHVBHF
    JOGFSFODF
     લఏͱԾઆͷϖΞ͔ΒɼͦΕ͕߹ཧ͔ໃ६͔தཱ͔Λ ൑ఆ͢ΔλεΫ %BUBTFU
    4UBOGPSE
    /BUVSBM
    -BOHVBHF
    *OGFSFODF
    4/-*
    EBUBTFU /4&Q /4&I
    
     .-1 1SFNJTF )ZQPUIFTJT &OUBJMNFOU$POUSBEJDUJPO/FVUSBM TIBSFE BUUFOUJPO
19. %PDVNFOU
    NPEFMMJOH
     จॻϨϕϧͷײ৘෼ྨλεΫ %BUBTFU
     *.%#
    Ϋϥε  :FMQ
    
    Ϋϥε ஈ໨ͷ/4&ͰจϕΫτϧΛɼஈ໨ ͷ/4&-45.ͰจॻϕΫτϧΛ࡞Γɼ ࠷ޙʹTPGUNBYͰ෼ྨ͢Δ /4& /4&I
    
     /4& 
     /4& /4&I
    
     -45. 
20. /FVSBM
    NBDIJOF
    USBOTMBUJPO
     %BUBTFU
    *84-5
    
    &OHMJTI(FSNBO &ODPEFS%FDPEFSϞσϧʹ/4&ΛదԠ /4&FOD -45.EFD
     .-1 &OHMJTI (FSNBO 5SBOTMBUJPO BUUFOUJPO /4&FOD /4&EFD
     .-1 &OHMJTI (FSNBO 5SBOTMBUJPO BUUFOUJPO TIBSFE -45.FOD -45.EFD .-1 &OHMJTI (FSNBO 5SBOTMBUJPO BUUFOUJPO
21. $PODMVTJPO 
    • SFBE
    DPNQPTF
    XSJUFͷػೳΛ࣋ͭ/FVSBM
    4FNBOUJD
    &ODPEFST /4& ΛఏҊ • .VMUJQMF4IBSFE
    NFNPSZ
    BDDFTTػೳʹΑΓɼಛఆͷλεΫʹԠͯ͡ ΞʔΩςΫνϟΛॊೈʹมߋͰ͖Δ • ͭͷ/-1λεΫʹ͓͍ͯɼ/4&Λ༻͍Δ͜ͱͰߴ͍ੑೳΛࣔͨ͠
22. 
    
23. /BUVSBM
    MBOHVBHF
    JOGFSFODF
    
    
24. 8PSE
    BTTPDJBUJPO
    HSBQI
     5IF
    HSBQI
    DPNQPOFOUT
    DMVTUFSFE
    BSPVOE
    UIF
    TFNBOUJDBMMZ
    SJDI
    XPSET B TJUT
    XBMM
    BVUVNO
    
    C
    5ISFF
    QVQQJFT
    UVC
    WFU ϑϨʔζ΍म০ؔ܎Λଊ͍͑ͯͦ͏
25.