Neural Semantic Encoders

66cc992074ab4522374e429c11fef225?s=47 youichiro
March 05, 2019

Neural Semantic Encoders

長岡技術科学大学
自然言語処理研究室
文献紹介(2019-03-05)
Neural Semantic Encoders
http://www.aclweb.org/anthology/E17-1038

66cc992074ab4522374e429c11fef225?s=128

youichiro

March 05, 2019
Tweet

Transcript

  1. Neural Semantic Encoders 29015 -3 Tsendsuren Munkhdalai and Hong Yu

    Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, pages 397–407, 2017
  2. "CTUSBDU • .FNPSZ/FUXPSLΛ֦ுͨ͠/FVSBM4FNBOUJD&ODPEFST /4&  ΛఏҊ Ø ͋͘·Ͱ୯ޠදݱ &ODPEFS ͳͷͰ༷ʑͳλεΫʹԠ༻Մೳ

    Ø NFNPSZVQEBUFػೳͷ௥Ճ Ø .VMUJQMFNFNPSZBDDFTT4IBSFENFNPSZBDDFTT • ͭͷ/-1λεΫʹ͓͍ͯ༗ޮੑΛࣔͨ͠ 
  3. .FNPSZ/FUXPLT • .FNPSZ/FUXPSLT +BTPO8FTUPOFUBM  ͸ʮڊେͳϝϞϦʯͱ ʮϝϞϦ΁ͷಡΈࠐΈॻ͖ࠐΈͳͲ͕Ͱ͖Δػೳʯ͔Βߏ੒͞ΕΔϝ ϞϦ෇͖χϡʔϥϧωοτϫʔΫͷ͜ͱ  (

    ( ) ) (     
  4. &OEUP&OE.FNPSZ/FUXPSLT 4BJOCBZBS4VLICBBUBSFUBM  

  5. /FVSBM4FNBOUJD&ODPEFST         

    !"#$: &' ℎ)$$*+ ,-.-*: ℎ'
  6. /FVSBM4FNBOUJD&ODPEFST         

    !"#$: &' ℎ)$$*+ ,-.-*: ℎ' ϝϞϦʹ͸͋Β͔͡Ίશ୯ޠΛ֨ೲͯ͠ ͓͘
  7. /FVSBM4FNBOUJD&ODPEFST         

    !"#$: &' ℎ)$$*+ ,-.-*: ℎ' ୯ޠ &' Λೖྗͯ͠ɼ ͭͷϕΫτϧ ℎ' Λग़ྗ͢Δ ˠ ޙஈͷϞσϧʹ౉͢
  8. /FVSBM4FNBOUJD&ODPEFST         

    !"#$: &' ℎ)$$*+ ,-.-*: ℎ' ϝϞϦͷத͔Βண໨͢Δ औΓग़͢ εϩο τΛࢦࣔ͢͠ϕΫτϧΛ࡞Δ ˠ /'
  9. /FVSBM4FNBOUJD&ODPEFST         

    !"#$: &' ℎ)$$*+ ,-.-*: ℎ' औΓग़ͨ͠ϝϞϦεϩοτͱೖྗ୯ޠΛ ૊Έ߹ΘͤͨϕΫτϧΛ࡞Δ ˠ /'
  10. /FVSBM4FNBOUJD&ODPEFST         

    !"#$: &' ℎ)$$*+ ,-.-*: ℎ' /'͔Βग़ྗϕΫτϧΛ࡞Δ ˠ ℎ' ಡΈࠐΜͩεϩοτͷҐஔʹ৽͍͠ঢ়ଶ Λ্ॻ͖͢Δ NFNPSZVPEBUF
  11. /FVSBM4FNBOUJD&ODPEFST 0 0 M = #$ , #& , …

    , #(        0 )* 0 )* +* 1 0 0 -0 - ,* , )* #-,* .* ℎ* 0*1$ 0* 0 0
  12. /FVSBM4FNBOUJD&ODPEFST 0 0 !" = $% , $' , …

    , $)        0 *" 0 *" +, 1 0 0 -0 - -, , *, $.,, /, ℎ, !,1% !, 0 0
  13. .VMUJQMFNFNPSZBDDFTT         !"#$:

    &' .FNPSZ. ℎ)$$*+ ,-.-*: ℎ' .FNPSZ. ü ෳ਺ͷϝϞϦΛࢀরͰ͖Δ
  14. 4IBSFENFNPSZBDDFTT         !"#$:

    &',) ℎ+$$,- ./0/,: ℎ',) .FNPSZ.O ü ڞ௨ͷϝϞϦΛࢀরͰ͖Δ        !"#$: &1,) ℎ+$$,- ./0/,: ℎ1,)
  15. &YQFSJNFOUT  ͭͷ/-1λεΫͰ࣮ݧΛߦͳͬͨ • 4FOUFODFDMBTTJGJDBUJPO • "OTXFSTFOUFODFTFMFDUJPO • /BUVSBMMBOHIVBHFJOGFSFODF •

    %PDVNFOUNPEFMMJOH • /FVSBMNBDIJOFUSBOTMBUJPO
  16. 4FOUFODFDMBTTJGJDBUJPO  %BUBTFU4UBOGPSE4FOUJNFOU5SFFCBOL ஋෼ྨ ϥϕϧ෼ྨ /4& .-1 4FOUFODF 4FOUJNFOUMBCFM 

  17. "OTXFSTFOUFODFTFMFDUJPO  ࣭໰จʹରͯ͠ީิจͷத͔Βਖ਼͍͠ճ౴จΛબ୒͢ΔλεΫ %BUBTFU8JLJ2" /4&B /4&R   .-1 "OTXFS

    2VFTUJPO :FT/P TIBSFE
  18. /BUVSBMMBOHVBHFJOGFSFODF  લఏͱԾઆͷϖΞ͔ΒɼͦΕ͕߹ཧ͔ໃ६͔தཱ͔Λ ൑ఆ͢ΔλεΫ %BUBTFU4UBOGPSE/BUVSBM-BOHVBHF*OGFSFODF 4/-* EBUBTFU /4&Q /4&I 

     .-1 1SFNJTF )ZQPUIFTJT &OUBJMNFOU$POUSBEJDUJPO/FVUSBM TIBSFE BUUFOUJPO
  19. %PDVNFOUNPEFMMJOH  จॻϨϕϧͷײ৘෼ྨλεΫ %BUBTFU  *.%# Ϋϥε  :FMQ Ϋϥε

    ஈ໨ͷ/4&ͰจϕΫτϧΛɼஈ໨ ͷ/4&-45.ͰจॻϕΫτϧΛ࡞Γɼ ࠷ޙʹTPGUNBYͰ෼ྨ͢Δ /4& /4&I   /4&   /4& /4&I   -45. 
  20. /FVSBMNBDIJOFUSBOTMBUJPO  %BUBTFU*84-5&OHMJTI(FSNBO &ODPEFS%FDPEFSϞσϧʹ/4&ΛదԠ /4&FOD -45.EFD  .-1 &OHMJTI (FSNBO

    5SBOTMBUJPO BUUFOUJPO /4&FOD /4&EFD  .-1 &OHMJTI (FSNBO 5SBOTMBUJPO BUUFOUJPO TIBSFE -45.FOD -45.EFD .-1 &OHMJTI (FSNBO 5SBOTMBUJPO BUUFOUJPO
  21. $PODMVTJPO  • SFBE DPNQPTF XSJUFͷػೳΛ࣋ͭ/FVSBM4FNBOUJD&ODPEFST /4& ΛఏҊ • .VMUJQMF4IBSFENFNPSZBDDFTTػೳʹΑΓɼಛఆͷλεΫʹԠͯ͡

    ΞʔΩςΫνϟΛॊೈʹมߋͰ͖Δ • ͭͷ/-1λεΫʹ͓͍ͯɼ/4&Λ༻͍Δ͜ͱͰߴ͍ੑೳΛࣔͨ͠
  22. 

  23. /BUVSBMMBOHVBHFJOGFSFODF 

  24. 8PSEBTTPDJBUJPOHSBQI  5IFHSBQIDPNQPOFOUTDMVTUFSFEBSPVOEUIFTFNBOUJDBMMZSJDIXPSET B TJUT XBMM BVUVNO C 5ISFF QVQQJFT

    UVC WFU ϑϨʔζ΍म০ؔ܎Λଊ͍͑ͯͦ͏
  25.