Addressing_Trobulesome_Words_in_Neural_Machine_Translation.pdf

Transcript

1. "EESFTTJOH
   5SPVCMFTPNF
   8PSET
   
   JO
   /FVSBM
   .BDIJOF
   5SBOTMBUJPO
   
   :BOH
   ;IBP
   +JBKVO
   ;IBOH
   ;IPOHKVO
   )F
   $IFOHRJOH
   ;POH
   BOE
   )VB
   8V
   
   &./-1
   QBHFT
    -JUFSBUVSF
   SFWJFX /BHBPLB
   6OJWFSTJUZ
   PG
   5FDIOPMPHZ
   
   
   
   5BLVNJ
   .BSVZBNB
2. "CTUSBDU ⾣
   $O&O
   %F&Oʹ͓͍ͯɺ຋༁඼࣭ͷվળΛ֬ೝ ⾣
   ॲཧͷ೉͍͠୯ޠ 5SPVCMFTPNF
   XPSE ͷఆٛ ⾣
   χϡʔϥϧػց຋༁ /.5 ʹ͓͍ͯɺॲཧͷ೉͍͠୯ޠΛద੾ʹ
   ɹ
   
   औΓѻ͓͏ͱ͍͏ࢼΈ ⾣
   5SPVCMFTPNF
   XPSEͱͦͷจ຺Λߟྀͨ͠ϝϞϦػߏ
3. 5SPVCMFTPNF
   8PSE
   %FpOJUJPO ⾣
   ॲཧͷखॱ  ༧Ίֶश͓͍ͯͨ͠/.5Λ༻͍ͯɺ
   ɹ
   HPME
   UBSHFU
   XPSEͷ֬཰Λࢉग़ 
   ୯ޠΞϥΠϝϯτπʔϧΛ༻͍ͯɺ
   ɹ
   HPME
   UBSHFU
   XPSEʹରԠͮ͘ɺ
   
   
   
   
   TPVSDF
   XPSEΛٻΊΔ 
   
   &YDFQUJPO
   DSJUFSJPOΛຬͨͨ͠৔߹ɺ
   ɹ
   ͦͷTPVSDF
   XPSEΛ
   
   
   
   
   USPVCMFTPNF
   XPSEͱΈͳ͢ ᶃ ᶄ ᶅ
4. 5SPVCMFTPNF
   8PSE
   %FpOJUJPO ⾣
   &YDFQUJPO
   $SJUFSJB 
   "CTPMVUF
   $SJUFSJPO 
   (BQ
   $SJUFSJPO 
   3BOLJOH
   $SJUFSJPO 1SPCBCJMJUZ
   
   EJTUSJCVUJPO Source sentence = (x1 , x2 , x3 ) Target sentence = (y1 , y2 , y3 ) 
   HPME
   TUBOEBSE
5. 5SPVCMFTPNF
   8PSE
   %FpOJUJPO ⾣
   &YDFQUJPO
   $SJUFSJB 
   "CTPMVUF
   $SJUFSJPO HPME
   TUBOEBSEͷग़ྗ֬཰ͷᮢ஋ PN i (yi ) < p0 1SPCBCJMJUZ
   
   EJTUSJCVUJPO Source sentence = (x1 , x2 , x3 ) Target sentence = (y1 , y2 , y3 ) 
   HPME
   TUBOEBSE PN 1 (y1 ) = 0.80 PN 2 (y2 ) = 0.31 PN 3 (y3 ) = 0.20 Y
   Y ͕USPVCMFTPNF
   XPSE ͳΒ͹ɺ p0 = 0.50 FH
6. 5SPVCMFTPNF
   8PSE
   %FpOJUJPO ⾣
   &YDFQUJPO
   $SJUFSJB ͱ
   
   
   
   
   
   
   
   ͷࠩͷᮢ஋ gi (yi ) = max(PN i (yi )) − PN i (yi ) 1SPCBCJMJUZ
   
   EJTUSJCVUJPO Source sentence = (x1 , x2 , x3 ) Target sentence = (y1 , y2 , y3 ) 
   HPME
   TUBOEBSE g1 (y1 ) = 0.80 − 0.80 = 0.00 g2 (y2 ) = 0.35 − 0.31 = 0.04 g3 (y3 ) = 0.75 − 0.20 = 0.55 Y ͕USPVCMFTPNF
   XPSE ͳΒ͹ɺ g0 = 0.10 FH max(PN i (yi )) PN i (yi ) gi (yi ) > g0 
   (BQ
   $SJUFSJPO
7. 5SPVCMFTPNF
   8PSE
   %FpOJUJPO ⾣
   &YDFQUJPO
   $SJUFSJB ग़ྗ֬཰ॱҐͷᮢ஋ 1SPCBCJMJUZ
   
   EJTUSJCVUJPO Source sentence = (x1 , x2 , x3 ) Target sentence = (y1 , y2 , y3 ) 
   HPME
   TUBOEBSE rank(y1 ) = 1 rank(y2 ) = 3 rank(y3 ) = 2 Y ͕USPVCMFTPNF
   XPSE ͳΒ͹ɺ rank0 = 2 FH rank(yi ) > rank0 
   3BOLJOH
   $SJUFSJPO         
8. *OUFHSBUJOH
   $POUFYUVBM
   .FNPSZ
   JOUP
   /.5 ⾣
   ຋༁ॲཧͷྲྀΕ 
   NFNPSZΑΓUSPVCMFTPNF
   XPSE
   Λநग़ 
   DPOUFYUVBM
   TJNJMBSJUZΛܭࢉ 
   NFNPSZ
   QSFEJDUFE
   QSPCBCJMJUZͷܭࢉ 
   ग़ྗ֬཰Λܭࢉ ᶃ ᶄ ᶅ ᶆ
9. *OUFHSBUJOH
   $POUFYUVBM
   .FNPSZ
   JOUP
   /.5 ⾣
   ຋༁ॲཧͷྲྀΕ 
   NFNPSZΑΓUSPVCMFTPNF
   XPSE
   Λநग़ ᶃ sm 
   USPVCMFTPNF
   TPVSDF
   XPSE tn 
   HPME
   UBSHFU
   XPSE c(sm , tn ) 
   IJEEFO
   TUBUF
   PG
   FODPEFS pL(sm , tn ) 
   MFYJDPO
   USBOTMBUJPO
   QSPCBCJMJUZ
10. *OUFHSBUJOH
    $POUFYUVBM
    .FNPSZ
    JOUP
    /.5 ⾣
    ຋༁ॲཧͷྲྀΕ 
    DPOUFYUVBM
    TJNJMBSJUZΛܭࢉ 
    NFNPSZ
    QSFEJDUFE
    QSPCBCJMJUZͷܭࢉ 
    ग़ྗ֬཰Λܭࢉ ᶅ ᶆ ᶄ
11. &YQFSJNFOUBM
    4FUUJOHT ⾣
    %BUBTFUT 5SBJOJOH
    -%$
    DPSQVT
    .
    TFOUFODF
    QBJST 7BMJEBUJPO
    /*45
     5FTU
    /*45
    
    
    
     ⾣
    .PEFMT #BTFMJOF
    /.5
    XJUI
    HMPCBM
    BUUFOUJPO "SUIVS
    /.5
    
    MFYJDBM
    USBOTMBUJPO
    9 .&.
    ఏҊख๏
12. 3FTVMUT ⾣
    ϝϞϦػߏͷޮՌ
13. 3FTVMUT ⾣
    &YDFQUJPO
    DSJUFSJBʹΑΔҧ͍
14. 3FTVMUT ⾣
    ௿ස౓ޠ
    ᐆດੑͷ͋ΔޠΛؚΉจͰͷධՁ -PX
    ग़ݱճ਺͕ճҎԼͷ୯ޠ "NC
    Τϯτϩϐʔ͕ᮢ஋Ҏ্ − K ∑ k=1 pL k logpL k > E0 (E0 = 1.5)
15. 3FTVMUT ⾣
    ௿ස౓ޠ
    ᐆດੑͷ͋ΔޠΛؚΉจͰͷධՁ #-&6είΞͷൺֱ
16. 3FTVMUT ⾣
    ௿ස౓ޠ
    ᐆດੑͷ͋ΔޠΛؚΉจͰͷධՁ 5XPSE
    USPVCMFTPNF
    XPSE &SSPS
    #BTFMJOFʹ͓͚Δؒҧͬͨग़ྗ਺ 3FDUJGZ
    ఏҊख๏ಋೖʹΑΓमਖ਼Ͱ͖ͨ਺ %FUFSJP
    ఏҊख๏ಋೖʹΑΓ૿͑ͨؒҧ͍ͷ਺
17. $PODMVTJPO ⾣
    $O&O
    %F&Oʹ͓͍ͯɺ຋༁඼࣭ͷվળΛ֬ೝ ⾣
    ॲཧͷ೉͍͠୯ޠ 5SPVCMFTPNF
    XPSE ͷఆٛ ⾣
    χϡʔϥϧػց຋༁ /.5 ʹ͓͍ͯɺॲཧͷ೉͍͠୯ޠΛద੾ʹ
    ɹ
    
    औΓѻ͓͏ͱ͍͏ࢼΈ ⾣
    5SPVCMFTPNF
    XPSEͱͦͷจ຺Λߟྀͨ͠ϝϞϦػߏ