Addressing_Trobulesome_Words_in_Neural_Machine_Translation.pdf

A3ea3bc5dde6ae2dd6eae71da9c418b0?s=47 MARUYAMA
May 28, 2019
33

 Addressing_Trobulesome_Words_in_Neural_Machine_Translation.pdf

A3ea3bc5dde6ae2dd6eae71da9c418b0?s=128

MARUYAMA

May 28, 2019
Tweet

Transcript

  1. "EESFTTJOH5SPVCMFTPNF8PSET JO/FVSBM.BDIJOF5SBOTMBUJPO :BOH;IBP +JBKVO;IBOH ;IPOHKVO)F $IFOHRJOH;POH BOE)VB8V &./-1 QBHFT -JUFSBUVSFSFWJFX

    /BHBPLB6OJWFSTJUZPG5FDIOPMPHZ5BLVNJ.BSVZBNB
  2. "CTUSBDU ⾣$O&O %F&Oʹ͓͍ͯɺ຋༁඼࣭ͷվળΛ֬ೝ ⾣ॲཧͷ೉͍͠୯ޠ 5SPVCMFTPNFXPSE ͷఆٛ ⾣χϡʔϥϧػց຋༁ /.5 ʹ͓͍ͯɺॲཧͷ೉͍͠୯ޠΛద੾ʹ ɹऔΓѻ͓͏ͱ͍͏ࢼΈ

    ⾣5SPVCMFTPNFXPSEͱͦͷจ຺Λߟྀͨ͠ϝϞϦػߏ
  3. 5SPVCMFTPNF8PSE%FpOJUJPO ⾣ॲཧͷखॱ  ༧Ίֶश͓͍ͯͨ͠/.5Λ༻͍ͯɺ ɹHPMEUBSHFUXPSEͷ֬཰Λࢉग़ ୯ޠΞϥΠϝϯτπʔϧΛ༻͍ͯɺ ɹHPMEUBSHFUXPSEʹରԠͮ͘ɺ TPVSDFXPSEΛٻΊΔ &YDFQUJPODSJUFSJPOΛຬͨͨ͠৔߹ɺ ɹͦͷTPVSDFXPSEΛ

    USPVCMFTPNFXPSEͱΈͳ͢ ᶃ ᶄ ᶅ
  4. 5SPVCMFTPNF8PSE%FpOJUJPO ⾣&YDFQUJPO$SJUFSJB "CTPMVUF$SJUFSJPO (BQ$SJUFSJPO 3BOLJOH$SJUFSJPO 1SPCBCJMJUZ EJTUSJCVUJPO Source sentence =

    (x1 , x2 , x3 ) Target sentence = (y1 , y2 , y3 ) HPMETUBOEBSE
  5. 5SPVCMFTPNF8PSE%FpOJUJPO ⾣&YDFQUJPO$SJUFSJB "CTPMVUF$SJUFSJPO HPMETUBOEBSEͷग़ྗ֬཰ͷᮢ஋ PN i (yi ) < p0

    1SPCBCJMJUZ EJTUSJCVUJPO Source sentence = (x1 , x2 , x3 ) Target sentence = (y1 , y2 , y3 ) HPMETUBOEBSE PN 1 (y1 ) = 0.80 PN 2 (y2 ) = 0.31 PN 3 (y3 ) = 0.20 Y Y ͕USPVCMFTPNFXPSE ͳΒ͹ɺ p0 = 0.50 FH
  6. 5SPVCMFTPNF8PSE%FpOJUJPO ⾣&YDFQUJPO$SJUFSJB ͱͷࠩͷᮢ஋ gi (yi ) = max(PN i (yi

    )) − PN i (yi ) 1SPCBCJMJUZ EJTUSJCVUJPO Source sentence = (x1 , x2 , x3 ) Target sentence = (y1 , y2 , y3 ) HPMETUBOEBSE g1 (y1 ) = 0.80 − 0.80 = 0.00 g2 (y2 ) = 0.35 − 0.31 = 0.04 g3 (y3 ) = 0.75 − 0.20 = 0.55 Y ͕USPVCMFTPNFXPSE ͳΒ͹ɺ g0 = 0.10 FH max(PN i (yi )) PN i (yi ) gi (yi ) > g0 (BQ$SJUFSJPO
  7. 5SPVCMFTPNF8PSE%FpOJUJPO ⾣&YDFQUJPO$SJUFSJB ग़ྗ֬཰ॱҐͷᮢ஋ 1SPCBCJMJUZ EJTUSJCVUJPO Source sentence = (x1 ,

    x2 , x3 ) Target sentence = (y1 , y2 , y3 ) HPMETUBOEBSE rank(y1 ) = 1 rank(y2 ) = 3 rank(y3 ) = 2 Y ͕USPVCMFTPNFXPSE ͳΒ͹ɺ rank0 = 2 FH rank(yi ) > rank0 3BOLJOH$SJUFSJPO         
  8. *OUFHSBUJOH$POUFYUVBM.FNPSZJOUP/.5 ⾣຋༁ॲཧͷྲྀΕ  NFNPSZΑΓUSPVCMFTPNFXPSEΛநग़ DPOUFYUVBMTJNJMBSJUZΛܭࢉ NFNPSZQSFEJDUFEQSPCBCJMJUZͷܭࢉ ग़ྗ֬཰Λܭࢉ ᶃ ᶄ ᶅ

  9. *OUFHSBUJOH$POUFYUVBM.FNPSZJOUP/.5 ⾣຋༁ॲཧͷྲྀΕ  NFNPSZΑΓUSPVCMFTPNFXPSEΛநग़ ᶃ sm USPVCMFTPNFTPVSDFXPSE tn HPMEUBSHFUXPSE c(sm

    , tn ) IJEEFOTUBUFPGFODPEFS pL(sm , tn ) MFYJDPOUSBOTMBUJPOQSPCBCJMJUZ
  10. *OUFHSBUJOH$POUFYUVBM.FNPSZJOUP/.5 ⾣຋༁ॲཧͷྲྀΕ DPOUFYUVBMTJNJMBSJUZΛܭࢉ NFNPSZQSFEJDUFEQSPCBCJMJUZͷܭࢉ ग़ྗ֬཰Λܭࢉ ᶅ ᶆ ᶄ

  11. &YQFSJNFOUBM4FUUJOHT ⾣%BUBTFUT 5SBJOJOH-%$DPSQVT .TFOUFODFQBJST 7BMJEBUJPO/*45 5FTU/*45  ⾣.PEFMT #BTFMJOF/.5XJUIHMPCBMBUUFOUJPO "SUIVS/.5

    MFYJDBMUSBOTMBUJPO 9 .&.ఏҊख๏
  12. 3FTVMUT ⾣ϝϞϦػߏͷޮՌ

  13. 3FTVMUT ⾣&YDFQUJPODSJUFSJBʹΑΔҧ͍

  14. 3FTVMUT ⾣௿ස౓ޠ ᐆດੑͷ͋ΔޠΛؚΉจͰͷධՁ -PXग़ݱճ਺͕ճҎԼͷ୯ޠ "NCΤϯτϩϐʔ͕ᮢ஋Ҏ্ − K ∑ k=1 pL

    k logpL k > E0 (E0 = 1.5)
  15. 3FTVMUT ⾣௿ස౓ޠ ᐆດੑͷ͋ΔޠΛؚΉจͰͷධՁ #-&6είΞͷൺֱ

  16. 3FTVMUT ⾣௿ස౓ޠ ᐆດੑͷ͋ΔޠΛؚΉจͰͷධՁ 5XPSEUSPVCMFTPNFXPSE &SSPS#BTFMJOFʹ͓͚Δؒҧͬͨग़ྗ਺ 3FDUJGZఏҊख๏ಋೖʹΑΓमਖ਼Ͱ͖ͨ਺ %FUFSJPఏҊख๏ಋೖʹΑΓ૿͑ͨؒҧ͍ͷ਺

  17. $PODMVTJPO ⾣$O&O %F&Oʹ͓͍ͯɺ຋༁඼࣭ͷվળΛ֬ೝ ⾣ॲཧͷ೉͍͠୯ޠ 5SPVCMFTPNFXPSE ͷఆٛ ⾣χϡʔϥϧػց຋༁ /.5 ʹ͓͍ͯɺॲཧͷ೉͍͠୯ޠΛద੾ʹ ɹऔΓѻ͓͏ͱ͍͏ࢼΈ

    ⾣5SPVCMFTPNFXPSEͱͦͷจ຺Λߟྀͨ͠ϝϞϦػߏ