Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Addressing_Trobulesome_Words_in_Neural_Machine_Translation.pdf

A3ea3bc5dde6ae2dd6eae71da9c418b0?s=47 MARUYAMA
May 28, 2019
0
48

 Addressing_Trobulesome_Words_in_Neural_Machine_Translation.pdf

A3ea3bc5dde6ae2dd6eae71da9c418b0?s=128

MARUYAMA

May 28, 2019
Tweet

More Decks by MARUYAMA

See All by MARUYAMA
vampire.pdf
A3ea3bc5dde6ae2dd6eae71da9c418b0?s=48 tmaru0204
0
69
Misspelling_Oblivious_Word_Embedding.pdf
A3ea3bc5dde6ae2dd6eae71da9c418b0?s=48 tmaru0204
0
52
Simple_Unsupervised_Summarization_by_Contextual_Matching.pdf
A3ea3bc5dde6ae2dd6eae71da9c418b0?s=48 tmaru0204
0
62
Controlling_Text_Complexity_in_Neural_Machine_Translation.pdf
A3ea3bc5dde6ae2dd6eae71da9c418b0?s=48 tmaru0204
0
61
20191028_literature-review.pdf
A3ea3bc5dde6ae2dd6eae71da9c418b0?s=48 tmaru0204
0
63
Hint-Based_Training_for_Non-Autoregressive_Machine_Translation.pdf
A3ea3bc5dde6ae2dd6eae71da9c418b0?s=48 tmaru0204
0
46
Soft_Contextual_Data_Augmentation_for_Neural_Machine_Translation_.pdf
A3ea3bc5dde6ae2dd6eae71da9c418b0?s=48 tmaru0204
0
61
An_Embarrassingly_Simple_Approach_for_Transfer_Learning_from_Pretrained_Language_Models_.pdf
A3ea3bc5dde6ae2dd6eae71da9c418b0?s=48 tmaru0204
0
56
Simple_Unsupervised_Keyphrase_Extraction_using_Sentence_Embeddings.pdf
A3ea3bc5dde6ae2dd6eae71da9c418b0?s=48 tmaru0204
0
85

Featured

See All Featured
The MySQL Ecosystem @ GitHub 2015
Be9caeb9d4ef9944d151af909063ed6e?s=48 samlambert
238
11k
"I'm Feeling Lucky" - Building Great Search Experiences for Today's Users (#IAC19)
3d4519fd34b76fb265fc0237f3792bd4?s=48 danielanewman
212
20k
Build your cross-platform service in a week with App Engine
5f83ef806155b403c3393160ce51f955?s=48 jlugia
219
17k
What’s in a name? Adding method to the madness
C0390e8b11079f8761c0cd3274ed61cb?s=48 productmarketing
11
1.5k
Why You Should Never Use an ORM
88bc30284c6a424b63a92aad27d0ba36?s=48 jnunemaker
PRO
47
5.6k
The Illustrated Children's Guide to Kubernetes
0438ae60cde4c7add6f9da48f28c15cc?s=48 chrisshort
14
35k
The Invisible Customer
4262b9cfacfb6b2c77f23e1d0310cd3e?s=48 myddelton
110
11k
A Philosophy of Restraint
C9b18d9dff88a9dd2393364c2b3b21bd?s=48 colly
192
14k
jQuery: Nuts, Bolts and Bling
9cde37f47e4a800ea081ea42de8d749a?s=48 dougneiner
56
6.4k
XXLCSS - How to scale CSS and keep your sanity
1b8ad785acdd1ce1c99914b1c2a4e10e?s=48 sugarenia
236
1M
Making the Leap to Tech Lead
32de0bd2ba869609d26fd052a4622778?s=48 cromwellryan
113
6.9k
Facilitating Awesome Meetings
245cee81a9c424266e5e401d844ea881?s=48 lara
29
3.9k

Transcript

  1. "EESFTTJOH5SPVCMFTPNF8PSET JO/FVSBM.BDIJOF5SBOTMBUJPO :BOH;IBP +JBKVO;IBOH ;IPOHKVO)F $IFOHRJOH;POH BOE)VB8V &./-1 QBHFT -JUFSBUVSFSFWJFX

    /BHBPLB6OJWFSTJUZPG5FDIOPMPHZ5BLVNJ.BSVZBNB

  2. "CTUSBDU ⾣$O&O %F&Oʹ͓͍ͯɺ຋༁඼࣭ͷվળΛ֬ೝ ⾣ॲཧͷ೉͍͠୯ޠ 5SPVCMFTPNFXPSE ͷఆٛ ⾣χϡʔϥϧػց຋༁ /.5 ʹ͓͍ͯɺॲཧͷ೉͍͠୯ޠΛద੾ʹ ɹऔΓѻ͓͏ͱ͍͏ࢼΈ

    ⾣5SPVCMFTPNFXPSEͱͦͷจ຺Λߟྀͨ͠ϝϞϦػߏ

  3. 5SPVCMFTPNF8PSE%FpOJUJPO ⾣ॲཧͷखॱ  ༧Ίֶश͓͍ͯͨ͠/.5Λ༻͍ͯɺ ɹHPMEUBSHFUXPSEͷ֬཰Λࢉग़ ୯ޠΞϥΠϝϯτπʔϧΛ༻͍ͯɺ ɹHPMEUBSHFUXPSEʹରԠͮ͘ɺ TPVSDFXPSEΛٻΊΔ &YDFQUJPODSJUFSJPOΛຬͨͨ͠৔߹ɺ ɹͦͷTPVSDFXPSEΛ

    USPVCMFTPNFXPSEͱΈͳ͢ ᶃ ᶄ ᶅ

  4. 5SPVCMFTPNF8PSE%FpOJUJPO ⾣&YDFQUJPO$SJUFSJB "CTPMVUF$SJUFSJPO (BQ$SJUFSJPO 3BOLJOH$SJUFSJPO 1SPCBCJMJUZ EJTUSJCVUJPO Source sentence =

    (x1 , x2 , x3 ) Target sentence = (y1 , y2 , y3 ) HPMETUBOEBSE

  5. 5SPVCMFTPNF8PSE%FpOJUJPO ⾣&YDFQUJPO$SJUFSJB "CTPMVUF$SJUFSJPO HPMETUBOEBSEͷग़ྗ֬཰ͷᮢ஋ PN i (yi ) < p0

    1SPCBCJMJUZ EJTUSJCVUJPO Source sentence = (x1 , x2 , x3 ) Target sentence = (y1 , y2 , y3 ) HPMETUBOEBSE PN 1 (y1 ) = 0.80 PN 2 (y2 ) = 0.31 PN 3 (y3 ) = 0.20 Y Y ͕USPVCMFTPNFXPSE ͳΒ͹ɺ p0 = 0.50 FH

  6. 5SPVCMFTPNF8PSE%FpOJUJPO ⾣&YDFQUJPO$SJUFSJB ͱͷࠩͷᮢ஋ gi (yi ) = max(PN i (yi

    )) − PN i (yi ) 1SPCBCJMJUZ EJTUSJCVUJPO Source sentence = (x1 , x2 , x3 ) Target sentence = (y1 , y2 , y3 ) HPMETUBOEBSE g1 (y1 ) = 0.80 − 0.80 = 0.00 g2 (y2 ) = 0.35 − 0.31 = 0.04 g3 (y3 ) = 0.75 − 0.20 = 0.55 Y ͕USPVCMFTPNFXPSE ͳΒ͹ɺ g0 = 0.10 FH max(PN i (yi )) PN i (yi ) gi (yi ) > g0 (BQ$SJUFSJPO

  7. 5SPVCMFTPNF8PSE%FpOJUJPO ⾣&YDFQUJPO$SJUFSJB ग़ྗ֬཰ॱҐͷᮢ஋ 1SPCBCJMJUZ EJTUSJCVUJPO Source sentence = (x1 ,

    x2 , x3 ) Target sentence = (y1 , y2 , y3 ) HPMETUBOEBSE rank(y1 ) = 1 rank(y2 ) = 3 rank(y3 ) = 2 Y ͕USPVCMFTPNFXPSE ͳΒ͹ɺ rank0 = 2 FH rank(yi ) > rank0 3BOLJOH$SJUFSJPO         

  8. *OUFHSBUJOH$POUFYUVBM.FNPSZJOUP/.5 ⾣຋༁ॲཧͷྲྀΕ  NFNPSZΑΓUSPVCMFTPNFXPSEΛநग़ DPOUFYUVBMTJNJMBSJUZΛܭࢉ NFNPSZQSFEJDUFEQSPCBCJMJUZͷܭࢉ ग़ྗ֬཰Λܭࢉ ᶃ ᶄ ᶅ

    ᶆ

  9. *OUFHSBUJOH$POUFYUVBM.FNPSZJOUP/.5 ⾣຋༁ॲཧͷྲྀΕ  NFNPSZΑΓUSPVCMFTPNFXPSEΛநग़ ᶃ sm USPVCMFTPNFTPVSDFXPSE tn HPMEUBSHFUXPSE c(sm

    , tn ) IJEEFOTUBUFPGFODPEFS pL(sm , tn ) MFYJDPOUSBOTMBUJPOQSPCBCJMJUZ

  10. *OUFHSBUJOH$POUFYUVBM.FNPSZJOUP/.5 ⾣຋༁ॲཧͷྲྀΕ DPOUFYUVBMTJNJMBSJUZΛܭࢉ NFNPSZQSFEJDUFEQSPCBCJMJUZͷܭࢉ ग़ྗ֬཰Λܭࢉ ᶅ ᶆ ᶄ

  11. &YQFSJNFOUBM4FUUJOHT ⾣%BUBTFUT 5SBJOJOH-%$DPSQVT .TFOUFODFQBJST 7BMJEBUJPO/*45 5FTU/*45  ⾣.PEFMT #BTFMJOF/.5XJUIHMPCBMBUUFOUJPO "SUIVS/.5

    MFYJDBMUSBOTMBUJPO 9 .&.ఏҊख๏

  12. 3FTVMUT ⾣ϝϞϦػߏͷޮՌ

  13. 3FTVMUT ⾣&YDFQUJPODSJUFSJBʹΑΔҧ͍

  14. 3FTVMUT ⾣௿ස౓ޠ ᐆດੑͷ͋ΔޠΛؚΉจͰͷධՁ -PXग़ݱճ਺͕ճҎԼͷ୯ޠ "NCΤϯτϩϐʔ͕ᮢ஋Ҏ্ − K ∑ k=1 pL

    k logpL k > E0 (E0 = 1.5)

  15. 3FTVMUT ⾣௿ස౓ޠ ᐆດੑͷ͋ΔޠΛؚΉจͰͷධՁ #-&6είΞͷൺֱ

  16. 3FTVMUT ⾣௿ස౓ޠ ᐆດੑͷ͋ΔޠΛؚΉจͰͷධՁ 5XPSEUSPVCMFTPNFXPSE &SSPS#BTFMJOFʹ͓͚Δؒҧͬͨग़ྗ਺ 3FDUJGZఏҊख๏ಋೖʹΑΓमਖ਼Ͱ͖ͨ਺ %FUFSJPఏҊख๏ಋೖʹΑΓ૿͑ͨؒҧ͍ͷ਺

  17. $PODMVTJPO ⾣$O&O %F&Oʹ͓͍ͯɺ຋༁඼࣭ͷվળΛ֬ೝ ⾣ॲཧͷ೉͍͠୯ޠ 5SPVCMFTPNFXPSE ͷఆٛ ⾣χϡʔϥϧػց຋༁ /.5 ʹ͓͍ͯɺॲཧͷ೉͍͠୯ޠΛద੾ʹ ɹऔΓѻ͓͏ͱ͍͏ࢼΈ

    ⾣5SPVCMFTPNFXPSEͱͦͷจ຺Λߟྀͨ͠ϝϞϦػߏ