Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Speaker Deck
PRO
Sign in
Sign up for free
Addressing_Trobulesome_Words_in_Neural_Machine_Translation.pdf
MARUYAMA
May 28, 2019
0
48
Addressing_Trobulesome_Words_in_Neural_Machine_Translation.pdf
MARUYAMA
May 28, 2019
Tweet
Share
More Decks by MARUYAMA
See All by MARUYAMA
vampire.pdf
tmaru0204
0
69
Misspelling_Oblivious_Word_Embedding.pdf
tmaru0204
0
52
Simple_Unsupervised_Summarization_by_Contextual_Matching.pdf
tmaru0204
0
62
Controlling_Text_Complexity_in_Neural_Machine_Translation.pdf
tmaru0204
0
61
20191028_literature-review.pdf
tmaru0204
0
63
Hint-Based_Training_for_Non-Autoregressive_Machine_Translation.pdf
tmaru0204
0
46
Soft_Contextual_Data_Augmentation_for_Neural_Machine_Translation_.pdf
tmaru0204
0
61
An_Embarrassingly_Simple_Approach_for_Transfer_Learning_from_Pretrained_Language_Models_.pdf
tmaru0204
0
56
Simple_Unsupervised_Keyphrase_Extraction_using_Sentence_Embeddings.pdf
tmaru0204
0
85
Featured
See All Featured
The MySQL Ecosystem @ GitHub 2015
samlambert
238
11k
"I'm Feeling Lucky" - Building Great Search Experiences for Today's Users (#IAC19)
danielanewman
212
20k
Build your cross-platform service in a week with App Engine
jlugia
219
17k
What’s in a name? Adding method to the madness
productmarketing
11
1.5k
Why You Should Never Use an ORM
jnunemaker
PRO
47
5.6k
The Illustrated Children's Guide to Kubernetes
chrisshort
14
35k
The Invisible Customer
myddelton
110
11k
A Philosophy of Restraint
colly
192
14k
jQuery: Nuts, Bolts and Bling
dougneiner
56
6.4k
XXLCSS - How to scale CSS and keep your sanity
sugarenia
236
1M
Making the Leap to Tech Lead
cromwellryan
113
6.9k
Facilitating Awesome Meetings
lara
29
3.9k
Transcript
"EESFTTJOH5SPVCMFTPNF8PSET JO/FVSBM.BDIJOF5SBOTMBUJPO :BOH;IBP +JBKVO;IBOH ;IPOHKVO)F $IFOHRJOH;POH BOE)VB8V &./-1 QBHFT -JUFSBUVSFSFWJFX
/BHBPLB6OJWFSTJUZPG5FDIOPMPHZ5BLVNJ.BSVZBNB
"CTUSBDU ⾣$O&O %F&Oʹ͓͍ͯɺ༁࣭ͷվળΛ֬ೝ ⾣ॲཧͷ͍͠୯ޠ 5SPVCMFTPNFXPSE ͷఆٛ ⾣χϡʔϥϧػց༁ /.5 ʹ͓͍ͯɺॲཧͷ͍͠୯ޠΛదʹ ɹऔΓѻ͓͏ͱ͍͏ࢼΈ
⾣5SPVCMFTPNFXPSEͱͦͷจ຺Λߟྀͨ͠ϝϞϦػߏ
5SPVCMFTPNF8PSE%FpOJUJPO ⾣ॲཧͷखॱ ༧Ίֶश͓͍ͯͨ͠/.5Λ༻͍ͯɺ ɹHPMEUBSHFUXPSEͷ֬Λࢉग़ ୯ޠΞϥΠϝϯτπʔϧΛ༻͍ͯɺ ɹHPMEUBSHFUXPSEʹରԠͮ͘ɺ TPVSDFXPSEΛٻΊΔ &YDFQUJPODSJUFSJPOΛຬͨͨ͠߹ɺ ɹͦͷTPVSDFXPSEΛ
USPVCMFTPNFXPSEͱΈͳ͢ ᶃ ᶄ ᶅ
5SPVCMFTPNF8PSE%FpOJUJPO ⾣&YDFQUJPO$SJUFSJB "CTPMVUF$SJUFSJPO (BQ$SJUFSJPO 3BOLJOH$SJUFSJPO 1SPCBCJMJUZ EJTUSJCVUJPO Source sentence =
(x1 , x2 , x3 ) Target sentence = (y1 , y2 , y3 ) HPMETUBOEBSE
5SPVCMFTPNF8PSE%FpOJUJPO ⾣&YDFQUJPO$SJUFSJB "CTPMVUF$SJUFSJPO HPMETUBOEBSEͷग़ྗ֬ͷᮢ PN i (yi ) < p0
1SPCBCJMJUZ EJTUSJCVUJPO Source sentence = (x1 , x2 , x3 ) Target sentence = (y1 , y2 , y3 ) HPMETUBOEBSE PN 1 (y1 ) = 0.80 PN 2 (y2 ) = 0.31 PN 3 (y3 ) = 0.20 Y Y ͕USPVCMFTPNFXPSE ͳΒɺ p0 = 0.50 FH
5SPVCMFTPNF8PSE%FpOJUJPO ⾣&YDFQUJPO$SJUFSJB ͱͷࠩͷᮢ gi (yi ) = max(PN i (yi
)) − PN i (yi ) 1SPCBCJMJUZ EJTUSJCVUJPO Source sentence = (x1 , x2 , x3 ) Target sentence = (y1 , y2 , y3 ) HPMETUBOEBSE g1 (y1 ) = 0.80 − 0.80 = 0.00 g2 (y2 ) = 0.35 − 0.31 = 0.04 g3 (y3 ) = 0.75 − 0.20 = 0.55 Y ͕USPVCMFTPNFXPSE ͳΒɺ g0 = 0.10 FH max(PN i (yi )) PN i (yi ) gi (yi ) > g0 (BQ$SJUFSJPO
5SPVCMFTPNF8PSE%FpOJUJPO ⾣&YDFQUJPO$SJUFSJB ग़ྗ֬ॱҐͷᮢ 1SPCBCJMJUZ EJTUSJCVUJPO Source sentence = (x1 ,
x2 , x3 ) Target sentence = (y1 , y2 , y3 ) HPMETUBOEBSE rank(y1 ) = 1 rank(y2 ) = 3 rank(y3 ) = 2 Y ͕USPVCMFTPNFXPSE ͳΒɺ rank0 = 2 FH rank(yi ) > rank0 3BOLJOH$SJUFSJPO
*OUFHSBUJOH$POUFYUVBM.FNPSZJOUP/.5 ⾣༁ॲཧͷྲྀΕ NFNPSZΑΓUSPVCMFTPNFXPSEΛநग़ DPOUFYUVBMTJNJMBSJUZΛܭࢉ NFNPSZQSFEJDUFEQSPCBCJMJUZͷܭࢉ ग़ྗ֬Λܭࢉ ᶃ ᶄ ᶅ
ᶆ
*OUFHSBUJOH$POUFYUVBM.FNPSZJOUP/.5 ⾣༁ॲཧͷྲྀΕ NFNPSZΑΓUSPVCMFTPNFXPSEΛநग़ ᶃ sm USPVCMFTPNFTPVSDFXPSE tn HPMEUBSHFUXPSE c(sm
, tn ) IJEEFOTUBUFPGFODPEFS pL(sm , tn ) MFYJDPOUSBOTMBUJPOQSPCBCJMJUZ
*OUFHSBUJOH$POUFYUVBM.FNPSZJOUP/.5 ⾣༁ॲཧͷྲྀΕ DPOUFYUVBMTJNJMBSJUZΛܭࢉ NFNPSZQSFEJDUFEQSPCBCJMJUZͷܭࢉ ग़ྗ֬Λܭࢉ ᶅ ᶆ ᶄ
&YQFSJNFOUBM4FUUJOHT ⾣%BUBTFUT 5SBJOJOH-%$DPSQVT .TFOUFODFQBJST 7BMJEBUJPO/*45 5FTU/*45 ⾣.PEFMT #BTFMJOF/.5XJUIHMPCBMBUUFOUJPO "SUIVS/.5
MFYJDBMUSBOTMBUJPO 9 .&.ఏҊख๏
3FTVMUT ⾣ϝϞϦػߏͷޮՌ
3FTVMUT ⾣&YDFQUJPODSJUFSJBʹΑΔҧ͍
3FTVMUT ⾣සޠ ᐆດੑͷ͋ΔޠΛؚΉจͰͷධՁ -PXग़ݱճ͕ճҎԼͷ୯ޠ "NCΤϯτϩϐʔ͕ᮢҎ্ − K ∑ k=1 pL
k logpL k > E0 (E0 = 1.5)
3FTVMUT ⾣සޠ ᐆດੑͷ͋ΔޠΛؚΉจͰͷධՁ #-&6είΞͷൺֱ
3FTVMUT ⾣සޠ ᐆດੑͷ͋ΔޠΛؚΉจͰͷධՁ 5XPSEUSPVCMFTPNFXPSE &SSPS#BTFMJOFʹ͓͚Δؒҧͬͨग़ྗ 3FDUJGZఏҊख๏ಋೖʹΑΓमਖ਼Ͱ͖ͨ %FUFSJPఏҊख๏ಋೖʹΑΓ૿͑ͨؒҧ͍ͷ
$PODMVTJPO ⾣$O&O %F&Oʹ͓͍ͯɺ༁࣭ͷվળΛ֬ೝ ⾣ॲཧͷ͍͠୯ޠ 5SPVCMFTPNFXPSE ͷఆٛ ⾣χϡʔϥϧػց༁ /.5 ʹ͓͍ͯɺॲཧͷ͍͠୯ޠΛదʹ ɹऔΓѻ͓͏ͱ͍͏ࢼΈ
⾣5SPVCMFTPNFXPSEͱͦͷจ຺Λߟྀͨ͠ϝϞϦػߏ