Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Sign up for free
Menu
Search
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Pricing
Search
Sign in
Sign up for free
Addressing_Trobulesome_Words_in_Neural_Machine_...
Search
MARUYAMA
May 28, 2019
190
0
Share
Embed
Copy iframe code
Copy JS code
Copy link
Start on current slide
Addressing_Trobulesome_Words_in_Neural_Machine_Translation.pdf
MARUYAMA
May 28, 2019
Transcript
17 slides
"EESFTTJOH5SPVCMFTPNF8PSET JO/FVSBM.BDIJOF5SBOTMBUJPO :BOH;IBP +JBKVO;IBOH ;IPOHKVO)F $IFOHRJOH;POH BOE)VB8V &./-1 QBHFT -JUFSBUVSFSFWJFX
/BHBPLB6OJWFSTJUZPG5FDIOPMPHZ5BLVNJ.BSVZBNB
"CTUSBDU ⾣$O&O %F&Oʹ͓͍ͯɺ༁࣭ͷվળΛ֬ೝ ⾣ॲཧͷ͍͠୯ޠ 5SPVCMFTPNFXPSE ͷఆٛ ⾣χϡʔϥϧػց༁ /.5 ʹ͓͍ͯɺॲཧͷ͍͠୯ޠΛదʹ ɹऔΓѻ͓͏ͱ͍͏ࢼΈ
⾣5SPVCMFTPNFXPSEͱͦͷจ຺Λߟྀͨ͠ϝϞϦػߏ
5SPVCMFTPNF8PSE%FpOJUJPO ⾣ॲཧͷखॱ ༧Ίֶश͓͍ͯͨ͠/.5Λ༻͍ͯɺ ɹHPMEUBSHFUXPSEͷ֬Λࢉग़ ୯ޠΞϥΠϝϯτπʔϧΛ༻͍ͯɺ ɹHPMEUBSHFUXPSEʹରԠͮ͘ɺ TPVSDFXPSEΛٻΊΔ &YDFQUJPODSJUFSJPOΛຬͨͨ͠߹ɺ ɹͦͷTPVSDFXPSEΛ
USPVCMFTPNFXPSEͱΈͳ͢ ᶃ ᶄ ᶅ
5SPVCMFTPNF8PSE%FpOJUJPO ⾣&YDFQUJPO$SJUFSJB "CTPMVUF$SJUFSJPO (BQ$SJUFSJPO 3BOLJOH$SJUFSJPO 1SPCBCJMJUZ EJTUSJCVUJPO Source sentence =
(x1 , x2 , x3 ) Target sentence = (y1 , y2 , y3 ) HPMETUBOEBSE
5SPVCMFTPNF8PSE%FpOJUJPO ⾣&YDFQUJPO$SJUFSJB "CTPMVUF$SJUFSJPO HPMETUBOEBSEͷग़ྗ֬ͷᮢ PN i (yi ) < p0
1SPCBCJMJUZ EJTUSJCVUJPO Source sentence = (x1 , x2 , x3 ) Target sentence = (y1 , y2 , y3 ) HPMETUBOEBSE PN 1 (y1 ) = 0.80 PN 2 (y2 ) = 0.31 PN 3 (y3 ) = 0.20 Y Y ͕USPVCMFTPNFXPSE ͳΒɺ p0 = 0.50 FH
5SPVCMFTPNF8PSE%FpOJUJPO ⾣&YDFQUJPO$SJUFSJB ͱͷࠩͷᮢ gi (yi ) = max(PN i (yi
)) − PN i (yi ) 1SPCBCJMJUZ EJTUSJCVUJPO Source sentence = (x1 , x2 , x3 ) Target sentence = (y1 , y2 , y3 ) HPMETUBOEBSE g1 (y1 ) = 0.80 − 0.80 = 0.00 g2 (y2 ) = 0.35 − 0.31 = 0.04 g3 (y3 ) = 0.75 − 0.20 = 0.55 Y ͕USPVCMFTPNFXPSE ͳΒɺ g0 = 0.10 FH max(PN i (yi )) PN i (yi ) gi (yi ) > g0 (BQ$SJUFSJPO
5SPVCMFTPNF8PSE%FpOJUJPO ⾣&YDFQUJPO$SJUFSJB ग़ྗ֬ॱҐͷᮢ 1SPCBCJMJUZ EJTUSJCVUJPO Source sentence = (x1 ,
x2 , x3 ) Target sentence = (y1 , y2 , y3 ) HPMETUBOEBSE rank(y1 ) = 1 rank(y2 ) = 3 rank(y3 ) = 2 Y ͕USPVCMFTPNFXPSE ͳΒɺ rank0 = 2 FH rank(yi ) > rank0 3BOLJOH$SJUFSJPO
*OUFHSBUJOH$POUFYUVBM.FNPSZJOUP/.5 ⾣༁ॲཧͷྲྀΕ NFNPSZΑΓUSPVCMFTPNFXPSEΛநग़ DPOUFYUVBMTJNJMBSJUZΛܭࢉ NFNPSZQSFEJDUFEQSPCBCJMJUZͷܭࢉ ग़ྗ֬Λܭࢉ ᶃ ᶄ ᶅ
ᶆ
*OUFHSBUJOH$POUFYUVBM.FNPSZJOUP/.5 ⾣༁ॲཧͷྲྀΕ NFNPSZΑΓUSPVCMFTPNFXPSEΛநग़ ᶃ sm USPVCMFTPNFTPVSDFXPSE tn HPMEUBSHFUXPSE c(sm
, tn ) IJEEFOTUBUFPGFODPEFS pL(sm , tn ) MFYJDPOUSBOTMBUJPOQSPCBCJMJUZ
*OUFHSBUJOH$POUFYUVBM.FNPSZJOUP/.5 ⾣༁ॲཧͷྲྀΕ DPOUFYUVBMTJNJMBSJUZΛܭࢉ NFNPSZQSFEJDUFEQSPCBCJMJUZͷܭࢉ ग़ྗ֬Λܭࢉ ᶅ ᶆ ᶄ
&YQFSJNFOUBM4FUUJOHT ⾣%BUBTFUT 5SBJOJOH-%$DPSQVT .TFOUFODFQBJST 7BMJEBUJPO/*45 5FTU/*45 ⾣.PEFMT #BTFMJOF/.5XJUIHMPCBMBUUFOUJPO "SUIVS/.5
MFYJDBMUSBOTMBUJPO 9 .&.ఏҊख๏
3FTVMUT ⾣ϝϞϦػߏͷޮՌ
3FTVMUT ⾣&YDFQUJPODSJUFSJBʹΑΔҧ͍
3FTVMUT ⾣සޠ ᐆດੑͷ͋ΔޠΛؚΉจͰͷධՁ -PXग़ݱճ͕ճҎԼͷ୯ޠ "NCΤϯτϩϐʔ͕ᮢҎ্ − K ∑ k=1 pL
k logpL k > E0 (E0 = 1.5)
3FTVMUT ⾣සޠ ᐆດੑͷ͋ΔޠΛؚΉจͰͷධՁ #-&6είΞͷൺֱ
3FTVMUT ⾣සޠ ᐆດੑͷ͋ΔޠΛؚΉจͰͷධՁ 5XPSEUSPVCMFTPNFXPSE &SSPS#BTFMJOFʹ͓͚Δؒҧͬͨग़ྗ 3FDUJGZఏҊख๏ಋೖʹΑΓमਖ਼Ͱ͖ͨ %FUFSJPఏҊख๏ಋೖʹΑΓ૿͑ͨؒҧ͍ͷ
$PODMVTJPO ⾣$O&O %F&Oʹ͓͍ͯɺ༁࣭ͷվળΛ֬ೝ ⾣ॲཧͷ͍͠୯ޠ 5SPVCMFTPNFXPSE ͷఆٛ ⾣χϡʔϥϧػց༁ /.5 ʹ͓͍ͯɺॲཧͷ͍͠୯ޠΛదʹ ɹऔΓѻ͓͏ͱ͍͏ࢼΈ
⾣5SPVCMFTPNFXPSEͱͦͷจ຺Λߟྀͨ͠ϝϞϦػߏ
More Decks by MARUYAMA
See All by MARUYAMA
vampire.pdf
tmaru0204
0
200
Misspelling_Oblivious_Word_Embedding.pdf
tmaru0204
0
210
Simple_Unsupervised_Summarization_by_Contextual_Matching.pdf
tmaru0204
0
200
Controlling_Text_Complexity_in_Neural_Machine_Translation.pdf
tmaru0204
0
180
20191028_literature-review.pdf
tmaru0204
0
170
Hint-Based_Training_for_Non-Autoregressive_Machine_Translation.pdf
tmaru0204
0
160
Soft_Contextual_Data_Augmentation_for_Neural_Machine_Translation_.pdf
tmaru0204
0
190
An_Embarrassingly_Simple_Approach_for_Transfer_Learning_from_Pretrained_Language_Models_.pdf
tmaru0204
0
170
Simple_Unsupervised_Keyphrase_Extraction_using_Sentence_Embeddings.pdf
tmaru0204
0
210
Featured
See All Featured
Six Lessons from altMBA
skipperchong
29
4.6k
Into the Great Unknown - MozCon
thekraken
41
2.8k
Everyday Curiosity
cassininazir
0
400
Unlocking the hidden potential of vector embeddings in international SEO
frankvandijk
0
1k
Effective software design: The role of men in debugging patriarchy in IT @ Voxxed Days AMS
baasie
1
610
How To Stay Up To Date on Web Technology
chriscoyier
790
250k
Designing Dashboards & Data Visualisations in Web Apps
destraynor
232
55k
Product Roadmaps are Hard
iamctodd
55
13k
Making Projects Easy
brettharned
120
6.9k
Easily Structure & Communicate Ideas using Wireframe
afnizarnur
194
17k
BBQ
matthewcrist
90
10k
DevOps and Value Stream Thinking: Enabling flow, efficiency and business value
helenjbeal
2
450