Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Extract_and_Edit_An_Alternative_to_Back-Translation_for_Unsupervised_Neural_Machine_Translation.pdf
Search
MARUYAMA
April 15, 2019
0
95
Extract_and_Edit_An_Alternative_to_Back-Translation_for_Unsupervised_Neural_Machine_Translation.pdf
MARUYAMA
April 15, 2019
Tweet
Share
More Decks by MARUYAMA
See All by MARUYAMA
vampire.pdf
tmaru0204
0
130
Misspelling_Oblivious_Word_Embedding.pdf
tmaru0204
0
130
Simple_Unsupervised_Summarization_by_Contextual_Matching.pdf
tmaru0204
0
130
Controlling_Text_Complexity_in_Neural_Machine_Translation.pdf
tmaru0204
0
120
20191028_literature-review.pdf
tmaru0204
0
120
Hint-Based_Training_for_Non-Autoregressive_Machine_Translation.pdf
tmaru0204
0
100
Soft_Contextual_Data_Augmentation_for_Neural_Machine_Translation_.pdf
tmaru0204
0
120
An_Embarrassingly_Simple_Approach_for_Transfer_Learning_from_Pretrained_Language_Models_.pdf
tmaru0204
0
110
Addressing_Trobulesome_Words_in_Neural_Machine_Translation.pdf
tmaru0204
0
110
Featured
See All Featured
Clear Off the Table
cherdarchuk
83
310k
Building a Scalable Design System with Sketch
lauravandoore
455
32k
Optimizing for Happiness
mojombo
370
69k
Sharpening the Axe: The Primacy of Toolmaking
bcantrill
16
1.4k
Understanding Cognitive Biases in Performance Measurement
bluesmoon
6
990
Optimising Largest Contentful Paint
csswizardry
7
2.3k
CSS Pre-Processors: Stylus, Less & Sass
bermonpainter
352
28k
The Psychology of Web Performance [Beyond Tellerrand 2023]
tammyeverts
5
1.5k
Unsuck your backbone
ammeep
663
57k
Design by the Numbers
sachag
274
18k
Building Better People: How to give real-time feedback that sticks.
wjessup
354
18k
Reflections from 52 weeks, 52 projects
jeffersonlam
344
19k
Transcript
&YUSBDUBOE&EJU"O"MUFSOBUJWFUP #BDL5SBOTMBUJPOGPS6OTVQFSWJTFE /FVSBM.BDIJOF5SBOTMBUJPO +JBXFJ8VBOE9JO8BOHBOE8JMMJBN:BOH8BOH/""$- -JUFSBUVSFSFWJFX /BHBPLB6OJWFSTJUZPG5FDIOPMPHZ5BLVNJ.BSVZBNB
"CTUSBDU ⾣ٯ༁ʹΑΓֶशͨ͠Ϟσϧʹൺɺ#-&6ΛQUҎ্վળ ⾣σʔλͷ࣭Λอͪͭͭڭࢣͳֶ͠शΛߦ͏zFYUSBDUFEJUΛఏҊ ⾣ػց༁ͷڭࢣͳֶ͠शख๏ͱͯ͠ɺٯ༁͕͋Δ ˠ͔͠͠ɺٖࣅతʹੜ͞Εͨσʔλͷ࣭͍
*OUSPEVDUJPO TU5SBOTMBUJPOͷ݁ՌΛ UT5SBOTMBUJPOʹೖྗ͠ɺ ݩͷTPVSDFTFOUFODFΛ ෮ݩͰ͖Δ͔ 3FDPOTUSVDUJPO-PTT &YUSBDU&EJUͰಘΒΕͨจͱൺɺ TU5SBOTMBUJPOͷग़ྗ݁Ռ͕ ͲͷఔTPVSDFTFOUFODFͷ ҙຯʹྨࣅ͍ͯ͠Δ͔
$PNQBSBUJWF 5SBOTMBUJPO-PTT
6OTVQFSWJTFE/.5 *OJUJBMJ[BUJPO ⾣Ұൠతʹ࣍ͷTUFQTͰߦΘΕΔ -BOHVBHFNPEFMJOH #BDLUSBOTMBUJPO ˡ&YUSBDUFEJU
6OTVQFSWJTFE/.5 ⾣*OJUJBMJ[BUJPO ݪݴޠଆͱతݴޠଆͷͭͷݴޠۭؒΛରԠ͚ͮΔ ɾ5IFJOGFSSFECJMJOHVBMEJDUJPOBSZ $POOFBVFUBM ɾ4IBSFE#1& -BNQMFFUBM
6OTVQFSWJTFE/.5 ⾣-BOHVBHFNPEFMJOH EFOPJTJOHBVUPFODPEJOHʹΑΓɺݪݴޠଆͱతݴޠଆͷ྆ํͷ ݴޠϞσϧΛֶश ɾθenc , θdec : encoder, decoderͷֶशύϥϝʔλ
ɾVs→s , Vt→t : encoder-decoder language model ɾC(ɾ): noise model (୯ޠͷܽམɾೖΕସ͑)
6OTVQFSWJTFE/.5 ⾣&YUSBDUFEJU
6OTVQFSWJTFE/.5 ⾣&YUSBDUFEJU &YUSBDU &EJU &WBMVBUF
6OTVQFSWJTFE/.5 ⾣&YUSBDUFEJU &YUSBDU ݪݴޠจʹࣅͨҙຯΛ࣋ͭతݴޠจΛऩू &YUSBDU es , et : shared
encoder͔ΒಘΒΕͨ sentence embeddings
6OTVQFSWJTFE/.5 ⾣&YUSBDUFEJU &EJU . FT FU ͷ.BYQPPMJOH &EJU es ,
et : shared encoder͔ΒಘΒΕͨ sentence embeddings
6OTVQFSWJTFE/.5 ⾣&YUSBDUFEJU &WBMVBUF \4JNJMBSJUZ T UcU㱨.`㱮\U ^ ^ͷ͏ͪɺ 4JNJMBSJUZ T
U Λ࠷େԽͤ͞ΔΑ͏ʹֶश &WBMVBUF rs , rt : shared encoder͔ΒಘΒΕͨsentence embeddings t*: Translation systemͷग़ྗ .` U
6OTVQFSWJTFE/.5 ⾣&YUSBDUFEJU ଛࣦؔ ωlm = ωcom = 1
&YQFSJNFOUTFUUJOHT ⾣%BUBTFUT ɾ&OHMJTI'SFODI (FSNBO 3VTTJBO 3PNBOJBO <5SBJO> ɾ8.5NPOPMJOHVBMOFXTDSBXMEBUBTFUT FO GS
EF SV ɾOFXTDSBXMEBUBTFU 8.5`NPOPMJOHVBMEBUBTFUT SP <5FTU> ɾOFXTUFTU FOGS ɾOFXTUFTU FOEF SV SP
3FTVMUT
"CMBUJPO4UVEZ ⾣5IF&⒎FDUPG&YUSBDUJPO/VNCFSL
⾣5IF2VBMJUZPG&YUSBDUJPO.PEFM "CMBUJPO4UVEZ ɾTVQFSWJTFEFYUSBDUJPONPEFM (SFHPJSFBOE-BOHMBJT ͱͷੑೳൺֱ ɾFOGSUSBOTMBUJPOEBUBTFUʹOPJTFΛ༩ ɾͦΕͧΕͷϞσϧ͕நग़ͨ͠UPQLจʹɺ࣮ࡍͷର༁จؚ͕·Ε͍ͯΔ͔Ͳ͏͔
"CMBUJPO4UVEZ ⾣5IF&⒎FDUPG$PNQBSBUJWF5SBOTMBUJPO ɾ$PNQBSBUJWFMPTTͱ.BYJNVNMJLFMJIPPEFTUJNBUJPO .-& MPTTͷൺֱ ɾ.-&MPTTͰɺFYUSBDUFEJUʹΑΓಘΒΕͨతݴޠଆͷจΛͦͷ·· ɹڭࢣσʔλͱͯ͠ར༻
$PODMVTJPO ⾣ٯ༁ʹΑΓֶशͨ͠Ϟσϧʹൺɺ#-&6ΛQUҎ্վળ ⾣ڭࢣͳֶ͠शͷͨΊͷDPNQBSBUJWFUSBOTMBUJPOMPTTΛఏҊ ⾣ٯ༁ͷସͱͳΔɺΑΓޮՌతͳख๏FYUSBDUFEJUΛఏҊ