Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Extract_and_Edit_An_Alternative_to_Back-Transla...
Search
MARUYAMA
April 15, 2019
0
120
Extract_and_Edit_An_Alternative_to_Back-Translation_for_Unsupervised_Neural_Machine_Translation.pdf
MARUYAMA
April 15, 2019
Tweet
Share
More Decks by MARUYAMA
See All by MARUYAMA
vampire.pdf
tmaru0204
0
170
Misspelling_Oblivious_Word_Embedding.pdf
tmaru0204
0
180
Simple_Unsupervised_Summarization_by_Contextual_Matching.pdf
tmaru0204
0
170
Controlling_Text_Complexity_in_Neural_Machine_Translation.pdf
tmaru0204
0
160
20191028_literature-review.pdf
tmaru0204
0
150
Hint-Based_Training_for_Non-Autoregressive_Machine_Translation.pdf
tmaru0204
0
130
Soft_Contextual_Data_Augmentation_for_Neural_Machine_Translation_.pdf
tmaru0204
0
160
An_Embarrassingly_Simple_Approach_for_Transfer_Learning_from_Pretrained_Language_Models_.pdf
tmaru0204
0
150
Addressing_Trobulesome_Words_in_Neural_Machine_Translation.pdf
tmaru0204
0
150
Featured
See All Featured
Navigating Team Friction
lara
187
15k
Building Flexible Design Systems
yeseniaperezcruz
328
39k
Scaling GitHub
holman
459
140k
Save Time (by Creating Custom Rails Generators)
garrettdimon
PRO
31
1.3k
Designing Dashboards & Data Visualisations in Web Apps
destraynor
231
53k
Raft: Consensus for Rubyists
vanstee
140
7k
Sharpening the Axe: The Primacy of Toolmaking
bcantrill
44
2.4k
Designing for Performance
lara
609
69k
CSS Pre-Processors: Stylus, Less & Sass
bermonpainter
357
30k
Design and Strategy: How to Deal with People Who Don’t "Get" Design
morganepeng
130
19k
Thoughts on Productivity
jonyablonski
69
4.7k
How to train your dragon (web standard)
notwaldorf
94
6.1k
Transcript
&YUSBDUBOE&EJU"O"MUFSOBUJWFUP #BDL5SBOTMBUJPOGPS6OTVQFSWJTFE /FVSBM.BDIJOF5SBOTMBUJPO +JBXFJ8VBOE9JO8BOHBOE8JMMJBN:BOH8BOH/""$- -JUFSBUVSFSFWJFX /BHBPLB6OJWFSTJUZPG5FDIOPMPHZ5BLVNJ.BSVZBNB
"CTUSBDU ⾣ٯ༁ʹΑΓֶशͨ͠Ϟσϧʹൺɺ#-&6ΛQUҎ্վળ ⾣σʔλͷ࣭Λอͪͭͭڭࢣͳֶ͠शΛߦ͏zFYUSBDUFEJUΛఏҊ ⾣ػց༁ͷڭࢣͳֶ͠शख๏ͱͯ͠ɺٯ༁͕͋Δ ˠ͔͠͠ɺٖࣅతʹੜ͞Εͨσʔλͷ࣭͍
*OUSPEVDUJPO TU5SBOTMBUJPOͷ݁ՌΛ UT5SBOTMBUJPOʹೖྗ͠ɺ ݩͷTPVSDFTFOUFODFΛ ෮ݩͰ͖Δ͔ 3FDPOTUSVDUJPO-PTT &YUSBDU&EJUͰಘΒΕͨจͱൺɺ TU5SBOTMBUJPOͷग़ྗ݁Ռ͕ ͲͷఔTPVSDFTFOUFODFͷ ҙຯʹྨࣅ͍ͯ͠Δ͔
$PNQBSBUJWF 5SBOTMBUJPO-PTT
6OTVQFSWJTFE/.5 *OJUJBMJ[BUJPO ⾣Ұൠతʹ࣍ͷTUFQTͰߦΘΕΔ -BOHVBHFNPEFMJOH #BDLUSBOTMBUJPO ˡ&YUSBDUFEJU
6OTVQFSWJTFE/.5 ⾣*OJUJBMJ[BUJPO ݪݴޠଆͱతݴޠଆͷͭͷݴޠۭؒΛରԠ͚ͮΔ ɾ5IFJOGFSSFECJMJOHVBMEJDUJPOBSZ $POOFBVFUBM ɾ4IBSFE#1& -BNQMFFUBM
6OTVQFSWJTFE/.5 ⾣-BOHVBHFNPEFMJOH EFOPJTJOHBVUPFODPEJOHʹΑΓɺݪݴޠଆͱతݴޠଆͷ྆ํͷ ݴޠϞσϧΛֶश ɾθenc , θdec : encoder, decoderͷֶशύϥϝʔλ
ɾVs→s , Vt→t : encoder-decoder language model ɾC(ɾ): noise model (୯ޠͷܽམɾೖΕସ͑)
6OTVQFSWJTFE/.5 ⾣&YUSBDUFEJU
6OTVQFSWJTFE/.5 ⾣&YUSBDUFEJU &YUSBDU &EJU &WBMVBUF
6OTVQFSWJTFE/.5 ⾣&YUSBDUFEJU &YUSBDU ݪݴޠจʹࣅͨҙຯΛ࣋ͭతݴޠจΛऩू &YUSBDU es , et : shared
encoder͔ΒಘΒΕͨ sentence embeddings
6OTVQFSWJTFE/.5 ⾣&YUSBDUFEJU &EJU . FT FU ͷ.BYQPPMJOH &EJU es ,
et : shared encoder͔ΒಘΒΕͨ sentence embeddings
6OTVQFSWJTFE/.5 ⾣&YUSBDUFEJU &WBMVBUF \4JNJMBSJUZ T UcU㱨.`㱮\U ^ ^ͷ͏ͪɺ 4JNJMBSJUZ T
U Λ࠷େԽͤ͞ΔΑ͏ʹֶश &WBMVBUF rs , rt : shared encoder͔ΒಘΒΕͨsentence embeddings t*: Translation systemͷग़ྗ .` U
6OTVQFSWJTFE/.5 ⾣&YUSBDUFEJU ଛࣦؔ ωlm = ωcom = 1
&YQFSJNFOUTFUUJOHT ⾣%BUBTFUT ɾ&OHMJTI'SFODI (FSNBO 3VTTJBO 3PNBOJBO <5SBJO> ɾ8.5NPOPMJOHVBMOFXTDSBXMEBUBTFUT FO GS
EF SV ɾOFXTDSBXMEBUBTFU 8.5`NPOPMJOHVBMEBUBTFUT SP <5FTU> ɾOFXTUFTU FOGS ɾOFXTUFTU FOEF SV SP
3FTVMUT
"CMBUJPO4UVEZ ⾣5IF&⒎FDUPG&YUSBDUJPO/VNCFSL
⾣5IF2VBMJUZPG&YUSBDUJPO.PEFM "CMBUJPO4UVEZ ɾTVQFSWJTFEFYUSBDUJPONPEFM (SFHPJSFBOE-BOHMBJT ͱͷੑೳൺֱ ɾFOGSUSBOTMBUJPOEBUBTFUʹOPJTFΛ༩ ɾͦΕͧΕͷϞσϧ͕நग़ͨ͠UPQLจʹɺ࣮ࡍͷର༁จؚ͕·Ε͍ͯΔ͔Ͳ͏͔
"CMBUJPO4UVEZ ⾣5IF&⒎FDUPG$PNQBSBUJWF5SBOTMBUJPO ɾ$PNQBSBUJWFMPTTͱ.BYJNVNMJLFMJIPPEFTUJNBUJPO .-& MPTTͷൺֱ ɾ.-&MPTTͰɺFYUSBDUFEJUʹΑΓಘΒΕͨతݴޠଆͷจΛͦͷ·· ɹڭࢣσʔλͱͯ͠ར༻
$PODMVTJPO ⾣ٯ༁ʹΑΓֶशͨ͠Ϟσϧʹൺɺ#-&6ΛQUҎ্վળ ⾣ڭࢣͳֶ͠शͷͨΊͷDPNQBSBUJWFUSBOTMBUJPOMPTTΛఏҊ ⾣ٯ༁ͷସͱͳΔɺΑΓޮՌతͳख๏FYUSBDUFEJUΛఏҊ