Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Sign up for free
Menu
Search
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Pricing
Search
Sign in
Sign up for free
Extract_and_Edit_An_Alternative_to_Back-Transla...
Search
MARUYAMA
April 15, 2019
130
0
Share
Embed
Copy iframe code
Copy JS code
Copy link
Start on current slide
Extract_and_Edit_An_Alternative_to_Back-Translation_for_Unsupervised_Neural_Machine_Translation.pdf
MARUYAMA
April 15, 2019
Transcript
18 slides
&YUSBDUBOE&EJU"O"MUFSOBUJWFUP #BDL5SBOTMBUJPOGPS6OTVQFSWJTFE /FVSBM.BDIJOF5SBOTMBUJPO +JBXFJ8VBOE9JO8BOHBOE8JMMJBN:BOH8BOH/""$- -JUFSBUVSFSFWJFX /BHBPLB6OJWFSTJUZPG5FDIOPMPHZ5BLVNJ.BSVZBNB
"CTUSBDU ⾣ٯ༁ʹΑΓֶशͨ͠Ϟσϧʹൺɺ#-&6ΛQUҎ্վળ ⾣σʔλͷ࣭Λอͪͭͭڭࢣͳֶ͠शΛߦ͏zFYUSBDUFEJUΛఏҊ ⾣ػց༁ͷڭࢣͳֶ͠शख๏ͱͯ͠ɺٯ༁͕͋Δ ˠ͔͠͠ɺٖࣅతʹੜ͞Εͨσʔλͷ࣭͍
*OUSPEVDUJPO TU5SBOTMBUJPOͷ݁ՌΛ UT5SBOTMBUJPOʹೖྗ͠ɺ ݩͷTPVSDFTFOUFODFΛ ෮ݩͰ͖Δ͔ 3FDPOTUSVDUJPO-PTT &YUSBDU&EJUͰಘΒΕͨจͱൺɺ TU5SBOTMBUJPOͷग़ྗ݁Ռ͕ ͲͷఔTPVSDFTFOUFODFͷ ҙຯʹྨࣅ͍ͯ͠Δ͔
$PNQBSBUJWF 5SBOTMBUJPO-PTT
6OTVQFSWJTFE/.5 *OJUJBMJ[BUJPO ⾣Ұൠతʹ࣍ͷTUFQTͰߦΘΕΔ -BOHVBHFNPEFMJOH #BDLUSBOTMBUJPO ˡ&YUSBDUFEJU
6OTVQFSWJTFE/.5 ⾣*OJUJBMJ[BUJPO ݪݴޠଆͱతݴޠଆͷͭͷݴޠۭؒΛରԠ͚ͮΔ ɾ5IFJOGFSSFECJMJOHVBMEJDUJPOBSZ $POOFBVFUBM ɾ4IBSFE#1& -BNQMFFUBM
6OTVQFSWJTFE/.5 ⾣-BOHVBHFNPEFMJOH EFOPJTJOHBVUPFODPEJOHʹΑΓɺݪݴޠଆͱతݴޠଆͷ྆ํͷ ݴޠϞσϧΛֶश ɾθenc , θdec : encoder, decoderͷֶशύϥϝʔλ
ɾVs→s , Vt→t : encoder-decoder language model ɾC(ɾ): noise model (୯ޠͷܽམɾೖΕସ͑)
6OTVQFSWJTFE/.5 ⾣&YUSBDUFEJU
6OTVQFSWJTFE/.5 ⾣&YUSBDUFEJU &YUSBDU &EJU &WBMVBUF
6OTVQFSWJTFE/.5 ⾣&YUSBDUFEJU &YUSBDU ݪݴޠจʹࣅͨҙຯΛ࣋ͭతݴޠจΛऩू &YUSBDU es , et : shared
encoder͔ΒಘΒΕͨ sentence embeddings
6OTVQFSWJTFE/.5 ⾣&YUSBDUFEJU &EJU . FT FU ͷ.BYQPPMJOH &EJU es ,
et : shared encoder͔ΒಘΒΕͨ sentence embeddings
6OTVQFSWJTFE/.5 ⾣&YUSBDUFEJU &WBMVBUF \4JNJMBSJUZ T UcU㱨.`㱮\U ^ ^ͷ͏ͪɺ 4JNJMBSJUZ T
U Λ࠷େԽͤ͞ΔΑ͏ʹֶश &WBMVBUF rs , rt : shared encoder͔ΒಘΒΕͨsentence embeddings t*: Translation systemͷग़ྗ .` U
6OTVQFSWJTFE/.5 ⾣&YUSBDUFEJU ଛࣦؔ ωlm = ωcom = 1
&YQFSJNFOUTFUUJOHT ⾣%BUBTFUT ɾ&OHMJTI'SFODI (FSNBO 3VTTJBO 3PNBOJBO <5SBJO> ɾ8.5NPOPMJOHVBMOFXTDSBXMEBUBTFUT FO GS
EF SV ɾOFXTDSBXMEBUBTFU 8.5`NPOPMJOHVBMEBUBTFUT SP <5FTU> ɾOFXTUFTU FOGS ɾOFXTUFTU FOEF SV SP
3FTVMUT
"CMBUJPO4UVEZ ⾣5IF&⒎FDUPG&YUSBDUJPO/VNCFSL
⾣5IF2VBMJUZPG&YUSBDUJPO.PEFM "CMBUJPO4UVEZ ɾTVQFSWJTFEFYUSBDUJPONPEFM (SFHPJSFBOE-BOHMBJT ͱͷੑೳൺֱ ɾFOGSUSBOTMBUJPOEBUBTFUʹOPJTFΛ༩ ɾͦΕͧΕͷϞσϧ͕நग़ͨ͠UPQLจʹɺ࣮ࡍͷର༁จؚ͕·Ε͍ͯΔ͔Ͳ͏͔
"CMBUJPO4UVEZ ⾣5IF&⒎FDUPG$PNQBSBUJWF5SBOTMBUJPO ɾ$PNQBSBUJWFMPTTͱ.BYJNVNMJLFMJIPPEFTUJNBUJPO .-& MPTTͷൺֱ ɾ.-&MPTTͰɺFYUSBDUFEJUʹΑΓಘΒΕͨతݴޠଆͷจΛͦͷ·· ɹڭࢣσʔλͱͯ͠ར༻
$PODMVTJPO ⾣ٯ༁ʹΑΓֶशͨ͠Ϟσϧʹൺɺ#-&6ΛQUҎ্վળ ⾣ڭࢣͳֶ͠शͷͨΊͷDPNQBSBUJWFUSBOTMBUJPOMPTTΛఏҊ ⾣ٯ༁ͷସͱͳΔɺΑΓޮՌతͳख๏FYUSBDUFEJUΛఏҊ
More Decks by MARUYAMA
See All by MARUYAMA
vampire.pdf
tmaru0204
0
200
Misspelling_Oblivious_Word_Embedding.pdf
tmaru0204
0
210
Simple_Unsupervised_Summarization_by_Contextual_Matching.pdf
tmaru0204
0
200
Controlling_Text_Complexity_in_Neural_Machine_Translation.pdf
tmaru0204
0
180
20191028_literature-review.pdf
tmaru0204
0
170
Hint-Based_Training_for_Non-Autoregressive_Machine_Translation.pdf
tmaru0204
0
160
Soft_Contextual_Data_Augmentation_for_Neural_Machine_Translation_.pdf
tmaru0204
0
190
An_Embarrassingly_Simple_Approach_for_Transfer_Learning_from_Pretrained_Language_Models_.pdf
tmaru0204
0
170
Addressing_Trobulesome_Words_in_Neural_Machine_Translation.pdf
tmaru0204
0
190
Featured
See All Featured
Navigating Team Friction
lara
192
16k
The Power of CSS Pseudo Elements
geoffreycrofte
82
6.6k
For a Future-Friendly Web
brad_frost
184
11k
Agile Leadership in an Agile Organization
kimpetersen
PRO
0
310
SEO Brein meetup: CTRL+C is not how to scale international SEO
lindahogenes
2
3k
How to Get Subject Matter Experts Bought In and Actively Contributing to SEO & PR Initiatives.
livdayseo
1
240
Why Mistakes Are the Best Teachers: Turning Failure into a Pathway for Growth
auna
0
360
Kristin Tynski - Automating Marketing Tasks With AI
techseoconnect
PRO
0
590
Agile that works and the tools we love
rasmusluckow
331
22k
Max Prin - Stacking Signals: How International SEO Comes Together (And Falls Apart)
techseoconnect
PRO
0
530
Measuring Dark Social's Impact On Conversion and Attribution
stephenakadiri
2
330
Refactoring Trust on Your Teams (GOTO; Chicago 2020)
rmw
35
3.9k