Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Extract_and_Edit_An_Alternative_to_Back-Transla...
Search
MARUYAMA
April 15, 2019
0
120
Extract_and_Edit_An_Alternative_to_Back-Translation_for_Unsupervised_Neural_Machine_Translation.pdf
MARUYAMA
April 15, 2019
Tweet
Share
More Decks by MARUYAMA
See All by MARUYAMA
vampire.pdf
tmaru0204
0
170
Misspelling_Oblivious_Word_Embedding.pdf
tmaru0204
0
180
Simple_Unsupervised_Summarization_by_Contextual_Matching.pdf
tmaru0204
0
170
Controlling_Text_Complexity_in_Neural_Machine_Translation.pdf
tmaru0204
0
160
20191028_literature-review.pdf
tmaru0204
0
140
Hint-Based_Training_for_Non-Autoregressive_Machine_Translation.pdf
tmaru0204
0
130
Soft_Contextual_Data_Augmentation_for_Neural_Machine_Translation_.pdf
tmaru0204
0
160
An_Embarrassingly_Simple_Approach_for_Transfer_Learning_from_Pretrained_Language_Models_.pdf
tmaru0204
0
150
Addressing_Trobulesome_Words_in_Neural_Machine_Translation.pdf
tmaru0204
0
150
Featured
See All Featured
Large-scale JavaScript Application Architecture
addyosmani
512
110k
Balancing Empowerment & Direction
lara
1
370
Optimising Largest Contentful Paint
csswizardry
37
3.3k
A Tale of Four Properties
chriscoyier
160
23k
Rebuilding a faster, lazier Slack
samanthasiow
82
9.1k
Imperfection Machines: The Place of Print at Facebook
scottboms
267
13k
Stop Working from a Prison Cell
hatefulcrawdad
270
20k
Learning to Love Humans: Emotional Interface Design
aarron
273
40k
Become a Pro
speakerdeck
PRO
28
5.4k
The Straight Up "How To Draw Better" Workshop
denniskardys
234
140k
How to Think Like a Performance Engineer
csswizardry
24
1.7k
The Cult of Friendly URLs
andyhume
79
6.5k
Transcript
&YUSBDUBOE&EJU"O"MUFSOBUJWFUP #BDL5SBOTMBUJPOGPS6OTVQFSWJTFE /FVSBM.BDIJOF5SBOTMBUJPO +JBXFJ8VBOE9JO8BOHBOE8JMMJBN:BOH8BOH/""$- -JUFSBUVSFSFWJFX /BHBPLB6OJWFSTJUZPG5FDIOPMPHZ5BLVNJ.BSVZBNB
"CTUSBDU ⾣ٯ༁ʹΑΓֶशͨ͠Ϟσϧʹൺɺ#-&6ΛQUҎ্վળ ⾣σʔλͷ࣭Λอͪͭͭڭࢣͳֶ͠शΛߦ͏zFYUSBDUFEJUΛఏҊ ⾣ػց༁ͷڭࢣͳֶ͠शख๏ͱͯ͠ɺٯ༁͕͋Δ ˠ͔͠͠ɺٖࣅతʹੜ͞Εͨσʔλͷ࣭͍
*OUSPEVDUJPO TU5SBOTMBUJPOͷ݁ՌΛ UT5SBOTMBUJPOʹೖྗ͠ɺ ݩͷTPVSDFTFOUFODFΛ ෮ݩͰ͖Δ͔ 3FDPOTUSVDUJPO-PTT &YUSBDU&EJUͰಘΒΕͨจͱൺɺ TU5SBOTMBUJPOͷग़ྗ݁Ռ͕ ͲͷఔTPVSDFTFOUFODFͷ ҙຯʹྨࣅ͍ͯ͠Δ͔
$PNQBSBUJWF 5SBOTMBUJPO-PTT
6OTVQFSWJTFE/.5 *OJUJBMJ[BUJPO ⾣Ұൠతʹ࣍ͷTUFQTͰߦΘΕΔ -BOHVBHFNPEFMJOH #BDLUSBOTMBUJPO ˡ&YUSBDUFEJU
6OTVQFSWJTFE/.5 ⾣*OJUJBMJ[BUJPO ݪݴޠଆͱతݴޠଆͷͭͷݴޠۭؒΛରԠ͚ͮΔ ɾ5IFJOGFSSFECJMJOHVBMEJDUJPOBSZ $POOFBVFUBM ɾ4IBSFE#1& -BNQMFFUBM
6OTVQFSWJTFE/.5 ⾣-BOHVBHFNPEFMJOH EFOPJTJOHBVUPFODPEJOHʹΑΓɺݪݴޠଆͱతݴޠଆͷ྆ํͷ ݴޠϞσϧΛֶश ɾθenc , θdec : encoder, decoderͷֶशύϥϝʔλ
ɾVs→s , Vt→t : encoder-decoder language model ɾC(ɾ): noise model (୯ޠͷܽམɾೖΕସ͑)
6OTVQFSWJTFE/.5 ⾣&YUSBDUFEJU
6OTVQFSWJTFE/.5 ⾣&YUSBDUFEJU &YUSBDU &EJU &WBMVBUF
6OTVQFSWJTFE/.5 ⾣&YUSBDUFEJU &YUSBDU ݪݴޠจʹࣅͨҙຯΛ࣋ͭతݴޠจΛऩू &YUSBDU es , et : shared
encoder͔ΒಘΒΕͨ sentence embeddings
6OTVQFSWJTFE/.5 ⾣&YUSBDUFEJU &EJU . FT FU ͷ.BYQPPMJOH &EJU es ,
et : shared encoder͔ΒಘΒΕͨ sentence embeddings
6OTVQFSWJTFE/.5 ⾣&YUSBDUFEJU &WBMVBUF \4JNJMBSJUZ T UcU㱨.`㱮\U ^ ^ͷ͏ͪɺ 4JNJMBSJUZ T
U Λ࠷େԽͤ͞ΔΑ͏ʹֶश &WBMVBUF rs , rt : shared encoder͔ΒಘΒΕͨsentence embeddings t*: Translation systemͷग़ྗ .` U
6OTVQFSWJTFE/.5 ⾣&YUSBDUFEJU ଛࣦؔ ωlm = ωcom = 1
&YQFSJNFOUTFUUJOHT ⾣%BUBTFUT ɾ&OHMJTI'SFODI (FSNBO 3VTTJBO 3PNBOJBO <5SBJO> ɾ8.5NPOPMJOHVBMOFXTDSBXMEBUBTFUT FO GS
EF SV ɾOFXTDSBXMEBUBTFU 8.5`NPOPMJOHVBMEBUBTFUT SP <5FTU> ɾOFXTUFTU FOGS ɾOFXTUFTU FOEF SV SP
3FTVMUT
"CMBUJPO4UVEZ ⾣5IF&⒎FDUPG&YUSBDUJPO/VNCFSL
⾣5IF2VBMJUZPG&YUSBDUJPO.PEFM "CMBUJPO4UVEZ ɾTVQFSWJTFEFYUSBDUJPONPEFM (SFHPJSFBOE-BOHMBJT ͱͷੑೳൺֱ ɾFOGSUSBOTMBUJPOEBUBTFUʹOPJTFΛ༩ ɾͦΕͧΕͷϞσϧ͕நग़ͨ͠UPQLจʹɺ࣮ࡍͷର༁จؚ͕·Ε͍ͯΔ͔Ͳ͏͔
"CMBUJPO4UVEZ ⾣5IF&⒎FDUPG$PNQBSBUJWF5SBOTMBUJPO ɾ$PNQBSBUJWFMPTTͱ.BYJNVNMJLFMJIPPEFTUJNBUJPO .-& MPTTͷൺֱ ɾ.-&MPTTͰɺFYUSBDUFEJUʹΑΓಘΒΕͨతݴޠଆͷจΛͦͷ·· ɹڭࢣσʔλͱͯ͠ར༻
$PODMVTJPO ⾣ٯ༁ʹΑΓֶशͨ͠Ϟσϧʹൺɺ#-&6ΛQUҎ্վળ ⾣ڭࢣͳֶ͠शͷͨΊͷDPNQBSBUJWFUSBOTMBUJPOMPTTΛఏҊ ⾣ٯ༁ͷସͱͳΔɺΑΓޮՌతͳख๏FYUSBDUFEJUΛఏҊ