Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Extract_and_Edit_An_Alternative_to_Back-Transla...
Search
MARUYAMA
April 15, 2019
0
120
Extract_and_Edit_An_Alternative_to_Back-Translation_for_Unsupervised_Neural_Machine_Translation.pdf
MARUYAMA
April 15, 2019
Tweet
Share
More Decks by MARUYAMA
See All by MARUYAMA
vampire.pdf
tmaru0204
0
180
Misspelling_Oblivious_Word_Embedding.pdf
tmaru0204
0
190
Simple_Unsupervised_Summarization_by_Contextual_Matching.pdf
tmaru0204
0
180
Controlling_Text_Complexity_in_Neural_Machine_Translation.pdf
tmaru0204
0
160
20191028_literature-review.pdf
tmaru0204
0
150
Hint-Based_Training_for_Non-Autoregressive_Machine_Translation.pdf
tmaru0204
0
140
Soft_Contextual_Data_Augmentation_for_Neural_Machine_Translation_.pdf
tmaru0204
0
170
An_Embarrassingly_Simple_Approach_for_Transfer_Learning_from_Pretrained_Language_Models_.pdf
tmaru0204
0
150
Addressing_Trobulesome_Words_in_Neural_Machine_Translation.pdf
tmaru0204
0
150
Featured
See All Featured
Code Reviewing Like a Champion
maltzj
525
40k
Become a Pro
speakerdeck
PRO
29
5.5k
Designing Dashboards & Data Visualisations in Web Apps
destraynor
231
53k
Unsuck your backbone
ammeep
671
58k
Writing Fast Ruby
sferik
628
62k
Practical Orchestrator
shlominoach
190
11k
The Myth of the Modular Monolith - Day 2 Keynote - Rails World 2024
eileencodes
26
3k
Making Projects Easy
brettharned
117
6.4k
The Language of Interfaces
destraynor
161
25k
Chrome DevTools: State of the Union 2024 - Debugging React & Beyond
addyosmani
7
850
Helping Users Find Their Own Way: Creating Modern Search Experiences
danielanewman
29
2.9k
Thoughts on Productivity
jonyablonski
70
4.8k
Transcript
&YUSBDUBOE&EJU"O"MUFSOBUJWFUP #BDL5SBOTMBUJPOGPS6OTVQFSWJTFE /FVSBM.BDIJOF5SBOTMBUJPO +JBXFJ8VBOE9JO8BOHBOE8JMMJBN:BOH8BOH/""$- -JUFSBUVSFSFWJFX /BHBPLB6OJWFSTJUZPG5FDIOPMPHZ5BLVNJ.BSVZBNB
"CTUSBDU ⾣ٯ༁ʹΑΓֶशͨ͠Ϟσϧʹൺɺ#-&6ΛQUҎ্վળ ⾣σʔλͷ࣭Λอͪͭͭڭࢣͳֶ͠शΛߦ͏zFYUSBDUFEJUΛఏҊ ⾣ػց༁ͷڭࢣͳֶ͠शख๏ͱͯ͠ɺٯ༁͕͋Δ ˠ͔͠͠ɺٖࣅతʹੜ͞Εͨσʔλͷ࣭͍
*OUSPEVDUJPO TU5SBOTMBUJPOͷ݁ՌΛ UT5SBOTMBUJPOʹೖྗ͠ɺ ݩͷTPVSDFTFOUFODFΛ ෮ݩͰ͖Δ͔ 3FDPOTUSVDUJPO-PTT &YUSBDU&EJUͰಘΒΕͨจͱൺɺ TU5SBOTMBUJPOͷग़ྗ݁Ռ͕ ͲͷఔTPVSDFTFOUFODFͷ ҙຯʹྨࣅ͍ͯ͠Δ͔
$PNQBSBUJWF 5SBOTMBUJPO-PTT
6OTVQFSWJTFE/.5 *OJUJBMJ[BUJPO ⾣Ұൠతʹ࣍ͷTUFQTͰߦΘΕΔ -BOHVBHFNPEFMJOH #BDLUSBOTMBUJPO ˡ&YUSBDUFEJU
6OTVQFSWJTFE/.5 ⾣*OJUJBMJ[BUJPO ݪݴޠଆͱతݴޠଆͷͭͷݴޠۭؒΛରԠ͚ͮΔ ɾ5IFJOGFSSFECJMJOHVBMEJDUJPOBSZ $POOFBVFUBM ɾ4IBSFE#1& -BNQMFFUBM
6OTVQFSWJTFE/.5 ⾣-BOHVBHFNPEFMJOH EFOPJTJOHBVUPFODPEJOHʹΑΓɺݪݴޠଆͱతݴޠଆͷ྆ํͷ ݴޠϞσϧΛֶश ɾθenc , θdec : encoder, decoderͷֶशύϥϝʔλ
ɾVs→s , Vt→t : encoder-decoder language model ɾC(ɾ): noise model (୯ޠͷܽམɾೖΕସ͑)
6OTVQFSWJTFE/.5 ⾣&YUSBDUFEJU
6OTVQFSWJTFE/.5 ⾣&YUSBDUFEJU &YUSBDU &EJU &WBMVBUF
6OTVQFSWJTFE/.5 ⾣&YUSBDUFEJU &YUSBDU ݪݴޠจʹࣅͨҙຯΛ࣋ͭతݴޠจΛऩू &YUSBDU es , et : shared
encoder͔ΒಘΒΕͨ sentence embeddings
6OTVQFSWJTFE/.5 ⾣&YUSBDUFEJU &EJU . FT FU ͷ.BYQPPMJOH &EJU es ,
et : shared encoder͔ΒಘΒΕͨ sentence embeddings
6OTVQFSWJTFE/.5 ⾣&YUSBDUFEJU &WBMVBUF \4JNJMBSJUZ T UcU㱨.`㱮\U ^ ^ͷ͏ͪɺ 4JNJMBSJUZ T
U Λ࠷େԽͤ͞ΔΑ͏ʹֶश &WBMVBUF rs , rt : shared encoder͔ΒಘΒΕͨsentence embeddings t*: Translation systemͷग़ྗ .` U
6OTVQFSWJTFE/.5 ⾣&YUSBDUFEJU ଛࣦؔ ωlm = ωcom = 1
&YQFSJNFOUTFUUJOHT ⾣%BUBTFUT ɾ&OHMJTI'SFODI (FSNBO 3VTTJBO 3PNBOJBO <5SBJO> ɾ8.5NPOPMJOHVBMOFXTDSBXMEBUBTFUT FO GS
EF SV ɾOFXTDSBXMEBUBTFU 8.5`NPOPMJOHVBMEBUBTFUT SP <5FTU> ɾOFXTUFTU FOGS ɾOFXTUFTU FOEF SV SP
3FTVMUT
"CMBUJPO4UVEZ ⾣5IF&⒎FDUPG&YUSBDUJPO/VNCFSL
⾣5IF2VBMJUZPG&YUSBDUJPO.PEFM "CMBUJPO4UVEZ ɾTVQFSWJTFEFYUSBDUJPONPEFM (SFHPJSFBOE-BOHMBJT ͱͷੑೳൺֱ ɾFOGSUSBOTMBUJPOEBUBTFUʹOPJTFΛ༩ ɾͦΕͧΕͷϞσϧ͕நग़ͨ͠UPQLจʹɺ࣮ࡍͷର༁จؚ͕·Ε͍ͯΔ͔Ͳ͏͔
"CMBUJPO4UVEZ ⾣5IF&⒎FDUPG$PNQBSBUJWF5SBOTMBUJPO ɾ$PNQBSBUJWFMPTTͱ.BYJNVNMJLFMJIPPEFTUJNBUJPO .-& MPTTͷൺֱ ɾ.-&MPTTͰɺFYUSBDUFEJUʹΑΓಘΒΕͨతݴޠଆͷจΛͦͷ·· ɹڭࢣσʔλͱͯ͠ར༻
$PODMVTJPO ⾣ٯ༁ʹΑΓֶशͨ͠Ϟσϧʹൺɺ#-&6ΛQUҎ্վળ ⾣ڭࢣͳֶ͠शͷͨΊͷDPNQBSBUJWFUSBOTMBUJPOMPTTΛఏҊ ⾣ٯ༁ͷସͱͳΔɺΑΓޮՌతͳख๏FYUSBDUFEJUΛఏҊ