Extract_and_Edit_An_Alternative_to_Back-Translation_for_Unsupervised_Neural_Machine_Translation.pdf

A3ea3bc5dde6ae2dd6eae71da9c418b0?s=47 MARUYAMA
April 15, 2019
44

 Extract_and_Edit_An_Alternative_to_Back-Translation_for_Unsupervised_Neural_Machine_Translation.pdf

A3ea3bc5dde6ae2dd6eae71da9c418b0?s=128

MARUYAMA

April 15, 2019
Tweet

Transcript

  1. &YUSBDUBOE&EJU"O"MUFSOBUJWFUP #BDL5SBOTMBUJPOGPS6OTVQFSWJTFE /FVSBM.BDIJOF5SBOTMBUJPO +JBXFJ8VBOE9JO8BOHBOE8JMMJBN:BOH8BOH/""$- -JUFSBUVSFSFWJFX /BHBPLB6OJWFSTJUZPG5FDIOPMPHZ5BLVNJ.BSVZBNB

  2. "CTUSBDU ⾣ٯ຋༁ʹΑΓֶशͨ͠Ϟσϧʹൺ΂ɺ#-&6ΛQUҎ্վળ ⾣σʔλͷ඼࣭Λอͪͭͭڭࢣͳֶ͠शΛߦ͏zFYUSBDUFEJUΛఏҊ ⾣ػց຋༁ͷڭࢣͳֶ͠शख๏ͱͯ͠ɺٯ຋༁͕͋Δ ˠ͔͠͠ɺٖࣅతʹੜ੒͞Εͨσʔλͷ඼࣭͸௿͍

  3. *OUSPEVDUJPO TU5SBOTMBUJPOͷ݁ՌΛ UT5SBOTMBUJPOʹೖྗ͠ɺ ݩͷTPVSDFTFOUFODFΛ ෮ݩͰ͖Δ͔ 3FDPOTUSVDUJPO-PTT &YUSBDU&EJUͰಘΒΕͨจͱൺ΂ɺ TU5SBOTMBUJPOͷग़ྗ݁Ռ͕ Ͳͷఔ౓TPVSDFTFOUFODFͷ ҙຯʹྨࣅ͍ͯ͠Δ͔

    $PNQBSBUJWF 5SBOTMBUJPO-PTT
  4. 6OTVQFSWJTFE/.5 *OJUJBMJ[BUJPO ⾣Ұൠతʹ࣍ͷTUFQTͰߦΘΕΔ -BOHVBHFNPEFMJOH #BDLUSBOTMBUJPO ˡ&YUSBDUFEJU

  5. 6OTVQFSWJTFE/.5 ⾣*OJUJBMJ[BUJPO ݪݴޠଆͱ໨తݴޠଆͷͭͷݴޠۭؒΛରԠ͚ͮΔ ɾ5IFJOGFSSFECJMJOHVBMEJDUJPOBSZ $POOFBVFUBM ɾ4IBSFE#1& -BNQMFFUBM

  6. 6OTVQFSWJTFE/.5 ⾣-BOHVBHFNPEFMJOH EFOPJTJOHBVUPFODPEJOHʹΑΓɺݪݴޠଆͱ໨తݴޠଆͷ྆ํͷ ݴޠϞσϧΛֶश ɾθenc , θdec : encoder, decoderͷֶशύϥϝʔλ

    ɾVs→s , Vt→t : encoder-decoder language model ɾC(ɾ): noise model (୯ޠͷܽམɾೖΕସ͑)
  7. 6OTVQFSWJTFE/.5 ⾣&YUSBDUFEJU

  8. 6OTVQFSWJTFE/.5 ⾣&YUSBDUFEJU &YUSBDU &EJU &WBMVBUF

  9. 6OTVQFSWJTFE/.5 ⾣&YUSBDUFEJU &YUSBDU ݪݴޠจʹࣅͨҙຯΛ࣋ͭ໨తݴޠจΛऩू &YUSBDU es , et : shared

    encoder͔ΒಘΒΕͨ sentence embeddings
  10. 6OTVQFSWJTFE/.5 ⾣&YUSBDUFEJU &EJU . FT FU ͷ.BYQPPMJOH &EJU es ,

    et : shared encoder͔ΒಘΒΕͨ sentence embeddings
  11. 6OTVQFSWJTFE/.5 ⾣&YUSBDUFEJU &WBMVBUF \4JNJMBSJUZ T UcU㱨.`㱮\U ^ ^ͷ͏ͪɺ 4JNJMBSJUZ T

    U Λ࠷େԽͤ͞ΔΑ͏ʹֶश &WBMVBUF rs , rt : shared encoder͔ΒಘΒΕͨsentence embeddings t*: Translation systemͷग़ྗ .` U
  12. 6OTVQFSWJTFE/.5 ⾣&YUSBDUFEJU ଛࣦؔ਺ ωlm = ωcom = 1

  13. &YQFSJNFOUTFUUJOHT ⾣%BUBTFUT ɾ&OHMJTI'SFODI (FSNBO 3VTTJBO 3PNBOJBO <5SBJO> ɾ8.5NPOPMJOHVBMOFXTDSBXMEBUBTFUT FO GS

    EF SV ɾOFXTDSBXMEBUBTFU 8.5`NPOPMJOHVBMEBUBTFUT SP <5FTU> ɾOFXTUFTU FOGS ɾOFXTUFTU FOEF SV SP
  14. 3FTVMUT

  15. "CMBUJPO4UVEZ ⾣5IF&⒎FDUPG&YUSBDUJPO/VNCFSL

  16. ⾣5IF2VBMJUZPG&YUSBDUJPO.PEFM "CMBUJPO4UVEZ ɾTVQFSWJTFEFYUSBDUJPONPEFM (SFHPJSFBOE-BOHMBJT  ͱͷੑೳൺֱ ɾFOGSUSBOTMBUJPOEBUBTFUʹOPJTFΛ෇༩ ɾͦΕͧΕͷϞσϧ͕நग़ͨ͠UPQLจʹɺ࣮ࡍͷର༁จؚ͕·Ε͍ͯΔ͔Ͳ͏͔

  17. "CMBUJPO4UVEZ ⾣5IF&⒎FDUPG$PNQBSBUJWF5SBOTMBUJPO ɾ$PNQBSBUJWFMPTTͱ.BYJNVNMJLFMJIPPEFTUJNBUJPO .-& MPTTͷൺֱ ɾ.-&MPTTͰ͸ɺFYUSBDUFEJUʹΑΓಘΒΕͨ໨తݴޠଆͷจΛͦͷ·· ɹڭࢣσʔλͱͯ͠ར༻

  18. $PODMVTJPO ⾣ٯ຋༁ʹΑΓֶशͨ͠Ϟσϧʹൺ΂ɺ#-&6ΛQUҎ্վળ ⾣ڭࢣͳֶ͠शͷͨΊͷDPNQBSBUJWFUSBOTMBUJPOMPTTΛఏҊ ⾣ٯ຋༁ͷ୅ସͱͳΔɺΑΓޮՌతͳख๏FYUSBDUFEJUΛఏҊ