Extract_and_Edit_An_Alternative_to_Back-Translation_for_Unsupervised_Neural_Machine_Translation.pdf

Transcript

1. &YUSBDU
   BOE
   &EJU
   "O
   "MUFSOBUJWF
   UP
   
   #BDL5SBOTMBUJPO
   GPS
   6OTVQFSWJTFE
   /FVSBM
   .BDIJOF
   5SBOTMBUJPO
   +JBXFJ
   8V
   BOE
   9JO
   8BOH
   BOE
   8JMMJBN
   :BOH
   8BOH
   /""$- -JUFSBUVSF
   SFWJFX /BHBPLB
   6OJWFSTJUZ
   PG
   5FDIOPMPHZ
   
   
   
   5BLVNJ
   .BSVZBNB

2. "CTUSBDU ⾣
   ٯ຋༁ʹΑΓֶशͨ͠Ϟσϧʹൺ΂ɺ#-&6ΛQUҎ্վળ ⾣
   σʔλͷ඼࣭Λอͪͭͭڭࢣͳֶ͠शΛߦ͏zFYUSBDUFEJUΛఏҊ ⾣
   ػց຋༁ͷڭࢣͳֶ͠शख๏ͱͯ͠ɺٯ຋༁͕͋Δ ˠ
   ͔͠͠ɺٖࣅతʹੜ੒͞Εͨσʔλͷ඼࣭͸௿͍

3. *OUSPEVDUJPO TU
   5SBOTMBUJPOͷ݁ՌΛ
   UT
   5SBOTMBUJPOʹೖྗ͠ɺ
   ݩͷTPVSDF
   TFOUFODFΛ
   ෮ݩͰ͖Δ͔ 3FDPOTUSVDUJPO
   -PTT &YUSBDU&EJUͰಘΒΕͨจͱൺ΂ɺ
   TU
   5SBOTMBUJPOͷग़ྗ݁Ռ͕
   Ͳͷఔ౓TPVSDF
   TFOUFODFͷ
   ҙຯʹྨࣅ͍ͯ͠Δ͔

   $PNQBSBUJWF
   5SBOTMBUJPO
   -PTT

4. 6OTVQFSWJTFE
   /.5 
   *OJUJBMJ[BUJPO ⾣
   Ұൠతʹ࣍ͷTUFQTͰߦΘΕΔ 
   -BOHVBHF
   NPEFMJOH 
   #BDLUSBOTMBUJPO ˡ
   &YUSBDUFEJU

5. 6OTVQFSWJTFE
   /.5 ⾣
   *OJUJBMJ[BUJPO ݪݴޠଆͱ໨తݴޠଆͷͭͷݴޠۭؒΛରԠ͚ͮΔ ɾ5IF
   JOGFSSFE
   CJMJOHVBM
   EJDUJPOBSZ
   $POOFBV
   FU
   BM
    ɾ4IBSFE
   #1&
   -BNQMF
   FU
   BM
   

6. 6OTVQFSWJTFE
   /.5 ⾣
   -BOHVBHF
   NPEFMJOH EFOPJTJOH
   BVUPFODPEJOHʹΑΓɺݪݴޠଆͱ໨తݴޠଆͷ྆ํͷ
   ݴޠϞσϧΛֶश ɾθenc , θdec : encoder, decoderͷֶशύϥϝʔλ

   ɾVs→s , Vt→t : encoder-decoder language model ɾC(ɾ): noise model (୯ޠͷܽམɾೖΕସ͑)

7. 6OTVQFSWJTFE
   /.5 ⾣
   &YUSBDUFEJU

8. 6OTVQFSWJTFE
   /.5 ⾣
   &YUSBDUFEJU &YUSBDU &EJU &WBMVBUF

9. 6OTVQFSWJTFE
   /.5 ⾣
   &YUSBDUFEJU &YUSBDU ݪݴޠจʹࣅͨҙຯΛ࣋ͭ໨తݴޠจΛऩू &YUSBDU
    es , et : shared

   encoder͔ΒಘΒΕͨ sentence embeddings

10. 6OTVQFSWJTFE
    /.5 ⾣
    &YUSBDUFEJU &EJU . FT
    FU ͷ.BYQPPMJOH &EJU
     es ,

    et : shared encoder͔ΒಘΒΕͨ sentence embeddings

11. 6OTVQFSWJTFE
    /.5 ⾣
    &YUSBDUFEJU &WBMVBUF \4JNJMBSJUZ T
    Uc
    U㱨.`㱮\U ^ ^ͷ͏ͪɺ
    4JNJMBSJUZ T

    
    U Λ࠷େԽͤ͞ΔΑ͏ʹֶश &WBMVBUF
     rs , rt : shared encoder͔ΒಘΒΕͨsentence embeddings t*: Translation systemͷग़ྗ .` U

12. 6OTVQFSWJTFE
    /.5 ⾣
    &YUSBDUFEJU ଛࣦؔ਺
     ωlm = ωcom = 1

13. &YQFSJNFOU
    TFUUJOHT ⾣
    %BUBTFUT ɾ&OHMJTI
    
    'SFODI
    (FSNBO
    3VTTJBO
    3PNBOJBO <
    5SBJO
    > ɾ8.5
    NPOPMJOHVBM
    OFXT
    DSBXM
    EBUBTFUT
    FO
    GS

    
    EF
    SV ɾOFXT
    DSBXM
    EBUBTFU
    
    8.5`
    NPOPMJOHVBM
    EBUBTFUT
    SP <
    5FTU
    > ɾOFXTUFTU
    
    FOGS ɾOFXTUFTU
    
    FOEF
    SV
    SP

14. 3FTVMUT

15. "CMBUJPO
    4UVEZ ⾣
    5IF
    &⒎FDU
    PG
    &YUSBDUJPO
    /VNCFS
    L
    

16. ⾣
    5IF
    2VBMJUZ
    PG
    &YUSBDUJPO
    .PEFM "CMBUJPO
    4UVEZ ɾTVQFSWJTFE
    FYUSBDUJPO
    NPEFM
    (SFHPJSF
    BOE
    -BOHMBJT
    
    ͱͷੑೳൺֱ ɾ
    FOGS
    USBOTMBUJPO
    EBUBTFUʹOPJTFΛ෇༩ ɾͦΕͧΕͷϞσϧ͕நग़ͨ͠UPQLจʹɺ࣮ࡍͷର༁จؚ͕·Ε͍ͯΔ͔Ͳ͏͔

17. "CMBUJPO
    4UVEZ ⾣
    5IF
    &⒎FDU
    PG
    $PNQBSBUJWF
    5SBOTMBUJPO ɾ$PNQBSBUJWF
    MPTTͱ.BYJNVN
    MJLFMJIPPE
    FTUJNBUJPO .-&
    MPTTͷൺֱ ɾ.-&
    MPTTͰ͸ɺFYUSBDUFEJUʹΑΓಘΒΕͨ໨తݴޠଆͷจΛͦͷ··
    ɹڭࢣσʔλͱͯ͠ར༻

18. $PODMVTJPO ⾣
    ٯ຋༁ʹΑΓֶशͨ͠Ϟσϧʹൺ΂ɺ#-&6ΛQUҎ্վળ ⾣
    ڭࢣͳֶ͠शͷͨΊͷDPNQBSBUJWF
    USBOTMBUJPO
    MPTTΛఏҊ ⾣
    ٯ຋༁ͷ୅ସͱͳΔɺΑΓޮՌతͳख๏FYUSBDUFEJUΛఏҊ