Neural Translation by David and Stefano - TMLS#3

A simple visual introduction to neural network models for text translation. All models are based on recurrent neural networks and can be applied to both word-level and character-level text translation. We start with the simplest possible approach and progressively make it more complex to overcome the difficulties encountered in this specific domain.

Speaker: David Malkin PhD and Stefano Peluchetti PhD from Reactive Inc.

Transcript

1. May 2016 /FVSBM
   5SBOTMBUJPO
2. Tokyo Machine Learning Society Presentation Reactive P2 /FVSBM
   5SBOTMBUJPO IPX`T JU HPJOH ݩؾ Ͱ͢ ͔
3. Tokyo Machine Learning Society Presentation Reactive P2 /FVSBM
   5SBOTMBUJPO WBSJBCMF
   MFOHUI
   XPSET WBSJBCMF
   MFOHUI
   JOQVU
4. Tokyo Machine Learning Society Presentation Reactive P2 -45. IFMMP          
5. Tokyo Machine Learning Society Presentation Reactive P2 -45. IPX           
6. Tokyo Machine Learning Society Presentation Reactive P2 -45. IPX           
7. Tokyo Machine Learning Society Presentation Reactive P2 -45. IPX           
8. Tokyo Machine Learning Society Presentation Reactive P2 -45. IPX           
9. Tokyo Machine Learning Society Presentation Reactive P2 -45. IPX           
10. Tokyo Machine Learning Society Presentation Reactive P2 -45. IFMMP          
11. Tokyo Machine Learning Society Presentation Reactive P2 -45.           IFMMP IPX BSF IFMMP IPX BSF          
12. Tokyo Machine Learning Society Presentation Reactive P2 4FRVFODF
    UP
    4FRVFODF
    5SBOTMBUJPO & IPX`T   ʜ & JU   ʜ HPJOH   ʜ 46.."3:  ݩؾ   ʜ & & % % % Ͱ͢   ʜ % ͔   ʜ
13. Tokyo Machine Learning Society Presentation Reactive P2 1IBTF
     & IPX`T & JU HPJOH 46.."3:  Q XPSE & & % % 4 4PGUNBY  Q XPSE % 4 4PGUNBY ݩؾ Q XPSE % 4 4PGUNBY Ͱ͢
14. Tokyo Machine Learning Society Presentation Reactive  ௐࢠ ΢χ   Ͱ͢ ͳ͍ ݩؾ ࢫ͍ ͕ Ͱ͢ ྑ͍ ͮ͘ ѱ͍ Α          ʜ
15. Tokyo Machine Learning Society Presentation Reactive    Ͱ͢ ͳ͍ ࢫ͍ ͕ Ͱ͢ ྑ͍ ͮ͘ ѱ͍ Α ௐࢠ ΢χ ݩؾ ௐ ࢠ ΢ χ ݩ ؾ Ͱ ͢    ྑ ͍    ࢫ ͍   
16. Tokyo Machine Learning Society Presentation Reactive    ௐࢠ ΢χ ݩؾ ௐ ࢠ ΢ χ ݩ ؾ Ͱ ͢  ྑ ͍  ࢫ ͍ 
17. Tokyo Machine Learning Society Presentation Reactive 4FRVFODF
    UP
    4FRVFODF
    5SBOTMBUJPO ) )PX ) XJMM ZPV ) ) TPMWF ) UIJT ) ) )5  ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) ) )  )
18. Tokyo Machine Learning Society Presentation Reactive 4FRVFODF
    UP
    4FRVFODF
    5SBOTMBUJPO )  ) ) ) ) ) ) ) ) ) ) ) ) 4 ) "UUFOUJPO
    
    'FFE
    'PSXBSE
    //
    4 ) "              $ $ $ $ $ $ $ $ $ $ $ $ $ "UUFOUJPO $POUFYU $
19. Tokyo Machine Learning Society Presentation Reactive 4FRVFODF
    UP
    4FRVFODF
    5SBOTMBUJPO ) 4 $ 4 : $ 4 $ 4 $ 4 : : : : 13&%*$5*0/
    
    'FFE
    'PSXBSE
    //
    4 1 :
20. Tokyo Machine Learning Society Presentation Reactive 3FGFSFODFT
    l-FBSOJOH
    1ISBTF
    3FQSFTFOUBUJPOT
    VTJOH
    3//
    &ODPEFS%FDPEFS
    GPS
    4UBUJTUJDBM
    .BDIJOF
    5SBOTMBUJPOz
    
    
    l/FVSBM
    .BDIJOF
    5SBOTMBUJPO
    CZ
    +PJOUMZ
    -FBSOJOH
    UP
    "MJHO
    BOE
    5SBOTMBUFz
    
    
    l"
    $IBSBDUFSMFWFM
    %FDPEFS
    XJUIPVU
    &YQMJDJU
    4FHNFOUBUJPO
    GPS
    /FVSBM
    .BDIJOF
    5SBOTMBUJPOz
    
    
    
21. Tokyo Machine Learning Society Presentation Reactive ENGLISH FRENCH ACTUAL FRENCH MODEL agenda~~~~ ordre des travaux~~~ ouuu~~~~~~~~~~~~~~~~ why?~~~~~~ pourquoi ?~~~~~~~~~~ uuuuuu~~~~~~~~~~~~~~ no.~~~~~~~ non.~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~ vote~~~~~~ votes~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~ why not?~~ pourquoi ?~~~~~~~~~~ uuuuuuuui~~~~~~~~~~~ can you?~~ pensez-vous pouvoir~ uuuuuuuu~~~~~~~~~~~~ agenda~~~~ ordre du jour~~~~~~~ ouuu~~~~~~~~~~~~~~~~ altener~~~ altener~~~~~~~~~~~~~ pp~~~~~~~~~~~~~~~~~~ save~~~~~~ save~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~ vote~~~~~~ votes~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~ 
22. Tokyo Machine Learning Society Presentation Reactive ENGLISH FRENCH ACTUAL FRENCH MODEL agenda~~~~ ordre des travaux~~~ rrrre ~~~~~~~~~~ why?~~~~~~ pourquoi ?~~~~~~~~~~ pourquoii?~~~~~~~~~~ no.~~~~~~~ non.~~~~~~~~~~~~~~~~ nnn.~~~~~~~~~~~~~~~~ vote~~~~~~ votes~~~~~~~~~~~~~~~ votes~~~~~~~~~~~~~~~ why not?~~ pourquoi ?~~~~~~~~~~ pourquoiiii~~~~~~~~~ can you?~~ pensez-vous pouvoir~ pouu i ii~~~~~~~~~~ agenda~~~~ ordre du jour~~~~~~~ rrrre ~~~~~~~~~~ altener~~~ altener~~~~~~~~~~~~~ seeiiii~~~~~~~~~~~~~ save~~~~~~ save~~~~~~~~~~~~~~~~ vot.~~~~~~~~~~~~~~~~ vote~~~~~~ votes~~~~~~~~~~~~~~~ votes~~~~~~~~~~~~~~~ 
23. Tokyo Machine Learning Society Presentation Reactive ENGLISH FRENCH ACTUAL FRENCH MODEL agenda~~~~ ordre des travaux~~~ ordre du jour~~~~~~~ why?~~~~~~ pourquoi ?~~~~~~~~~~ pourquoi?~~~~~~~~~~~ no.~~~~~~~ non.~~~~~~~~~~~~~~~~ non.~~~~~~~~~~~~~~~~ vote~~~~~~ votes~~~~~~~~~~~~~~~ votes~~~~~~~~~~~~~~~ why not?~~ pourquoi ?~~~~~~~~~~ pourquoi ~~~~~~~~~~~ can you?~~ pensez-vous pouvoir~ pou uoeee ~~~~~~~~~ agenda~~~~ ordre du jour~~~~~~~ ordre du jour~~~~~~~ altener~~~ altener~~~~~~~~~~~~~ rrres~~~~~~~~~~~~~~~ save~~~~~~ save~~~~~~~~~~~~~~~~ vots~~~~~~~~~~~~~~~~ vote~~~~~~ votes~~~~~~~~~~~~~~~ votes~~~~~~~~~~~~~~~ 
24. Tokyo Machine Learning Society Presentation Reactive ENGLISH FRENCH ACTUAL FRENCH MODEL agenda~~~~ ordre des travaux~~~ ordre du jour~~~~~~~ why?~~~~~~ pourquoi ?~~~~~~~~~~ pourquoi?~~~~~~~~~~~ no.~~~~~~~ non.~~~~~~~~~~~~~~~~ non.~~~~~~~~~~~~~~~~ vote~~~~~~ votes~~~~~~~~~~~~~~~ votes~~~~~~~~~~~~~~~ why not?~~ pourquoi ?~~~~~~~~~~ pourquoi ~~~~~~~~~~~ can you?~~ pensez-vous pouvoir~ penseztroupououvoi~~ agenda~~~~ ordre du jour~~~~~~~ ordre du jour~~~~~~~ altener~~~ altener~~~~~~~~~~~~~ altener~~~~~~~~~~~~~ save~~~~~~ save~~~~~~~~~~~~~~~~ save~~~~~~~~~~~~~~~~ vote~~~~~~ votes~~~~~~~~~~~~~~~ votes~~~~~~~~~~~~~~~