Learning Semantic Sentence Embeddings using Pair-wise Discriminator

弊研究室で行なったCOLING2018読み会の発表資料です。

Transcript

1. $0-*/(
   :0.*,"*
   UI
   0DU
    -FBSOJOH
   4FNBOUJD
   4FOUFODF
   &NCFEEJOHT VTJOH
   1BJSXJTF
   %JTDSJNJOBUPS (3"%6"5&
   4$)00-
   PG
   */'03."5*0/
   4$*&/$&
   BOE
   5&$)/0-0(:
   04","
   6/*7 +6/:"
   5",":"."
2. 1BQFS
   */'0 #BESJ /BSBZBOB
   1BUSP 7JOPE
   ,VNBS
   ,VSNJ 4BOEFFQ
   ,VNBS 7JOBZ
   /BNCPPEJSJ l-FBSOJOH
   4FNBOUJD
   4FOUFODF
   &NCFEEJOHT VTJOH
   4FRVFOUJBM
   1BJSXJTF
   %JTDSJNJOBUPSz *O
   1SPDFFEJOHT
   PG
   $0-*/( ݴ͍׵͑ੜ੒ϞσϧΛվྑͯͭ͠Α͍ &NCFEEJOH
   ͕ಘΒΕΔΑ͏ʹͨ͠Α
   
3. $POUFOUT  *OUSPEVDUJPO  .FUIPET  &YQFSJNFOUT  $PODMVTJPO
   
4. #BDLHSPVOE • ҙຯ͕͍ۙจ͸͖ۙͮɼҙຯ͕ԕ͍จ͸ԕ͔͟ΔΑ͏ͳ จͷ෼ࢄදݱʢ&NCFEEJOHʣΛಘ͍ͨ • ݴ͍׵͑ੜ੒͕ղ͚ΔΑ͏ͳදݱΛ֫ಘ͢Ε͹ྑ͍ • 4FR4FR
   Ͱ͸σίʔμͷηϧ͝ͱʹϩεΛܭࢉ͢ΔͨΊɼ จશମ͕ਖ਼͘͠ੜ੒͞ΕΔอূ͕ͳ͍ ˠࢀরจͱੜ੒จͷҙຯతͳۙ͞Λอূ͢Δϩε͕ཉ͍͠
   
5. $POUSJCVUJPOT • ࢀরจͱੜ੒จͷҙຯతͳۙ͞Λอূ͢Δ QBJSXJTF
   EJTDSJNJOBUPS
   MPTT
   HMPCBM
   MPTT
   ͷఏҊ • 4FOUJNFOU
   "OBMZTJT
   ʹ΋ద༻ՄೳͰ͋Δ͜ͱΛࣔ͢ 4UBOGPSE
   4FOUJNFOU
   5SFFCBOL
   Ͱ 405"
   
6. $POUFOUT  *OUSPEVDUJPO  .FUIPET  &YQFSJNFOUT  $PODMVTJPO
   
7. .PEFM
   0WFSWJFX • ௨ৗͷ 4FR4FR ʹɼੜ੒݁Ռͱ (SPVOE
   USVUI
   ͷΤϯίʔυ݁Ռ͕ ۙ͘ͳΔΑ͏ʹֶश͢Δ 1BJSXJTF
   %JTDSJNJOBUPS
   Λ௥Ճ
   1BJSXJTF
   %JTDSJNJOBUPS &ODPEFS %FDPEFS -45. &ODPEFS (SPVOE
   5SVUI -PDBM
   MPTT DSPTTFOUSPQZ (MPCBM
   MPTT 4FOUFODF
   &NCFEEJOH
8. -PTT
   GVODUJPOT
    • -PDBMMPTT τʔΫϯ͝ͱͷ $SPTT&OUSPQZʢ௨ৗͷ 4FR4FR
   ͷֶशͱಉ༷ʣ
   1BJSXJTF
   %JTDSJNJOBUPS &ODPEFS %FDPEFS -45. &ODPEFS (SPVOE
   5SVUI -PDBM
   MPTT $SPTT&OUSPQZ (MPCBM
   MPTT 4FOUFODF
   &NCFEEJOH
9. -PTT
   GVODUJPOT
    • (MPCBMMPTT ੜ੒จͱࢀরจͷ &NCFEEJOH
   ͷ಺ੵΛ࠷େԽ ੜ੒จͱόον಺ͷࢀরจҎ֎ͷจͷ &NCFEEJOH
   ͷ಺ੵΛ࠷খԽ
   1BJSXJTF
   %JTDSJNJOBUPS &ODPEFS %FDPEFS -45. &ODPEFS (SPVOE
   5SVUI -PDBM
   MPTT DSPTTFOUSPQZ (MPCBM
   MPTT 4FOUFODF
   &NCFEEJOH
10. -PTT
    GVODUJPOT
     • ϩεؔ਺ʢ5PUBMʣ MPDBM
    MPTT
    ͱ HMPCBM
    MPTT
    ͷ࿨
    1BJSXJTF
    %JTDSJNJOBUPS &ODPEFS %FDPEFS -45. &ODPEFS (SPVOE
    5SVUI -PDBM
    MPTT DSPTTFOUSPQZ (MPCBM
    MPTT 4FOUFODF
    &NCFEEJOH
11. $POUFOUT  *OUSPEVDUJPO  .FUIPET  &YQFSJNFOUT  $PODMVTJPO 
    
12. &YQFSJNFOUT
    PWFSWJFX ͭͷλεΫͰ࣮ݧ • 1BSBQISBTF
    HFOFSBUJPO • 4FOUJNFOU
    "OBMZTJT
    
    
13. 1BSBQISBTF
    (FOFSBUJPO • σʔληοτ 2VPSB EBUBTFU
    
    2VPSB ͷ࣭໰จ͔Βॏෳͨ͠ϖΞΛநग़ʢ,ʣ • ࣮ݧઃఆ ,Λ UFTU
    ηοτɼ,Λ WBMJEBUJPO
    ηοτͱͯ͠༻͍Δ ੜ੒จͱࢀরจͷྨࣅ౓Λ #-&6
    ౳ͰධՁ
    
14. 1BSBQISBTF
    (FOFSBUJPO Ø#BTFMJOFʢ4FR4FRʣ ͱͷൺֱ %JTDSJNJOBUPS
    ͱ &ODPEFS
    ͸ॏΈΛڞ༗ͨ͠ํ͕ྑ͍ σʔληοτͷαΠζʹؔΘΒͣɼ&%%-(
    TIBSFE
    ͕ͭΑ͍
    
15. 1BSBQISBTF
    (FOFSBUJPO Øଞͷख๏ͱͷൺֱ
    
16. 4FOUJNFOU
    "OBMZTJT • σʔληοτ 4UBOGPSE
    4FOUJNFOU
    5SFFCBOL 5SBJO
    ,
    7BMJE
    ,
    5FTU
    , • ࣮ݧઃఆ GJOFHSBJOFE
    DMBTTJGJDBUJPO
    UBTL
    ஋෼ྨ
    ͰධՁ \7FSZ
    OFHBUJWF
    /FHBUJWF
    /FVUSBM
    1PTJUJWF
    7FSZ
    QPTJUJWF^ ධՁࢦඪ͸ &SSPS
    3BUF
    
17. 4FOUJNFOU
    "OBMZTJT Ø݁Ռ
    
18. $POUFOUT  *OUSPEVDUJPO  .FUIPET  &YQFSJNFOUT  $PODMVTJPO
    
19. $PODMVTJPO • 4FR4FR
    ϕʔεͷݴ͍׵͑ੜ੒ϞσϧʹΑΔจ &NCFEEJOH
    ख๏ʹର͠ɼ ࢀরจͱੜ੒จͷҙຯతͳۙ͞Λอূ͢Δ QBJSXJTF
    EJTDSJNJOBUPS
    MPTT
    HMPCBM
    MPTT
    Λ௥Ճ • 1BSBQISBTF
    (FOFSBUJPO
    ΍ 4FOUJNFOU
    "OBMZTJT
    Ͱ 405"
    
20. ͓ؾ࣋ͪʢ͋ͱͰফ͢εϥΠυʣ Ø 405"
    Λେʑతʹओு͢Δʹ͸͍Ζ͍Ζͱऑ͍ؾ͕͢Δʜ • 1BSBQISBTF
    ͸ଞͷσʔληοτ΋ࢼ͢΂͖Ͱ͸ʜ • 4FOUJNFOU
    ͸ 2VJDL5IPVHIUT ͱ͔ *OGFS4FOU ͱൺֱ͢΂͖Ͱ͸ʜ ͳΜͰ GJOFHSBJOFE DMBTTFT
    ͚ͩͳͷ͔΋Θ͔Βͳ͍ʜ CJOBSZ
    ࡌͤͯ͘ΕͨΒ 25
    ͷ࿦จͱൺֱͰ͖Δͷʹʜ Ø %JTDSJNJOBUPS ػߏࣗମ͸ 7"&
    ͷ ,-DPMMBQTF
    ཈੍ͱ͔ʹ΋࢖͑ͦ͏