Generating More Interesting Responses in Neural Conversation Models with Distributional Constraints

弊研究室で行なったEMNLP2018読み会の発表資料です。

Transcript

1. &./-1
   :0.*,"*
   UI
   %FD
    (FOFSBUJOH
   .PSF
   *OUFSFTUJOH
   3FTQPOTFT
   JO
   /FVSBM
   $POWFSTBUJPO
   .PEFMT
   XJUI %JTUSJCVUJPOBM
   $POTUSBJOUT (3"%6"5&
   4$)00-
   PG
   */'03."5*0/
   4$*&/$&
   BOE
   5&$)/0-0(:
   04","
   6/*7 +6/:"
   5",":"."
2. 1BQFS
   */'0 "TIVUPTI #BIFUJ
   "MBO
   3JUUFS
   +JXFJ -J
   BOE
   #JMM
   %PMBO l(FOFSBUJOH
   .PSF
   *OUFSFTUJOH
   3FTQPOTFT
   JO
   /FVSBM
   $POWFSTBUJPO
   .PEFMT
   XJUI
   %JTUSJCVUJPOBM
   $POTUSBJOUTz *O
   1SPDFFEJOHT
   PG
   &./-1 ೖྗൃ࿩ͱͷτϐοΫҰ؏ੑ΍ҙຯతྨࣅ౓͕ߴ͍Ԡ౴จΛੜ੒͢ΔΑ͏ʹ ੍໿Λ͔͚ͯσίʔυ͢Δ͜ͱͰɼ%VMM
   3FTQPOTF
   ໰୊ʹରॲͨ͠Α
   
3. $POUFOUT  *OUSPEVDUJPO  .FUIPET  &YQFSJNFOUT  $PODMVTJPO
   
4. *TTVF Ø࠷໬ਪఆʹΑΔ UFYUUPUFYU
   ੜ੒Ϟσϧ͸ೖྗจͱग़ྗจ͕ ΄΅ҰରҰରԠͷλεΫʢػց຋༁ͳͲʣͰ͸༗༻ ØԠ౴ੜ੒͸ଟରଟͷͨΊ࠷໬ਪఆͰ͸ʮͦ͏Ͱ͢Ͷʯʮ͸͍ʯ ͳͲͷ EVMM
   SFTQPOTF
   Λੜ੒͕ͪ͠
   
5. *TTVF ØަࠩΤϯτϩϐʔଛࣦ ʹΑΔԠ౴ੜ੒ ʹ͓͚Δσίʔυྫ • τϐοΫϫʔυʹൺ΂ɼ ετοϓϫʔυͷ໬౓͕ ߴ͘ͳΓ͕ͪ • ස౓ͷ௿͍ޠ͸ ੜ੒ͮ͠Β͍
   
6. 0WFSWJFX (PBM
   ೖྗൃ࿩ͱରԠͨ͠಺༰ޠΛଟؚ͘ΉΑ͏ͳԠ౴จͷੜ੒ 1SPQPTBM
   ೖྗจ 9
   ग़ྗจ : ͱͨ͠ͱ͖ɼ • ग़ྗจͱԠ౴จͷτϐοΫͷҰ؏ੑ !"# $ % & , $ % ( • ग़ྗจͱԠ౴จͷҙຯతྨࣅ౓ !"# )#* & , )#* ( Λߟྀͨ͠Ԡ౴Λੜ੒͢ΔΑ͏ʹॏΈ෇͚͠ͳ͕Βσίʔυ͢Δ
   
7. $POUFOUT  *OUSPEVDUJPO  .FUIPET  &YQFSJNFOUT  $PODMVTJPO
   
8. "QQSPBDI ØτϐοΫͷҰ؏ੑ͕ߴ͘ͳΔΑ͏ʹग़ྗʢτϐοΫ੍໿ʣ Øग़ྗͱԠ౴ͷྨࣅ౓͕ߴ͘ͳΔΑ͏ʹग़ྗʢҙຯ੍໿ʣ
   τϐοΫྨࣅ౓ ҙຯతྨࣅ౓
9. ࣮ࡍͷσίʔυ͸୯ޠ୯ҐͰஞ࣍తʹग़ྗΛߦ͏ͨΊɼ τϐοΫ੍໿ɾҙຯ੍໿ʹ༻͍ΔείΞ͸ҎԼͷͭͷ৚݅ʀ • σίʔυ్தͷෆ׬શͳจΛར༻Մೳ • ܭࢉίετ͕௿͍ Λຬͨ͢ඞཁ͕͋Δ %FDPEJOH
   XJUI
   %JTUSJCVUJPOBM $POTUSBJOUT*O
   
10. ! " # ͷਪఆʹ͸ )..-%"
    ϞσϧΛ࠾༻ จதͷ֤୯ޠʹ͍ͭͯɼͦͷ୯ޠ͕ • τϐοΫ 5
    Ͱ͋Δ֬཰ • τϐοΫϫʔυͰ͋Δ֬཰ ͷੵΛͱΓɼτϐοΫϫʔυͷ਺ʢظ଴஋ʣͰॏΈ෇͚ฏۉ τϐοΫྨࣅ౓
    
11. ! " # ͷਪఆʹ͸ )..-%"
    ϞσϧΛ࠾༻ ཁ͢Δʹɿ୯ޠ͝ͱͷτϐοΫਪఆ݁ՌΛ଍͠߹Θͤ ˠσίʔυதͷෆ׬શͳจʹ΋ஞ࣍తʹద༻Մೳ ྨࣅ౓ؔ਺ ∆ !("|' , ! ) ' ʹ͸୯७ͳυοτੵΛ࠾༻ τϐοΫྨࣅ౓ 
    
12. จͷ෼ࢄදݱ !"# $ Λ "SPSB
    Β <>
    ͷख๏Λ༻͍ͯಘΔɿ  ୯ޠͷϢχάϥϜ֬཰ % &
    ύϥϝʔλ '
    ୯ޠ෼ࢄදݱ () Λ༻͍ͯɼ(* = ∑ )∈* . ./0 ) () Λܭࢉ  (* ͔ΒୈҰओ੒෼ ཁૉΛҾ͘ ୈҰओ੒෼͸ػೳޠͷӨڹ͕େ͖͍ʢΒ͍͠ʣ ࣮ࡍʹ͸܇࿅ίʔύεΛ༻͍ͯओ੒෼෼ੳΛࣄલʹߦ͍ɼ σίʔυ࣌ʹ͸ͦͷ݁ՌΛར༻ʢίʔυΛݟΔݶΓʣ ҙຯతྨࣅ౓ .0 7 27,- 2 .1 2 1 1 0 .1, A .
13. จͷ෼ࢄදݱ !"# $ Λ "SPSB
    Β <>
    ͷख๏Λ༻͍ͯಘΔɿ ཁ͢Δʹɿ୯ޠ͝ͱͷ෼ࢄදݱͷॏΈ෇͖ฏۉ ˠσίʔυ࣌ʹ΋ஞ࣍తʹૉૣ͘ܭࢉՄೳ ྨࣅ౓ؔ਺ % &' , &) ʹ͸୯७ͳυοτੵΛ࠾༻ ҙຯతྨࣅ౓ .0 7 27,- 2 .1 2 1 1 0 .1, A .
14. $POUFOUT  *OUSPEVDUJPO  .FUIPET  &YQFSJNFOUT  $PODMVTJPO
    
15. 4FUUJOHT • σʔληοτ ܇࿅σʔλɿ0QFO4VCUJUMFTʢ.
    VS
    QBJSTʣ ςετσʔλɿ$PSOFMM
    .PWJF
    %JBMPHVF
    $PSQVTʢ
    QBJST ʣ • ϕʔεϥΠϯ ..*
    Ԡ౴ͱೖྗͷ૬ޓ৘ใྔΛ࠷େԽ͢ΔϞσϧ ! = argmax( log + ! , + - 5"4FR4FR
    τϐοΫਪఆ݁ՌΛ "UUFOUJPO
    ܭࢉʹ༻͍ΔϞσϧ
    
16. 3FTVMUT
    "VUPNBUJD
    &WBMVBUJPO #-&6
    Λେ͖͘Լ͛ͣʹԠ౴ͷଟ༷ੑΛ޲্ ετοϓϫʔυ཈੍ʹ΋ޮՌ͋Γ
    
17. 3FTVMUT
    )VNBO ਓखධՁͰ΋Ԡ౴ͷ໬΋Β͠͞ΛଛͶͣʹ಺༰ͷॆ࣮౓͕޲্
    
18. $POUFOUT  *OUSPEVDUJPO  .FUIPET  &YQFSJNFOUT  $PODMVTJPO
    
19. $PODMVTJPO Ø ෼ࢄදݱͷܗͰग़ྗͱԠ౴ͷτϐοΫྨࣅ౓ͱҙຯతྨࣅ౓Λ ࢉग़͠ɼσίʔυ࣌ͷ੍໿ͱͯ͠૊ΈࠐΜͩ Ø Ԡ౴ͷ໬΋Β͠͞ΛԼ͛Δ͜ͱͳ͘ɼଟ༷ੑΛ޲্ͤͨ͞