Share
Embed
Transcript
19 slides
-
ܻ ӝ അप Q: ٮо ఃૉঙ тѢঠ R: झಅైח ԙ
٘࣊ࠁࣁਃ. Q: ٮо ఃૉঙ тѢঠ R: જ दр ࠁղࣁਃ! -
ܻ ӝ അप Q: ٮо ఃૉঙ тѢঠ R: झಅైח ԙ
٘࣊ࠁࣁਃ. Q: ٮо ఃૉঙ тѢঠ R: જ दр ࠁղࣁਃ! ௪ܻ entityܳ ೞѱ ߈ೞח (బஎܳ Ҋ ח) ߸ ೞ݅, బஎח ࠗೠ ߸ -
ؘఠ݅ ਊ೧ࢲ ޙী ध/బஎܳ ਸ ࣽ হਸө? Ӓ۞ա, ੌ߈ੋ ച
ؘఠࣇ Wikipedia, IMDB ١ী ١ೞח entityী ೠ ࠁܳ ݽف Ҋ ח ঋ. -
Ѣ ݽٚ Knowledge entityܳ Ҋ ח ؘఠо ઓೞ؊ۄب 1. णೞӝী
ցޖ Ҋ 2. (࠺तೠ entityী ೧) ࠛਃೠ ച ಁఢ ઓ ؘఠ݅ ਊ೧ࢲ ޙী ध/బஎܳ ਸ ࣽ হਸө? -
Grounded Response Generation 1. ࠛਃೠ ച ಁఢ णਸ ೖೞҊ 2.
ӝઓ ച ؘఠࣇਵ۽ ੌ߈ചػ ച ಁఢਸ णೞ Q: Looking forward to trying @pizzalibretto tonight! My expectations are high. R: Get the Rocco salad. Can you eat calamari? Q: I’m at California Academy of Sciences R: Make sure you catch the show at the Planetarium. Tickets are usually limited. Q: I just bought: […] 4-3-inch portable GPS navigator for my wife, shh, don’t tell her. R: I heard this brand loses battery power. -
Datasets Non-conversational Data: Foursquare • ࠘ 11ѐ بद, بब बࠗ
1.1M tips ਊ • 10ѐ ݅ tipsܳ ߉ Ҕ ઁ৻ • Handle (username) Twitter ؘఠীࢲ ١ೞח tips݅ ࢎਊ Conversational Data: Twitter • 23Mѐ 3ఢ ച Grounded Conversation Datasets (from Twitter) • Foursquare ؘఠী ١ೞח twitter handle ఢী ١ೞח 2ఢ ച (߸ Foursquare ؘఠ ղী হח twitter handle ߸ੌ ҃ী݅ ࢎਊ) • LM perplexity, chi-square, randomਵ۽ пп 15K ୶ • 10Kܳ human judge → 4Kܳ dev/test setਵ۽ ࢎਊ -
1. purely conversational data (S, R) (w/o Facts Encoder) 2.
Full model with facts ({f_1, …, f_k, S}, R) (w/ Facts Encoder) 1. ੌ߈ ച ؘఠ݅ਸ ਊೠ pretrain оמ 2. ف taskী ਬোೠ ؘఠ ࢎਊ 3. 2ߣ taskীࢲ R=F۽ فݶ autoencoderۢ ز (ખ؊ బஎܳ ݆ ߸) Multi-task learning -
Facts Encoder Memory Network (Sukhbaatar et al. 2015)ܳ ੌࠗ ߸ഋ೧ࢲ
ࢎਊ Fact set Input sentence: ҅ী BoW न RNN Encoder hidden state ࢎਊ Dialog Encoder ∑ Decoder u Facts Encoder -
Decoding & Reranking log P(R|S, F) + λ log P(S|R)
+ γ|R| Beam-search Decoder (B=200) N-best response ࢤࢿ (reranking) Reranking Score ҅: -
Facts task Full model ({f_1, …, f_n, S}, R) NoFacts
task w/o fact encoder (S, R) AutoEncoder task Facts taskীࢲ Rਸ f۽ Ү ({f_1, …, f_n, S}, f_i) Experiments Seq2Seq NoFacts + 23M general conversation dataset (no multi) MTask NoFacts (23M general + 1M grounded) MTask-R NoFacts (23M general) + Facts (1M grounded) MTask-F NoFacts (23M general) + AutoEncoder (1M grounded) MTask-RF NoFacts (23M general) + Facts (1M grounded) + AutoEncoder (1M grounded) Task setting Model setting -
Discussion • MemNN ഝਊ оמࢿ (MemNN ഝਊ оמࢿ) ֢ૉо
݆ पઁ ച ؘఠীࢲب MemNN ੜ زೡ Ѫੋо? (context ӡо ӡ য ٸ) • पઁ ਊਸ ਤ೧ࢲח KBܳ conversation modelীࢲ যڌѱ ഝਊೞҊ ח ؊ ܻࢲ о ਃ೧ ࠁ • ֤ޙীࢲ ਊೠ ध(ࣗ) ৻ بݫੋਵ۽ ഛਸ ਤ೧ࢲח पઁ۽ח contextually relevant factsܳ ഛೞח بݫੋ݅ఀ णী ਊ೧ঠ ೞ જ ݽ؛ ࢿמ աয় ঋਸ ө? (֤ޙীࢲ ೞח Ѫ݅ఀ scalable ೠо?) • informativeೞ ঋ ੌ߈ ച ௬ܻ౭ח যו بੌө?