A Knowledge-Grounded Neural Conversation Model

Transcript

1. A Knowledge-Grounded Neural Conversation Model ӣࢶਔ AAAI 2018, Microsoft
2. Fully data-driven conversational modelਸ ݅٘ח Ѫ਷ ݒ਋ য۵׮.
3. ਋ܻ੄ ӝ؀ അप Q: ੉ٮо ఃૉঙ݋ тѢঠ R: झಅైח ԙ ٘࣊ࠁࣁਃ. Q: ੉ٮо ఃૉঙ݋ тѢঠ R: જ਷ दр ࠁղࣁਃ!
4. ਋ܻ੄ ӝ؀ അप Q: ੉ٮо ఃૉঙ݋ тѢঠ R: झಅైח ԙ ٘࣊ࠁࣁਃ. Q: ੉ٮо ఃૉঙ݋ тѢঠ R: જ਷ दр ࠁղࣁਃ! ௪ܻ੄ entityܳ ੸੺ೞѱ ߈৔ೞח (௑బஎܳ ׸Ҋ ੓ח) ׹߸ ੸੺ೞ૑݅, ௑బஎח ࠗ઒ೠ ׹߸
5. Ӓېࢲ? Slot-filling ١ਸ ੉ਊೞ૑݅, ੌ߈ച(generalization)৬ ഛ੢ࢿ(scalability)ী ೠ҅о ੓׮.
6. ؘ੉ఠ݅ ੉ਊ೧ࢲ ޙ੢ী ૑ध/௑బஎܳ ׸ਸ ࣽ হਸө?
7. ؘ੉ఠ݅ ੉ਊ೧ࢲ ޙ੢ী ૑ध/௑బஎܳ ׸ਸ ࣽ হਸө? Ӓ۞ա, ੌ߈੸ੋ ؀ച ؘ੉ఠࣇ਷ Wikipedia, IMDB ١ী ١੢ೞח entityী ؀ೠ ੿ࠁܳ ݽف ׸Ҋ ੓૑ח ঋ׮.
8. Ѣ੄ ݽٚ Knowledge entityܳ ׸Ҋ ੓ח ؘ੉ఠо ઓ੤ೞ؊ۄب 1. ೟णೞӝী ցޖ ௼Ҋ 2. (࠺तೠ entityী ؀೧) ࠛ೙ਃೠ ؀ച ಁఢ੉ ઓ੤ ؘ੉ఠ݅ ੉ਊ೧ࢲ ޙ੢ী ૑ध/௑బஎܳ ׸ਸ ࣽ হਸө?
9. Grounded Response Generation 1. ࠛ೙ਃೠ ؀ച ಁఢ ೟णਸ ೖೞҊ 2. ӝઓ੄ ؀ച ؘ੉ఠࣇਵ۽ ੌ߈ചػ ؀ച ಁఢਸ ೟णೞ੗ Q: Looking forward to trying @pizzalibretto tonight! My expectations are high. R: Get the Rocco salad. Can you eat calamari? Q: I’m at California Academy of Sciences R: Make sure you catch the show at the Planetarium. Tickets are usually limited. Q: I just bought: […] 4-3-inch portable GPS navigator for my wife, shh, don’t tell her. R: I heard this brand loses battery power.
10. Knowledge-Grounded Model Architecture
11. Datasets Non-conversational Data: Foursquare • ࠘޷ 11ѐ بद, بब ઺बࠗ 1.1M tips ੉ਊ • 10ѐ ޷݅ tipsܳ ߉਷ Ҕ਷ ઁ৻ • Handle (username)੉ Twitter ؘ੉ఠীࢲ ١੢ೞח tips݅ ࢎਊ Conversational Data: Twitter • 23Mѐ੄ 3ఢ ؀ച Grounded Conversation Datasets (from Twitter) • Foursquare ؘ੉ఠী ١੢ೞח twitter handle੉ ୐ ఢী ١੢ೞח 2ఢ ؀ച
 (׹߸਷ Foursquare ؘ੉ఠ ղী হח twitter handle੄ ׹߸ੌ ҃਋ী݅ ࢎਊ) • LM perplexity, chi-square, randomਵ۽ пп 15K ୶୹ • ੉ ઺ 10Kܳ human judge → 4Kܳ dev/test setਵ۽ ࢎਊ
12. 1. purely conversational data (S, R) (w/o Facts Encoder) 2. Full model with facts ({f_1, …, f_k, S}, R) (w/ Facts Encoder) 1. ੌ߈ ؀ച ؘ੉ఠ݅ਸ ੉ਊೠ pretrain оמ 2. ف taskী ਬোೠ ؘ੉ఠ ࢎਊ 3. 2ߣ taskীࢲ R=F۽ فݶ autoencoder୊ۢ ੘ز
 (ખ؊ ௑బஎܳ ݆੉ ׸਷ ׹߸) Multi-task learning
13. Facts Encoder Memory Network (Sukhbaatar et al. 2015)ܳ ੌࠗ ߸ഋ೧ࢲ ࢎਊ Fact set Input sentence: ҅࢑ী BoW ؀न RNN Encoder੄ hidden state ࢎਊ Dialog Encoder ∑ Decoder u Facts Encoder
14. Decoding & Reranking log P(R|S, F) + λ log P(S|R) + γ|R| Beam-search Decoder (B=200) N-best response ࢤࢿ (reranking) Reranking Score ҅࢑:
15. Facts task Full model ({f_1, …, f_n, S}, R) NoFacts task w/o fact encoder (S, R) AutoEncoder task Facts taskীࢲ Rਸ f۽ Ү୓ ({f_1, …, f_n, S}, f_i) Experiments Seq2Seq NoFacts + 23M general conversation dataset (no multi) MTask NoFacts (23M general + 1M grounded) MTask-R NoFacts (23M general) + Facts (1M grounded) MTask-F NoFacts (23M general) + AutoEncoder (1M grounded) MTask-RF NoFacts (23M general) + Facts (1M grounded) + AutoEncoder (1M grounded) Task setting Model setting
16. Results PPL & BLEU
17. Results Human Evaluation
18. Examples (MTask-RF) Negative examples
19. Discussion • MemNN੄ ഝਊ оמࢿ (MemNN ੗୓੄ ഝਊ оמࢿ)
 ֢੉ૉо ݆਷ पઁ ؀ച ؘ੉ఠীࢲب MemNN੉ ੜ ੘زೡ Ѫੋо? (context ӡ੉о ӡ য૕ ٸ)
 • पઁ ੸ਊਸ ਤ೧ࢲח KBܳ conversation modelীࢲ যڌѱ ഝਊೞҊ ੓ח૑ ؊ ܻࢲ஖ о ೙ਃ೧ ࠁ੐ • ֤ޙীࢲ ੉ਊೠ ध׼(੢ࣗ) ৻੄ بݫੋਵ۽ ഛ੢ਸ ਤ೧ࢲח पઁ۽ח contextually relevant factsܳ ഛ੢ೞח بݫੋ݅ఀ ೟णী ੉ਊ೧ঠ ೞ૑ જ਷ ݽ؛ ࢿמ੉ աয়૑ ঋਸ ө? (֤ޙীࢲ ઱੢ೞח Ѫ݅ఀ scalable ೠо?) • informativeೞ૑ ঋ਷ ੌ߈ ؀ച੄ ௬ܻ౭ח যו ੿بੌө?