Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Towards Universal Dialog State Tracking

Towards Universal Dialog State Tracking

Scatter Lab Inc.

August 28, 2019

Transcript

25 slides
  1. Towards Universal Dialogue State Tracking Overview • “Towards Universal Dialogue

    State Tracking” • Liliang Ren, Kaige Xie, Lu Chen and Kai Yu (Shanghai Jiao Tong University) • EMNLP 2018 Oral • Contributions • Dialogue State Trackingਸ ਤೠ “StateNet” ઁউ • DSTC2, WoZ datasetীࢲ state-of-the-art ࢿמ
  2. Dialogue System 1. Introduction Natural Language Understanding (NLU) Dialogue Management

    (DM) Natural Language Generation (NLG) Response Query Dialogue Management (DM)
  3. Dialogue Management 1. Introduction • DM਷ ௼ѱ 2о૑ ޷࣌ਸ ыҊ

    ੓਺ 1. Dialogue State Tracking (DST) • അ੤ ؀ചо যڌѱ ൗ۞оҊ ੓ח૑ ౵ঈ • ૑Әө૑ ؀ചܳ ా೧ ঳਷ ੿ࠁܳ ӝ߈ਵ۽ ࢚కܳ ҳೞҊ ਬ૑ 2. Decision Making • അ੤ ࢚కী ঌݏ਷ ঘ࣌ਸ Policyী ٮۄ Ѿ੿
  4. Dialogue State Tracking 1. Introduction • ؀ച੄ ࢚కܳ ୶੸ೞח NLP,

    Dialogue ࠙ঠ੄ ؀಴੸ੋ ޙઁ • Task-oriented Dialogueܳ ੹੤۽ ೣ • ݾ੸ = User੄ Goalী ب׳ೞҊ Requestܳ ೧Ѿ೧઱ח Ѫ • Ex. ۨझషی ୶ୌਸ ਤೠ ؀ച दझమ • ݾ੸ = Userо ਗೞח ۨझషی੄ ઑѤਸ ଺ইղח Ѫ • DSTח ੉۠ Goalਸ ੜ ଺ইоח૑ ݒ ఢ݃׮ ഛੋೞח Ѫ
  5. Dialogue State Tracking Challenge 2 (DSTC2) 1. Introduction • Target

    Label = (act, slot, value) • act = {inform, request} • slot of inform = {food, pricerange, name, area} • food = {korean, chinese, japanese, french, halal, …} # 91ѐ • pricerange = {cheap, moderate, expensive} • name = {оѱ ੉ܴ} # 113ѐ • area = {north, south, east, west, centre} • slot of request = slot of inform + {addr, phone, postcode, signature} • Evaluation Metric = (Joint) Accuracy
  6. Limitations of Previous Works 1. Introduction • Slot੄ Valuesо ز੸ਵ۽

    ߸ೞח ജ҃ীࢲ ੜ ੘زೞ૑ ঋ਺ • ҙҟ ੿ࠁ दझమী ࢜۽਍ Value(ۨझషی, ഐభ ١)੉ ୶оؼ ࣻ ੓਺ • Slot੄ ѐࣻী ࠺۹೧ࢲ ݽ؛ ౵ۄ޷ఠ ௼ӝо ழ૗ • Slot݃׮ ݽ؛ਸ ݅ٞ • Hand-crafted Lexicon Featureܳ ӝ߈ਵ۽ ೣ • ী۞ ੹౵, ࣘب ੷ೞ
  7. StateNet: A Universal Dialogue State Tracker 2. Method Utterance Machine

    Act Slot Values for Slot Probability Distribution of Values
  8. StateNet: A Universal Dialogue State Tracker 2. Method “ೠध੉ જਸ

    Ѫ эই” request(food) food {ೠध, ઺ध, …} [ೠध: 0.8, ઺ध: 0.1 …]
  9. Utterance Representation 2. Method • Multi-scale Receptors Layerܳ ా೧ n-gram

    ӝ߈੄ repr ࢤࢿ
 = 1-gram, 2-gram, …, n-gram੄ reprܳ ׮ Sumೠ Ѫ • K-gram੄ Representation • kѐ ױਤ੄ seq of wordsܳ ҳࢿೞח word vector ೤ • ਤ੄ word vector ೤ਸ ݽف concat • Concat ӝ߈੉ӝ ٸޙী kী ٮۄࢲ ߭ఠ ӡ੉о ׳ۄઉࢲ Linear కਕࢲ ӡ੉ ݏ୺
  10. Machine Act Representation 2. Method • Machine Actח ૒੹ दझమ

    ߊച੄ ӝ߈੉ غח Dialogue Act • e.g., welcomemsg, request, canthelp, inform, offer ١ • ਤ੄ ױয(pre-defined)ী ؀೧ࢲ Bag-of-words۽ ಴അ • ੉റ ো࢑ਸ ਤ೧ Linear కਕࢲ ରਗਸ Utterance Representationҗ ݏ୺
  11. Slot Information Decoding 2. Method • Slotী ؀ೠ ੿ࠁܳ ೣԋ

    ֍যષ • Slot੄ ੉ܴ ੗୓ী ؀ೠ pre-trained word embeddingਸ ഝਊ • ৉द ରਗਸ ݏ୾઱ӝ ਤ೧ Linear క਑
  12. Turn-level Feature Vector 2. Method • খࢲ ҳೠ Utterance, Machine

    Act, Slotਸ ઙ೤ೠ Vector • Utterance৬ Machine Actח concatೞҊ
 Ӓ Ѿҗী Slotਸ point-wise multiplicationೣ • ୭ઙ੸ਵ۽ ҳೠ Turn-level Feature Vectorח LSTM੄ Input੉ ؽ
  13. 2-Norm Distance 2. Method • LSTM੄ output, ૊ Context৬ Slotਸ

    Ҋ۰ೠ reprਸ ҳೣ • ੉ী ؀೧ࢲ оמೠ ݽٚ Valueٜҗ੄ ਬࢎب(L2)ܳ ҳೣ • Ѿҗী ؀೧ࢲ Softmax ஂ೧ࢲ ୭ઙ Slotী ؀ೠ Value Prediction • Slotী ؀ೠ Valueо (ই૒) হਸ ࣻ ੓ӝী “None”ਸ Value Setী ୶о • ૑Әө૑੄ җ੿ਸ п ఢ݃׮ ݽٚ Slot(inform)ী ؀೧ ૓೯ [ೠध: 0.8, ઺ध: 0.1 …] {ೠध, ઺ध, …} food Context
  14. Performance: Joint Goal Accuracy 3. Experiments • Parameter Sharing among

    the Different Slots • ੉੹ োҳח Slot݃׮ ݽ؛ਸ ٜ݅঻חؘ ੉ܳ ӓࠂ • Initialization with a Pre-trained Model • ೞա੄ Slotী ؀೧ࢲ Pre-trainingਸ ೞҊ, 
 Ѿҗ Weight۽ ݽٚ Slotী ؀ೠ ݽ؛ਸ Initialization
  15. Initialization with a Pre-trained Model 3. Experiments • ੹୓ Slot

    ઺ foodী ؀೧ࢲ pre-trainingਸ ೮ਸ ٸ о੢ જও਺ • ׮ܲ Slotী ࠺೧ foodо о੢ য۵ӝ ٸޙী pre-trainingী ؀ೠ
 ੉ٙ੉ о੢ ௾ ѱ ইקө ୶ஏ • ੌઙ੄ Weak Slotী ؀ೠ Boosting ѐ֛ਵ۽ ࢤп೧ࠅ ࣻ ੓਺
 (Ӓր ੜ ޅ ݏ୶ח Ѣ(Weakness)ী ؀೧ ೟ण۝ਸ ט۰ࢲ ࢿמਸ ֫ੋ ࣅ = Boosting)
  16. Contributions 4. Conclusion • ੉੹ ݽ؛ٜ੄ ೠ҅ܳ ӓࠂೞח StateNet ઁউ

    • ز੸ਵ۽ Slot੄ Valueо ߸ೡ ٸ ੜ ੘زೞ૑ ঋ਺ • Slotী ٮܲ ѐ߹ ݽ؛۽ ੋೠ ੹୓ ౵ۄ޷ఠ ন ૐо • Hand-crafted Lexicon Feature ࢎਊ • DSTC2, WoZ datasetী ؀೧ࢲ state-of-the-art ࢿמਸ ࠁ੐
  17. хࢎ೤פ׮✌ ୶о ૕ޙ ژח ҾӘೠ ੼੉ ੓׮ݶ ঱ઁٚ ইې োۅ୊۽

    োۅ ઱ࣁਃ! ੉઱ഘ (ML Engineer, Pingpong) Email. [email protected] Facebook. @roomylee Linked in. @roomylee

More Decks by Scatter Lab Inc.

Other Decks in Research