Budzianowski et al. - EMNLP 2018 - MultiWOZ - A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling

F16d24f8c3767910d0ef9dd3093ae016?s=47 tosho
December 10, 2018

Budzianowski et al. - EMNLP 2018 - MultiWOZ - A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling

F16d24f8c3767910d0ef9dd3093ae016?s=128

tosho

December 10, 2018
Tweet

Transcript

  1. MultiWOZ – A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue

    Modeling Tosho Hirasawa
  2. 0. Overview • -6<+?E$> • 4L3I/%) H2  • :@Multi-Domain

    Wizard-of-Oz (MultiWOZ) • KJ • ("*72 GA/9F8 #!= 5 ,1 • 0.BD&* ' &(*;C
  3. 1. Introduction • Conversational Artificial Intelligence • human-level *)&($ •

     #%' ! •   Seneff and Polifroni, 2000 • "Raux et al., 2005 • Amazon AlexaRam et al., 2018
  4. 1. Introduction    • \T@F [C0*%0# RA •

    2DKU • =W:J • ?6) 8V • OXN3A • PH517 E2E ,"/LI  • <];Z17MYB( >E • &!-0Q • " 9 • [C$+0_4D • GS5'.-0^
  5. 1. Introduction ,    , 2017

  6. 2. Related Works • >K&.(%3/9 ! • Machine-to-Machine • *5/4+"O6K"R

    • HLJ-$) T DM6K\E  ]X • Human-to-Machine • 7:=@^Y'(*0UZ9";I • G OE! :B • HLJ^Y'(*0 YS?,1$5&.(NI  • Human-to-Human • G<QW &(+< • Twitter, Reddit, Ubuntu 6K"_8NI! • HLJ6KC[ AP#-*'25 FV 
  7. 3. Data Collection Set-up • Wizard-of-Oz E4 • Dialogue Task:

    • *,-@ ontology  random sampling !'#%"8(6  • User Side: • (6=1 97CF.;A  • System (Wizard) Side: • $ 2:  97/D  • Wizard/User (6>, (6JG+ • (6)I30<  • (6H5&?B)I30 
  8. 3. Data Collection Set-up • Annotation of Dialogue Acts •

    Dialogue Act = intent + slot-value pairs • intent: inform / request • slot-value: domain, price, … • Amazon Mechanical Turk +!" &$ dialogue acts .)  • !" &$ '- /(  • % ,*0.8843#0
  9. 4. MultiWOZ Dialogue Corpus •     

    : domain
  10. 4. MultiWOZ Dialogue Corpus : expensive : domain

  11. 4. MultiWOZ Dialogue Corpus •   (turns in a

    dialogue) • 8.93 (single-domain), 15.39 (multi-domain) • 115,434 turns • >70%   10 turns  •  (sentence length) • 11.75 (user), 15.12 (wizard)
  12. 4. MultiWOZ Dialogue Corpus • Dialogue Acts • 60% turns

     action  • %#  •  "$    • %# !"$  
  13. 4. MultiWOZ Dialogue Corpus •     •

      • Multi-Domain,  Dialogue Act
  14. 5. MultiWOZ as a New Benchmark • Dialogue modelling task

    • Dialogue State Tracking • (,# '/ • &,.5-0)1 ontology • Dialogue-Context-to-Text Generation • (,Dialogue State, # '/ • &,!16 • Cam676/MultiWOZ 28 •  %    $"+* • RNN 473 • Cam676: GRU • MultiWOZ: LSTM
  15. 5. MultiWOZ as a New Benchmark • Dialogue-Act-to-Text Generation •

    Structured meaning representation (Dialogue Act?) •   • Semantically Conditioned LSTM (Wen+, 2015)  • SFX  MultiWOZ  restaurant   • SER = (missing slots + redundant slots) / total slots Wen+, 2015
  16. 6. Conclusion • )1"&7*  8  E2E #$20 

    • Modular-based (+%' • MultiWOZ 3 46   • !-53.    github /,