Budzianowski et al. - EMNLP 2018 - MultiWOZ - A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling

Transcript

1. MultiWOZ – A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modeling Tosho Hirasawa
2. 0. Overview • -6<+?E$> • 4L3I/%) H2  • :@Multi-Domain Wizard-of-Oz (MultiWOZ) • KJ • ("*72 GA/9F8 #!= 5 ,1 • 0.BD&* ' &(*;C
3. 1. Introduction • Conversational Artificial Intelligence • human-level *)&($ •  #%' ! •   Seneff and Polifroni, 2000 • "Raux et al., 2005 • Amazon AlexaRam et al., 2018
4. 1. Introduction    • \T@F [C0*%0# RA • 2DKU • =W:J • ?6) 8V • OXN3A • PH517 E2E ,"/LI  • <];Z17M
   YB( >E • &!-0Q • " 9 • [C$+0_4D • GS5'.-0^
5. 1. Introduction ,    , 2017
6. 2. Related Works • >K&.(%3/9 ! • Machine-to-Machine • *5/4+"O6K"R • HLJ-$) T DM6K\E  ]X • Human-to-Machine • 7:=@^Y'(*0UZ9";I • G OE! :B • HLJ^Y'(*0 YS?,1$5&.(NI  • Human-to-Human • G<QW &(+< • Twitter, Reddit, Ubuntu 6K"_8NI! • HLJ6KC[ AP#-*'25 FV 
7. 3. Data Collection Set-up • Wizard-of-Oz E4 • Dialogue Task: • *,-@ ontology  random sampling !'#%"8(6  • User Side: • (6=1 97CF.;A  • System (Wizard) Side: • $ 2:  97/D  • Wizard/User (6>, (6JG+ • (6)I30<  • (6H5&?B)I30 
8. 3. Data Collection Set-up • Annotation of Dialogue Acts • Dialogue Act = intent + slot-value pairs • intent: inform / request • slot-value: domain, price, … • Amazon Mechanical Turk +!" &$ dialogue acts .)  • !" &$ '- /(  • % ,*0.8843#0
9. 4. MultiWOZ Dialogue Corpus •      : domain
10. 4. MultiWOZ Dialogue Corpus : expensive : domain
11. 4. MultiWOZ Dialogue Corpus •   (turns in a dialogue) • 8.93 (single-domain), 15.39 (multi-domain) • 115,434 turns • >70%   10 turns  •  (sentence length) • 11.75 (user), 15.12 (wizard)
12. 4. MultiWOZ Dialogue Corpus • Dialogue Acts • 60% turns  action  • %#  •  "$    • %# !"$  
13. 4. MultiWOZ Dialogue Corpus •     •   • Multi-Domain,  Dialogue Act
14. 5. MultiWOZ as a New Benchmark • Dialogue modelling task • Dialogue State Tracking • (,# '/ • &,.5-0)1 ontology • Dialogue-Context-to-Text Generation • (,Dialogue State, # '/ • &,!16 • Cam676/MultiWOZ 28 •  %    $"+* • RNN 473 • Cam676: GRU • MultiWOZ: LSTM
15. 5. MultiWOZ as a New Benchmark • Dialogue-Act-to-Text Generation • Structured meaning representation (Dialogue Act?) •   • Semantically Conditioned LSTM (Wen+, 2015)  • SFX  MultiWOZ  restaurant   • SER = (missing slots + redundant slots) / total slots Wen+, 2015
16. 6. Conclusion • )1"&7* 
    8  E2E #$20  • Modular-based (+%' • MultiWOZ 3 46   • !
    -53.    github /,