Budzianowski et al. - EMNLP 2018 - MultiWOZ - A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling

Transcript

1. MultiWOZ – A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue

   Modeling Tosho Hirasawa

2. 0. Overview • -6<+?E$> • 4L3I/%) H2  • :@Multi-Domain

   Wizard-of-Oz (MultiWOZ) • KJ • ("*72 GA/9F8 #!= 5 ,1 • 0.BD&* ' &(*;C

3. 1. Introduction • Conversational Artificial Intelligence • human-level *)&($ •

    #%' ! •   Seneff and Polifroni, 2000 • "Raux et al., 2005 • Amazon AlexaRam et al., 2018

4. 1. Introduction    • \T@F [C0*%0# RA •

   2DKU • =W:J • ?6) 8V • OXN3A • PH517 E2E ,"/LI  • <];Z17M
   YB( >E • &!-0Q • " 9 • [C$+0_4D • GS5'.-0^

5. 1. Introduction ,    , 2017

6. 2. Related Works • >K&.(%3/9 ! • Machine-to-Machine • *5/4+"O6K"R

   • HLJ-$) T DM6K\E  ]X • Human-to-Machine • 7:=@^Y'(*0UZ9";I • G OE! :B • HLJ^Y'(*0 YS?,1$5&.(NI  • Human-to-Human • G<QW &(+< • Twitter, Reddit, Ubuntu 6K"_8NI! • HLJ6KC[ AP#-*'25 FV 

7. 3. Data Collection Set-up • Wizard-of-Oz E4 • Dialogue Task:

   • *,-@ ontology  random sampling !'#%"8(6  • User Side: • (6=1 97CF.;A  • System (Wizard) Side: • $ 2:  97/D  • Wizard/User (6>, (6JG+ • (6)I30<  • (6H5&?B)I30 

8. 3. Data Collection Set-up • Annotation of Dialogue Acts •

   Dialogue Act = intent + slot-value pairs • intent: inform / request • slot-value: domain, price, … • Amazon Mechanical Turk +!" &$ dialogue acts .)  • !" &$ '- /(  • % ,*0.8843#0

9. 4. MultiWOZ Dialogue Corpus •     

   : domain

10. 4. MultiWOZ Dialogue Corpus : expensive : domain

11. 4. MultiWOZ Dialogue Corpus •   (turns in a

    dialogue) • 8.93 (single-domain), 15.39 (multi-domain) • 115,434 turns • >70%   10 turns  •  (sentence length) • 11.75 (user), 15.12 (wizard)

12. 4. MultiWOZ Dialogue Corpus • Dialogue Acts • 60% turns

     action  • %#  •  "$    • %# !"$  

13. 4. MultiWOZ Dialogue Corpus •     •

      • Multi-Domain,  Dialogue Act

14. 5. MultiWOZ as a New Benchmark • Dialogue modelling task

    • Dialogue State Tracking • (,# '/ • &,.5-0)1 ontology • Dialogue-Context-to-Text Generation • (,Dialogue State, # '/ • &,!16 • Cam676/MultiWOZ 28 •  %    $"+* • RNN 473 • Cam676: GRU • MultiWOZ: LSTM

15. 5. MultiWOZ as a New Benchmark • Dialogue-Act-to-Text Generation •

    Structured meaning representation (Dialogue Act?) •   • Semantically Conditioned LSTM (Wen+, 2015)  • SFX  MultiWOZ  restaurant   • SER = (missing slots + redundant slots) / total slots Wen+, 2015

16. 6. Conclusion • )1"&7* 
    8  E2E #$20 

    • Modular-based (+%' • MultiWOZ 3 46   • !
    -53.    github /,