Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Sign up for free
Menu
Search
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Pricing
Search
Sign in
Sign up for free
Budzianowski et al. - EMNLP 2018 - MultiWOZ - A...
Search
tosho
December 10, 2018
Research
380
0
Share
Embed
Copy iframe code
Copy JS code
Copy link
Start on current slide
Budzianowski et al. - EMNLP 2018 - MultiWOZ - A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling
tosho
December 10, 2018
Transcript
16 slides
MultiWOZ – A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue
Modeling Tosho Hirasawa
0. Overview • -6<+?E$> • 4L3I/%) H2 • :@Multi-Domain
Wizard-of-Oz (MultiWOZ) • KJ • ("*72 GA/9F8 #!= 5 ,1 • 0.BD&* ' &(*;C
1. Introduction • Conversational Artificial Intelligence • human-level *)&($ •
#%' ! • Seneff and Polifroni, 2000 • "Raux et al., 2005 • Amazon AlexaRam et al., 2018
1. Introduction • \T@F [C0*%0# RA •
2DKU • =W:J • ?6) 8V • OXN3A • PH517 E2E ,"/LI • <];Z17MYB( >E • &!-0Q • " 9 • [C$+0_4D • GS5'.-0^
1. Introduction , , 2017
2. Related Works • >K&.(%3/9 ! • Machine-to-Machine • *5/4+"O6K"R
• HLJ-$) T DM6K\E ]X • Human-to-Machine • 7:=@^Y'(*0UZ9";I • G OE! :B • HLJ^Y'(*0 YS?,1$5&.(NI • Human-to-Human • G<QW &(+< • Twitter, Reddit, Ubuntu 6K"_8NI! • HLJ6KC[ AP#-*'25 FV
3. Data Collection Set-up • Wizard-of-Oz E4 • Dialogue Task:
• *,-@ ontology random sampling !'#%"8(6 • User Side: • (6=1 97CF.;A • System (Wizard) Side: • $ 2: 97/D • Wizard/User (6>, (6JG+ • (6)I30< • (6H5&?B)I30
3. Data Collection Set-up • Annotation of Dialogue Acts •
Dialogue Act = intent + slot-value pairs • intent: inform / request • slot-value: domain, price, … • Amazon Mechanical Turk +!" &$ dialogue acts .) • !" &$ '- /( • % ,*0.8843#0
4. MultiWOZ Dialogue Corpus •
: domain
4. MultiWOZ Dialogue Corpus : expensive : domain
4. MultiWOZ Dialogue Corpus • (turns in a
dialogue) • 8.93 (single-domain), 15.39 (multi-domain) • 115,434 turns • >70% 10 turns • (sentence length) • 11.75 (user), 15.12 (wizard)
4. MultiWOZ Dialogue Corpus • Dialogue Acts • 60% turns
action • %# • "$ • %# !"$
4. MultiWOZ Dialogue Corpus • •
• Multi-Domain, Dialogue Act
5. MultiWOZ as a New Benchmark • Dialogue modelling task
• Dialogue State Tracking • (,# '/ • &,.5-0)1 ontology • Dialogue-Context-to-Text Generation • (,Dialogue State, # '/ • &,!16 • Cam676/MultiWOZ 28 • % $"+* • RNN 473 • Cam676: GRU • MultiWOZ: LSTM
5. MultiWOZ as a New Benchmark • Dialogue-Act-to-Text Generation •
Structured meaning representation (Dialogue Act?) • • Semantically Conditioned LSTM (Wen+, 2015) • SFX MultiWOZ restaurant • SER = (missing slots + redundant slots) / total slots Wen+, 2015
6. Conclusion • )1"&7* 8 E2E #$20
• Modular-based (+%' • MultiWOZ 3 46 • !-53. github /,
More Decks by tosho
See All by tosho
LayerXにおけるセキュリティ管理の現在地と次の一手
tosho
0
710
Experts, Errors, and Context: A Large-Scale Study of Human Evaluation for Machine Translation
tosho
0
350
Good for Misconceived Reasons: An Empirical Revisiting on the Need for Visual Context in Multimodal Machine Translation
tosho
0
410
Shaham and Levy, 2021. Neural Machine Translation without Embeddings. NAACL2021
tosho
0
150
Liu et al., 2021. Pay Attention to MLPs. arXiv
tosho
0
210
Huang et al. 2020 Unsupervised Multimodal Neural Machine Translation with Pseudo Visual Pivoting
tosho
0
510
Ive, Madhyastha, Specia_2019_EMNLP_Deep Copycat Networks for Text-to-Text Generation
tosho
0
190
Tan, Bansal_2019_EMNLP_LXMERT Learning Cross-Modality Encoder Representations from Transformers
tosho
0
310
Tsai et al._2019_ACL_Multimodal Transformer for Unaligned Multimodal Language Sequences
tosho
0
480
Other Decks in Research
See All in Research
HAKARI-Bench - 実運用視点での情報検索モデル評価ベンチマーク
hotchpotch
1
870
HackSick vol.7 LT資料【LLMアーキテクチャ入門・事前学習時の躓き所解説】 スパースなAttention・状態空間モデル
rikkabotan7
0
240
Bilan travail en élevage : première restitution des enquêtes 2025-2026
institutdelelevage
PRO
0
280
PHTalks Bengaluru - SSRF When All Else Fails
dk999
0
1.2k
Top 4セキュリティ会議に一通り論文が採択されるまでの道のりと展望 / Top 4 Security Grand Slam
takaomurakami
0
1.1k
Santé et fertilité des sols : ce que nous disent les pratiques dans les systèmes de polyculture-élevage
institutdelelevage
PRO
0
160
ハードウェア研究で国際トップ会議を目指す!IROS 2027での論文採択を目指して
ayatokanada
7
4.2k
Vers un système bovin allaitant multiperformant :leviers actionnés et trajectoire de réduction de l'empreinte carbone à travers 4 années d'expérimentation à FERM'INOV
institutdelelevage
PRO
0
150
Bilan travail en élevage : première restitution des enquêtes 2025-2026
institutdelelevage
PRO
0
190
Karkada さんの論文 × 2 の紹介: (1) Closed-Form Training Dynamics Reveal Learned Features and Linear Structure in Word2Vec-like Models, (2) Symmetry in language statistics shapes the geometry of model representations
eumesy
PRO
1
1k
The Conductor Pattern -Multi-Granularity Feedback for Creative Agents-
shibuiwilliam
2
120
ros2-perf-multihost: 分散システムにおける客観的なアーキテクチャ評価フレームワーク
takasehideki
0
340
Featured
See All Featured
A better future with KSS
kneath
240
18k
HU Berlin: Industrial-Strength Natural Language Processing with spaCy and Prodigy
inesmontani
PRO
0
810
Writing Fast Ruby
sferik
630
63k
Building a Modern Day E-commerce SEO Strategy
aleyda
45
9.3k
Mobile First: as difficult as doing things right
swwweet
225
10k
sira's awesome portfolio website redesign presentation
elsirapls
0
510
Principles of Awesome APIs and How to Build Them.
keavy
128
18k
Imperfection Machines: The Place of Print at Facebook
scottboms
270
14k
Marketing Yourself as an Engineer | Alaka | Gurzu
gurzu
0
370
Between Models and Reality
mayunak
4
580
Ten Tips & Tricks for a 🌱 transition
stuffmc
1
280
From π to Pie charts
rasagy
1
450