Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Budzianowski et al. - EMNLP 2018 - MultiWOZ - A...
Search
tosho
December 10, 2018
Research
0
340
Budzianowski et al. - EMNLP 2018 - MultiWOZ - A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling
tosho
December 10, 2018
Tweet
Share
More Decks by tosho
See All by tosho
Experts, Errors, and Context: A Large-Scale Study of Human Evaluation for Machine Translation
tosho
0
300
Good for Misconceived Reasons: An Empirical Revisiting on the Need for Visual Context in Multimodal Machine Translation
tosho
0
350
Shaham and Levy, 2021. Neural Machine Translation without Embeddings. NAACL2021
tosho
0
120
Liu et al., 2021. Pay Attention to MLPs. arXiv
tosho
0
170
Huang et al. 2020 Unsupervised Multimodal Neural Machine Translation with Pseudo Visual Pivoting
tosho
0
460
Ive, Madhyastha, Specia_2019_EMNLP_Deep Copycat Networks for Text-to-Text Generation
tosho
0
140
Tan, Bansal_2019_EMNLP_LXMERT Learning Cross-Modality Encoder Representations from Transformers
tosho
0
220
Tsai et al._2019_ACL_Multimodal Transformer for Unaligned Multimodal Language Sequences
tosho
0
380
Zhou et al. 2019. Density Matching for Bilingual Word Embedding. NAACL
tosho
3
280
Other Decks in Research
See All in Research
なめらかなシステムと運用維持の終わらぬ未来 / dicomo2025_coherently_fittable_system
monochromegane
0
2.8k
AIグラフィックデザインの進化:断片から統合(One Piece)へ / From Fragment to One Piece: A Survey on AI-Driven Graphic Design
shunk031
0
440
Stealing LUKS Keys via TPM and UUID Spoofing in 10 Minutes - BSides 2025
anykeyshik
0
110
RHO-1: Not All Tokens Are What You Need
sansan_randd
1
170
投資戦略202508
pw
0
560
PhD Defense 2025: Visual Understanding of Human Hands in Interactions
tkhkaeio
1
190
カスタマーサクセスの視点からAWS Summitの展示を考える~製品開発で活用できる勘所~
masakiokuda
2
190
AIによる画像認識技術の進化 -25年の技術変遷を振り返る-
hf149
7
4k
単施設でできる臨床研究の考え方
shuntaros
0
2.7k
2025/7/5 応用音響研究会招待講演@北海道大学
takuma_okamoto
1
180
AWSで実現した大規模日本語VLM学習用データセット "MOMIJI" 構築パイプライン/buiding-momiji
studio_graph
2
510
[CV勉強会@関東 CVPR2025] VLM自動運転model S4-Driver
shinkyoto
2
480
Featured
See All Featured
Sharpening the Axe: The Primacy of Toolmaking
bcantrill
44
2.5k
Performance Is Good for Brains [We Love Speed 2024]
tammyeverts
12
1.1k
GitHub's CSS Performance
jonrohan
1032
460k
Evolution of real-time – Irina Nazarova, EuRuKo, 2024
irinanazarova
8
920
Helping Users Find Their Own Way: Creating Modern Search Experiences
danielanewman
29
2.9k
How GitHub (no longer) Works
holman
315
140k
Agile that works and the tools we love
rasmusluckow
330
21k
What's in a price? How to price your products and services
michaelherold
246
12k
Code Reviewing Like a Champion
maltzj
525
40k
Why Our Code Smells
bkeepers
PRO
339
57k
The MySQL Ecosystem @ GitHub 2015
samlambert
251
13k
The Pragmatic Product Professional
lauravandoore
36
6.9k
Transcript
MultiWOZ – A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue
Modeling Tosho Hirasawa
0. Overview • -6<+?E$> • 4L3I/%) H2 • :@Multi-Domain
Wizard-of-Oz (MultiWOZ) • KJ • ("*72 GA/9F8 #!= 5 ,1 • 0.BD&* ' &(*;C
1. Introduction • Conversational Artificial Intelligence • human-level *)&($ •
#%' ! • Seneff and Polifroni, 2000 • "Raux et al., 2005 • Amazon AlexaRam et al., 2018
1. Introduction • \T@F [C0*%0# RA •
2DKU • =W:J • ?6) 8V • OXN3A • PH517 E2E ,"/LI • <];Z17MYB( >E • &!-0Q • " 9 • [C$+0_4D • GS5'.-0^
1. Introduction , , 2017
2. Related Works • >K&.(%3/9 ! • Machine-to-Machine • *5/4+"O6K"R
• HLJ-$) T DM6K\E ]X • Human-to-Machine • 7:=@^Y'(*0UZ9";I • G OE! :B • HLJ^Y'(*0 YS?,1$5&.(NI • Human-to-Human • G<QW &(+< • Twitter, Reddit, Ubuntu 6K"_8NI! • HLJ6KC[ AP#-*'25 FV
3. Data Collection Set-up • Wizard-of-Oz E4 • Dialogue Task:
• *,-@ ontology random sampling !'#%"8(6 • User Side: • (6=1 97CF.;A • System (Wizard) Side: • $ 2: 97/D • Wizard/User (6>, (6JG+ • (6)I30< • (6H5&?B)I30
3. Data Collection Set-up • Annotation of Dialogue Acts •
Dialogue Act = intent + slot-value pairs • intent: inform / request • slot-value: domain, price, … • Amazon Mechanical Turk +!" &$ dialogue acts .) • !" &$ '- /( • % ,*0.8843#0
4. MultiWOZ Dialogue Corpus •
: domain
4. MultiWOZ Dialogue Corpus : expensive : domain
4. MultiWOZ Dialogue Corpus • (turns in a
dialogue) • 8.93 (single-domain), 15.39 (multi-domain) • 115,434 turns • >70% 10 turns • (sentence length) • 11.75 (user), 15.12 (wizard)
4. MultiWOZ Dialogue Corpus • Dialogue Acts • 60% turns
action • %# • "$ • %# !"$
4. MultiWOZ Dialogue Corpus • •
• Multi-Domain, Dialogue Act
5. MultiWOZ as a New Benchmark • Dialogue modelling task
• Dialogue State Tracking • (,# '/ • &,.5-0)1 ontology • Dialogue-Context-to-Text Generation • (,Dialogue State, # '/ • &,!16 • Cam676/MultiWOZ 28 • % $"+* • RNN 473 • Cam676: GRU • MultiWOZ: LSTM
5. MultiWOZ as a New Benchmark • Dialogue-Act-to-Text Generation •
Structured meaning representation (Dialogue Act?) • • Semantically Conditioned LSTM (Wen+, 2015) • SFX MultiWOZ restaurant • SER = (missing slots + redundant slots) / total slots Wen+, 2015
6. Conclusion • )1"&7* 8 E2E #$20
• Modular-based (+%' • MultiWOZ 3 46 • !-53. github /,