Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Budzianowski et al. - EMNLP 2018 - MultiWOZ - A...
Search
tosho
December 10, 2018
Research
0
340
Budzianowski et al. - EMNLP 2018 - MultiWOZ - A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling
tosho
December 10, 2018
Tweet
Share
More Decks by tosho
See All by tosho
Experts, Errors, and Context: A Large-Scale Study of Human Evaluation for Machine Translation
tosho
0
300
Good for Misconceived Reasons: An Empirical Revisiting on the Need for Visual Context in Multimodal Machine Translation
tosho
0
350
Shaham and Levy, 2021. Neural Machine Translation without Embeddings. NAACL2021
tosho
0
120
Liu et al., 2021. Pay Attention to MLPs. arXiv
tosho
0
170
Huang et al. 2020 Unsupervised Multimodal Neural Machine Translation with Pseudo Visual Pivoting
tosho
0
460
Ive, Madhyastha, Specia_2019_EMNLP_Deep Copycat Networks for Text-to-Text Generation
tosho
0
140
Tan, Bansal_2019_EMNLP_LXMERT Learning Cross-Modality Encoder Representations from Transformers
tosho
0
220
Tsai et al._2019_ACL_Multimodal Transformer for Unaligned Multimodal Language Sequences
tosho
0
380
Zhou et al. 2019. Density Matching for Bilingual Word Embedding. NAACL
tosho
3
280
Other Decks in Research
See All in Research
Language Models Are Implicitly Continuous
eumesy
PRO
0
230
Google Agent Development Kit (ADK) 入門 🚀
mickey_kubo
2
1.7k
日本語新聞記事を用いた大規模言語モデルの暗記定量化 / LLMC2025
upura
0
180
最適化と機械学習による問題解決
mickey_kubo
0
170
利用シーンを意識した推薦システム〜SpotifyとAmazonの事例から〜
kuri8ive
1
250
在庫管理のための機械学習と最適化の融合
mickey_kubo
3
1.1k
VectorLLM: Human-like Extraction of Structured Building Contours via Multimodal LLMs
satai
4
200
SSII2025 [SS1] レンズレスカメラ
ssii
PRO
2
1.1k
能動適応的実験計画
masakat0
2
810
2025/7/5 応用音響研究会招待講演@北海道大学
takuma_okamoto
1
180
Cross-Media Information Spaces and Architectures
signer
PRO
0
240
AWSで実現した大規模日本語VLM学習用データセット "MOMIJI" 構築パイプライン/buiding-momiji
studio_graph
2
500
Featured
See All Featured
Designing Dashboards & Data Visualisations in Web Apps
destraynor
231
53k
The Cost Of JavaScript in 2023
addyosmani
53
8.9k
How to train your dragon (web standard)
notwaldorf
96
6.2k
A Modern Web Designer's Workflow
chriscoyier
696
190k
Creating an realtime collaboration tool: Agile Flush - .NET Oxford
marcduiker
31
2.2k
Code Reviewing Like a Champion
maltzj
525
40k
Practical Tips for Bootstrapping Information Extraction Pipelines
honnibal
PRO
23
1.4k
Principles of Awesome APIs and How to Build Them.
keavy
126
17k
The Straight Up "How To Draw Better" Workshop
denniskardys
236
140k
Chrome DevTools: State of the Union 2024 - Debugging React & Beyond
addyosmani
7
840
RailsConf & Balkan Ruby 2019: The Past, Present, and Future of Rails at GitHub
eileencodes
139
34k
Performance Is Good for Brains [We Love Speed 2024]
tammyeverts
12
1.1k
Transcript
MultiWOZ – A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue
Modeling Tosho Hirasawa
0. Overview • -6<+?E$> • 4L3I/%) H2 • :@Multi-Domain
Wizard-of-Oz (MultiWOZ) • KJ • ("*72 GA/9F8 #!= 5 ,1 • 0.BD&* ' &(*;C
1. Introduction • Conversational Artificial Intelligence • human-level *)&($ •
#%' ! • Seneff and Polifroni, 2000 • "Raux et al., 2005 • Amazon AlexaRam et al., 2018
1. Introduction • \T@F [C0*%0# RA •
2DKU • =W:J • ?6) 8V • OXN3A • PH517 E2E ,"/LI • <];Z17MYB( >E • &!-0Q • " 9 • [C$+0_4D • GS5'.-0^
1. Introduction , , 2017
2. Related Works • >K&.(%3/9 ! • Machine-to-Machine • *5/4+"O6K"R
• HLJ-$) T DM6K\E ]X • Human-to-Machine • 7:=@^Y'(*0UZ9";I • G OE! :B • HLJ^Y'(*0 YS?,1$5&.(NI • Human-to-Human • G<QW &(+< • Twitter, Reddit, Ubuntu 6K"_8NI! • HLJ6KC[ AP#-*'25 FV
3. Data Collection Set-up • Wizard-of-Oz E4 • Dialogue Task:
• *,-@ ontology random sampling !'#%"8(6 • User Side: • (6=1 97CF.;A • System (Wizard) Side: • $ 2: 97/D • Wizard/User (6>, (6JG+ • (6)I30< • (6H5&?B)I30
3. Data Collection Set-up • Annotation of Dialogue Acts •
Dialogue Act = intent + slot-value pairs • intent: inform / request • slot-value: domain, price, … • Amazon Mechanical Turk +!" &$ dialogue acts .) • !" &$ '- /( • % ,*0.8843#0
4. MultiWOZ Dialogue Corpus •
: domain
4. MultiWOZ Dialogue Corpus : expensive : domain
4. MultiWOZ Dialogue Corpus • (turns in a
dialogue) • 8.93 (single-domain), 15.39 (multi-domain) • 115,434 turns • >70% 10 turns • (sentence length) • 11.75 (user), 15.12 (wizard)
4. MultiWOZ Dialogue Corpus • Dialogue Acts • 60% turns
action • %# • "$ • %# !"$
4. MultiWOZ Dialogue Corpus • •
• Multi-Domain, Dialogue Act
5. MultiWOZ as a New Benchmark • Dialogue modelling task
• Dialogue State Tracking • (,# '/ • &,.5-0)1 ontology • Dialogue-Context-to-Text Generation • (,Dialogue State, # '/ • &,!16 • Cam676/MultiWOZ 28 • % $"+* • RNN 473 • Cam676: GRU • MultiWOZ: LSTM
5. MultiWOZ as a New Benchmark • Dialogue-Act-to-Text Generation •
Structured meaning representation (Dialogue Act?) • • Semantically Conditioned LSTM (Wen+, 2015) • SFX MultiWOZ restaurant • SER = (missing slots + redundant slots) / total slots Wen+, 2015
6. Conclusion • )1"&7* 8 E2E #$20
• Modular-based (+%' • MultiWOZ 3 46 • !-53. github /,