Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Open-Retrieval Conversational Question Answering
Search
Scatter Lab Inc.
July 24, 2020
Research
0
2.3k
Open-Retrieval Conversational Question Answering
Scatter Lab Inc.
July 24, 2020
Tweet
Share
More Decks by Scatter Lab Inc.
See All by Scatter Lab Inc.
zeta introduction
scatterlab
0
1.8k
SimCLR: A Simple Framework for Contrastive Learning of Visual Representations
scatterlab
0
4.1k
Adversarial Filters of Dataset Biases
scatterlab
0
2.2k
Sparse, Dense, and Attentional Representations for Text Retrieval
scatterlab
0
2.3k
Weight Poisoning Attacks on Pre-trained Models
scatterlab
0
2.2k
Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval
scatterlab
0
2.5k
Beyond Accuracy: Behavioral Testing of NLP Models with CheckList
scatterlab
0
2.3k
What Can Neural Networks Reason About?
scatterlab
0
2.2k
Exploring the Limits of Transfer Learning with Unified Text-to-Text Transformer
scatterlab
0
2.2k
Other Decks in Research
See All in Research
Mechanistic Interpretability:解釈可能性研究の新たな潮流
koshiro_aoki
1
460
説明可能な機械学習と数理最適化
kelicht
0
190
論文読み会 SNLP2025 Learning Dynamics of LLM Finetuning. In: ICLR 2025
s_mizuki_nlp
0
280
多言語カスタマーインタビューの“壁”を越える~PMと生成AIの共創~ 株式会社ジグザグ 松野 亘
watarumatsuno
0
130
Generative Models 2025
takahashihiroshi
25
14k
Hiding What from Whom? A Critical Review of the History of Programming languages for Music
tomoyanonymous
0
200
J-RAGBench: 日本語RAGにおける Generator評価ベンチマークの構築
koki_itai
0
760
AWSで実現した大規模日本語VLM学習用データセット "MOMIJI" 構築パイプライン/buiding-momiji
studio_graph
2
740
Language Models Are Implicitly Continuous
eumesy
PRO
0
300
2025/7/5 応用音響研究会招待講演@北海道大学
takuma_okamoto
1
220
ip71_contraflow_reconfiguration
stkmsd
0
110
CoRL2025速報
rpc
1
2.1k
Featured
See All Featured
Gamification - CAS2011
davidbonilla
81
5.5k
Embracing the Ebb and Flow
colly
88
4.9k
Exploring the Power of Turbo Streams & Action Cable | RailsConf2023
kevinliebholz
35
6.1k
Practical Tips for Bootstrapping Information Extraction Pipelines
honnibal
PRO
23
1.5k
Build your cross-platform service in a week with App Engine
jlugia
232
18k
Speed Design
sergeychernyshev
32
1.2k
Performance Is Good for Brains [We Love Speed 2024]
tammyeverts
12
1.2k
Dealing with People You Can't Stand - Big Design 2015
cassininazir
367
27k
個人開発の失敗を避けるイケてる考え方 / tips for indie hackers
panda_program
115
20k
Distributed Sagas: A Protocol for Coordinating Microservices
caitiem20
333
22k
Unsuck your backbone
ammeep
671
58k
Helping Users Find Their Own Way: Creating Modern Search Experiences
danielanewman
30
2.9k
Transcript
Open-Retrieval Conversational Question Answering ࢲ࢚ (ܻࢲ ࢎ౭झ, ೝಯ)
ѐਃ Open-Retrieval Conversational Question Answering
ѐਃ ѐਃ • SIGIR 20 • Chen Qu, Liu Yang,
Cen Chen, Minghui Qiu, W. Bruce Croft, Mohit Iyyer • University of Massachusetts Amherst, Ant Financial, Alibaba Group • Conversational searchਸ ਤ೧ ConvQAܳ open retrieval settingਵ۽ ഛೞח Ѫ ਃ োҳ ਃ
ѐਃ ѐਃ • Conversational search information retrieval Ҿӓੋ ݾী ೞա
• ୭Ӕ োҳٜ conversational searchܳ response rankingҗ conversational question answering۽ ೧Ѿ • ױࣽ ߸ਸ য candidate setীࢲ ҊܰѢա য passageীࢲ spanਸ ࢶఖ • ח conversational searchীࢲ retrieval ӝୡੋ ഝਸ ޖदೞח ߑध • ࠄ ֤ޙ open-retrieval conversational question answering(ORConvQA) settingਸ ઁউೞৈ ޙઁܳ ೧Ѿ
ѐਃ ѐਃ • ORConvQAী ೠ োҳܳ ਤ೧ OR-QuAC ؘఠ ࣇਸ
ٜ݅ਵݴ ORConvQAܳ ਤೠ end-to-end दझమਸ ҳ୷ೞݴ ےझನݠ ӝ߈ retriever, reranker ৬ reader ١ਸ ನೣ • OR-QuACܳ ࢚ਵ۽ ೠ ֤ޙ प learnable retriever ਃࢿਸ ૐݺ • ژೠ ݽٚ दझమ ҳࢿ ਃࣗ(retriever, reranker ৬ reader)ীࢲ history modelingਸ ࢎਊೞݶ दझమ ѱ ѐࢶ ؼ ࣻ ਸ ࠁ
Dataset Open-Retrieval Conversational Question Answering
ORConvQA? Dataset • conversational search systemsਸ ҳ୷ೞӝ ਤೠ ୶о ױ҅۽ࢲ
߸ਸ Ҋܰ ӝ ী retrieve evidenceܳ large collection۽ ࠗఠ Ѩ࢝ 1. ࠁܳ ҳೞח ചܳ ઁҕ(information seeker৬ information provider)৬ ೞח QuAC dataset 2. QuAC ޙਸ context-independentೞѱ द ࢿೠ CANARD dataset 3. Wikipedia passage
Dataset
CANARD? Dataset • QuAC dialogsח self-containedೞ ঋח ড חؘ ח
ࠛ৮ೠ ୡӝ ޙਵ۽ ੋ೧ ߊࢤ • ܳ ٜয seekerীѱ a Chinese polymathic scientistੋ Zhang Hengী ೧ ߓۄҊ ೮חؘ ޙ "җҗ ӝࣿҗ যڃ ҙ ҅о णפө?” • ۞ೠ ࠛౠೞҊ ݽഐೠ ୡӝ ޙ ചܳ ೧ࢳೞӝ য۵ѱ ೞӝ ٸޙী ҕѐ Ѩ࢝ ജ҃ীࢲ ޙઁܳ ঠӝ • CANARD ؘఠ ࣁীࢲ ઁҕೞח context-independent rewritesਵ۽ ೞৈ ޙઁܳ ೧Ѿ, Ӓۢ "Zhang Heng җ ӝ ࣿҗ যڃ ҙ҅о णפө?"۽ ޙ
CANARD? Dataset • ߣ૩ ޙী ೧ࢲ݅ Үܳ ࣻ೯ೞݶ ച
ղীࢲ history dependenciesਸ Ӓ۽ ਬೞݶࢲ ചо self-contained • QuAC test set ҕѐغয ঋӝ ٸޙী QuAC dev setਸ ਊೞৈ CANARD test setਸ ݅ٞ • ژೠ QuAC train set 10%ܳ dev۽ ഝਊ. • CANARDী হח QuAC ޙ ತӝ೮ਵݴ ܳ ਊೠ ࢤ ؘఠ ੋ OR-QuAC ؘఠ ా҅ח җ э.
Model Open-Retrieval Conversational Question Answering
ݽ؛ Retriever, Reranker, Reader۽ ա Model
ݽ؛ Retriever, Reranker, Reader۽ ա Model
Passage Retriever Dataset • Passage Encoder • Question Encoder •
Retrieval Score
Retrieval score ӝળਵ۽ ࢚ਤ top-Kѐ ޙࢲܳ rerank৬ reader۽ ׳ Model
ݽ؛ Retriever, Reranker, Reader۽ ա Model
Reranker& Reader Encoding Dataset • Input • Contextualized Representations •
sequence representation
Reranker& Reader Dataset • Sequence Representation • Reranker (W_rr is
vector) • Reader (span prediction)
Training Open-Retrieval Conversational Question Answering
Retriever pretraining Training • retrieval scores for the batch •
to maximize the probability of the gold passage for each question • Pretraining loss Pretraning റী passage encoderח offlineਵ۽ ك. Faissܳ ࢎਊ೧ࢲ Ѿҗܳ ࡳই১.
Concurrent Learning Training • Retriever loss • Reranker loss •
Reader loss
Inference Training • Retrieval Ѿҗ Top-K ޙࢲܳ ݽف ੋಌ۠झ ೞৈ
п ޙࢲ߹ spanਸ ஏ • Retriever loss + Reranker loss + Reader lossо ઁੌ ޙࢲ spanਸ ୭ઙ ਵ۽ ஏ
RESULTS Open-Retrieval Conversational Question Answering
Competing Method RESULTS • DrQA : TF-IDF + RNN based
reader • BERTserini : BM25 + BERT reader • ORConvQA without history : our method + window size 0 • ORConvQA : our method • Evaluation Metric : word level F1, human equivalence score (HEQ), Mean Reciprocal Rank(MRR), Recall
DrQA < BERTserini < Ours w/o hist < Ours RESULTS
Ablation study RESULTS
History windows size ઑ RESULTS
хࢎפ✌ ୶о ޙ ژח ҾӘೠ ݶ ઁٚ ইې োۅ۽
োۅ ࣁਃ! ࢲ࢚ (ܻࢲ ࢎ౭झ, ೝಯ)
[email protected]
Linked in. @pingpong