Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Open-Retrieval Conversational Question Answering
Search
Scatter Lab Inc.
July 24, 2020
Research
0
2.3k
Open-Retrieval Conversational Question Answering
Scatter Lab Inc.
July 24, 2020
Tweet
Share
More Decks by Scatter Lab Inc.
See All by Scatter Lab Inc.
zeta introduction
scatterlab
0
1.8k
SimCLR: A Simple Framework for Contrastive Learning of Visual Representations
scatterlab
0
4.1k
Adversarial Filters of Dataset Biases
scatterlab
0
2.2k
Sparse, Dense, and Attentional Representations for Text Retrieval
scatterlab
0
2.3k
Weight Poisoning Attacks on Pre-trained Models
scatterlab
0
2.2k
Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval
scatterlab
0
2.5k
Beyond Accuracy: Behavioral Testing of NLP Models with CheckList
scatterlab
0
2.3k
What Can Neural Networks Reason About?
scatterlab
0
2.2k
Exploring the Limits of Transfer Learning with Unified Text-to-Text Transformer
scatterlab
0
2.2k
Other Decks in Research
See All in Research
AI in Enterprises - Java and Open Source to the Rescue
ivargrimstad
0
530
投資戦略202508
pw
0
560
ip71_contraflow_reconfiguration
stkmsd
0
110
MetaEarth: A Generative Foundation Model for Global-Scale Remote Sensing Image Generation
satai
4
300
問いを起点に、社会と共鳴する知を育む場へ
matsumoto_r
PRO
0
650
Vision and LanguageからのEmbodied AIとAI for Science
yushiku
PRO
1
550
電通総研の生成AI・エージェントの取り組みエンジニアリング業務向けAI活用事例紹介
isidaitc
1
1k
[論文紹介] Intuitive Fine-Tuning
ryou0634
0
120
IMC の細かすぎる話 2025
smly
2
670
能動適応的実験計画
masakat0
2
840
Agentic AIとMCPを利用したサービス作成入門
mickey_kubo
0
640
AlphaEarth Foundations: An embedding field model for accurate and efficient global mapping from sparse label data
satai
3
320
Featured
See All Featured
Building Applications with DynamoDB
mza
96
6.6k
Practical Tips for Bootstrapping Information Extraction Pipelines
honnibal
PRO
23
1.5k
The World Runs on Bad Software
bkeepers
PRO
71
11k
The Pragmatic Product Professional
lauravandoore
36
6.9k
Producing Creativity
orderedlist
PRO
347
40k
Bash Introduction
62gerente
615
210k
Faster Mobile Websites
deanohume
310
31k
[RailsConf 2023 Opening Keynote] The Magic of Rails
eileencodes
31
9.7k
RailsConf & Balkan Ruby 2019: The Past, Present, and Future of Rails at GitHub
eileencodes
140
34k
Build The Right Thing And Hit Your Dates
maggiecrowley
37
2.9k
GraphQLとの向き合い方2022年版
quramy
49
14k
Thoughts on Productivity
jonyablonski
70
4.9k
Transcript
Open-Retrieval Conversational Question Answering ࢲ࢚ (ܻࢲ ࢎ౭झ, ೝಯ)
ѐਃ Open-Retrieval Conversational Question Answering
ѐਃ ѐਃ • SIGIR 20 • Chen Qu, Liu Yang,
Cen Chen, Minghui Qiu, W. Bruce Croft, Mohit Iyyer • University of Massachusetts Amherst, Ant Financial, Alibaba Group • Conversational searchਸ ਤ೧ ConvQAܳ open retrieval settingਵ۽ ഛೞח Ѫ ਃ োҳ ਃ
ѐਃ ѐਃ • Conversational search information retrieval Ҿӓੋ ݾী ೞա
• ୭Ӕ োҳٜ conversational searchܳ response rankingҗ conversational question answering۽ ೧Ѿ • ױࣽ ߸ਸ য candidate setীࢲ ҊܰѢա য passageীࢲ spanਸ ࢶఖ • ח conversational searchীࢲ retrieval ӝୡੋ ഝਸ ޖदೞח ߑध • ࠄ ֤ޙ open-retrieval conversational question answering(ORConvQA) settingਸ ઁউೞৈ ޙઁܳ ೧Ѿ
ѐਃ ѐਃ • ORConvQAী ೠ োҳܳ ਤ೧ OR-QuAC ؘఠ ࣇਸ
ٜ݅ਵݴ ORConvQAܳ ਤೠ end-to-end दझమਸ ҳ୷ೞݴ ےझನݠ ӝ߈ retriever, reranker ৬ reader ١ਸ ನೣ • OR-QuACܳ ࢚ਵ۽ ೠ ֤ޙ प learnable retriever ਃࢿਸ ૐݺ • ژೠ ݽٚ दझమ ҳࢿ ਃࣗ(retriever, reranker ৬ reader)ীࢲ history modelingਸ ࢎਊೞݶ दझమ ѱ ѐࢶ ؼ ࣻ ਸ ࠁ
Dataset Open-Retrieval Conversational Question Answering
ORConvQA? Dataset • conversational search systemsਸ ҳ୷ೞӝ ਤೠ ୶о ױ҅۽ࢲ
߸ਸ Ҋܰ ӝ ী retrieve evidenceܳ large collection۽ ࠗఠ Ѩ࢝ 1. ࠁܳ ҳೞח ചܳ ઁҕ(information seeker৬ information provider)৬ ೞח QuAC dataset 2. QuAC ޙਸ context-independentೞѱ द ࢿೠ CANARD dataset 3. Wikipedia passage
Dataset
CANARD? Dataset • QuAC dialogsח self-containedೞ ঋח ড חؘ ח
ࠛ৮ೠ ୡӝ ޙਵ۽ ੋ೧ ߊࢤ • ܳ ٜয seekerীѱ a Chinese polymathic scientistੋ Zhang Hengী ೧ ߓۄҊ ೮חؘ ޙ "җҗ ӝࣿҗ যڃ ҙ ҅о णפө?” • ۞ೠ ࠛౠೞҊ ݽഐೠ ୡӝ ޙ ചܳ ೧ࢳೞӝ য۵ѱ ೞӝ ٸޙী ҕѐ Ѩ࢝ ജ҃ীࢲ ޙઁܳ ঠӝ • CANARD ؘఠ ࣁীࢲ ઁҕೞח context-independent rewritesਵ۽ ೞৈ ޙઁܳ ೧Ѿ, Ӓۢ "Zhang Heng җ ӝ ࣿҗ যڃ ҙ҅о णפө?"۽ ޙ
CANARD? Dataset • ߣ૩ ޙী ೧ࢲ݅ Үܳ ࣻ೯ೞݶ ച
ղীࢲ history dependenciesਸ Ӓ۽ ਬೞݶࢲ ചо self-contained • QuAC test set ҕѐغয ঋӝ ٸޙী QuAC dev setਸ ਊೞৈ CANARD test setਸ ݅ٞ • ژೠ QuAC train set 10%ܳ dev۽ ഝਊ. • CANARDী হח QuAC ޙ ತӝ೮ਵݴ ܳ ਊೠ ࢤ ؘఠ ੋ OR-QuAC ؘఠ ా҅ח җ э.
Model Open-Retrieval Conversational Question Answering
ݽ؛ Retriever, Reranker, Reader۽ ա Model
ݽ؛ Retriever, Reranker, Reader۽ ա Model
Passage Retriever Dataset • Passage Encoder • Question Encoder •
Retrieval Score
Retrieval score ӝળਵ۽ ࢚ਤ top-Kѐ ޙࢲܳ rerank৬ reader۽ ׳ Model
ݽ؛ Retriever, Reranker, Reader۽ ա Model
Reranker& Reader Encoding Dataset • Input • Contextualized Representations •
sequence representation
Reranker& Reader Dataset • Sequence Representation • Reranker (W_rr is
vector) • Reader (span prediction)
Training Open-Retrieval Conversational Question Answering
Retriever pretraining Training • retrieval scores for the batch •
to maximize the probability of the gold passage for each question • Pretraining loss Pretraning റী passage encoderח offlineਵ۽ ك. Faissܳ ࢎਊ೧ࢲ Ѿҗܳ ࡳই১.
Concurrent Learning Training • Retriever loss • Reranker loss •
Reader loss
Inference Training • Retrieval Ѿҗ Top-K ޙࢲܳ ݽف ੋಌ۠झ ೞৈ
п ޙࢲ߹ spanਸ ஏ • Retriever loss + Reranker loss + Reader lossо ઁੌ ޙࢲ spanਸ ୭ઙ ਵ۽ ஏ
RESULTS Open-Retrieval Conversational Question Answering
Competing Method RESULTS • DrQA : TF-IDF + RNN based
reader • BERTserini : BM25 + BERT reader • ORConvQA without history : our method + window size 0 • ORConvQA : our method • Evaluation Metric : word level F1, human equivalence score (HEQ), Mean Reciprocal Rank(MRR), Recall
DrQA < BERTserini < Ours w/o hist < Ours RESULTS
Ablation study RESULTS
History windows size ઑ RESULTS
хࢎפ✌ ୶о ޙ ژח ҾӘೠ ݶ ઁٚ ইې োۅ۽
োۅ ࣁਃ! ࢲ࢚ (ܻࢲ ࢎ౭झ, ೝಯ)
[email protected]
Linked in. @pingpong