Upgrade to Pro — share decks privately, control downloads, hide ads and more …
Sign up for free

Real-Time Open-Domain Question Answering with D...

Sponsored · Your Podcast. Everywhere. Effortlessly. Share. Educate. Inspire. Entertain. You do you. We'll handle the rest. →
Avatar for Scatter Lab Inc. Scatter Lab Inc.
July 17, 2019
Research
940
0

Real-Time Open-Domain Question Answering with Dense-Sparse Phrase Index

Avatar for Scatter Lab Inc.

Scatter Lab Inc.

July 17, 2019

More Decks by Scatter Lab Inc.

See All by Scatter Lab Inc.
zeta introduction
Avatar for Scatter Lab Inc. scatterlab
0
2.2k
SimCLR: A Simple Framework for Contrastive Learning of Visual Representations
Avatar for Scatter Lab Inc. scatterlab
0
4.6k
Adversarial Filters of Dataset Biases
Avatar for Scatter Lab Inc. scatterlab
0
2.3k
Sparse, Dense, and Attentional Representations for Text Retrieval
Avatar for Scatter Lab Inc. scatterlab
0
2.4k
Weight Poisoning Attacks on Pre-trained Models
Avatar for Scatter Lab Inc. scatterlab
0
2.2k
Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval
Avatar for Scatter Lab Inc. scatterlab
0
2.6k
Beyond Accuracy: Behavioral Testing of NLP Models with CheckList
Avatar for Scatter Lab Inc. scatterlab
0
2.4k
Open-Retrieval Conversational Question Answering
Avatar for Scatter Lab Inc. scatterlab
0
2.4k
What Can Neural Networks Reason About?
Avatar for Scatter Lab Inc. scatterlab
0
2.3k

Other Decks in Research

See All in Research
GLIM とMegaParticles：正規分布近似の限界とタイトカップリング＆パーティクルフィルタの進展 / GLIM and MegaParticles : Progress of the distribution representation in SLAM
Avatar for koide3 koide3
0
950
20260624 NLP colloquium: 単一のhubテキストがCLIPを壊す：hubnessによる埋め込みの脆弱性特定
Avatar for Hiroyuki Deguchi de9uch1
2
290
[CV勉強会＠関東 CVPR2026] PSDesigner: Automated Graphic Design with a Human-Like Creative Workflow / kantocv 67th CVPR 2026
Avatar for Shunsuke KITADA shunk031
0
370
進化？迷走？CasualConc ファミリーアプリの現在地 @ 英語コーパス学会 2026
Avatar for Yasu Imao casualconc
0
140
2026年 オープンキャンパス 研究室紹介
Avatar for Jun Kurihara junkurihara
0
250
ふとした出会いで生まれたSkillが、 社内利用1位になるまで
Avatar for earlgrayMK mikimhk
22
25k
完全自律LLMエージェントでDIVER OSINT CTF 2026に挑戦してみた
Avatar for RabotniKuma analokmaus
0
770
横浜市長（山中氏）の言動にかかる第三者による調査報告書
Avatar for Sayaka Isizuka sishi2026
0
640
論文読み会 SNLP2026 Tau2-Bench: Evaluating Conversational Agents in a Dual-Control Environment
Avatar for S s_mizuki_nlp
0
300
2025年度秋葉原ウォーカブルプロジェクト調査報告 「アキバらしいウォーカブル」とは何か
Avatar for 日本大学 都市計画研究室（泉山ゼミ） izumiyama_lab
1
250
量子サマースクール2026「量子計算機アーキテクチャ分野の概観」
Avatar for Yosuke Ueno youten622
1
970
長時間動画QAにおけるマルチエージェント推論 ・SVAgent: Storyline-Guided Long Video Understanding via Cross-Modal Multi-Agent Collaboration
Avatar for 村川卓也 murakawatakuya
1
220

Featured

See All Featured
Scaling GitHub
Avatar for Zach Holman holman
464
140k
The #1 spot is gone: here's how to win anyway
Avatar for Tamara Novitovic tamaranovitovic
4
1.2k
Sharpening the Axe: The Primacy of Toolmaking
Avatar for Bryan Cantrill bcantrill
46
3k
A Tale of Four Properties
Avatar for Chris Coyier chriscoyier
163
24k
Six Lessons from altMBA
Avatar for Skipper Chong Warson skipperchong
29
4.5k
Groundhog Day: Seeking Process in Gaming for Health
Avatar for Sebastian Deterding codingconduct
0
410
Code Reviewing Like a Champion
Avatar for maltzj maltzj
529
40k
HTML-Aware ERB: The Path to Reactive Rendering @ RubyCon 2026, Rimini, Italy
Avatar for Marco Roth marcoroth
5
810
Leadership Guide Workshop - DevTernity 2021
Avatar for David Neal reverentgeek
1
400
Reality Check: Gamification 10 Years Later
Avatar for Sebastian Deterding codingconduct
0
2.3k
How to Align SEO within the Product Triangle To Get 
Buy-In & Support - #RIMC
Avatar for Aleyda Solis aleyda
2
1.8k
SEO in 2025: How to Prepare for the Future of Search
Avatar for Michael King ipullrank
3
3.9k

Transcript

  1. Real-Time Open-Domain Question Answering With Dense-Sparse Phrase Index 스캐터랩(ScatterLab) Machine

    Learning Engineer 김준성 ੌ࢚؀ച ੋҕ૑מ

  2. Motivation Query When is the Christmas? Christmas Jejus Christmas Tree

    Christmas Eve … Document Retrieval Reading Comprehension ৘ࣻ੄ ఍ࢤਸ ୷ೞೞח ೯ࢎо द੘ػ Ѫ਷ 4ࣁӝ ޖ۵ࠗఠ৓׮. ׼दীח ௼ܻझ݃झ੄ զ૞ о ੿ഛೞѱ ੿೧૑૑ ঋওӝ ٸޙী 1ਘ 6ੌ, 3ਘ 21ੌ, 12ਘ 25ੌ о਍ؘ ೞܖо ࢶ੿غ঻ ׮. 12ਘ 25ੌ੉ ௼ܻझ݃झ۽ ੗ܻ੟਷ Ѫ਷ AD 354֙, ੉റ੉׮. 12ਘ 25ੌ

  3. Motivation Query When is the Christmas? Christmas Jejus Christmas Tree

    Christmas Eve … Document Retrieval Reading Comprehension ৘ࣻ੄ ఍ࢤਸ ୷ೞೞח ೯ࢎо द੘ػ Ѫ਷ 4ࣁӝ ޖ۵ࠗఠ৓׮. ׼दীח ௼ܻझ݃झ੄ զ૞ о ੿ഛೞѱ ੿೧૑૑ ঋওӝ ٸޙী 1ਘ 6ੌ, 3ਘ 21ੌ, 12ਘ 25ੌ о਍ؘ ೞܖо ࢶ੿غ঻ ׮. 12ਘ 25ੌ੉ ௼ܻझ݃झ۽ ੗ܻ੟਷ Ѫ਷ AD 354֙, ੉റ੉׮. 12ਘ 25ੌ TF-IDF Searching SQuAD

  4. Motivation Query When is the Christmas? Christmas Jejus Christmas Tree

    Christmas Eve … Document Retrieval Reading Comprehension ৘ࣻ੄ ఍ࢤਸ ୷ೞೞח ೯ࢎо द੘ػ Ѫ਷ 4ࣁӝ ޖ۵ࠗఠ৓׮. ׼दীח ௼ܻझ݃झ੄ զ૞ о ੿ഛೞѱ ੿೧૑૑ ঋওӝ ٸޙী 1ਘ 6ੌ, 3ਘ 21ੌ, 12ਘ 25ੌ о਍ؘ ೞܖо ࢶ੿غ঻ ׮. 12ਘ 25ੌ੉ ௼ܻझ݃झ۽ ੗ܻ੟਷ Ѫ਷ AD 354֙, ੉റ੉׮. 12ਘ 25ੌ TF-IDF Searching SQuAD Error Propagation Ӓܻझ݃झ ࣛ۽؀୏ … X

  5. Motivation Query When is the Christmas? Christmas Jejus Christmas Tree

    Christmas Eve … Document Retrieval Reading Comprehension ৘ࣻ੄ ఍ࢤਸ ୷ೞೞח ೯ࢎо द੘ػ Ѫ਷ 4ࣁӝ ޖ۵ࠗఠ৓׮. ׼दীח ௼ܻझ݃झ੄ զ૞ о ੿ഛೞѱ ੿೧૑૑ ঋওӝ ٸޙী 1ਘ 6ੌ, 3ਘ 21ੌ, 12ਘ 25ੌ о਍ؘ ೞܖо ࢶ੿غ঻ ׮. 12ਘ 25ੌ੉ ௼ܻझ݃झ۽ ੗ܻ੟਷ Ѫ਷ AD 354֙, ੉റ੉׮. 12ਘ 25ੌ TF-IDF Searching SQuAD Super Slow Document Size ࣻরѤ੄ WIKi Model Size BERT-Large 10-100 sec/query ࢲ࠺झೡ ࣻ হ਺

  6. Motivation Super Slow SQuAD-BERT ௼ܻझ݃झח ঱ઁঠ | ৘ࣻ੄ ఍ࢤਸ ୷ೞೞח..

    ௼ܻझ݃झח ঱ઁঠ | ӝةҮ ੋٜীѱ ੓য о੢… ௼ܻझ݃झח ঱ઁঠ | োੋٜ੄ о੢ ೥ೠ…. ଺Ҋ੗ ೞח Articleਸ Caching ೡ ࣻ হӝ ٸޙী Query ী ٮۄࢲ ݒߣ ҅࢑೧ঠೣ 12ਘ 25ੌ

  7. Models Query Embedding [CLS] What day is it the Christmas?

    Query Encoding

  8. Models Christmas is an annual festival commemorating the birth of

    Jesus Christ, observed on December 25.. Phrase Encoding

  9. Models Christmas is an annual festival commemorating the birth of

    Jesus Christ, observed on December 25 Start Span Embed End Span Embed December 25 Phrase Encoding

  10. Models Christmas is an annual festival commemorating the birth of

    Jesus Christ, observed on December 25 Start Span Embed End Span Embed annual festival commemorating the birth of Jesus Christ Phrase Encoding

  11. Sparse Encoding Models ੌ߈੸ੋ Document Retrieval + Reading Comprehension झఋੌ੉

    ইצ ߄۽ Query -> Span ՙܻ Distribution Matching ೣ (RMM э਷?)

  12. Sparse Encoding Models START EMBED / END EMBED / COHERENCE

    EMBED = DENSE EMBEDDING

  13. Sparse Encoding Models ৵ Coherenceо ৵ ٜযо૑??

  14. Models Dense Encoding Token Embed dimਸ 4١࠙ ೧ࢲ 1,2,3,4 ۽

    ա׃ Start_embed1, End_embed2, dot[Start_embed3, End_embed4] Concat

  15. Sparse Encoding 2-Gram TF-IDFܳ ੉ਊ೧ࢲ Sparse Embeddingਸ ݅ٞ + Paragraph-Level

    Sparse Embedding ب ؊ೣ Models

  16. ੌ߈੸ਵ۽ח ݽٚ Document ৬ ࠺Ү೧ࢲ argmaxܳ ೞח ߑߨ ࢎਊ Training

    Training Efficiency

  17. Training Training Efficiency ೞ૑݅ ਋ܻח ݏח ௪ܻী ؀೧ࢲ Positive ೞѱ

    ೟णೞݶ ؽ (RMMীࢲ Positive Sample) ઑӘ؊ ࣻ೟੸ਵ۽ ੿ܻೞݶ ؊ ബਯ੸ਵ۽ Loss ܳ ҅࢑ೡ ࣻ ੓਺

  18. Result SQuAD 1.1 BERTࠁ׮ח ࢿמ੉ 8% ੿ب ծ਷Ѫ੉ ࢎप ೞ૑݅

    प ࢲ࠺झ۽ ٜ݅য૓ ݽ؛ীࢲח
 ӝઓ SOTA (DrQA) ؀࠺ ࢿמ਷ 4% ֫Ҋ
 ࣘبח DrQA 58ߓ ࡅܰҊ BERTࠁ׮ 6000ߓ ੿ب ࡅܴ