Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Real-Time Open-Domain Question Answering with D...

Sponsored · Your Podcast. Everywhere. Effortlessly. Share. Educate. Inspire. Entertain. You do you. We'll handle the rest. →
Avatar for Scatter Lab Inc. Scatter Lab Inc.
July 17, 2019
Research
920
0

Real-Time Open-Domain Question Answering with Dense-Sparse Phrase Index

Avatar for Scatter Lab Inc.

Scatter Lab Inc.

July 17, 2019

More Decks by Scatter Lab Inc.

See All by Scatter Lab Inc.
zeta introduction
Avatar for Scatter Lab Inc. scatterlab
0
1.9k
SimCLR: A Simple Framework for Contrastive Learning of Visual Representations
Avatar for Scatter Lab Inc. scatterlab
0
4.4k
Adversarial Filters of Dataset Biases
Avatar for Scatter Lab Inc. scatterlab
0
2.3k
Sparse, Dense, and Attentional Representations for Text Retrieval
Avatar for Scatter Lab Inc. scatterlab
0
2.3k
Weight Poisoning Attacks on Pre-trained Models
Avatar for Scatter Lab Inc. scatterlab
0
2.2k
Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval
Avatar for Scatter Lab Inc. scatterlab
0
2.6k
Beyond Accuracy: Behavioral Testing of NLP Models with CheckList
Avatar for Scatter Lab Inc. scatterlab
0
2.3k
Open-Retrieval Conversational Question Answering
Avatar for Scatter Lab Inc. scatterlab
0
2.3k
What Can Neural Networks Reason About?
Avatar for Scatter Lab Inc. scatterlab
0
2.3k

Other Decks in Research

See All in Research
2026年度 生成AI を活用した論文執筆ガイド／ワークショップ / 2026 Academic Year Guide to Writing Papers Using Generative AI - Workshop
Avatar for Kenji Saito ks91
PRO
0
190
重要だけど測れていないもの：高齢者ケアの見えない課題
Avatar for テオリア・テクノロジーズ theoriatec2024
0
420
敵対生成プロンプト同時探索による内省型プロンプト最適化
Avatar for Kotaro Inoue kinoue_smarthr
0
320
第12回人と環境にやさしい交通をめざす全国大会／熊本都市圏「車1割削減、渋滞半減、公共交通2倍」をめざして
Avatar for Traffic Brain trafficbrain
0
140
RS-Agent: Automating Remote Sensing Tasks through Intelligent Agent
Avatar for SatAI.challenge satai
3
400
Fukui Shibiten 39 - AI Art
Avatar for IWABUCHI Yu(u)ki butchi butchi
0
150
Φ-Sat-2のAutoEncoderによる情報圧縮系論文
Avatar for SatAI.challenge satai
4
860
2025年度秋葉原ウォーカブルプロジェクト調査報告 「アキバらしいウォーカブル」とは何か
Avatar for 日本大学 都市計画研究室（泉山ゼミ） izumiyama_lab
1
100
業界横断 副業コンプライアンス調査 三者（副業者・本業先・発注者）におけるトラブル認知ギャップの構造分析
Avatar for fkske fkske
0
1.3k
typst の使い方：言語学を研究する学生のために
Avatar for Tomoya gitomochang
0
520
Scalable dynamic origin-destination demand estimation enhanced by high-resolution satellite imagery data
Avatar for SatAI.challenge satai
3
350
Sleuthcon Keynote - How Cybercriminals (ab)use AI
Avatar for Thomas Roccia fr0gger
0
260

Featured

See All Featured
The Success of Rails: Ensuring Growth for the Next 100 Years
Avatar for Eileen M. Uchitelle eileencodes
47
8.2k
Learning to Love Humans: Emotional Interface Design
Avatar for Aarron Walter aarron
275
41k
State of Search Keynote: SEO is Dead Long Live SEO
Avatar for Ryan Jones ryanjones
0
220
Balancing Empowerment & Direction
Avatar for Lara Hogan lara
6
1.2k
10 Git Anti Patterns You Should be Aware of
Avatar for Lemi Orhan Ergin lemiorhan
PRO
659
62k
Claude Code どこまでも/ Claude Code Everywhere
Avatar for nwiizo nwiizo
65
57k
The Curious Case for Waylosing
Avatar for Cassini Nazir cassininazir
1
440
Design and Strategy: How to Deal with People Who Don’t "Get" Design
Avatar for Morgane Peng morganepeng
133
19k
Ecommerce SEO: The Keys for Success Now & Beyond - #SERPConf2024
Avatar for Aleyda Solis aleyda
1
2.1k
The Art of Programming - Codeland 2020
Avatar for Erika Heidi erikaheidi
57
14k
Distributed Sagas: A Protocol for Coordinating Microservices
Avatar for Caitie McCaffrey caitiem20
333
23k
Mind Mapping
Avatar for Hélio Medeiros helmedeiros
PRO
1
290

Transcript

  1. Real-Time Open-Domain Question Answering With Dense-Sparse Phrase Index 스캐터랩(ScatterLab) Machine

    Learning Engineer 김준성 ੌ࢚؀ച ੋҕ૑מ

  2. Motivation Query When is the Christmas? Christmas Jejus Christmas Tree

    Christmas Eve … Document Retrieval Reading Comprehension ৘ࣻ੄ ఍ࢤਸ ୷ೞೞח ೯ࢎо द੘ػ Ѫ਷ 4ࣁӝ ޖ۵ࠗఠ৓׮. ׼दীח ௼ܻझ݃झ੄ զ૞ о ੿ഛೞѱ ੿೧૑૑ ঋওӝ ٸޙী 1ਘ 6ੌ, 3ਘ 21ੌ, 12ਘ 25ੌ о਍ؘ ೞܖо ࢶ੿غ঻ ׮. 12ਘ 25ੌ੉ ௼ܻझ݃झ۽ ੗ܻ੟਷ Ѫ਷ AD 354֙, ੉റ੉׮. 12ਘ 25ੌ

  3. Motivation Query When is the Christmas? Christmas Jejus Christmas Tree

    Christmas Eve … Document Retrieval Reading Comprehension ৘ࣻ੄ ఍ࢤਸ ୷ೞೞח ೯ࢎо द੘ػ Ѫ਷ 4ࣁӝ ޖ۵ࠗఠ৓׮. ׼दীח ௼ܻझ݃झ੄ զ૞ о ੿ഛೞѱ ੿೧૑૑ ঋওӝ ٸޙী 1ਘ 6ੌ, 3ਘ 21ੌ, 12ਘ 25ੌ о਍ؘ ೞܖо ࢶ੿غ঻ ׮. 12ਘ 25ੌ੉ ௼ܻझ݃झ۽ ੗ܻ੟਷ Ѫ਷ AD 354֙, ੉റ੉׮. 12ਘ 25ੌ TF-IDF Searching SQuAD

  4. Motivation Query When is the Christmas? Christmas Jejus Christmas Tree

    Christmas Eve … Document Retrieval Reading Comprehension ৘ࣻ੄ ఍ࢤਸ ୷ೞೞח ೯ࢎо द੘ػ Ѫ਷ 4ࣁӝ ޖ۵ࠗఠ৓׮. ׼दীח ௼ܻझ݃झ੄ զ૞ о ੿ഛೞѱ ੿೧૑૑ ঋওӝ ٸޙী 1ਘ 6ੌ, 3ਘ 21ੌ, 12ਘ 25ੌ о਍ؘ ೞܖо ࢶ੿غ঻ ׮. 12ਘ 25ੌ੉ ௼ܻझ݃झ۽ ੗ܻ੟਷ Ѫ਷ AD 354֙, ੉റ੉׮. 12ਘ 25ੌ TF-IDF Searching SQuAD Error Propagation Ӓܻझ݃झ ࣛ۽؀୏ … X

  5. Motivation Query When is the Christmas? Christmas Jejus Christmas Tree

    Christmas Eve … Document Retrieval Reading Comprehension ৘ࣻ੄ ఍ࢤਸ ୷ೞೞח ೯ࢎо द੘ػ Ѫ਷ 4ࣁӝ ޖ۵ࠗఠ৓׮. ׼दীח ௼ܻझ݃झ੄ զ૞ о ੿ഛೞѱ ੿೧૑૑ ঋওӝ ٸޙী 1ਘ 6ੌ, 3ਘ 21ੌ, 12ਘ 25ੌ о਍ؘ ೞܖо ࢶ੿غ঻ ׮. 12ਘ 25ੌ੉ ௼ܻझ݃झ۽ ੗ܻ੟਷ Ѫ਷ AD 354֙, ੉റ੉׮. 12ਘ 25ੌ TF-IDF Searching SQuAD Super Slow Document Size ࣻরѤ੄ WIKi Model Size BERT-Large 10-100 sec/query ࢲ࠺झೡ ࣻ হ਺

  6. Motivation Super Slow SQuAD-BERT ௼ܻझ݃झח ঱ઁঠ | ৘ࣻ੄ ఍ࢤਸ ୷ೞೞח..

    ௼ܻझ݃झח ঱ઁঠ | ӝةҮ ੋٜীѱ ੓য о੢… ௼ܻझ݃झח ঱ઁঠ | োੋٜ੄ о੢ ೥ೠ…. ଺Ҋ੗ ೞח Articleਸ Caching ೡ ࣻ হӝ ٸޙী Query ী ٮۄࢲ ݒߣ ҅࢑೧ঠೣ 12ਘ 25ੌ

  7. Models Query Embedding [CLS] What day is it the Christmas?

    Query Encoding

  8. Models Christmas is an annual festival commemorating the birth of

    Jesus Christ, observed on December 25.. Phrase Encoding

  9. Models Christmas is an annual festival commemorating the birth of

    Jesus Christ, observed on December 25 Start Span Embed End Span Embed December 25 Phrase Encoding

  10. Models Christmas is an annual festival commemorating the birth of

    Jesus Christ, observed on December 25 Start Span Embed End Span Embed annual festival commemorating the birth of Jesus Christ Phrase Encoding

  11. Sparse Encoding Models ੌ߈੸ੋ Document Retrieval + Reading Comprehension झఋੌ੉

    ইצ ߄۽ Query -> Span ՙܻ Distribution Matching ೣ (RMM э਷?)

  12. Sparse Encoding Models START EMBED / END EMBED / COHERENCE

    EMBED = DENSE EMBEDDING

  13. Sparse Encoding Models ৵ Coherenceо ৵ ٜযо૑??

  14. Models Dense Encoding Token Embed dimਸ 4١࠙ ೧ࢲ 1,2,3,4 ۽

    ա׃ Start_embed1, End_embed2, dot[Start_embed3, End_embed4] Concat

  15. Sparse Encoding 2-Gram TF-IDFܳ ੉ਊ೧ࢲ Sparse Embeddingਸ ݅ٞ + Paragraph-Level

    Sparse Embedding ب ؊ೣ Models

  16. ੌ߈੸ਵ۽ח ݽٚ Document ৬ ࠺Ү೧ࢲ argmaxܳ ೞח ߑߨ ࢎਊ Training

    Training Efficiency

  17. Training Training Efficiency ೞ૑݅ ਋ܻח ݏח ௪ܻী ؀೧ࢲ Positive ೞѱ

    ೟णೞݶ ؽ (RMMীࢲ Positive Sample) ઑӘ؊ ࣻ೟੸ਵ۽ ੿ܻೞݶ ؊ ബਯ੸ਵ۽ Loss ܳ ҅࢑ೡ ࣻ ੓਺

  18. Result SQuAD 1.1 BERTࠁ׮ח ࢿמ੉ 8% ੿ب ծ਷Ѫ੉ ࢎप ೞ૑݅

    प ࢲ࠺झ۽ ٜ݅য૓ ݽ؛ীࢲח
 ӝઓ SOTA (DrQA) ؀࠺ ࢿמ਷ 4% ֫Ҋ
 ࣘبח DrQA 58ߓ ࡅܰҊ BERTࠁ׮ 6000ߓ ੿ب ࡅܴ