Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Search Infrastructure using Lambda Architecture
Search
Ananth Packkildurai
May 31, 2017
Programming
1
330
Search Infrastructure using Lambda Architecture
Search
Ananth Packkildurai
May 31, 2017
Tweet
Share
More Decks by Ananth Packkildurai
See All by Ananth Packkildurai
Data Contracts & Domain Ownership
vananth22
0
110
Data Catalogs - Rebuild the Broken Promise
vananth22
0
84
Functional Data Engineering - A Blueprint for adopting functional principles in data pipeline
vananth22
0
540
Back To The Future: Emerging Trends in Data Engineering
vananth22
0
1.2k
Murron: A Reliable Monitoring Pipeline
vananth22
0
390
The_journey_towards_Pinot.pdf
vananth22
0
220
Reliable_Event_Pipeline___scale.pdf
vananth22
0
200
Operating Data Pipeline with Airflow @ Slack
vananth22
1
2.5k
Streaming data pipelines @ Slack
vananth22
2
2.4k
Other Decks in Programming
See All in Programming
すべてのコンテキストを、 ユーザー価値に変える
applism118
3
1.3k
Modern Angular with Signals and Signal Store:New Rules for Your Architecture @enterJS Advanced Angular Day 2025
manfredsteyer
PRO
0
220
Goで作る、開発・CI環境
sin392
0
230
たった 1 枚の PHP ファイルで実装する MCP サーバ / MCP Server with Vanilla PHP
okashoi
1
260
PipeCDのプラグイン化で目指すところ
warashi
1
280
Flutterで備える!Accessibility Nutrition Labels完全ガイド
yuukiw00w
0
160
The Modern View Layer Rails Deserves: A Vision For 2025 And Beyond @ RailsConf 2025, Philadelphia, PA
marcoroth
1
170
0626 Findy Product Manager LT Night_高田スライド_speaker deck用
mana_takada
0
170
Porting a visionOS App to Android XR
akkeylab
0
470
AI時代の『改訂新版 良いコード/悪いコードで学ぶ設計入門』 / ai-good-code-bad-code
minodriven
14
5k
Quand Symfony, ApiPlatform, OpenAI et LangChain s'allient pour exploiter vos PDF : de la théorie à la production…
ahmedbhs123
0
190
スタートアップの急成長を支えるプラットフォームエンジニアリングと組織戦略
sutochin26
1
5.8k
Featured
See All Featured
Agile that works and the tools we love
rasmusluckow
329
21k
The Straight Up "How To Draw Better" Workshop
denniskardys
234
140k
[Rails World 2023 - Day 1 Closing Keynote] - The Magic of Rails
eileencodes
35
2.4k
Understanding Cognitive Biases in Performance Measurement
bluesmoon
29
1.8k
Producing Creativity
orderedlist
PRO
346
40k
Fireside Chat
paigeccino
37
3.5k
Build your cross-platform service in a week with App Engine
jlugia
231
18k
GitHub's CSS Performance
jonrohan
1031
460k
Raft: Consensus for Rubyists
vanstee
140
7k
Code Review Best Practice
trishagee
69
18k
A Tale of Four Properties
chriscoyier
160
23k
GraphQLの誤解/rethinking-graphql
sonatard
71
11k
Transcript
Search infrastructure using Lambda Architecture Ananth Packkildurai
Outline 1.Search @ Slack 2.Lambda Architecture 3.Search infrastructure overview 4.FCC
Design patterns
Search @ Slack
http://www.mckinsey.com/industries/high-tech/our-insights/the-social-economy https://slack.engineering/search-at-slack-431f8c80619e
Lambda Architecture
Stream processing trade-offs Pick 2 Speed Accuracy Volume
Lambda Architecture http://lambda-architecture.net/
Search Infrastructure overview
Search infrastructure overview
! Online sharding by Collections ! Offline sharding by CompositeId
Router Sharding strategy
mutating Index & Delta Replication • What happens if someone
edits the message that indexed in batch mode? • A snapshot occurs at time T1, the index built, and the core comes online at T2.Between T1 & T2, How to handle the update?
FCC Pattern
Fork Pattern
Contract
Clone
Any Questions? Thanks!