Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Search Infrastructure using Lambda Architecture
Search
Ananth Packkildurai
May 31, 2017
Programming
1
330
Search Infrastructure using Lambda Architecture
Search
Ananth Packkildurai
May 31, 2017
Tweet
Share
More Decks by Ananth Packkildurai
See All by Ananth Packkildurai
Data Contracts & Domain Ownership
vananth22
0
110
Data Catalogs - Rebuild the Broken Promise
vananth22
0
83
Functional Data Engineering - A Blueprint for adopting functional principles in data pipeline
vananth22
0
530
Back To The Future: Emerging Trends in Data Engineering
vananth22
0
1.2k
Murron: A Reliable Monitoring Pipeline
vananth22
0
390
The_journey_towards_Pinot.pdf
vananth22
0
220
Reliable_Event_Pipeline___scale.pdf
vananth22
0
190
Operating Data Pipeline with Airflow @ Slack
vananth22
1
2.5k
Streaming data pipelines @ Slack
vananth22
2
2.4k
Other Decks in Programming
See All in Programming
RubyKaigiで得られる10の価値 〜Ruby話を聞くことだけが RubyKaigiじゃない〜
tomohiko9090
0
110
PT AI без купюр
v0lka
0
200
DevTalks 25 - Create your own AI-infused Java apps with ease
kdubois
2
130
抽象データ型について学んだ
ryounasso
0
210
バランスを見極めよう!実装の意味を明示するための型定義 TSKaigi 2025 Day2 (5/24)
whatasoda
2
780
Cursor Meetup Tokyo ゲノミクスとCursor: 進化と制約のあいだ
koido
1
430
Feature Flag 自動お掃除のための TypeScript プログラム変換
azrsh
PRO
4
630
Devinで実践する!AIエージェントと協働する開発組織の作り方
masahiro_nishimi
6
2.6k
Practical Domain-Driven Design - Workshop at NDC 2025
mufrid
0
130
漸進。
ssssota
0
1.2k
バリデーションライブラリ徹底比較
nayuta999999
1
460
從零到一:搭建你的第一個 Observability 平台
blueswen
0
230
Featured
See All Featured
Bootstrapping a Software Product
garrettdimon
PRO
307
110k
The Power of CSS Pseudo Elements
geoffreycrofte
76
5.8k
Fashionably flexible responsive web design (full day workshop)
malarkey
407
66k
Music & Morning Musume
bryan
47
6.6k
Git: the NoSQL Database
bkeepers
PRO
430
65k
Why You Should Never Use an ORM
jnunemaker
PRO
56
9.4k
Improving Core Web Vitals using Speculation Rules API
sergeychernyshev
15
890
Helping Users Find Their Own Way: Creating Modern Search Experiences
danielanewman
29
2.6k
It's Worth the Effort
3n
184
28k
The Pragmatic Product Professional
lauravandoore
35
6.7k
Distributed Sagas: A Protocol for Coordinating Microservices
caitiem20
331
21k
Six Lessons from altMBA
skipperchong
28
3.8k
Transcript
Search infrastructure using Lambda Architecture Ananth Packkildurai
Outline 1.Search @ Slack 2.Lambda Architecture 3.Search infrastructure overview 4.FCC
Design patterns
Search @ Slack
http://www.mckinsey.com/industries/high-tech/our-insights/the-social-economy https://slack.engineering/search-at-slack-431f8c80619e
Lambda Architecture
Stream processing trade-offs Pick 2 Speed Accuracy Volume
Lambda Architecture http://lambda-architecture.net/
Search Infrastructure overview
Search infrastructure overview
! Online sharding by Collections ! Offline sharding by CompositeId
Router Sharding strategy
mutating Index & Delta Replication • What happens if someone
edits the message that indexed in batch mode? • A snapshot occurs at time T1, the index built, and the core comes online at T2.Between T1 & T2, How to handle the update?
FCC Pattern
Fork Pattern
Contract
Clone
Any Questions? Thanks!