Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Speaker Deck
PRO
Sign in
Sign up
for free
Search Infrastructure using Lambda Architecture
Ananth Packkildurai
May 31, 2017
Programming
1
170
Search Infrastructure using Lambda Architecture
Search
Ananth Packkildurai
May 31, 2017
Tweet
Share
More Decks by Ananth Packkildurai
See All by Ananth Packkildurai
vananth22
0
140
vananth22
0
61
vananth22
0
51
vananth22
1
1.3k
vananth22
2
1.2k
vananth22
0
1.1k
Other Decks in Programming
See All in Programming
viteinfinite
0
200
pdone
0
210
akatsukinewgrad
0
180
rishitdagli
0
180
aratayokoyama
0
200
makicamel
1
180
anchorcable
1
120
akatsukinewgrad
0
190
mizdra
7
4.8k
daiki1020
0
1.1k
manfredsteyer
PRO
0
260
wasabeef
1
560
Featured
See All Featured
thoeni
4
550
chriscoyier
780
240k
jlugia
216
16k
eitanlees
111
9.9k
denniskardys
220
120k
pauljervisheath
196
15k
dougneiner
119
7.8k
lara
590
61k
sferik
609
54k
orderedlist
PRO
329
35k
myddelton
109
11k
holman
461
280k
Transcript
Search infrastructure using Lambda Architecture Ananth Packkildurai
Outline 1.Search @ Slack 2.Lambda Architecture 3.Search infrastructure overview 4.FCC
Design patterns
Search @ Slack
http://www.mckinsey.com/industries/high-tech/our-insights/the-social-economy https://slack.engineering/search-at-slack-431f8c80619e
Lambda Architecture
Stream processing trade-offs Pick 2 Speed Accuracy Volume
Lambda Architecture http://lambda-architecture.net/
Search Infrastructure overview
Search infrastructure overview
! Online sharding by Collections ! Offline sharding by CompositeId
Router Sharding strategy
mutating Index & Delta Replication • What happens if someone
edits the message that indexed in batch mode? • A snapshot occurs at time T1, the index built, and the core comes online at T2.Between T1 & T2, How to handle the update?
FCC Pattern
Fork Pattern
Contract
Clone
Any Questions? Thanks!