Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Search Infrastructure using Lambda Architecture

Ananth Packkildurai
May 31, 2017
Programming
1
310

Search Infrastructure using Lambda Architecture

Search

Ananth Packkildurai

May 31, 2017
Tweet

More Decks by Ananth Packkildurai

See All by Ananth Packkildurai
Data Contracts & Domain Ownership
vananth22
0
93
Data Catalogs - Rebuild the Broken Promise
vananth22
0
79
Functional Data Engineering - A Blueprint for adopting functional principles in data pipeline
vananth22
0
500
Back To The Future: Emerging Trends in Data Engineering
vananth22
0
1.2k
Murron: A Reliable Monitoring Pipeline
vananth22
0
380
The_journey_towards_Pinot.pdf
vananth22
0
210
Reliable_Event_Pipeline___scale.pdf
vananth22
0
180
Operating Data Pipeline with Airflow @ Slack
vananth22
1
2.4k
Streaming data pipelines @ Slack
vananth22
2
2.3k

Other Decks in Programming

See All in Programming
令和7年版 あなたが使ってよいフロントエンド機能とは
mugi_uno
10
5.1k
ecspresso, ecschedule, lambroll を PipeCDプラグインとして動かしてみた （プロトタイプ）　/ Running ecspresso, ecschedule, and lambroll as PipeCD Plugins (prototype)
tkikuc
2
1.8k
Внедряем бюджетирование, или Как сделать хорошо?
lamodatech
0
940
ESLintプラグインを使用してCDKのセオリーを適用する
yamanashi_ren01
2
230
各クラウドサービスにおける.NETの対応と見解
ymd65536
0
250
Azure AI Foundryのご紹介
qt_luigi
1
190
Alba: Why, How and What's So Interesting
okuramasafumi
0
210
今年のアップデートで振り返るCDKセキュリティのシフトレフト/2024-cdk-security-shift-left
tomoki10
0
360
PHPUnitしか使ってこなかった 一般PHPerがPestに乗り換えた実録
mashirou1234
0
420
React 19でお手軽にCSS-in-JSを自作する
yukukotani
5
560
いりゃあせ、PHPカンファレンス名古屋2025 / Welcome to PHP Conference Nagoya 2025
ttskch
1
170
return文におけるstd::moveについて
onihusube
1
1.4k

Featured

See All Featured
Why You Should Never Use an ORM
jnunemaker
PRO
54
9.1k
実際に使うSQLの書き方 徹底解説 / pgcon21j-tutorial
soudai
173
51k
Designing Dashboards & Data Visualisations in Web Apps
destraynor
230
52k
The Web Performance Landscape in 2024 [PerfNow 2024]
tammyeverts
3
360
[RailsConf 2023] Rails as a piece of cake
palkan
53
5.1k
jQuery: Nuts, Bolts and Bling
dougneiner
62
7.6k
Let's Do A Bunch of Simple Stuff to Make Websites Faster
chriscoyier
507
140k
Testing 201, or: Great Expectations
jmmastey
41
7.2k
Intergalactic Javascript Robots from Outer Space
tanoku
270
27k
Product Roadmaps are Hard
iamctodd
PRO
50
11k
The Cult of Friendly URLs
andyhume
78
6.1k
The Power of CSS Pseudo Elements
geoffreycrofte
74
5.4k

Transcript

  1. Search infrastructure using Lambda Architecture Ananth Packkildurai

  2. Outline 1.Search @ Slack 2.Lambda Architecture 3.Search infrastructure overview 4.FCC

    Design patterns

  3. Search @ Slack

  4. http://www.mckinsey.com/industries/high-tech/our-insights/the-social-economy https://slack.engineering/search-at-slack-431f8c80619e

  5. Lambda Architecture

  6. Stream processing trade-offs Pick 2 Speed Accuracy Volume

  7. Lambda Architecture http://lambda-architecture.net/

  8. Search Infrastructure overview

  9. Search infrastructure overview

  10. ! Online sharding by Collections ! Offline sharding by CompositeId

    Router Sharding strategy

  11. mutating Index & Delta Replication • What happens if someone

    edits the
 message that indexed in 
 batch mode? • A snapshot occurs at time T1, 
 the index built, and the core 
 comes online at T2.Between T1 & T2, How to handle the update?

  12. FCC Pattern

  13. Fork Pattern

  14. Contract

  15. Clone

  16. Any Questions? Thanks!