Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Search Infrastructure using Lambda Architecture

Ananth Packkildurai
May 31, 2017
Programming
1
250

Search Infrastructure using Lambda Architecture

Search

Ananth Packkildurai

May 31, 2017
Tweet

More Decks by Ananth Packkildurai

See All by Ananth Packkildurai
Back To The Future: Emerging Trends in Data Engineering
vananth22
0
1k
Murron: A Reliable Monitoring Pipeline
vananth22
0
250
The_journey_towards_Pinot.pdf
vananth22
0
130
Reliable_Event_Pipeline___scale.pdf
vananth22
0
87
Operating Data Pipeline with Airflow @ Slack
vananth22
1
1.8k
Streaming data pipelines @ Slack
vananth22
2
1.7k
measuring api performance using druid
vananth22
0
1.4k

Other Decks in Programming

See All in Programming
Git 未経験者 が GitHub Actions で もがいた話 / 20220914 git beginner uses github actions
mackey0225
1
420
Web UI Feuerwerk mit SVG und der Web Animation API
joergneumann
0
370
Attacking web without JS - CSS injection
aszx87410
0
1.9k
2年かけました！大規模サービスをJava製CMSからPHP+Laravelの構成にリプレイスし、運用している話 / Replaced service from Java CMS to PHP and Laravel architecture
yumechi
1
480
Ruby Pattern Matching
bkuhlmann
0
480
API Platform 3
dunglas
2
3.1k
controller-runtime Deep Dive
bells17
2
110
Layout Grid for Browser | FigmaのLayout Gridを拡張機能で実装してブラウザに持ち込んでみた話
hanzochang
0
120
[API Platform Con] Domain Driven Design with API Platform 3
mtarld
0
3k
Swift Concurrency時代のiOSアプリの作り方
koher
13
2.2k
入門Hello world（PHPカンファレンス2022）
yutaron
0
270
フレームワークの機能を使わずに標準関数使ったら障害起こした話/phpcon-2022
asumikam
1
470

Featured

See All Featured
ピンチをチャンスに：未来をつくるプロダクトロードマップ #pmconf2020
aki_i
25
16k
Optimizing for Happiness
mojombo
364
64k
Art, The Web, and Tiny UX
lynnandtonic
281
18k
Principles of Awesome APIs and How to Build Them.
keavy
114
15k
Git: the NoSQL Database
bkeepers
PRO
417
60k
The Mythical Team-Month
searls
210
40k
Keith and Marios Guide to Fast Websites
keithpitt
405
21k
Art Directing for the Web. Five minutes with CSS Template Areas
malarkey
196
9.6k
Automating Front-end Workflow
addyosmani
1351
200k
Raft: Consensus for Rubyists
vanstee
128
5.5k
Java REST API Framework Comparison - PWX 2021
mraible
PRO
12
5k
Imperfection Machines: The Place of Print at Facebook
scottboms
254
12k

Transcript

  1. Search infrastructure using Lambda Architecture Ananth Packkildurai

  2. Outline 1.Search @ Slack 2.Lambda Architecture 3.Search infrastructure overview 4.FCC

    Design patterns

  3. Search @ Slack

  4. http://www.mckinsey.com/industries/high-tech/our-insights/the-social-economy https://slack.engineering/search-at-slack-431f8c80619e

  5. Lambda Architecture

  6. Stream processing trade-offs Pick 2 Speed Accuracy Volume

  7. Lambda Architecture http://lambda-architecture.net/

  8. Search Infrastructure overview

  9. Search infrastructure overview

  10. ! Online sharding by Collections ! Offline sharding by CompositeId

    Router Sharding strategy

  11. mutating Index & Delta Replication • What happens if someone

    edits the
 message that indexed in 
 batch mode? • A snapshot occurs at time T1, 
 the index built, and the core 
 comes online at T2.Between T1 & T2, How to handle the update?

  12. FCC Pattern

  13. Fork Pattern

  14. Contract

  15. Clone

  16. Any Questions? Thanks!