Tweet
Tweet

Slide 1

Slide 1 text

Search infrastructure using Lambda Architecture Ananth Packkildurai

Slide 2

Slide 2 text

Outline 1.Search @ Slack 2.Lambda Architecture 3.Search infrastructure overview 4.FCC Design patterns

Slide 3

Slide 3 text

Search @ Slack

Slide 4

Slide 4 text

http://www.mckinsey.com/industries/high-tech/our-insights/the-social-economy https://slack.engineering/search-at-slack-431f8c80619e

Slide 5

Slide 5 text

Lambda Architecture

Slide 6

Slide 6 text

Stream processing trade-offs Pick 2 Speed Accuracy Volume

Slide 7

Slide 7 text

Lambda Architecture http://lambda-architecture.net/

Slide 8

Slide 8 text

Search Infrastructure overview

Slide 9

Slide 9 text

Search infrastructure overview

Slide 10

Slide 10 text

! Online sharding by Collections ! Offline sharding by CompositeId Router Sharding strategy

Slide 11

Slide 11 text

mutating Index & Delta Replication • What happens if someone edits the
 message that indexed in 
 batch mode? • A snapshot occurs at time T1, 
 the index built, and the core 
 comes online at T2.Between T1 & T2, How to handle the update?

Slide 12

Slide 12 text

FCC Pattern

Slide 13

Slide 13 text

Fork Pattern

Slide 14

Slide 14 text

Contract

Slide 15

Slide 15 text

Clone

Slide 16

Slide 16 text

Any Questions? Thanks!