Search Infrastructure using Lambda Architecture

Search

Transcript

1. Search infrastructure using Lambda Architecture Ananth Packkildurai
2. Outline 1.Search @ Slack 2.Lambda Architecture 3.Search infrastructure overview 4.FCC Design patterns
3. Search @ Slack
4. http://www.mckinsey.com/industries/high-tech/our-insights/the-social-economy https://slack.engineering/search-at-slack-431f8c80619e
5. Lambda Architecture
6. Stream processing trade-offs Pick 2 Speed Accuracy Volume
7. Lambda Architecture http://lambda-architecture.net/
8. Search Infrastructure overview
9. Search infrastructure overview
10. ! Online sharding by Collections ! Offline sharding by CompositeId Router Sharding strategy
11. mutating Index & Delta Replication • What happens if someone edits the
 message that indexed in 
 batch mode? • A snapshot occurs at time T1, 
 the index built, and the core 
 comes online at T2.Between T1 & T2, How to handle the update?
12. FCC Pattern
13. Fork Pattern
14. Contract
15. Clone
16. Any Questions? Thanks!