Elastic{ON} 2018 - Helping Grubhub Diners Find the Perfect Meal

Transcript

1. 1 Helping Grubhub Diners Find the Perfect Meal
3. Our Journey to Elasticsearch 3
4. Where we were before Elasticsearch • Used Solr • Deployed servers under an ELB • Multi datacenter hot-hot • Re-build indices and ship them into production with 0 downtime • We were growing and needed to change …. 4
5. Elasticsearch Features Distributed Index 5 Data node Shard Data node Shard Data node Shard
6. Elasticsearch Features Data sources 6 Java Client / Hadoop Connector S3 Cassandra SQS/SNS Kinesis Elasticsearch
7. Elasticsearch Features Relational Data 7 Spatial Temporal Analytics Food
8. Deployment, Discover & Upgrades 8
9. Elasticsearch Deployment 9 AWS Region AWS Availability Zone Elasticsearch • AWS Cloud Plugin • Netflix Eureka • Metrics in Datadog • Tags in Eureka • Eureka aware Client • Index & Node Discovery • Shard allocation aware • Snapshots in S3 Master Node Master Node Master Node Master Node Data Nodes Data Nodes Data Nodes Data Nodes Eureka Eureka Eureka Eureka App Master Node App S3
10. Elasticsearch Snapshots • Snapshots for everyone • Emulating a Production dataset: ◦ Snapshot of the market ◦ Test out a sort • Performance testing: ◦ Index in production ◦ Replay production load 10
11. Search and Relevance 11
12. Data Collection and Feedback 12 Search Cassandra S3 Elasticsearch A B TEST Search Logs Clickstream Events Impressions Data Science
13. Improving Search Features • Create consumable data • What are your KPIs? • What user attributes can you use to influence KPIs? • Decouple moving parts to allow for independent testing • Measure, make changes and improve 13
14. Tuning for Production 14
15. Metrics that matter 15 • Index store size and memory • Active searcher threads • Balance Read and Write • Query distribution / Profiler
16. Balancing Shards 16
17. Location Based Sharding 17 • Balance data in each shard. • Tuned for index size. • Tuned for query throughput. • Indices behind an alias to allow for greater flexibility.
18. Beyond Elasticsearch 18 • Impressions and click tracking • Relevance tuning • Data analytics • Machine learning • A/B testing