Walmart: Near Real Time Analytics

Did you know that Walmart sells about 156 bananas per second? Thanks to the Elastic Stack, they do. What started as an engineering hack to prove the banana myth is now a fully-operational self-service analytics platform for Walmart’s data scientists.

Kevin Conaway l Engineer l Walmart
Srinivas Singanamalla l Engineer l Walmart

Transcript

1. Walmart Technology 03/07/2017 Near Real Time Retail Analytics @ Walmart Kevin Conaway, Engineer Srinivas Singanamalla, Engineer
2. Agenda 2 1 About Us 2 Elasticsearch and X-Pack 3 4 Questions / Feedback Retail Use Cases @Walmart
3. Walmart in a nutshell 3 F1
4. Walmart Technology - East 4
5. Walmart Store 5
6. Inventory Receiving Returns Point of Sale
7. 7 This is a sample image Project Background IBM / Toshiba TLog Register
8. Near Real Time - Simplified POS POS
9. Why Elasticsearch?
10. Elasticsearch 10 Indices 60 1800 5TB 6B Shards Data Documents 18 Nodes
11. 11 m1 m2 m3 m2 m1 m0 Tick tuple Tick tuples introduces Latency ack Bulk Ingestion m0 fail Index.refresh_interval: 60s Shards: 15
12. 12 Shield Marvel Watcher X-Pack
13. 13 This is a sample image • Securing Financial data • LDAP Authentication • Document Level Security • Role Based Access • Message Auditing Shield
14. Marvel 14
15. 15 This is a sample image • Cluster Alerts • Different States: Red, Yellow and Green • Watch Popular metrics: CPU, Disk Usage, Ingestion Rate • Business Use case Alerts • Sales of a UPC is below a threshold • Stores revenue > x dollars Watcher: Health of the Cluster
16. 16 Document Design • Sale Items • UPC • Price • Store Number • Total Sales
18. 18 This is a sample image Use Cases • Price Consistency • Gift Cards Sales • Maximizing Warranty Revenue • Cash Availability in Stores
19. 19 This is a sample image Price Consistency • National Price • Price/UPC Consistency • Monitor in Near Real Time • Cigarettes, gas, gift cards
20. Price Anomaly 20
21. 21 Gift Cards Sales Trends
22. 22 Location Location Location
23. 23 This is a sample image Hourly Warranty Count
24. 24 Cash Availability in Stores
25. • So do we actually sell 156 bananas / second? Lets see Let’s go Bananas 25
26. • 3,941 Scans • 9,022.79 pounds Let’s go Bananas 26
27. • 3,941 Scans • 9,022.79 pounds Let’s go Bananas 27 • 6.5 oz per banana • 370 bananas / second! Same day, we sold 4,614,464.44 pounds of bananas, or