Walmart: Near Real Time Analytics

Did you know that Walmart sells about 156 bananas per second? Thanks to the Elastic Stack, they do. What started as an engineering hack to prove the banana myth is now a fully-operational self-service analytics platform for Walmart’s data scientists.

Kevin Conaway l Engineer l Walmart
Srinivas Singanamalla l Engineer l Walmart

Transcript

1. Walmart Technology
   03/07/2017
   Near Real Time Retail Analytics @ Walmart
   Kevin Conaway, Engineer
   Srinivas Singanamalla, Engineer
   

   View Slide

2. Agenda
   2
   1 About Us
   2 Elasticsearch and X-Pack
   3
   4 Questions / Feedback
   Retail Use Cases @Walmart
   

   View Slide

3. Walmart in a nutshell
   3
   F1
   

   View Slide

4. Walmart Technology - East
   4
   

   View Slide

5. Walmart Store
   5
   

   View Slide

6. Inventory
   Receiving
   Returns
   Point of Sale
   

   View Slide

7. 7
   This is a
   sample image
   Project Background
   IBM / Toshiba
   TLog Register
   

   View Slide

8. Near Real Time - Simplified
   POS
   POS
   

   View Slide

9. Why Elasticsearch?
   

   View Slide

10. Elasticsearch
    10
    Indices
    60 1800 5TB 6B
    Shards Data Documents
    18
    Nodes
    

    View Slide

11. 11
    m1 m2
    m3
    m2
    m1
    m0
    Tick tuple
    Tick tuples introduces Latency
    ack
    Bulk Ingestion
    m0
    fail
    Index.refresh_interval: 60s
    Shards: 15
    

    View Slide

12. 12
    Shield Marvel Watcher
    X-Pack
    

    View Slide

13. 13
    This is a
    sample image
    • Securing Financial data
    • LDAP Authentication
    • Document Level Security
    • Role Based Access
    • Message Auditing
    Shield
    

    View Slide

14. Marvel
    14
    

    View Slide

15. 15
    This is a
    sample image
    • Cluster Alerts
    • Different States: Red, Yellow and Green
    • Watch Popular metrics: CPU, Disk Usage, Ingestion Rate
    • Business Use case Alerts
    • Sales of a UPC is below a threshold
    • Stores revenue > x dollars
    Watcher: Health of the Cluster
    

    View Slide

16. 16
    Document Design
    • Sale Items
    • UPC
    • Price
    • Store Number
    • Total Sales
    

    View Slide

17. View Slide

18. 18
    This is a
    sample image
    Use Cases
    • Price Consistency
    • Gift Cards Sales
    • Maximizing Warranty Revenue
    • Cash Availability in Stores
    

    View Slide

19. 19
    This is a
    sample image
    Price Consistency
    • National Price
    • Price/UPC Consistency
    • Monitor in Near Real Time
    • Cigarettes, gas, gift cards
    

    View Slide

20. Price Anomaly
    20
    

    View Slide

21. 21
    Gift Cards Sales Trends
    

    View Slide

22. 22
    Location Location Location
    

    View Slide

23. 23
    This is a
    sample image
    Hourly Warranty Count
    

    View Slide

24. 24
    Cash Availability in Stores
    

    View Slide

25. • So do we actually sell 156 bananas / second? Lets see
    Let’s go Bananas
    25
    

    View Slide

26. • 3,941 Scans
    • 9,022.79 pounds
    Let’s go Bananas
    26
    

    View Slide

27. • 3,941 Scans
    • 9,022.79 pounds
    Let’s go Bananas
    27
    • 6.5 oz per banana
    • 370 bananas / second!
    Same day, we sold 4,614,464.44 pounds of bananas, or
    

    View Slide

28. View Slide