Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Walmart: Near Real Time Analytics

Walmart: Near Real Time Analytics

Did you know that Walmart sells about 156 bananas per second? Thanks to the Elastic Stack, they do. What started as an engineering hack to prove the banana myth is now a fully-operational self-service analytics platform for Walmart’s data scientists.

Kevin Conaway l Engineer l Walmart
Srinivas Singanamalla l Engineer l Walmart

Dd9d954997353b37b4c2684f478192d3?s=128

Elastic Co

March 07, 2017
Tweet

Transcript

  1. Walmart Technology 03/07/2017 Near Real Time Retail Analytics @ Walmart

    Kevin Conaway, Engineer Srinivas Singanamalla, Engineer
  2. Agenda 2 1 About Us 2 Elasticsearch and X-Pack 3

    4 Questions / Feedback Retail Use Cases @Walmart
  3. Walmart in a nutshell 3 F1

  4. Walmart Technology - East 4

  5. Walmart Store 5

  6. Inventory Receiving Returns Point of Sale

  7. 7 This is a sample image Project Background IBM /

    Toshiba TLog Register
  8. Near Real Time - Simplified POS POS

  9. Why Elasticsearch?

  10. Elasticsearch 10 Indices 60 1800 5TB 6B Shards Data Documents

    18 Nodes
  11. 11 m1 m2 m3 m2 m1 m0 Tick tuple Tick

    tuples introduces Latency ack Bulk Ingestion m0 fail Index.refresh_interval: 60s Shards: 15
  12. 12 Shield Marvel Watcher X-Pack

  13. 13 This is a sample image • Securing Financial data

    • LDAP Authentication • Document Level Security • Role Based Access • Message Auditing Shield
  14. Marvel 14

  15. 15 This is a sample image • Cluster Alerts •

    Different States: Red, Yellow and Green • Watch Popular metrics: CPU, Disk Usage, Ingestion Rate • Business Use case Alerts • Sales of a UPC is below a threshold • Stores revenue > x dollars Watcher: Health of the Cluster
  16. 16 Document Design • Sale Items • UPC • Price

    • Store Number • Total Sales
  17. None
  18. 18 This is a sample image Use Cases • Price

    Consistency • Gift Cards Sales • Maximizing Warranty Revenue • Cash Availability in Stores
  19. 19 This is a sample image Price Consistency • National

    Price • Price/UPC Consistency • Monitor in Near Real Time • Cigarettes, gas, gift cards
  20. Price Anomaly 20

  21. 21 Gift Cards Sales Trends

  22. 22 Location Location Location

  23. 23 This is a sample image Hourly Warranty Count

  24. 24 Cash Availability in Stores

  25. • So do we actually sell 156 bananas / second?

    Lets see Let’s go Bananas 25
  26. • 3,941 Scans • 9,022.79 pounds Let’s go Bananas 26

  27. • 3,941 Scans • 9,022.79 pounds Let’s go Bananas 27

    • 6.5 oz per banana • 370 bananas / second! Same day, we sold 4,614,464.44 pounds of bananas, or
  28. 28