Finding the Right Tool for the Job: Elastic at The Home Depot

The Home Depot has moved to Elastic solutions for product search, event logging/visualization, percolation for discounting rules management, etc. Elasticsearch has become the most trusted tool in our arsenal.

Joe Davis and Adron Hall | Elastic{ON} Tour Seattle | December 3, 2015

Transcript

1. Finding the Right Tool For the Job: Elastic @ Home Depot Joe Davis, Senior IT Architect/Manager | Home Depot Adron Hall | Home Depot 1
2. Overview
3. Talk  Overview • Who  we  are/What  we  do   • How  we  use  Elastic  tools   • Business  Scenario:  Supplier  promotions  —  How  we  used  Elastic  to  solve  a   major  pain  point  for  our  users  and  stakeholders   • Hacking  business  strategy  with  Elastic   • Questions?? 3
4. Who  we  are/What  we  do
5. Who  we  are/What  we  do:  The  Home  Depot  Quote  Center • 80+  associates  working  out  of  Vancouver,  WA   • Facilitate  special  order  quoting/sales  to  “PRO”  customers   • Phone  support/sales   • Special  order  merchandising/product  data  management   • Product  development  (Web-­‐based  solutions  for  vendors  and  associates)   • Annually  $Billions  quoted,  ~$1/2  Billion  sold   5
7. How  we  use  Elastic  tools
8. How  we  use  Elastic  tools • Product  search   • Associate  workflows   • Vendor  workflows   • Customer  Lookup   • Messaging  analytics  —  Logstash/Kibana  (In  development)   • Promotional  rules  engine!?!?     • Query  and  Document  examples   The  possibilities  are  limited  only  by  your  imagination!   8
9. …more  importantly,  WHY  do  we  use  Elastic? 9 • It  just  works   • Configuration  is  a  breeze   • It  is  crazy  flexible   • It  scales  so  easily   • It  handles  server  loss  so  gracefully   • It’s  like  adding  a  10x  programmer  to  your  staff   • It’s  our  Swiss  Army  knife
11. Business  Scenario:  ProDirect  supplier  promotions • Supplier-­‐based  promotions   • “PRO”-­‐only  customer  base   • The  program  was  languishing   • Binders  of  promotions…   • Complex  rulesets   • How  do  we  turn  average,  high-­‐turnover  associates  into  experts:   • Automating  the  insanity  with  the  help  of  Percolator   • Our  PRO  marketing/merchandising  teams  are  now  our  biggest  fans 11
14. How  does  a  promotional  rules  engine  like  this  work? 14 • Establish  input  context   • More  information  means  more  choices  in…   • Decide  which  rules  apply  to  the  given  context   • Retrieve  messages,  adjustments,  etc.  that  apply  to  these  rules   • Client  displays  adjustments  and  translates  messages
15. How  Percolate  works  -­‐  index  of  queries PUT  /my-­‐index/.percolator/123   {          query:{                  bool:{                          must:[  {                                  terms:  {                                          productId:[                                                  427092,427093                                          ]  }                                    }                    }]   }}} 15
16. How  Percolate  works  -­‐  Documents  in  place  of  queries GET  /my-­‐index/my-­‐type/_percolate   {          “doc”:  {                  “storeId”:3406,                  “productId”:427093,                  "name":"Universal  Security  Instruments  Smoke  Alarm  w/  Batteries”,                  "additionalDescription":"1-­‐UL  Listed  USI-­‐1227L”,                  "unit":"EA",                  “count":5          }   } 16
17. How  Percolate  works  -­‐  results {          "took"  :  3,          "_shards"  :  {  …  },          "total"  :  1,          "matches"  :  [                    {                      "_index"  :  "my-­‐index",                      "_id"  :  "123"                  }          ]   } 17
18. It’s  a  little  more  complicated  in  practice,  but  you  get  the  message… {"storeId":3406,"domains":[{"domain":79,"name":"Smoke  &  CO  Alarms","jobsiteDeliveryEnabled":true,"count":null}],"categories":{"categoryId":0,"name":"All  Departments","parentId":0,"children":[{"categoryId":322,"name":"D27  -­‐  Electrical","parentId":0,"children":[{"categoryId":368,"name":"Safety  and   Security","parentId":0,"children":[{"categoryId":373,"name":"Smoke   and   Fire","parentId":0,"children":[],"domainId":null,"productCount":5}],"domainId":null,"productCount":5}],"domainId":null,"productCount":5}],"domainId":null,"productCount":5},"branches":[{"branchId":7813,"branchName":"Owings   Mills,   MD","vendorId":1491,"vendorName":"USI   Electric","brandName":"USI   Electric","branchPhone":"4103632218","branchPhoneExt":"6218","branchFax":null,"latitude":39.4395949,"longitude":-­‐76.7747897,"jobsiteDeliverySettings":{"deliveryRanges":[{"fee":0.0000,"maxMiles": 10000,"freeThreshold":null,"leadTime":5}],"deliveryAreaExclusions":[],"services":[],"jobsiteDistance":null,"customLeadTime":null,"customFee":null,"deliveryAreaExcluded":false},"storeDeliverySettings":{"minimumChargeThreshold":null,"minimumCharge":null,"standardLeadTime":3,"cadenceLeadTime": 3,"deliveryScheduleType":"Standard","cadenceWeek":[]}},{"branchId":-­‐1,"branchName":"The   Home   Depot   -­‐   Store   3406","vendorId":-­‐1,"vendorName":"The   Home   Depot","brandName":"The   Home   Depot","branchPhone":null,"branchPhoneExt":null,"branchFax":null,"latitude":0.0,"longitude": 0.0,"jobsiteDeliverySettings":null,"storeDeliverySettings":{"minimumChargeThreshold":0.0,"minimumCharge":0.0,"standardLeadTime":0,"cadenceLeadTime":null,"deliveryScheduleType":"Standard","cadenceWeek":null}}],"results":[{"type":"Group","id":68488,"productId":393012,"productGroupId": 68488,"domain":79,"skuId":1001547444,"superskuId":null,"mvendorNumber":null,"name":"Universal  Security  Instruments  Battery  Operated  Ionization  Flame,  Smoke  Alarm  w/  Batteries","additionalDescription":"1-­‐UL  Listed  USI-­‐1227L","completeName":"Universal  Security  Instruments  Battery  Operated   Ionization   Flame,   Smoke   Alarm   w/   Batteries","shortName":null,"baseQuantity":1.0,"packageTypeId":null,"unit":"EA","branchProducts":[{"branchId":7813,"pricingTierId":1,"deliverToOptions":["Store","Jobsite"],"deliverTo":"Store","unitPrice":6.58,"cost":3.95,"baseUnitPrice":6.58,"unit":"EA","unitsOf": 24.0,"leadTimeAdder":0,"onHandQuantity":null,"onOrderQuantity":null,"asOfDateTime":"2015-­‐12-­‐02T00:00:00Z","productId":393012,"priceDescription":null,"modelNumber":"USI-­‐1227L","stockProductId":null,"skuId":1001547444,"mvendorNumber": 60085502,"uomIncrementUnit":null,"uomValueIncrement":0.0,"uomMaxValue":0.0,"uomMinValue":0.0,"uomValue":0.0,"hasPalletFee":false,"minOrderQuantity":0,"selected":false,"stale":true,"ivpList":{"ivpValues":null},"proDirectList":null,"xData":"<Data><CostBehaviorType>1</ CostBehaviorType><DisplayBehaviorType>4</DisplayBehaviorType><UoMIncrementUnit   /><UoMValueIncrement>0.0000</UoMValueIncrement><UoMMaxValue>0.0000</UoMMaxValue><UoMMinValue>0.0000</UoMMinValue><IVPList   /></Data>"}],"images":[{"primaryPath":"// content.hdquotecenter.com/cdn/public/ergon/141/861/71/528/JA3FDAMNJ3W4UJKTRIVBQOIU3D.jpg","thumbnailPath":"//content.hdquotecenter.com/cdn/public/ergon/141/861/71/528/JA3FDAMNJ3W4UJKTRIVBQOIU3D.jpg"}],"productIds":[]},{"type":"Group","id":68498,"productId": 393014,"productGroupId":68498,"domain":79,"skuId":1001547444,"superskuId":null,"mvendorNumber":null,"name":"Universal  Security  Instruments  Hardwired  Dual-­‐Sensor  Smoke  Alarm  w/  Strobe  Light","additionalDescription":"1-­‐UL  Listed,  ADA  Compliant  USI-­‐2413","completeName":"Universal  Security   Instruments   Hardwired   Dual-­‐Sensor   Smoke   Alarm   w/   Strobe   Light","shortName":null,"baseQuantity":1.0,"packageTypeId":null,"unit":"EA","branchProducts":[{"branchId":7813,"pricingTierId":1,"deliverToOptions":["Store","Jobsite"],"deliverTo":"Store","unitPrice":86.47,"cost":51.88,"baseUnitPrice": 86.47,"unit":"EA","unitsOf":1.0,"leadTimeAdder":0,"onHandQuantity":null,"onOrderQuantity":null,"asOfDateTime":"2015-­‐12-­‐02T00:00:00Z","productId":393014,"priceDescription":null,"modelNumber":"USI-­‐2413","stockProductId":null,"skuId":1001547444,"mvendorNumber": 60085502,"uomIncrementUnit":null,"uomValueIncrement":0.0,"uomMaxValue":0.0,"uomMinValue":0.0,"uomValue":0.0,"hasPalletFee":false,"minOrderQuantity":0,"selected":false,"stale":true,"ivpList":{"ivpValues":null},"proDirectList":null,"xData":"<Data><CostBehaviorType>1</ CostBehaviorType><DisplayBehaviorType>4</DisplayBehaviorType><UoMIncrementUnit   /><UoMValueIncrement>0.0000</UoMValueIncrement><UoMMaxValue>0.0000</UoMMaxValue><UoMMinValue>0.0000</UoMMinValue><IVPList   /></Data>"}],"images":[{"primaryPath":"// content.hdquotecenter.com/cdn/public/ergon/878/316/269/456/YW3MJMGRQ210EG4GMHNTNAJDOB.jpg","thumbnailPath":"//content.hdquotecenter.com/cdn/public/ergon/878/316/269/456/YW3MJMGRQ210EG4GMHNTNAJDOB.jpg"}],"productIds":[]},{"type":"Group","id":68497,"productId": 393015,"productGroupId":68497,"domain":79,"skuId":1001547444,"superskuId":null,"mvendorNumber":null,"name":"Universal   Security   Instruments   Hardwired   Dual-­‐Sensor   Heat   Alarm","additionalDescription":"Interconnected   1-­‐UL   Listed   USI-­‐2430","completeName":"Universal   Security   Instruments   Hardwired   Dual-­‐Sensor   Heat   Alarm","shortName":null,"baseQuantity":1.0,"packageTypeId":null,"unit":"EA","branchProducts":[{"branchId":7813,"pricingTierId":1,"deliverToOptions":["Store","Jobsite"],"deliverTo":"Store","unitPrice":20.83,"cost":12.50,"baseUnitPrice":20.83,"unit":"EA","unitsOf": 1.0,"leadTimeAdder":0,"onHandQuantity":null,"onOrderQuantity":null,"asOfDateTime":"2015-­‐12-­‐02T00:00:00Z","productId":393015,"priceDescription":null,"modelNumber":"USI-­‐2430","stockProductId":null,"skuId":1001547444,"mvendorNumber": 60085502,"uomIncrementUnit":null,"uomValueIncrement":0.0,"uomMaxValue":0.0,"uomMinValue":0.0,"uomValue":0.0,"hasPalletFee":false,"minOrderQuantity":0,"selected":false,"stale":true,"ivpList":{"ivpValues":null},"proDirectList":null,"xData":"<Data><CostBehaviorType>1</ CostBehaviorType><DisplayBehaviorType>4</DisplayBehaviorType><UoMIncrementUnit   /><UoMValueIncrement>0.0000</UoMValueIncrement><UoMMaxValue>0.0000</UoMMaxValue><UoMMinValue>0.0000</UoMMinValue><IVPList   /></Data>"}],"images":[{"primaryPath":"// content.hdquotecenter.com/cdn/public/ergon/935/560/561/796/5WOMJGG3HT2TEAOTWBCAKAHCPD.jpg","thumbnailPath":"//content.hdquotecenter.com/cdn/public/ergon/935/560/561/796/5WOMJGG3HT2TEAOTWBCAKAHCPD.jpg"}],"productIds":[]},{"type":"Group","id":68496,"productId": 393013,"productGroupId":68496,"domain":79,"skuId":1001547444,"superskuId":null,"mvendorNumber":null,"name":"Universal  Security  Instruments  Hardwired  Dual-­‐Sensor  Flame,  Smoke  Alarm","additionalDescription":"1-­‐UL  Listed  USI-­‐960","completeName":"Universal  Security  Instruments  Hardwired   Dual-­‐Sensor   Flame,   Smoke   Alarm","shortName":null,"baseQuantity":1.0,"packageTypeId":null,"unit":"EA","branchProducts":[{"branchId":7813,"pricingTierId":1,"deliverToOptions":["Store","Jobsite"],"deliverTo":"Store","unitPrice":18.00,"cost":10.80,"baseUnitPrice":18.00,"unit":"EA","unitsOf": 1.0,"leadTimeAdder":0,"onHandQuantity":null,"onOrderQuantity":null,"asOfDateTime":"2015-­‐12-­‐02T00:00:00Z","productId":393013,"priceDescription":null,"modelNumber":"USI-­‐960","stockProductId":null,"skuId":1001547444,"mvendorNumber": 60085502,"uomIncrementUnit":null,"uomValueIncrement":0.0,"uomMaxValue":0.0,"uomMinValue":0.0,"uomValue":0.0,"hasPalletFee":false,"minOrderQuantity":0,"selected":false,"stale":true,"ivpList":{"ivpValues":null},"proDirectList":null,"xData":"<Data><CostBehaviorType>1</ CostBehaviorType><DisplayBehaviorType>4</DisplayBehaviorType><UoMIncrementUnit   /><UoMValueIncrement>0.0000</UoMValueIncrement><UoMMaxValue>0.0000</UoMMaxValue><UoMMinValue>0.0000</UoMMinValue><IVPList   /></Data>"}],"images":[{"primaryPath":"// content.hdquotecenter.com/cdn/public/ergon/434/661/440/722/BTJVMJFGA21BE3WDJLP0IWT2SD.jpg","thumbnailPath":"//content.hdquotecenter.com/cdn/public/ergon/434/661/440/722/BTJVMJFGA21BE3WDJLP0IWT2SD.jpg"}],"productIds":[]},{"type":"Group","id":76158,"productId": 427091,"productGroupId":76158,"domain":79,"skuId":1001547444,"superskuId":null,"mvendorNumber":null,"name":"Universal  Security  Instruments  120-­‐Volt  Hardwired  Photoelectric  Smoke  Alarm","additionalDescription":"Interconnected  1-­‐UL  Listed  2-­‐pack  USI-­‐3204-­‐2P-­‐3CC","completeName":"Universal   Security   Instruments   120-­‐Volt   Hardwired   Photoelectric   Smoke   Alarm","shortName":null,"baseQuantity":1.0,"packageTypeId":null,"unit":"EA","branchProducts":[{"branchId":7813,"pricingTierId":1,"deliverToOptions":["Store","Jobsite"],"deliverTo":"Store","unitPrice":31.98,"cost":19.19,"baseUnitPrice": 31.98,"unit":"EA","unitsOf":3.0,"leadTimeAdder":0,"onHandQuantity":null,"onOrderQuantity":null,"asOfDateTime":"2015-­‐12-­‐02T00:00:00Z","productId":427091,"priceDescription":null,"modelNumber":"USI-­‐3204-­‐2P-­‐3CC","stockProductId":null,"skuId":1001547444,"mvendorNumber": 60085502,"uomIncrementUnit":null,"uomValueIncrement":0.0,"uomMaxValue":0.0,"uomMinValue":0.0,"uomValue":0.0,"hasPalletFee":false,"minOrderQuantity":0,"selected":false,"stale":true,"ivpList":{"ivpValues":null},"proDirectList":null,"xData":"<Data><CostBehaviorType>1</ CostBehaviorType><DisplayBehaviorType>4</DisplayBehaviorType><UoMIncrementUnit   /><UoMValueIncrement>0.0000</UoMValueIncrement><UoMMaxValue>0.0000</UoMMaxValue><UoMMinValue>0.0000</UoMMinValue><IVPList   /></Data>"}],"images":[{"primaryPath":"// content.hdquotecenter.com/cdn/public/ergon/587/53/219/252/2AAKH4TBL1SWULSQCOFUYUNM1E.jpg","thumbnailPath":"//content.hdquotecenter.com/cdn/public/ergon/587/53/219/252/2AAKH4TBL1SWULSQCOFUYUNM1E.jpg"}],"productIds":[]}],"alternateSearches":[],"searchQuery":"USI   Alarm","filters":[{"title":"Source","filters":[{"attributeValueId":1491,"attributeId":-­‐1,"value":"USI   Electric","rank":99,"name":"USI   Electric","filterType":"Source","selected":false,"count":0}],"rankCount":0,"count":0},{"title":"Discounts   &   Promotions","filters":[{"attributeValueId":86947,"attributeId": 270,"value":"Bulk   Pricing","rank":99,"name":"Bulk   Pricing","filterType":"Discounts   &   Promotions","selected":false,"count":5}],"rankCount":0,"count":5},{"title":"Brand","filters":[{"attributeValueId":75537,"attributeId":6,"value":"Universal   Security   Instruments","rank":99,"name":"Universal   Security   Instruments","filterType":"Brand","selected":false,"count":5}],"rankCount":0,"count":5},{"title":"PowerSource","filters":[{"attributeValueId":85915,"attributeId":244,"value":"120-­‐Volt   Hardwired","rank":99,"name":"120-­‐Volt   Hardwired","filterType":"PowerSource","selected":false,"count":1}, {"attributeValueId":75642,"attributeId":244,"value":"Battery   Operated","rank":99,"name":"Battery   Operated","filterType":"PowerSource","selected":false,"count":1},{"attributeValueId":75644,"attributeId":244,"value":"Hardwired","rank": 99,"name":"Hardwired","filterType":"PowerSource","selected":false,"count":3}],"rankCount":0,"count":5},{"title":"Item  Type","filters":[{"attributeValueId":75556,"attributeId":152,"value":"Smoke  and  Fire  Alarms","rank":99,"name":"Smoke  and  Fire  Alarms","filterType":"Item  Type","selected":false,"count": 5}],"rankCount":0,"count":5},{"title":"Alarm   Type","filters":[{"attributeValueId":75625,"attributeId":240,"value":"Flame,   Smoke","rank":99,"name":"Flame,   Smoke","filterType":"Alarm   Type","selected":false,"count":2},{"attributeValueId":75626,"attributeId":240,"value":"Heat","rank": 99,"name":"Heat","filterType":"Alarm   Type","selected":false,"count":1},{"attributeValueId":75627,"attributeId":240,"value":"Smoke","rank":99,"name":"Smoke","filterType":"Alarm   Type","selected":false,"count":2}],"rankCount":0,"count":5},{"title":"Alarm   Sensor   Type","filters":[{"attributeValueId": 75628,"attributeId":241,"value":"Dual-­‐Sensor","rank":99,"name":"Dual-­‐Sensor","filterType":"Alarm   Sensor   Type","selected":false,"count":3},{"attributeValueId":75629,"attributeId":241,"value":"Ionization","rank":99,"name":"Ionization","filterType":"Alarm   Sensor   Type","selected":false,"count":1}, {"attributeValueId":75630,"attributeId":241,"value":"Photoelectric","rank":99,"name":"Photoelectric","filterType":"Alarm  Sensor  Type","selected":false,"count":1}],"rankCount":0,"count":5}],"count":5} 18
19. Hacking  business  strategy
20. Hacking  business  strategy • Our  business  partners  LOVE  us  because  we  deliver  “the  impossible”   • Don’t  go  build  a  promotional  discounting  rules  engine—but,  you  CAN   • Compelling  open-­‐ended  architecture  of  Elastic—the  Swiss  Army  knife  of  data   • Solve  complex  problems  quickly   • Prepare  for  scale   • Elastic  enables   • Rapid  NoSQL  prototyping  with  minimal  configuration   • Easy  and  massive  scale   • “But,  it  was  only  supposed  to  be  a  prototype.” 20
21. What  will  we  do  next  with  Elastic? • Shallow  graph  lookup?   • Product  relationships  (cross-­‐sell/up-­‐sell)   • Compatibles  and  alternatives   • If  I  want  a  specific  vinyl  fencing  panel,  that  automatically  decides  which   connector  types  I  need  and  limits  my  choice  of  posts!  Too  many  returns!!   • Just  in  time  messaging   • Warning!!  If  you  buy  different  brands  of  roofing  shingles  and  felt  rolls,  you   may  void  the  30-­‐year  warranty!!   • Kappa  speed  layer?   • CQRS  read-­‐only  channel? 21
22. We’re  Hiring!! 22 • Lead  BI/DW/Analytics  Engineer   • joseph_davis@homedepot.com   • @TheBigScaryGuy   • QUESTIONS??
23. Questions??