Elastic{ON} 2018 - Nuts and Bolts of Product Search - How Fastenal Refreshes Large Indexes Nightly

Transcript

1. Fastenal February 28, 2018 The Nuts and Bolts of Product Search How Fastenal Refreshes Large Indexes Nightly Kelly Sauke, Enterprise Architect Nicole Albee, DevOps Engineer Alarka Sanyal, Software Developer
2. Topics • Who is Fastenal? • Infrastructure • Elasticsearch Cluster • Data Structure • Elasticsearch Indexing
3. Source: 2017 Fastenal Annual Report
4. LOCAL SERVICE U S S T O R E L O C A T I O N S 3 0 M I N U T E D R I V E 6 0 M I N U T E D R I V E 2,500+ Store Locations ~19k Employees 14 Distribution Hubs 7,000+ Delivery Vehicles ~500 Onsite Locations
6. INDUSTRIAL SERVICES Calibration & Repair Hose Fabrication Hoist Repair Gas Detection Repair Custom Packaging Custom Chain Sling Bandsaw Blades Tool Repair Tool & Cutter Regrind Stud Cutting Special Assemblies Lifting & Rigging V A L U E - A D D W I T H I N T H E S U P P L Y C H A I N
7. Elasticsearch Origin Story (2014) February Loaded Data for first time Product, Customer Warehouse (v1.0.0) Initial Discussions With First Developer March First Development Cluster Online April Official Product Search Project Charter Fastenal Becomes Elasticsearch Inc. Support Customer (v.1.1) Built Ugly POC
8. Use Cases Within Fastenal • Product Search • What we're talking about today • Enterprise Search • Vendor • Customer • Vending • Operations – Telemetry • Logs • Performance Analytics • Product Search telemetry data
9. Product Search www.fastenal.ca
10. Product Search – Category List
11. Product Search – Product Family
12. Product Search – Counter Book
13. Product Search – Product List
14. Product Search – Product Detail
15. Product Search – Recommendations
16. Infrastructure Hardware and Software
17. Hardware and Software Stack
18. Elastic Software Usage Search Operations
19. Elasticsearch VMs 1000 GB 200 GB 64 GB 12 vCPU 1000 GB
20. Production Environment Search Product Operations 1 6 18 Multi 6 3 Enterprise
21. Future • Elastic software upgrade • Dedicated node roles • Masters • Data • Coordinating • Dedicated hardware • Direct attached SSD • Leverage other software from Elastic • Machine Learning • Graph • APM
22. Elasticsearch Cluster Stats of the indices
23. CLUSTER
24. Data Structure Mapping the data into Elasticsearch
25. Anatomy of a Product Category tree Related items CAD URLs Product Info Product attributes
26. Category index Ø Product classification tree Ø Category Image Ø Category Description Ø Metadata related to servicing search requests Ø No child docs Root Fasteners Bolts Eye Bolts U-Bolts Screws Wood Screws Lag Screws Safety Protective Garments Sleeves Hand Protection Glove Bags Level One Level Two Level Three
27. Catalog Index Ø During the building of the product index the catalog index is used to calculate which products belong to which catalogs based on the business rules Ø Initial Structure Ø 1 parent type doc Ø 1 child doc Ø 1 grand child Ø Challenges with grandchild docs Ø Inefficient Ø Cause: Slow search response time Ø Effect: Slow indexing rate of product index Ø Complex query structure Ø Cause: Nested "has_child" query Ø Effect: Increased dev and test time Ø Current Structure Ø 1 parent type doc Ø 1 child doc
28. Product Index v1
29. Product Index v2 Advantages
30. Elasticsearch Indexing Adding the data to Elasticsearch Indices
31. What is Gearman? Ø According to gearman.org, “Gearman provides a generic application framework to farm out work to other machines or processes that are better suited to do the work.” Ø It has three main parts: Ø Client Ø Job server Ø Worker Ref: http://gearman.org/
32. Building blocks
33. Indexing Data
34. Indexing Data Server 1 Server 2 Server 3
35. Indexing Data Server 1 Server 2 Server 3
36. Indexing Data Server 1 Server 2 Server 3
37. Indexing Data Server 1 Server 2 Server 3 Search product category catalog … Logging
38. Indexing Data Server 1 Server 2 Server 3 Worker type 1 Worker type 2 Worker type 3 Worker type 1 queue Worker type 2 queue Worker type 3 queue Search product category catalog … Logging
39. Indexing Data Server 1 Server 2 Server 3 Worker type 1 Worker type 2 Worker type 3 (PHP Code) Index builder Worker type 1 queue Worker type 2 queue Worker type 3 queue Search product category catalog … Logging
40. Search product category catalog … Logging Indexing Data Server 1 Server 2 Server 3 (PHP Code) Query databases Stream data Index builder Enhance data from Elasticsearch Worker type 1 Worker type 2 Worker type 3 Worker type 1 queue Worker type 2 queue Worker type 3 queue
41. Search product category catalog … Logging Indexing Data Server 1 Server 2 Server 3 Query databases Stream data Enhance data from Elasticsearch Worker type 1 Worker type 2 Worker type 3 Worker type 1 queue Worker type 2 queue Worker type 3 queue (PHP Code) Index builder
42. Search product category catalog … Logging Indexing Data Server 1 Server 2 Server 3 Worker type 1 Worker type 2 Worker type 3 Worker type 1 queue Worker type 2 queue Worker type 3 queue (PHP Code) Index builder Query databases Stream data Enhance data from Elasticsearch
43. Search product category catalog … Logging Indexing Data Server 1 Server 2 Server 3 Index search data into Elasticsearch Index log data into Elasticsearch Worker type 1 Worker type 2 Worker type 3 Worker type 1 queue Worker type 2 queue Worker type 3 queue (PHP Code) Index builder Query databases Stream data Enhance data from Elasticsearch
44. Search product category catalog … Logging Indexing Data Server 1 Server 2 Server 3 Index search data into Elasticsearch Enhance data from Elasticsearch Index log data into Elasticsearch Worker type 1 Worker type 2 Worker type 3 Worker type 1 queue Worker type 2 queue Worker type 3 queue (PHP Code) Index builder Query databases Stream data Enhance data from Elasticsearch
45. Search product category catalog … Logging Indexing Data Server 1 Server 2 Server 3 Index search data into Elasticsearch Enhance data from Elasticsearch Index log data into Elasticsearch Worker type 1 Worker type 2 Worker type 3 Worker type 1 queue Worker type 2 queue Worker type 3 queue (PHP Code) Index builder Query databases Stream data Enhance data from Elasticsearch
46. Search product category catalog … Logging Indexing Data Server 1 Server 2 Server 3 Index search data into Elasticsearch Enhance data from Elasticsearch Index log data into Elasticsearch Worker type 1 Worker type 2 Worker type 3 Worker type 1 queue Worker type 2 queue Worker type 3 queue (PHP Code) Index builder Query databases Stream data Enhance data from Elasticsearch
47. Indexing Data v1
48. Indexing Data v1 Create
49. Indexing Data v1 Create
50. Indexing Data v1 Create Multiple updates
51. Indexing Data v1 Create Multiple updates
52. Indexing Data v2
53. Data for “helper” index Indexing Data v2 Data for “main” index
54. Data for “helper” index Indexing Data v2 Data for “main” index
55. Indexing Data v2 Data for “main” index Data for “helper” index Scroll “helper” index
56. Indexing Data v2 Data for “main” index Data for “helper” index Scroll “helper” index
57. Product index helper index Initial create SKUs Single update by scrolling helper index 10+ updates Sleep Time Initial create SKUs Initial create SKUs Single update by scrolling helper index helper index Time Time V1: 24+ hours V2: 18+ hours Current: < 12 hours
58. Additional Insights Ø Suggester limitations Ø "index" : "no" (2.X) to save disk space Ø Elastic Data modeling Future Ø Avoid updates for full index build Ø Really large docs Ø Get rid of parent-child/nested docs Ø Over-tokenize fields
59. Ask us Questions at AMA