Elastic{ON} 2018 - Surfacing the Products You Love on Spring

Transcript

1. Surfacing the Products You Love on Spring Julie Qiu Spring @jqiu25
2. @jqiu25 ES Building Search at Spring
3. 3 @jqiu25
6. Julie Qiu Engineering Lead Catalog Platform Spring @jqiu25
7. Get product data into database Search challenges @jqiu25
8. Get product data into database Keep database in sync with Elasticsearch Search challenges @jqiu25
9. 2,000+ brands 500,000+ products 1 checkout experience @jqiu25
12. How do I find the products that I want?
14. HEELS BLUE DRESS Finding the perfect outfit @jqiu25
16. Finding the perfect outfit @jqiu25
17. Finding the perfect outfit Search is hard @jqiu25
18. Finding the perfect outfit Search is hard Price matters @jqiu25
19. Finding the perfect outfit Search is hard Price matters Too many shoes @jqiu25
20. Finding the perfect outfit @jqiu25
21. Finding the perfect outfit Search is hard Price matters Too many shoes What is a “blue dress”? @jqiu25
22. Finding the perfect outfit Search is hard Price matters Too many shoes What is a “blue dress”? No need for words @jqiu25
23. Finding the perfect outfit Search is hard Price matters Too many shoes What is a “blue dress”? No need for words This time, it’s personal @jqiu25
24. Search is hard: Finding a pair of heels
25. Let’s search for a pair of heels. heels @jqiu25
27. 14,559 results @jqiu25
28. @jqiu25
29. @jqiu25
30. @jqiu25
31. @jqiu25
32. @jqiu25
33. ES Search is only half the story @jqiu25
34. • Get product data from suppliers Search is kind of hard @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
35. Getting product data @jqiu25
36. Some product data is pushed @jqiu25
37. Some product data is pulled @jqiu25
38. Product data ingestion Catalog Ingestion System @jqiu25
39. • Get product data from suppliers • Sync Postgres with Elasticsearch Search is kind of hard @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
40. Sync database with Elasticsearch ES repopulate (nightly) ES Catalog Ingestion System @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
41. Products Index Alias products (alias) products_20180131030000 products_20180131030000 products_20180129030000 products_group (index_group) (index) @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
42. Alexandre Birman Elouise Suede Platform Sandals $695 $278 @jqiu25
43. @jqiu25
44. • Get product data from suppliers • Sync database with Elasticsearch • Real-time updates Search is kind of hard @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
45. Real-time updates @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
46. Real-time updates Consistent Data Channon Heeled Sandal by Club Monaco @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
47. Consistent Data Channon Heeled Sandal by Club Monaco Regularly Updated $298 Now $139.30 Sizes: • 38 • 39 • 40 Unavailable Real-time updates @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
48. Real-time updates ES repopulate (nightly) ES Catalog Ingestion System @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
49. Real-time updates ES repopulate (nightly) ES ES sync (real-time) Catalog Ingestion System @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
50. Club Monaco Channon Heeled Sandal $298 @jqiu25
51. Club Monaco Channon Heeled Sandal $298 Club Monaco Channon Heeled Sandal $298 $199 @jqiu25
52. Price Matters: Finding a pair of heels on sale
53. • Mark products on sale Price Matters @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
54. Mark products on sale ES repopulate (nightly) ES ES sync (real-time) Catalog Ingestion System @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
55. Mark products on sale @jqiu25
56. Mark products on sale Sale Start and End Dates @jqiu25
57. Mark products on sale Sale Amount @jqiu25
59. • Mark products on sale • Store prices in Elasticsearch Price Matters @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
60. Store prices in Elasticsearch Sale Tagger ES repopulate (nightly) ES ES sync (real-time) Catalog Ingestion System @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
61. • Mark products on sale • Store prices in Elasticsearch • Compute sale prices Price Matters @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
62. Compute sale prices { "price":"409", "prices":[ { "active_dates":{ "start":"2000-01-01T00:00:00Z", "end":"2100-01-01T00:00:00Z" }, "price":"409", "discount_amount":48, } ] } @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
63. Compute sale prices Consumer API ES @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
65. Too many shoes: Refining search results
66. Let’s search for a pair of blue heels. blue heels @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
68. What I expect when I search for “blue heels” Too Many Shoes Price Matters Search Is Hard Adding NLP Visual Browse Personalized Experience @jqiu25
69. Baby Powder Candied Veg Royal Silk Actual colors of these heels @jqiu25 Too Many Shoes Price Matters Search Is Hard Adding NLP Visual Browse Personalized Experience
70. • Categorize data Too many shoes @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
71. Categorizing data: Sometimes it’s simple @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
72. What brand are these heels? @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
73. Vince Camuto Shoes of Prey Club Monaco What brand are these heels? @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
74. Categorizing data: Sometimes it’s simple Sometimes it’s complicated @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
75. What color are these heels? @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
76. What color are these heels? Candied Veg Royal Silk Baby Powder @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
77. Color Categorization API Categorize data @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
78. • Categorize data • Aggregate documents Too many shoes @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
79. Aggregate documents @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal Consumer API ES
80. • Categorize data • Aggregate documents • Display results Too many shoes @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
82. @jqiu25 We found a pair of heels! Club Monaco Channon Heeled Sandal $298 $199
84. Let’s search for a blue dress blue dress @jqiu25
86. blue dress shirt blue dress shoes @jqiu25
87. What is a blue dress? (for humans) @jqiu25
88. What is a blue dress? (for machines) @jqiu25
89. What is a blue dress? Incorporating Natural Language Processing
90. @jqiu25
91. • Get Twiggle product features Incorporating NLP @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
92. Get Twiggle product features Catalog Ingestion System ES Elasticsearch Cron Jobs NSQ @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
93. • Get Twiggle product features • Index Twiggle’s product ontology Incorporating NLP @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
94. Index Twiggle’s product ontology { “$concept_sm”: [“Dress”, “FashionProduct”], “$color_o”: {"$concept_sm": [”Blue”]}, “design_patterns_o”: {"$concept_sm": [”Floral”, “Pattern”]}, “material_so”: {"$concept_sm": [”Silk”]}, “persona_so”: {"$concept_sm": [”Women”]}, /* …more features… */ } @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
95. • Get Twiggle product features • Index Twiggle’s product ontology • Interpret search queries Incorporating NLP @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
96. Interpreting search queries @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal Consumer API ES
97. Interpret search queries “blue dress with a floral pattern under 1600” { “@combine”: [ {“$concept_sm”: {“@eq”: “Dress”}}, {“$color_sm”: {“@eq”: “Blue”}}. {“$$price”: {“usd”: {“@lte”:1600}}}. {“design_patterns_o”:{ “$concept_sm”: {”@eq”: “Floral”} }}}, ] } @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
99. What I picture wearing in my mind @jqiu25
100. Assorted dresses @jqiu25
101. Sort of what I want @jqiu25
102. No need for words: Similarity and visual browse
103. • Classify product attributes Visual browse @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
104. • Name • Brand • Price Classify product attributes • Size • Category • Color @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
105. • Classify product attributes • Generate similar products Visual browse @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
106. Sort of what I want @jqiu25
107. Generate similar products @jqiu25 Emilio Pucci Ruffle Sleeve Dress $1,280
108. Generate similar products @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal Consumer API ES
109. • Classify product attributes • Generate similar products • Browse by similar products Visual browse @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
111. This time, it’s personal: We look good in different blue dresses
112. • Classify product attributes (better) This time, it’s personal @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
113. Attributes we know • Name • Brand • Price • Size • Category • Color @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
114. • Style • Occasion • Season Attributes we want to know • Shape • Trend • … and more! @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
115. How do we define style? @jqiu25
116. Catalog Ingestion System ES Elasticsearch Sync Jobs Attribute Tagging System @jqiu25 NSQ Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
117. • Classify product attributes (better) • Understand user preferences This time, it’s personal @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
118. Understand style preferences @jqiu25
119. Understand style preferences @jqiu25
120. Understand style preferences @jqiu25
121. Track user events Roland Mouret ALLERSTON DRESS - BLUE $2,885 $1,440 @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
122. Track user events Products viewed Roland Mouret ALLERSTON DRESS - BLUE $2,885 $1,440 @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
123. Track user events Favorited products Products viewed Roland Mouret ALLERSTON DRESS - BLUE $2,885 $1,440 @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
124. Track user events Brands loved Favorited products Products viewed Roland Mouret ALLERSTON DRESS - BLUE $2,885 $1,440 @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
125. • Classify product attributes (better) • Understand user preferences • Personalize results This time, it’s personal @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
126. Personalize results Popular Products ES Personalized Products User Preferences User Events @jqiu25 Search Is Hard Price Matters Too Many Shoes Adding NLP Visual Browse It’s Personal
127. Blue dresses personalized for me! women > clothing > dresses Roland Mouret ALLERSTON DRESS - BLUE $2,885 $1,440 Roland Mouret GERZON GOWN – NAVY WOOL $3,200 Laundry by Shelli Segal MATTE CREPE MERMAID GOWN $295 $177 @jqiu25
128. We found a blue dress! Roland Mouret ALLERSTON DRESS - BLUE $2,885 $1,440 @jqiu25
130. @jqiu25 ES Building Search at Spring
131. It’s more than looking at raw data.
132. It’s more than keyword matching.
133. It’s about intent.
134. Thanks! Julie Qiu @jqiu25 Promo Link: bit.ly/spring-elasticon