BDS18___Machine_learning_for_product_matching__the_fashion_use_case.pdf

Transcript

1. ML FOR PRODUCT MATCHING The fashion use case Javier Ordóñez | @fjordonz Alicia Pérez | @alipeji
2. StyleSage Powering retail’s speed-to-market with AI Assortment Pricing Promotions Trends
3. Data Science @StyleSage 400,000,000 products + 1,000,000 new per week Image Recognition NLP Time Series
4. Data Science @StyleSage 400,000,000 products + 1,000,000 new per week Image Recognition NLP Time Series
5. Product matching The Gate’s prediction
7. PRODUCT MATCHING Definition
8. PRODUCT MATCHING images
9. PRODUCT MATCHING text
10. GOALS prices
11. GOALS prices
12. GOALS assortment
13. GOALS organized inventory ??
14. Universal identifiers (UPC, GTIN, ISBN, etc) could work
15. In fashion business, in very rare occasions we have a barcode :( Universal identifiers (UPC, GTIN, ISBN, etc) could work
16. The barcode role
19. Product matching ≠ Product similarity
20. 1. Standard taxonomies Comparing apples with apples
21. Unstructured data
22. Unstructured data
23. Unstructured data
24. Image based classifier
25. Jackets Shoes Dresses ? ? ? Classifying products
26. 0.90 dresses 0.08 t-shirt 0.02 jacket Deep neural networks
27. Attribute extraction from image Category: Dress
28. Attribute extraction from image Category: Dress Color: Red
29. Attribute extraction from image Category: Dress Print: Floral Color: Red
30. Attribute extraction from image Category: Dress Print: Floral Color: Red Sleeve: Short
31. Attribute extraction from image Category: Dress Print: Floral Color: Red Sleeve: Short Length: Medium
32. Attribute extraction from image Category: Dress Print: Floral Color: Red Sleeve: Short Length: Medium Neck: Turtle
33. Attribute extraction from image Category: Dress Print: Floral Color: Red Sleeve: Short Length: Medium Neck: Turtle Type: Day
34. Attribute extraction from image Category: Dress Print: Floral Color: Red Sleeve: Short Length: Medium Neck: Turtle Type: Day Ruffles: No
35. Attribute extraction from image Category: Dress Print: Floral Color: Red Sleeve: Short Length: Medium Neck: Turtle Type: Day Ruffles: No
36. Attribute extraction from image Category: Dress Print: Floral Color: Red Sleeve: Short Length: Medium Neck: V Type: Day Ruffles: No
37. Attribute extraction from image Category: Dress Print: Floral Color: Red Sleeve: Short Length: Medium Neck: V Type: Day Ruffles: No
38. Is this a match?
39. Category: Dress Print: Polka dots Print: Polka dots
40. Category: Dress Sequins: No Sleeve: Long Print: Polka dots Sleeve: Long Print: Polka dots
41. Category: Dress Sleeve: Long Sequins: No Length: Long Sleeve: Long Print: Polka dots Length: Long Sleeve: Long Print: Polka dots
42. Color: #0f1719 Category: Dress Sleeve: Long Sequins: No Category: Dress Length: Long Sleeve: Long Print: Polka dots Category: Dress Length: Long Sleeve: Long Print: Polka dots
43. Color: #0f1719 Category: Dress Sleeve: Long Print: Polka dots Sequins: No Category: Dress Length: Long Ruffles: Yes Sleeve: Long Print: Polka dots Category: Dress Length: Long Sleeve: Long Print: Polka dots
44. Color: #0f1719 Category: Dress Sleeve: Long Print: Polka dots Sequins: No Neck: Turtle Color: #0f1719 Category: Dress Length: Long Ruffles: Yes Sleeve: Long Print: Polka dots Color: #0f1719 Category: Dress Length: Long Sleeve: Long Print: Polka dots
45. Color: #0f1719 Category: Dress Length: Long Sleeve: Long Print: Polka dots Sequins: No Neck: Turtle Color: #0f1719 Category: Dress Length: Long Ruffles: Yes Sleeve: Long Print: Polka dots Neck: Turtle Color: #0f1719 Category: Dress Length: Long Sleeve: Long Print: Polka dots Neck: Turtle
46. Color: #0f1719 Category: Dress Length: Long Ruffles: Yes Sleeve: Long Print: Polka dots Sequins: No Neck: Turtle Color: #0f1719 Category: Dress Length: Long Sleeve: Long Print: Polka dots Sequins: No Neck: Turtle Color: #0f1719 Category: Dress Length: Long Ruffles: Yes Sleeve: Long Print: Polka dots Sequins: No Neck: Turtle Color: #0f1719 Category: Dress Length: Long Sleeve: Long Print: Polka dots Sequins: No Neck: Turtle
48. It’s not always so easy... Which is the product?
49. It’s not always so easy... Pant or pijama? Which is the product?
50. It’s not always so easy... Pant or pijama? Which is the product? Product or detail?
51. Text based classifier
52. Text based classifier
53. Text based classifier
54. Text processing Practical sophisticated this season hot trend jacket . . String to vector Hashes Label
55. It’s not always so easy... Title / Description is often ambiguous and low granularity Puffer or Bomber Jacket?
56. It’s not always so easy... Title / Description is often ambiguous and low granularity Different spellings of the same things spandex = elastane? Underarmour = Under Armour? XXL = 2XL? Blue = Sky?
57. It’s not always so easy... Title / Description is often ambiguous and low granularity Different spellings of the same things Inconsistent languages and missing data
58. From the web to the taxonomy
59. From the web to the taxonomy
60. From the web to the taxonomy
61. … and now we can compare
62. … and now we can compare
63. 2. Feeding the algorithm What are our instances and features
64. Defining a match is...
65. Defining a match is... ≠
66. Defining a match is... = ≠
67. = ≠ … a supervised binary classification task Defining a match is...
68. Defining a match is... … a supervised binary classification task match not match
69. Standard characteristics Neck: V Sleeve: Short Occasion: Evening Occasion: Day Casual Sleeve: Sleeveless Neck: V
70. Text similarity Neck: V Sleeve: Short Occasion: Evening Occasion: Day Casual Sleeve: Sleeveless Neck: V
71. Text similarity Practical sophisticated this season hot trend jacket . . Dot product Euclidean Manhattan Cosine
72. Color similarity Neck: V Sleeve: Short Occasion: Evening Occasion: Day Casual Sleeve: Sleeveless Neck: V
73. Color density from image #191970
74. Color density from image #191970 Color name normalization Midnight blue = Dark blue
75. Color density from image #191970 Color name normalization Midnight blue = Dark blue Color code dictionary Dark blue = #191970
76. CIELAB ΔE* The International Commission on Illumination (CIE) calls their distance metric ΔE*
77. Material similarity Neck: V Sleeve: Short Occasion: Evening Occasion: Day Casual Sleeve: Sleeveless Neck: V
78. rayon viscose lyocell rayon viscose lyocell polyester latex goretex polyurethane polyester latex goretex polyurethane Rayon based Synthetic
79. rayon viscose lyocell rayon viscose lyocell polyester latex goretex polyurethane polyester latex goretex polyurethane Rayon based Synthetic 17% 0% 100% 100%
80. Attribute similarity Neck: V Sleeve: Short Occasion: Evening Occasion: Day Casual Sleeve: Sleeveless Neck: V
81. It’s not always so easy... Category: Dress Length: Long Ruffles: No Print: Solid Sequins: No Neck: V Sleeve: Short Occasion: Evening
82. Occasion: Day Casual Category: Dress Length: Short Ruffles: No Sleeve: Sleeveless Print: Solid Sequins: No Neck: V Category: Dress Length: Midi Ruffles: No Print: Solid Sequins: No Neck: V Sleeve: Short Occasion: Evening
83. Sleeve: Short Similarity sorting heuristics Sleeve: Long Sleeve: Sleeveless Sleeve: Strapless
84. Sleeve: Short Sleeve: Long Sleeve: Sleeveless Sleeve: Strapless 0.88 0.80 Similarity sorting heuristics
85. Image similarity Neck: V Sleeve: Short Occasion: Evening Occasion: Day Casual Sleeve: Sleeveless Neck: V
86. 0.82 SIAMESE NEURAL NETWORK
87. 0.79 0.82 0.91 0.54 0.91 0.99 1.0 0.13 0.86 0.76 0.98 0.82 0.88 0.78 Text Color Material Attributes Image
88. Features ... ... ... label Features ... ... ... label
89. Category: Dress
90. Category: Dress Missing pictures
91. Category: Dress Missing pictures Wrong extracted color
92. Category: Dress Missing pictures Wrong extracted color Unknown material
93. Category: Dress Missing pictures Wrong extracted color Unknown material Poor attributes
94. Category: Dress 0.79 0.82 0.91 0.54 ... ... ... ... ... 0.76 ... ... ... ... Text Color Material Attributes Image
95. 3. Model design The Choosing
99. WEAK LEARNERS
100. WEAK LEARNERS
101. WEAK LEARNERS ENSEMBLE MODEL
102. ENSEMBLE MODEL Robust model Missing features Unbalanced data Good first approach XGBoost
103. 4. Training Fantastic Matches and Where to Find Them
104. Data sources
105. Data sources
106. Data sources
107. Data sources
108. Data sources Semi-supervised approach match match match
109. 5. Evaluating Proper metrics for proper unbalanced tasks
110. 30% overlap
111. 30% overlap 3 pairs are matches 97 pairs are not matches
113. Accuracy = 12/15 = 80% not matches
114. matches
115. matches Precision Precision = ⅖ = 40%
116. matches Precision Precision = ⅖ = 40% Recall Recall = ⅔ = 66%
117. 6. Deploying The Good, the Bad and the Ugly
118. Retailer 1 850,000 products Retailer 2 380,000 products 323,000,000,000 pairwise combinations
119. Neck: V Sleeve: Short Occasion: Evening Occasion: Day Casual Sleeve: Sleeveless Neck: V SCALING & CLUSTERING
120. Neck: V Sleeve: Short Occasion: Evening Occasion: Day Casual Sleeve: Sleeveless Neck: V BRAND GENDER CATEGORY SCALING & CLUSTERING
121. SCALING & CLUSTERING
122. Heuristics & tips Very specific color names not matching Midnight blue [Sea blue, Sky blue] vs Blue [Sad blue, Midnight blue]
123. Heuristics & tips Very specific color names not matching Very precise material information not matching Polyester 97%, Cotton 3% vs Cotton 100%
124. Heuristics & tips Very specific color names not matching Very precise material information not matching Very different prices not matching
125. Heuristics & tips Very specific color names not matching Very precise material information not matching Very different prices not matching Very rare occasions we have a barcode Only the 5% !
126. Heuristics & tips Very specific color names not matching Very precise material information not matching Very different prices not matching Very rare occasions we have a barcode Think at which level of category you define clusters Jeans is better than trousers Dresses is better than party dresses
127. Lessons Standard taxonomy is key
128. Lessons Tools are important Tools are important Standard taxonomy is key
129. Lessons Tools are important Barcodes are scarce good features Standard taxonomy is key
130. Lessons Tools are important More QA than expected Standard taxonomy is key Barcodes are scarce good features
131. Lessons Tools are important More QA than expected Standard taxonomy is key Features & data > Model Barcodes are scarce good features
132. ML FOR PRODUCT MATCHING The fashion use case javier.ordonez@stylesage.co ? alicia@stylesage.co