Smart Searcher

SheHack - Hackathon at Gojek

Transcript

1. Smart Searcher A virtual assistant to find exactly what you need Nancy Chauhan Team: OhMyCode!
2. Hello! I am Nancy Chauhan • SDE @ Grofers Engg. • B.Tech ECE from IIIT Una I love to make cool things ranging from hardware to infra. Find me at @nancy_nano_ on Twitter 2
3. “Finding items in online shopping is hard, no matter how good the filters are… It can never beat the efficiency of a store guide. Until… AI! 3
4. 74% of shoppers Rate ease of product selection as an important for a shopping portal $ 3,000,000,000 Loss to industry due to user drop off 38% users drop If they find search layout unattractive 4 https://www.disruptiveadvertising.com/ppc/ecommerce/2018-ecommerce-statistics/
5. Smart Searcher Will... • Find your heart’s wish from a never ending catalogue of products with nothing more than a picture of something similar • Will also recommend accessories that are a match to that dress you want to wear to that party 5
6. Behind the scenes… 6
7. Tech Stack 7 • Image Classification and similarity comparison powered by Tensorflow. • API served by Flask • Mobile App in Ionic/React
8. How it works? 8 • Compares user’s image with pre-calculated vector representations from InceptionV4 • Generate a list of candidate products based on cosine similarity 2 Compare • Classifies user’s image to what category it is • Uses InceptionV4-ResNet50 to classify into one of the classes in ImageNet • Determine which category to search for 1 Classify • Depending on user’s request, serve the product with highest similarity 3 Recommend
9. 9 1. Listed products are pre-processed at batch and data is available offline. 2. A model trained on ImageNet decides the category of a product and sends it to the appropriate pipeline. 3. Pipeline for various categories may vary such as, we may extract color information for garments, or estimate sizes for mobile phones etc. 4. Using all extracted features, we build a search space from our catalogue and compare it for similarity using a cosine vector similarity algorithms. Vectors are generated from the model at step 2. 5. Show results to user!
11. ImageNet 11 • Image database organized according to the WordNet hierarchy. • Cover almost all categories of products sold online
12. Compared to traditional recommender systems 12 • Doesn’t require tracking user’s every purchase to generate recommendations. However, such data can greatly refine predictions • Users can easily describe what they want… “A picture is worth a thousand words” • Can scale to hundreds of categories with additional pipelines for classification
13. 13 Feasibility • Scalable due to moderately intensive resource needs • Majority of users shop using mobile phones, where clicking photos is easy
14. Thank you ! 14