Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Smart Searcher

Nancy Chauhan
November 08, 2019

Smart Searcher

SheHack - Hackathon at Gojek

Nancy Chauhan

November 08, 2019
Tweet

More Decks by Nancy Chauhan

Other Decks in Technology

Transcript

  1. Hello! I am Nancy Chauhan • SDE @ Grofers Engg.

    • B.Tech ECE from IIIT Una I love to make cool things ranging from hardware to infra. Find me at @nancy_nano_ on Twitter 2
  2. “Finding items in online shopping is hard, no matter how

    good the filters are… It can never beat the efficiency of a store guide. Until… AI! 3
  3. 74% of shoppers Rate ease of product selection as an

    important for a shopping portal $ 3,000,000,000 Loss to industry due to user drop off 38% users drop If they find search layout unattractive 4 https://www.disruptiveadvertising.com/ppc/ecommerce/2018-ecommerce-statistics/
  4. Smart Searcher Will... • Find your heart’s wish from a

    never ending catalogue of products with nothing more than a picture of something similar • Will also recommend accessories that are a match to that dress you want to wear to that party 5
  5. Tech Stack 7 • Image Classification and similarity comparison powered

    by Tensorflow. • API served by Flask • Mobile App in Ionic/React
  6. How it works? 8 • Compares user’s image with pre-calculated

    vector representations from InceptionV4 • Generate a list of candidate products based on cosine similarity 2 Compare • Classifies user’s image to what category it is • Uses InceptionV4-ResNet50 to classify into one of the classes in ImageNet • Determine which category to search for 1 Classify • Depending on user’s request, serve the product with highest similarity 3 Recommend
  7. 9 1. Listed products are pre-processed at batch and data

    is available offline. 2. A model trained on ImageNet decides the category of a product and sends it to the appropriate pipeline. 3. Pipeline for various categories may vary such as, we may extract color information for garments, or estimate sizes for mobile phones etc. 4. Using all extracted features, we build a search space from our catalogue and compare it for similarity using a cosine vector similarity algorithms. Vectors are generated from the model at step 2. 5. Show results to user!
  8. 10

  9. ImageNet 11 • Image database organized according to the WordNet

    hierarchy. • Cover almost all categories of products sold online
  10. Compared to traditional recommender systems 12 • Doesn’t require tracking

    user’s every purchase to generate recommendations. However, such data can greatly refine predictions • Users can easily describe what they want… “A picture is worth a thousand words” • Can scale to hundreds of categories with additional pipelines for classification
  11. 13 Feasibility • Scalable due to moderately intensive resource needs

    • Majority of users shop using mobile phones, where clicking photos is easy