Hierarchical Product Classification @Shopify Kshetrajna Raghavan Data Science @ Shopify

Overview ● What does Product Understanding mean @Shopify ? ● Problem Formulation and Model architecture. ● Infrastructure and Scaling ● How merchants interact with ML and feedback loops ● What next?

What is Product Understanding? Build a product metadata identification service that effectively extracts useful product information for downstream applications or analytical purposes.

Walk down memory lane …

Category: Dog Bed Color: Gray Shape: Rectangular Fill Material: Memory Foam * * *

Problem Formulation: Category Prediction Taxonomy / Controlled Lists Machine Learning to categorize products

Google Product Taxonomy

Google Product Taxonomy

Model Architecture ● Multi Task - Multi Class Learning: ○ Each level of the taxonomy is a separate learning task. ○ Each task is a multi class classification problem. ● Transfer Learning: Pretrained models for both text and image features. ○ Multi lingual BERT for text ○ MobileNet V2 for Images ● Subclass Model Architecture to make different pieces of the model deployable either as a standalone model or in combination with other models together. ● Tensorflow Transform to perform stateful transformations during preprocessing.

Model Architecture: Training ● Parent nodes help child node predictions. ● 7 levels/tasks in total spanning over 5500 nodes ● Data parallelism using distributed Tensorflow across multiple machines/GPUs ● Uses Shopify’s ML platform which is built on Google Cloud Platform. ● Taxonomy unaware during training!

Inference Scaling Requirements ● Shopify has multiple billions of products historically. ● Tens of billions of images ● Tens of million of products created/updated daily. ● Multiple downstream consumers. Ex: Search, Product Sync, Admin Page, Analytics. ● Real time , Streaming and Batch applications.

Model Architecture: Inference Model Image_Text2Pred Input: Raw Text & Raw Image Output: Prediction per level Model: Image_Text2Prob Input: Raw Text & Raw Image Output: All Probabilities Model: Text2Emb Input: Raw Text Output: Text Emb Model: Image2Emb Input: Raw Image Output: Img Emb Model: Emb2Pred Input: Img, Text Emb Output:Prediction per level Model: Emb2Prob Input: Img,Text Emb Output: All Probabilities Model: Prob2Pred Input: All Prob Output: Prediction per level

Taxonomy Structure During Inference ● Raw predictions contain confidence scores for every node in the taxonomy. ● Greedy prediction logic with thresholding to enforce the taxonomy structure. Raw Predictions: Array on confidence scores/per level Choose level 1 prediction with highest score Filter to only immediate descendants of level 1 predicted category Choose level 2 descendant with highest score Continue till hitting Leaf Node or Level 7

Infrastructure/Tools

ML from Notebooks to Prod

Ray Train for Training Ray Actor Pools for Batch Inference Microservices for Online Inference

Let's continue that story from before ..

Metrics ● Model Metrics: ○ Hierarchical Precision ○ Hierarchical Recall ○ Coverage ○ Relative Lowest common ancestor ● Product Metrics: ○ Adoption ○ Merchant acceptance

What next ? ● Evolve and adapt the taxonomy ● Varying logic for probability to prediction for different downstream consumers. ● Automate threshold tuning based on merchant feedback.

Vision and Next Steps ● Develop connected taxonomies of categories and attributes. Category: Electric Guitar Attributes: Color: {Red, Green, Blue, Black, Brown} # Strings: {6,7,12} Fretboard Material: {Rosewood, Ebony , Maple} Category: Refrigerator Attributes: Color: {Black, White, Stainless Steel} Door Type: {French, Side by Side, Top Freezer} Lock Type: {Electronic, Manual} ● Expand model to infer additional attributes.

thank you