Brickmaster: Scenario-Wise Recommender Engine @ TECHPULSE 2023

by LINE Developers Taiwan

Slide 1

Slide 1 text

Slide 2

Slide 2 text

Agenda › Introduction › What is Scenario › Issue › Goal › Brickmaster › Architecture › Flow

Slide 3

Slide 3 text

What is Scenario?

Slide 4

Slide 4 text

Scenario is a Browsing Route Main Page Product Recommendation

Slide 5

Slide 5 text

Recommendation Scenario Charts Point Search Keyword Shop

Slide 6

Slide 6 text

Issue Issue 1 Outdated Easily Item

Slide 7

Slide 7 text

Issue 1: Items become out of date Popularity Time Item still be recommended ! Introduction Growth Mature Decline

Slide 8

Slide 8 text

Issue Issue 2 Lack of Recency User Intention Issue 1 Outdated Easily Item

Slide 9

Slide 9 text

Issue 2: Users' intentions become over time

Slide 10

Slide 10 text

Issue Issue 1 Outdated Easily Item Issue 2 Lack of Recency User Intention Issue 3 Ambiguous Reasoning Business

Slide 11

Slide 11 text

Issue 3: Hard to measure potential impacts OA Web APP Search View Click Clickout Order CTR GMV Engagement Action Performance Channel ?

Slide 12

Slide 12 text

Solve 3 issues + Knowledge Sharing Goal

Slide 13

Slide 13 text

Brickmaster Two-Stage Information Retrieval

Slide 14

Slide 14 text

LINE Platform Hive + Spark IU Serving Verda Machine Learning MLU

Slide 15

Slide 15 text

Tech Stack Computing - PySpark - PyTorch   (PyTorch Lightning) - Modin + Ray Storage - HDFS - Hive - Redis Serving - FastAPI - BentoML - Dash Scheduler - Airflow ML lifecycle - MLflow

Slide 16

Slide 16 text

Architecture Candidate Generation Ranking Feature Generation User   Engagement Recommendations User   Web/App  Log User  Profile Context Item  Profile Serving Brickmaster

Slide 17

Slide 17 text

Architecture Candidate Generation Ranking Feature Generation User   Engagement Recommendations User   Web/App  Log User  Profile Context Item  Profile Serving Brickmaster Candidate Generation Ranking Feature Generation Serving

Slide 18

Slide 18 text

Flow of Brickmaster Feature  Generation Billions  of Features Thousands  of Items Hundreds   of Items Candidate  Generation Ranking Serving

Slide 19

Slide 19 text

Flow of Brickmaster 19 Feature  Generation Candidate  Generation Ranking Serving Recall Precision Billions  of Features Thousands  of Items Hundreds   of Items

Slide 20

Slide 20 text

Flow of Brickmaster Feature  Generation Candidate  Generation Ranking Serving Recall Precision Billions  of Features Thousands  of Items Hundreds   of Items

Slide 21

Slide 21 text

Recall Precision Scalability and Flexibility Feature  Generation Candidate  Generation Ranking Serving Enlarge User Population Rankers for BIZ Idea Billions  of Features Thousands  of Items Hundreds   of Items Scenario  Feature

Slide 22

Slide 22 text

Slide 23

Slide 23 text

Feature Generation User, Product, Context and Scenario User Pro fi le › Age › Area › Gender User Browsing › Click › Like › Search › UTM Product Pro fi le › Age › Gender › Text › Shop › Category Product Price › Mean › Median › IQR › Price change Date Time › Festival event › Next EC event › Holiday Scenario › Clickout › Impression › Last clickout time › Last view time

Slide 24

Slide 24 text

Slide 25

Slide 25 text

Slide 26

Slide 26 text

Slide 27

Slide 27 text

Slide 28

Slide 28 text

Slide 29

Slide 29 text

Feature Generation Product Features Product Token, Embedding NLP-based  Classifier Model-based Classifier Product  Name Product  Text  Feature Product  Gender  Feature Product  Text  Feature

Slide 30

Slide 30 text

Slide 31

Slide 31 text

Candidate Generation Feature Engineering User  Product  Feature Training Data Generation Processed  Dataset Modeling  Dataset DNN Model Training Candidate  Generator   Matcher  (LSH) Model Inference User & Product Id Mapping User  Product  Candidate User & Product Embedding Imputation   Transformation Scaling Negative Sampling

Slide 32

Slide 32 text

Slide 33

Slide 33 text

Slide 34

Slide 34 text

Slide 35

Slide 35 text

Slide 36

Slide 36 text

Ranking Feature Engineering User  Product  Candidate Training Data Generation Processed  Dataset Modeling  Dataset Scenario A  Feature DNN Model Training Ranker A Model Inference Personalized  Ranking A Imputation   Transformation Scaling Negative Sampling

Slide 37

Slide 37 text

Slide 38

Slide 38 text

Slide 39

Slide 39 text

Serving Personalized  Ranking A Personalized  Ranking Ｂ Redis Production  API Web/APP Demo Site Experimental  API

Slide 40

Slide 40 text

Serving Personalized  Ranking A Personalized  Ranking Ｂ Redis Production  API Web/APP Demo Site Experimental  API

Slide 41

Slide 41 text

Health Checking Original DataSet Transformed  Dataset Check DAG Success Check Data Splitting Training Data Generation Feature Engineering Imputation Outlier Detection Noti fi cation

Slide 42

Slide 42 text

Health Checking Original DataSet Transformed  Dataset Check DAG Success Check Data Splitting Training Data Generation Feature Engineering Imputation Outlier Detection Noti fi cation Pipeline  Level

Slide 43

Slide 43 text

Health Checking Original DataSet Transformed  Dataset Check DAG Success Check Data Splitting Training Data Generation Feature Engineering Imputation Outlier Detection Noti fi cation Data  Level

Slide 44

Slide 44 text

Ranker Type User Intention

Slide 45

Slide 45 text

Personalized Recommendation › Match business goal (e.g. clickout, GMV) › We use proximal concept to solve business problems Business Goal Science Clickout User  Engagement Min(Next Clickout Time) Max(Session Duration)

Slide 46

Slide 46 text

Slide 47

Slide 47 text

Ranker Type Inspiration

Slide 48

Slide 48 text

Trending Inspire user to   fi nd out potential needs Popularity Introduction Growth Mature Decline

Slide 49

Slide 49 text

Recap Design of Brickmaster Business Target Stage User Preference KPI -Oriented Objective 1st stage - Candidate Generation 2nd stage - Scenario Rankers

Slide 50

Slide 50 text

Recap Design of Brickmaster Business Target Stage User Preference KPI -Oriented Objective 1st stage - Candidate Generation 2nd stage - Scenario Rankers

Slide 51

Slide 51 text

Thank you