Brickmaster: Scenario-Wise Recommender Engine @ TECHPULSE 2023

- Speaker: Vila Lin
- Event: http://techpulse.line.me/

Brickmaster 是 LINE 購物於 2022 年推出的情境感知深度學習推薦系統。本系統將解決傳統推薦系統常遇到的三個問題：推薦商品過時、無法跟上使用者喜好以及與組織 KPI 脫鉤。系統架構也提供可再用性與知識共享，將大幅減少所需開發人力與時間。

Transcript

2. Agenda › Introduction › What is Scenario › Issue › Goal › Brickmaster › Architecture › Flow
3. What is Scenario?
4. Scenario is a Browsing Route Main Page Product Recommendation
5. Recommendation Scenario Charts Point Search Keyword Shop
6. Issue Issue 1 Outdated Easily Item
7. Issue 1: Items become out of date Popularity Time Item still be recommended ! Introduction Growth Mature Decline
8. Issue Issue 2 Lack of Recency User Intention Issue 1 Outdated Easily Item
9. Issue 2: Users' intentions become over time
10. Issue Issue 1 Outdated Easily Item Issue 2 Lack of Recency User Intention Issue 3 Ambiguous Reasoning Business
11. Issue 3: Hard to measure potential impacts OA Web APP Search View Click Clickout Order CTR GMV Engagement Action Performance Channel ?
12. Solve 3 issues + Knowledge Sharing Goal
13. Brickmaster Two-Stage Information Retrieval
14. LINE Platform Hive + Spark IU Serving Verda Machine Learning MLU
15. Tech Stack Computing - PySpark - PyTorch 
 (PyTorch Lightning) - Modin + Ray Storage - HDFS - Hive - Redis Serving - FastAPI - BentoML - Dash Scheduler - Airflow ML lifecycle - MLflow
16. Architecture Candidate Generation Ranking Feature Generation User 
 Engagement Recommendations User 
 Web/App
 Log User
 Profile Context Item
 Profile Serving Brickmaster
17. Architecture Candidate Generation Ranking Feature Generation User 
 Engagement Recommendations User 
 Web/App
 Log User
 Profile Context Item
 Profile Serving Brickmaster Candidate Generation Ranking Feature Generation Serving
18. Flow of Brickmaster Feature
 Generation Billions
 of Features Thousands
 of Items Hundreds 
 of Items Candidate
 Generation Ranking Serving
19. Flow of Brickmaster 19 Feature
 Generation Candidate
 Generation Ranking Serving Recall Precision Billions
 of Features Thousands
 of Items Hundreds 
 of Items
20. Flow of Brickmaster Feature
 Generation Candidate
 Generation Ranking Serving Recall Precision Billions
 of Features Thousands
 of Items Hundreds 
 of Items
21. Recall Precision Scalability and Flexibility Feature
 Generation Candidate
 Generation Ranking Serving Enlarge User Population Rankers for BIZ Idea Billions
 of Features Thousands
 of Items Hundreds 
 of Items Scenario
 Feature
22. Recall Precision Scalability and Flexibility Feature
 Generation Candidate
 Generation Ranking Serving Enlarge User Population Rankers for BIZ Idea Billions
 of Features Thousands
 of Items Hundreds 
 of Items Scenario
 Feature
23. Feature Generation User, Product, Context and Scenario User Pro fi le › Age › Area › Gender User Browsing › Click › Like › Search › UTM Product Pro fi le › Age › Gender › Text › Shop › Category Product Price › Mean › Median › IQR › Price change Date Time › Festival event › Next EC event › Holiday Scenario › Clickout › Impression › Last clickout time › Last view time
24. Feature Generation User, Product, Context and Scenario User Pro fi le › Age › Area › Gender User Browsing › Click › Like › Search › UTM Product Pro fi le › Age › Gender › Text › Shop › Category Product Price › Mean › Median › IQR › Price change Date Time › Festival event › Next EC event › Holiday Scenario › Clickout › Impression › Last clickout time › Last view time Product Price › Mean › Median › IQR › Price change
25. Feature Generation User, Product, Context and Scenario User Pro fi le › Age › Area › Gender User Browsing › Click › Like › Search › UTM Product Pro fi le › Age › Gender › Text › Shop › Category Product Price › Mean › Median › IQR › Price change Date Time › Festival event › Next EC event › Holiday Scenario › Clickout › Impression › Last clickout time › Last view time Product Price › Mean › Median › IQR › Price change
26. Feature Generation User, Product, Context and Scenario User Pro fi le › Age › Area › Gender User Browsing › Click › Like › Search › UTM Product Pro fi le › Age › Gender › Text › Shop › Category Product Price › Mean › Median › IQR › Price change Date Time › Festival event › Next EC event › Holiday Scenario › Clickout › Impression › Last clickout time › Last view time Product Price › Mean › Median › IQR › Price change
27. Feature Generation User, Product, Context and Scenario User Pro fi le › Age › Area › Gender User Browsing › Click › Like › Search › UTM Product Pro fi le › Age › Gender › Text › Shop › Category Product Price › Mean › Median › IQR › Price change Date Time › Festival event › Next EC event › Holiday Scenario › Clickout › Impression › Last clickout time › Last view time Product Price › Mean › Median › IQR › Price change
28. Feature Generation User, Product, Context and Scenario User Pro fi le › Age › Area › Gender User Browsing › Click › Like › Search › UTM Product Pro fi le › Age › Gender › Text › Shop › Category Product Price › Mean › Median › IQR › Price change Date Time › Festival event › Next EC event › Holiday Scenario › Clickout › Impression › Last clickout time › Last view time Product Price › Mean › Median › IQR › Price change
29. Feature Generation Product Features Product Token, Embedding NLP-based
 Classifier Model-based Classifier Product
 Name Product
 Text
 Feature Product
 Gender
 Feature Product
 Text
 Feature
30. Feature Generation Product Features Product Token, Embedding NLP-based
 Classifier Model-based Classifier Product
 Name Product
 Text
 Feature Product
 Gender
 Feature Product
 Text
 Feature
31. Candidate Generation Feature Engineering User
 Product
 Feature Training Data Generation Processed
 Dataset Modeling
 Dataset DNN Model Training Candidate
 Generator 
 Matcher
 (LSH) Model Inference User & Product Id Mapping User
 Product
 Candidate User & Product Embedding Imputation 
 Transformation Scaling Negative Sampling
32. Candidate Generation Feature Engineering User
 Product
 Feature Training Data Generation Processed
 Dataset Modeling
 Dataset DNN Model Training Candidate
 Generator 
 Matcher
 (LSH) Model Inference User & Product Id Mapping User
 Product
 Candidate User & Product Embedding Imputation 
 Transformation Scaling Negative Sampling
33. Candidate Generation Feature Engineering User
 Product
 Feature Training Data Generation Processed
 Dataset Modeling
 Dataset DNN Model Training Candidate
 Generator 
 Matcher
 (LSH) Model Inference User & Product Id Mapping User
 Product
 Candidate User & Product Embedding Imputation 
 Transformation Scaling Negative Sampling
34. Candidate Generation Feature Engineering User
 Product
 Feature Training Data Generation Processed
 Dataset Modeling
 Dataset DNN Model Training Candidate
 Generator 
 Matcher
 (LSH) Model Inference User & Product Id Mapping User
 Product
 Candidate User & Product Embedding Imputation 
 Transformation Scaling Negative Sampling
35. Candidate Generation Feature Engineering User
 Product
 Feature Training Data Generation Processed
 Dataset Modeling
 Dataset DNN Model Training Candidate
 Generator 
 Matcher
 (LSH) Model Inference User & Product Id Mapping User
 Product
 Candidate User & Product Embedding Imputation 
 Transformation Scaling Negative Sampling
36. Ranking Feature Engineering User
 Product
 Candidate Training Data Generation Processed
 Dataset Modeling
 Dataset Scenario A
 Feature DNN Model Training Ranker A Model Inference Personalized
 Ranking A Imputation 
 Transformation Scaling Negative Sampling
37. Ranking Feature Engineering User
 Product
 Candidate Training Data Generation Processed
 Dataset Modeling
 Dataset Scenario A
 Feature DNN Model Training Ranker A Model Inference Personalized
 Ranking A Imputation 
 Transformation Scaling Negative Sampling
38. Ranking Feature Engineering User
 Product
 Candidate Training Data Generation Processed
 Dataset Modeling
 Dataset Scenario A
 Feature DNN Model Training Ranker A Model Inference Personalized
 Ranking A Imputation 
 Transformation Scaling Negative Sampling
39. Serving Personalized
 Ranking A Personalized
 Ranking Ｂ Redis Production
 API Web/APP Demo Site Experimental
 API
40. Serving Personalized
 Ranking A Personalized
 Ranking Ｂ Redis Production
 API Web/APP Demo Site Experimental
 API
41. Health Checking Original DataSet Transformed
 Dataset Check DAG Success Check Data Splitting Training Data Generation Feature Engineering Imputation Outlier Detection Noti fi cation
42. Health Checking Original DataSet Transformed
 Dataset Check DAG Success Check Data Splitting Training Data Generation Feature Engineering Imputation Outlier Detection Noti fi cation Pipeline
 Level
43. Health Checking Original DataSet Transformed
 Dataset Check DAG Success Check Data Splitting Training Data Generation Feature Engineering Imputation Outlier Detection Noti fi cation Data
 Level
44. Ranker Type User Intention
45. Personalized Recommendation › Match business goal (e.g. clickout, GMV) › We use proximal concept to solve business problems Business Goal Science Clickout User
 Engagement Min(Next Clickout Time) Max(Session Duration)
46. Personalized Recommendation › Match business goal (e.g. clickout, GMV) › We use proximal concept to solve business problems Business Goal Science Clickout User
 Engagement Min(Next Clickout Time) Max(Session Duration)
47. Ranker Type Inspiration
48. Trending Inspire user to 
 fi nd out potential needs Popularity Introduction Growth Mature Decline
49. Recap Design of Brickmaster Business Target Stage User Preference KPI -Oriented Objective 1st stage - Candidate Generation 2nd stage - Scenario Rankers
50. Recap Design of Brickmaster Business Target Stage User Preference KPI -Oriented Objective 1st stage - Candidate Generation 2nd stage - Scenario Rankers
51. Thank you