ML Life Cycle for LINE SHOPPING Recommender

Event: iThome Hello World Dev Conference
Speaker: Rei Huang

Transcript

2. ML Life Cycle for LINE Shopping Recommender
3. 01 02 03 04 Transform as ML Problem Modeling Execute A/B test Problem Definition 05 Evaluation CONTENT 06 Decision
4. Rei Huang Machine Learning Engineer Education • Bachelor in Mathematics @ CCU • Master in Mathematics @ NTHU Work Experience • E.Sun Commercial Bank (2016 ~ 2022) • LINE Taiwan EC (2022 ~ Now)
5. Sketch of ML Life Cycle Business Issue Problem Definition ML problem ML model building A/B test Evaluation Decision
6. Identify the core issue Focus on the primary problem 01 Problem Definition
7. 鎖定問題，對症下藥！ Problem Definition 設計產品 折扣券 提升用戶 體驗
8. Analyze by the funnel chart Problem Definition Pageview Click Transfer page Orders Importance Sparsity
9. Describe problem by ML statement 02 Transform our Problem as a ML Problem
10. Transform our Problem as a ML Problem Precision@10 電話行銷 簡訊通知 email Precision@10k Recall@500k
11. Transform our Problem as a ML Problem Which one do you want to click? ( ) ， ？future days Regession or Classifiction?
12. End-to-end model building 03 Modeling
13. Feature Engineering Model Data Modeling Model Training Model Inference Offline Evaluation Modeling Cycle
14. Design an effective experiment 04 Execute A/B testing
15. Execute A/B test Save to Database Request Recommendation experimental control 核心概念 1. 均勻分布 2. 執行同一個實驗，user 的組別是固定的 3. 同一個實驗，若增加實驗比率，實驗組不變 4. 高計算效率 experiment_id ratio Ref: A/B Test Bucketing using Hashing Sketch of Design Version Control
16. Execute A/B test Ref: A/B Test Bucketing using Hashing Core of hash function 每次進入網站時分組為 A or B 同一實驗調整 ratio 但組別重分配 同一 user 對到不同 model logic 均勻分布 全母體均勻 or acitve user 均勻 A control B control A control B control Session based User based 高計算效率
17. Confirm the performance 05 Evaluation
18. Evaluation Online experimental control Within a certain time Extract data CTR Precision MRR CVR ML metric business metric …
19. Evaluation Others Feature correlation Ref: https://byjus.com/maths/inversely-proportional/ Monitoring
20. What’s the next? 06 Decision
21. What’s wrong with us? Orders Clicks Model Performance Model Tuning Bad Good Does model solve click? 這時候才調整嗎？ Monitoring
22. ML Life Cycle Business Issue Problem Definition ML problem ML model building A/B test Evaluation Decision
23. Ideal Corporation Mode Conclusion 項目標題文字 Engineering Product Owner