關於我和Data Dev 變成家人的那件事

Event: 政治大學企業參訪
Speaker: Maggie Lee

Transcript

1. ᮫ԙզ࿨%BUB
   %FW Ꮣ੒Ոਓతಹ݅ࣄ Data Dev/ Maggie Lee 2023.04.21
2. Maggie Lee Machine Learning Engineer @LINE Data Dev 政⼤中⽂學⼠ 雙政⼤資科 Research Assistant @CKIP Lab LINE TECH FRESH @LINE Data Dev Software Graphics intern @Intel CCG 政⼤資科所碩⼠ @LKT lab 害怕⼀鏡到底的⼈⽣ 討厭社會對中⽂系的偏⾒ 連滾帶爬 跌跌撞撞 誤打誤撞 被鄰座強制開啟新世界 跨出舒適圈探索未知 成⻑總是有失有得 沒有極限的研究⽣活 跨領域研究的酸甜苦辣 每天都是⼀場新挑戰 視野⼤展開 努⼒點滿MLE技能 Work Life balance才是終極⽬標
3. Data Dev LINE Family Services LINE SHOPPING LINE SPOT LINE MUSIC LINE Sticker LINE VOOM LINE Reward Fact Checker LINE HELP TW LINE Travel NLP CV MarTech NER Classifier Duplica-on Detector Auto comple-on Keyword Extraction Related Search Text Genera-on User Tagging Data Analy,cs Recom- mendation CLV/ RFM LINE TODAY 3 Data Dev的任務有哪些 Image Search Social Intelligence UpliB Modeling Sales forecasting STT
4. ML/DL Business Knowledge Data Scientist Data Engineer Data Visualization Statistics Data Dev成員組成 Data Dev Machine Learning Engineer Product Engineer
5. Big Data infra Backend Data Scientist Data Engineer SQL ETL Data Dev成員組成 Data Dev Machine Learning Engineer Product Engineer
6. MLOPs backend Data Scientist Data Engineer ML/DL ETL Data Dev成員組成 Data Dev Machine Learning Engineer Product Engineer
7. DS DE MLE PM Biz DS DE DS DS DE DS MLE Data prepara1on Scaling Performance Model decay Data dri= EDA Model build Hyper-parameter tuning Evalua1on Feature Engineering Error analysis 一個ML專案會經過… MLE MLE MLE DE DS
8. MLE的使命是什麼 DEV: Develop/Testing/Deployment OPS: CI/CD/CT
9. 9 如何實踐MLOPs experiments Source code Pipeline CI Pipeline CD Automated Pipeline Trained model Model CD Con9nuous Training Monitoring Prediction service Develop produc9on Source code pipeline model package
10. 10 Data preparation Model training Model Evalua2on Model Valida2on Model Analysis Experiment
11. 11 如何實踐MLOPs experiments Source code Pipeline CI Pipeline CD Automated Pipeline Trained model Model CD Con9nuous Training Monitoring Predic9on service Develop produc9on Source code pipeline model package
12. 12 experiments Source code Pipeline CI Pipeline CD Automated Pipeline Trained model Model CD Monitoring Predic9on service 如何實踐MLOPs Con9nuous Training Develop production Source code pipeline model package
13. 13 experiments Source code Pipeline CI Pipeline CD Automated Pipeline Trained model Model CD Monitoring Predic9on service 如何實踐MLOPs Continuous Training Develop produc9on Source code pipeline model package
14. 14 experiments Source code Pipeline CI Pipeline CD Automated Pipeline Trained model Model CD Monitoring Prediction service 如何實踐MLOPs Con9nuous Training Develop produc9on Source code pipeline model package
15. 15 Packaging Containerizing Deployment Model Service package管理 Bundle file Docker Image Model file pipeline Object Storage harbor Docker build/push Meta data
16. 16 experiments Source code Pipeline CI Pipeline CD Automated Pipeline Trained model Model CD Monitoring Prediction service 如何實踐MLOPs Con9nuous Training Develop produc9on
17. • 建立儀表板管理資料管道狀態 à 確認ETL是否成功 • 紀錄table的來源及schema à 加速debug的效率 • 針對特定重要指標進行監測(例如：UU/AU) à 確認資料源頭是否異常 Data Observation 一旦出現異常，可即時發送警告信或訊息 除了系統health check 我們還可以做什麼
18. 18 Case – SmartText portal
19. 19 如果每次都要自己手動重新訓練好麻煩 有沒有辦法直接拿到現成的模型？ 有了良好的模型 要怎麼讓使用者知道並使用呢？ With ML background 你們的模型好酷！ 但我對機器學習沒有概念 要怎麼應用呢... 可以簡單到用手指頭點一點就好了？ With no ML background
20. 20 如果我們可以做到…
21. 21 團隊如何分配任務 任務量 當前⼯作量 任務適配度 OKR 任務緊急程度 當前任務優先序
22. 22 團隊如何分配任務 Objective Key Result 與團隊⽬標有⼀致性/具備影響⼒/有時間限制 Object Improve service stability Key Result 1. Build data observation dashboard for 2+ service in Q3 2. Design outage or service accident info process Example
23. 23 團隊如何分配任務 任務緊急程度 當前任務優先序 任務適配度 OKR 任務量 當前⼯作量
24. 24 團隊如何分配任務 任務統計 完成 進行中 待辦事項 5 4 2 優先序⾼ 優先序低 high high low low medium medium medium medium medium low
25. 25 團隊如何分配任務 任務量 當前⼯作量 任務適配度 OKR 任務緊急程度 當前任務優先序
26. 26 團隊如何分配任務 Leo 1 4 1 Nina 3 6 1 Maggie 5 4 2
27. 2"
    5JNF