1001 號 到 333 號

LINE 台灣資料工程團隊的資料工程師，也是前一屆轉正成功的前實習生 Shandy Yu 帶來 「從 1001 號到 333 號」的分享

Transcript

1. 從 1001 號 到 333 號
2. Shandy •陽明交通大學 百川學士學位學程 111級 •Data Scientist @LINE Taiwan •TECH FRESH @LINE Taiwan •Consulting Service Intern @Microsoft Taiwan •Research Assistant @NYCU CIRDA •Software Engineer Intern @ITRI Yu
3. Source from: LINE Engineering Blog 跟大部分人一樣.... 我們都曾經坐在台下！
4. LINE Family Services LINE TODAY LINE SHOPPING LINE SPOT LINE INVOICE LINE STICKER LINE VOOM LINE TRAVEL Data Dev NLP Generative AI Data Analytics NER Classifier Duplication Detector Auto completion Keyword Extraction Related Search Text Generation User Tagging Uplift Modeling Recom- mendation CLV Source from: Penny LINE OA Data Dev 在做什麼 ?
5. NLPaaS —— SmartText SmartText 1.0 Automatic NLP Classifier Multi-label Classifier Topic Detection SmartText 2.0 Generative NLP Summarization Paraphrasing Question-Answering Beyond NLP Image Search Image Generation Audio Interaction
6. LINE Family Services LINE TODAY LINE SHOPPING LINE SPOT LINE INVOICE LINE STICKER LINE VOOM LINE TRAVEL Data Dev NLP Generative AI Data Analytics NER Classifier Duplication Detector Auto completion Keyword Extraction Related Search Text Generation User Tagging Uplift Modeling Recom- mendation CLV Source from: Penny LINE OA Data Dev 在做什麼 ?
7. 7 • Build and optimize data pipeline architecture • Assemble large, complex data sets that meet requirements Data Engineer Data Analyst Big data infra, SQL, ETL, message queuing • Interpret data, analyze results using statistical techniques • Identify, analyze, and interpret trends or patterns in complex data sets Statistics, Data Visualization, Business Knowledge SKILL RESPONSIBILITY Pipeline Biz • Select appropriate datasets and data representation methods • Research and implement appropriate ML algorithms Data Scientist Machine learning, deep learning, CV, NLP, Speech Model ML Engineer • Build and scale machine learning infrastructure • Monitor model performance System infrastructure design, DevOps Service 常見的組織分佈
8. As a Data Scientist @Data Dev ???
9. As a Data Scientist @Data Dev Data Scientist • Build prediction model • Study cutting-edged technologies 30% Source from: Tomaz Bratanic.(2021 Jul 20) Turn a Harry Potter Book into a Knowledge Graph https://neo4j.com/developer-blog/turn-a-harry-potter-book-into-a-knowledge-graph/
10. As a Data Scientist @Data Dev Data Analyst • Discover the data • Design BI platform 30% 30%
11. As a Data Scientist @Data Dev Data Engineer • Clean the data • Design the ETL • Design the automatic pipeline 30% 30% 20% Extract Load Transform Source DB Data Warehouse
12. As a Data Scientist @Data Dev Others • Requirements Discussion • Internal/External Speech • Training 30% 30% 20% 10%
13. VOOM Hashtag Analysis 1 6 1 6
14. VOOM Hashtag Analysis
15. Dependency Between Pipeline Post Hashtag Creator User Behavior delay AS-IS
16. Dependency Between Pipeline TO-BE IU User Behavior Performance Creator Performance Post Performance Hashtag Performance
17. Hackathon Cross-functional Team Work Time Management User Rule
18. Learning Materials Study Group Online Learning Platform Workshop
19. Life in LINE is not only work