陽明交大資工系-企業參訪：資料工程團隊介紹

by LINE Developers Taiwan

Slide 1

Slide 1 text

陽明交⼤資⼯系企業參訪 Ray Lu, Data Dev Team 2023.04

Slide 2

Slide 2 text

LINE Taiwan, Data Engineer 2011 - 2015：北科⼤電機系 2015 - 2018：交⼤資⼯所 2018 - 2021：資策會數轉所 2021 - present：LINE Taiwan Ray Lu

Slide 3

Slide 3 text

01 02 03 Contents Data Dev 介紹我在 Data Dev ⼼路歷程

Slide 4

Slide 4 text

Data Dev 介紹 • Data Dev 的任務 • Data Dev 的成員組成 • Data Dev 的分⼯合作

Slide 5

Slide 5 text

Data Dev 介紹 Data Dev 的任務 Data Dev LINE Family Services LINE SHOPPING LINE SPOT LINE MUSIC LINE Sticker LINE VOOM LINE Reward Fact Checker LINE HELP TW LINE Travel NLP Knowledge Graph Uplift Modeling NER Classifier Duplication Detector Auto completion Keyword Extraction Related Search Text Generation User Tagging Data Analytics Recom- mendation CLV LINE TODAY

Slide 6

Slide 6 text

Data Dev 的成員組成 • Build and optimize d ata pipeline architec ture • Assemble large, com plex data sets that meet requirements Data Engineer Data Analyst Big data infra, SQL, E TL, message queuing • Interpret data, analy ze results using stati stical techniques • Identify, analyze, an d interpret trends or patterns in complex data sets Statistics, Data Visual ization, Business Kno wledge SKILL RESPONSIBILITY • Select appropriate d atasets and data rep resentation methods • Research and imple ment appropriate M L algorithms Data Scientist Machine learning, dee p learning, CV, NLP, S peech ML Svc Engineer • Build and scale mac hine learning infrastr ucture • Monitor model perfo rmance System infrastructure design, DevOps Data Dev 介紹

Slide 7

Slide 7 text

DS DE MLE DA PM Biz DS DE DS DS DE DA MLE Data preparation Scaling Performance Model decay Data drift EDA Model build Hyper-parameter t uning Evaluation Feature Engineering Error analysis MLE MLE MLE DE 資料探索與準備開發/訓練/測試包裝/部署/監控 Data Dev 的分⼯合作 Data Dev 介紹

Slide 8

Slide 8 text

我在 Data Dev • CLOVA FaceSign • CLOVA ChatBot • Speech-To-Text, STT • Image Search

Slide 9

Slide 9 text

我在 Data Dev CLOVA FaceSign

Slide 10

Slide 10 text

我在 Data Dev CLOVA Chatbot

Slide 11

Slide 11 text

我在 Data Dev Speech-To-Text, STT

Slide 12

Slide 12 text

我在 Data Dev Speech-To-Text, STT 1. 排程不再凌亂一目瞭然 2. 快速掌握排程任務狀態 3. 方便參數管理

Slide 13

Slide 13 text

我在 Data Dev Speech-To-Text, STT @bentoml.env(auto_pip_dependencies=True) class ExamplePredictionService(bentoml.BentoService): @bentoml.api(DataframeHandler) def predict(self, df): return self.artifacts.model.predict(df)

Slide 14

Slide 14 text

我在 Data Dev

Slide 15

Slide 15 text

⼼路歷程 • 動機 • 技能樹 • 累積經驗 • ⾯試

Slide 16

Slide 16 text

⼼路歷程動機 – 從電機到資⼯ ※Source from︓https://atceiling.blogspot.com/2019/08/arduino517-led_12.html

Slide 17

Slide 17 text

⼼路歷程動機 – 踏入資料科學 ※Source from︓https://www.ewant.org ※Source from︓https://github.com/cysmith/neural-style-tf

Slide 18

Slide 18 text

⼼路歷程技能樹

Slide 19

Slide 19 text

⼼路歷程技能樹 ※Source from︓https://ithelp.ithome.com.tw/articles/10228219