陽明交大資工系-企業參訪：資料工程團隊介紹

Event: 陽明交大資工系-企業參訪
Speaker: Ray Lu

Transcript

1. 陽明交⼤資⼯系
   企業參訪
   Ray Lu, Data Dev Team
   2023.04
   

   View Slide

2. LINE Taiwan, Data Engineer
   2011 - 2015：北科⼤電機系
   2015 - 2018：交⼤資⼯所
   2018 - 2021：資策會數轉所
   2021 - present：LINE Taiwan
   Ray Lu
   

   View Slide

3. 01
   02
   03
   Contents
   Data Dev 介紹
   我在 Data Dev
   ⼼路歷程
   

   View Slide

4. Data Dev 介紹
   • Data Dev 的任務
   • Data Dev 的成員組成
   • Data Dev 的分⼯合作
   

   View Slide

5. Data Dev 介紹
   Data Dev 的任務
   Data Dev
   LINE
   Family
   Services
   LINE
   SHOPPING
   LINE
   SPOT
   LINE
   MUSIC
   LINE
   Sticker
   LINE
   VOOM
   LINE
   Reward
   Fact
   Checker
   LINE
   HELP TW
   LINE
   Travel
   NLP
   Knowledge
   Graph
   Uplift
   Modeling
   NER
   Classifier
   Duplication
   Detector
   Auto
   completion
   Keyword
   Extraction
   Related
   Search
   Text
   Generation
   User
   Tagging
   Data
   Analytics
   Recom-
   mendation
   CLV
   LINE
   TODAY
   

   View Slide

6. Data Dev 的成員組成
   • Build and optimize d
   ata pipeline architec
   ture
   • Assemble large, com
   plex data sets that
   meet requirements
   Data Engineer Data Analyst
   Big data infra, SQL, E
   TL, message queuing
   • Interpret data, analy
   ze results using stati
   stical techniques
   • Identify, analyze, an
   d interpret trends or
   patterns in complex
   data sets
   Statistics, Data Visual
   ization, Business Kno
   wledge
   SKILL RESPONSIBILITY
   • Select appropriate d
   atasets and data rep
   resentation methods
   • Research and imple
   ment appropriate M
   L algorithms
   Data Scientist
   Machine learning, dee
   p learning, CV, NLP, S
   peech
   ML Svc Engineer
   • Build and scale mac
   hine learning infrastr
   ucture
   • Monitor model perfo
   rmance
   System infrastructure
   design, DevOps
   Data Dev 介紹
   

   View Slide

7. DS
   DE MLE
   DA
   PM Biz DS
   DE DS DS DE DA
   MLE
   Data
   preparation Scaling
   Performance
   Model decay
   Data drift
   EDA Model build
   Hyper-parameter t
   uning Evaluation
   Feature
   Engineering Error analysis
   MLE
   MLE MLE DE
   資料探索與準備 開發/訓練/測試 包裝/部署/監控
   Data Dev 的分⼯合作
   Data Dev 介紹
   

   View Slide

8. 我在 Data Dev
   • CLOVA FaceSign
   • CLOVA ChatBot
   • Speech-To-Text, STT
   • Image Search
   

   View Slide

9. 我在 Data Dev
   CLOVA FaceSign
   

   View Slide

10. 我在 Data Dev
    CLOVA Chatbot
    

    View Slide

11. 我在 Data Dev
    Speech-To-Text, STT
    

    View Slide

12. 我在 Data Dev
    Speech-To-Text, STT
    1. 排程不再凌亂一目瞭然
    2. 快速掌握排程任務狀態
    3. 方便參數管理
    

    View Slide

13. 我在 Data Dev
    Speech-To-Text, STT
    @bentoml.env(auto_pip_dependencies=True)
    class ExamplePredictionService(bentoml.BentoService):
    @bentoml.api(DataframeHandler)
    def predict(self, df):
    return self.artifacts.model.predict(df)
    

    View Slide

14. 我在 Data Dev
    

    View Slide

15. ⼼路歷程
    • 動機
    • 技能樹
    • 累積經驗
    • ⾯試
    

    View Slide

16. ⼼路歷程
    動機 – 從電機到資⼯
    ※Source from︓https://atceiling.blogspot.com/2019/08/arduino517-led_12.html
    

    View Slide

17. ⼼路歷程
    動機 – 踏入資料科學
    ※Source from︓https://www.ewant.org
    ※Source from︓https://github.com/cysmith/neural-style-tf
    

    View Slide

18. ⼼路歷程
    技能樹
    

    View Slide

19. ⼼路歷程
    技能樹
    ※Source from︓https://ithelp.ithome.com.tw/articles/10228219
    

    View Slide

20. ⼼路歷程
    Side Project
    

    View Slide

21. 實習
    ※Source from︓https://www.seinsights.asia/article/4724
    ⼼路歷程
    

    View Slide

22. 比賽
    ※Source from︓https://tbrain.trendmicro.com.tw/
    ⼼路歷程
    

    View Slide

23. THANK YOU
    

    View Slide