Upgrade to Pro — share decks privately, control downloads, hide ads and more …

End-To-End MLOps Platform at LINE

End-To-End MLOps Platform at LINE

Sun Hyeong Hong (LINE Plus / MLU Core TF / Platform Product Manager)

https://tech-verse.me/ja/sessions/49
https://tech-verse.me/en/sessions/49
https://tech-verse.me/ko/sessions/49

Tech-Verse2022

November 17, 2022
Tweet

More Decks by Tech-Verse2022

Other Decks in Technology

Transcript

  1. Sunhyeong hong (홍선형)
    What to eat
    Today
    MLU
    Beginner
    planner
    Golf
    Learning English
    Travel
    Platform Product
    Manager
    What to
    drink
    Today

    View full-size slide

  2. Intro
    Jason Allen's art created on Midjourney titled “Space Opera Theater”

    View full-size slide

  3. Intro
    Rain
    Go Out Stay In
    Too Hot Cable Signal?
    Shopping Movie Coffee Shop TV Shows
    Mode to predict Hong’s Weekend Plan
    No Yes

    View full-size slide

  4. Intro
    Is it enough? ML-Model
    development

    View full-size slide

  5. Intro
    Data sourcing
    Streaming
    Sourcing
    Ingestion
    Data
    management
    Validation
    Analysis
    Segmentation
    Feature
    engineering
    ML-Model
    development
    Building
    Training
    Evaluation
    Versioning
    metadata
    Application
    Model
    deployment
    Model monitoring
    Serving API/UI
    Load balancing
    Infrastructure
    Configuration Containerization
    Logging
    monitoring
    CI/CD
    Pipelines
    Authentication Hardware/GPUs
    No. you have to do all this

    View full-size slide

  6. Intro
    These are the skills you need to do ML
    Business
    Programming Statices, ML/AI
    Communication
    The business
    guy
    The “if”
    guy
    The hacker
    The number
    cruncher
    The data
    translator
    The ML/AI
    engineer
    The perfect
    Data
    scientist
    The
    salesperson
    The Data
    nerd
    The
    statistician The
    Storyteller
    The good
    consultant
    The competence
    Science
    professor

    View full-size slide

  7. Intro
    These are the skills you need to do ML
    Business
    Programming Statices, ML/AI
    Communication
    Are you okay...?

    View full-size slide

  8. Agenda
    - What is Machine Learning Pipeline?
    - How to Build an End-to-End a Machine
    Learning Pipeline?
    - Building an End-to-End Pipeline using MLU
    - Next MLU?

    View full-size slide

  9. What is Machine Learning Pipeline?

    View full-size slide

  10. How to Build an End-to-End a Machine
    Learning Pipeline?
    Data Extraction
    and Analysis
    Data
    Preparation
    Model Training
    Model Evaluation
    and Validation
    Trained
    Model
    Manual experiment steps
    Offline data
    Manual ML Pipeline

    View full-size slide

  11. How to Build an End-to-End a Machine
    Learning Pipeline?
    Data Extraction
    and Analysis
    Data
    Preparation
    Model Training
    Model Evaluation
    and Validation
    Model Serving
    Trained
    Model
    Manual experiment steps
    Offline data
    Model registry
    Ops
    Manual ML Pipeline
    ML

    View full-size slide

  12. How to Build an End-to-End a Machine
    Learning Pipeline?
    It's a new model
    ML Ops

    View full-size slide

  13. How to Build an End-to-End a Machine
    Learning Pipeline?
    It's a new model
    I checked, but It doesn’t work
    ML Ops

    View full-size slide

  14. How to Build an End-to-End a Machine
    Learning Pipeline?
    It's a new model
    What? I have no problem
    I checked, but It doesn’t work
    ML Ops

    View full-size slide

  15. How to Build an End-to-End a Machine
    Learning Pipeline?
    It's a new model
    What? I have no problem
    I checked, but It doesn’t work
    what python do you use?
    ML Ops

    View full-size slide

  16. How to Build an End-to-End a Machine
    Learning Pipeline?
    It's a new model
    What? I have no problem
    Ah, My Python version
    is not compatible
    , so I uploaded it to 3.8.
    I checked, but It doesn’t work
    what python do you use?
    ML Ops

    View full-size slide

  17. How to Build an End-to-End a Machine
    Learning Pipeline?
    It's a new model
    What? I have no problem
    Ah, My Python version
    is not compatible
    , so I uploaded it to 3.8.
    I checked, but It doesn’t work
    what python do you use?
    Oh…
    Please send requirements.txt
    ML Ops

    View full-size slide

  18. How to Build an End-to-End a Machine
    Learning Pipeline?
    Automation ML pipeline
    Data
    Extraction
    Data
    Preparation
    Model
    Training
    Model
    Evaluation
    Model
    Serving
    Trained
    Model
    Code Repository
    Model registry
    Model
    Monitoring
    Data
    Validation
    Model
    Validation
    Feature
    Store
    Data Analysis
    And Experimentation
    Now
    Code
    Model registry

    View full-size slide

  19. How to Build an End-to-End a Machine
    Learning Pipeline?
    Automation ML pipeline
    Data
    Extraction
    Data
    Preparation
    Model
    Training
    Model
    Evaluation
    Model
    Serving
    Trained
    Model
    Automated pipeline
    Code Repository
    Model registry
    Model
    Monitoring
    Data
    Validation
    Model
    Validation
    Feature
    Store
    Data Analysis
    And Experimentation
    Now
    Code
    Model registry

    View full-size slide

  20. How to Build an End-to-End a Machine
    Learning Pipeline?
    Automation ML pipeline
    Data
    Extraction
    Data
    Preparation
    Model
    Training
    Model
    Evaluation
    Model
    Serving
    Trained
    Model
    Automated pipeline
    Code Repository
    Model registry
    Model
    Monitoring
    Data
    Validation
    Model
    Validation
    Feature
    Store
    Data Analysis
    And Experimentation
    Now
    Code
    Model registry

    View full-size slide

  21. LINE's MLOps platform
    Machine Learning Universe

    View full-size slide

  22. Building an End-to-End Pipeline using MLU
    VOOM For you recommendation system

    View full-size slide

  23. Machine Learning Model
    Building an End-to-End Pipeline using MLU
    VOOM For you recommendation system
    VOOM public Post
    Create a
    recommended
    candidate group
    Recommendation
    candidate data
    filtering and quality
    verification
    Recommendation
    model Training

    View full-size slide

  24. Machine Learning Model
    Building an End-to-End Pipeline using MLU
    VOOM For you recommendation system
    Create a
    recommended
    candidate group
    Recommendation
    candidate data
    filtering and quality
    verification
    Recommendation
    model Training
    Recommendation
    model deployment
    VOOM public Post
    Recommendation
    model monitoring

    View full-size slide

  25. Machine Learning Model
    Building an End-to-End Pipeline using MLU
    VOOM For you recommendation system
    Create a
    recommended
    candidate group
    Recommendation
    candidate data
    filtering and quality
    verification
    Recommendation
    model Training
    Recommendation
    model deployment
    VOOM public Post
    User Feedback
    Recommendation
    model monitoring

    View full-size slide

  26. Continuous Data and Model updates

    View full-size slide

  27. ML Model Logging and Tracking

    View full-size slide

  28. ML Model API Packaging

    View full-size slide

  29. Server resource optimization

    View full-size slide

  30. Different development environment

    View full-size slide

  31. Building an End-to-End Pipeline using MLU
    The VOOM recommendation model process within the MLU architecture
    Data
    preprocessing
    Model training
    Feature
    Store
    Data
    Interactive Computing Mountable Filesystem
    Model
    validation

    View full-size slide

  32. Building an End-to-End Pipeline using MLU
    The VOOM recommendation model process within the MLU architecture
    Data
    preprocessing
    Model training
    Feature
    Store
    Data
    Interactive Computing
    Experiment Management
    Mountable Filesystem
    Model
    validation

    View full-size slide

  33. Building an End-to-End Pipeline using MLU
    The VOOM recommendation model process within the MLU architecture
    Data
    preprocessing
    Model training
    Feature
    Store
    Data
    Interactive Computing
    Experiment Management
    Mountable Filesystem
    Model
    validation
    Model
    packaging
    Model
    deployment
    Packaging Management

    View full-size slide

  34. Building an End-to-End Pipeline using MLU
    The VOOM recommendation model process within the MLU architecture
    Data
    preprocessing
    Model training
    Feature
    Store
    Data
    Interactive Computing
    Experiment Management
    Mountable Filesystem
    Model
    validation
    Model
    packaging
    Model
    deployment
    Workflow Management
    Packaging Management

    View full-size slide

  35. Building an End-to-End Pipeline using MLU
    The VOOM recommendation model process within the MLU architecture
    Data
    preprocessing
    Model training
    Feature
    Store
    Data
    Interactive Computing
    Experiment Management
    Mountable Filesystem
    Model
    validation
    Model
    packaging
    Model
    deployment
    Workflow Management
    Packaging Management
    Model Serving
    Model
    Monitoring
    Serving Management

    View full-size slide

  36. Building an End-to-End Pipeline using MLU
    The dashboard of the MLU portal

    View full-size slide

  37. Building an End-to-End Pipeline using MLU
    Monitoring screen of MLU serving

    View full-size slide

  38. Building an End-to-End Pipeline using MLU
    Pipeline operation accidents
    90%↓
    100%↓
    ! 95%↓
    !

    View full-size slide

  39. Building an End-to-End Pipeline using MLU
    Improve model performance CTR 25% → 38% (13%↑)
    Pipeline operation accidents
    90%↓
    100%↓
    ! 95%↓
    !

    View full-size slide

  40. Building an End-to-End Pipeline using MLU
    Provisioning

    View full-size slide

  41. Provisioning ML Ecosystem Minimal Code
    Building an End-to-End Pipeline using MLU

    View full-size slide

  42. Building an End-to-End Pipeline using MLU
    Provisioning ML Ecosystem GUI Platform

    View full-size slide

  43. Building an End-to-End Pipeline using MLU
    MLU Usage Status - As of October ‘22
    Cluster
    GPU: 504 Cores
    CPU: 14,886 Cores
    MEM: 52TB

    View full-size slide

  44. Building an End-to-End Pipeline using MLU
    MLU Usage Status - As of October ‘22
    Cluster
    GPU: 504 Cores
    CPU: 14,886 Cores
    MEM: 52TB
    User
    Total Count: 1,111
    Company: 33
    Team: 386

    View full-size slide

  45. Building an End-to-End Pipeline using MLU
    MLU Usage Status - As of October ‘22
    Cluster
    GPU: 504 Cores
    CPU: 14,886 Cores
    MEM: 52TB
    User
    Total Count: 1,111
    Company: 33
    Team: 386
    Pipeline
    Pipeline : 300
    Serving : 107

    View full-size slide

  46. - Efficient Data management tools
    - An active community where everyone can share their knowledge
    - A Public repository for sharing Models and Datasets
    Next MLU

    View full-size slide

  47. - Efficient Data management tools
    - An active community where everyone can share their knowledge
    - A Public repository for sharing Models and Datasets
    Next MLU

    View full-size slide

  48. Released just yesterday
    MLU MARKET PLACE

    View full-size slide

  49. ML Engineer
    MLOps

    View full-size slide

  50. ML Engineer
    Marketer
    Planner
    Service Engineer
    Designer
    Server Engineer
    MLU

    View full-size slide