Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Mise en place d'un CI/CD pour vos modèles de machine learning

Mise en place d'un CI/CD pour vos modèles de machine learning

La mise en production de models de Machine learning réponds aux mêmes contraintes que la mise en production d'applications classique. Dans cette sessions venez découvrir comment vos data scientists pourront mettre en production leur algorithms de machine learning avec des outils comme code comme code build, step functions et sagemaker.

Transcript

  1. © 2019, Amazon Web Services, Inc. or its Affiliates. Marc

    Cabocel Startup Solutions Architect November 28th, 2019 Industrialisation du Machine Learning Mise en place d’une CI/CD pour vos modèles de machine learning
  2. © 2019, Amazon Web Services, Inc. or its Affiliates. Customer

    Goals Business Outcomes I N N O V A T I O N M O D E R N I Z A T I O N & M I G R A T I O N Successful enterprises follow a common path F O U N D A T I O N
  3. © 2019, Amazon Web Services, Inc. or its Affiliates. Challenges

  4. © 2019, Amazon Web Services, Inc. or its Affiliates. Definition

    of DevOps Strong focus on automation and monitoring at all steps of software construction. Software Operations Software Development
  5. © 2019, Amazon Web Services, Inc. or its Affiliates. Definition

    of MLOps Strong focus on automation and monitoring at all steps of ML System construction. ML System Operations ML System Development
  6. © 2019, Amazon Web Services, Inc. or its Affiliates. Specific

    problem areas Development Deployment Operation ML System Construction Process Common anti-patterns in …
  7. © 2019, Amazon Web Services, Inc. or its Affiliates. Specific

    problem areas Development Deployment Operation ML System Construction Process Common anti-patterns in …
  8. © 2019, Amazon Web Services, Inc. or its Affiliates. Development

    Anti Pattern: Superhero-Dependence ML Researcher ML Full Stack Data Engineer DevOps Engineer Infrastructure Architect Product Manager Exec Partner POC Production
  9. © 2019, Amazon Web Services, Inc. or its Affiliates. Development

    Anti Pattern: Superhero-Dependence ML Full Stack Scalability?
  10. © 2019, Amazon Web Services, Inc. or its Affiliates. Scalable

    Team Development Anti Pattern: Superhero-Dependence ML Researcher Data Engineer Infra & Ops Engineer Product Manager Business Leader
  11. © 2019, Amazon Web Services, Inc. or its Affiliates. Development

    Anti Pattern: Blackbox Problem ML Researcher Engineer POC Production
  12. © 2019, Amazon Web Services, Inc. or its Affiliates. Development

    Anti Pattern: Blackbox Problem ML Researcher Engineer POC Production
  13. © 2019, Amazon Web Services, Inc. or its Affiliates. Development

    Anti Pattern: Deep Embedded Failure „just changing one feature“ A change of • Feature • Hyper-parameter • Regularization • Learning Rate • Sampling • Thresholds can affect the whole result.
  14. © 2019, Amazon Web Services, Inc. or its Affiliates. Semantically

    Interpretable Models Development Anti Pattern: Deep Embedded Failure Model I Model II Model III Ensemble Model Result: Better robustness and easier troubleshooting
  15. © 2019, Amazon Web Services, Inc. or its Affiliates. Specific

    problem areas Development Deployment Operation ML System Construction Process Common anti-patterns in …
  16. © 2019, Amazon Web Services, Inc. or its Affiliates. Deployment

    Anti Pattern: No Deployment Transparency
  17. © 2019, Amazon Web Services, Inc. or its Affiliates. Deployment

    Anti Pattern: No ML Lifecycle Management ML Full Stack POC Production
  18. © 2019, Amazon Web Services, Inc. or its Affiliates. Can

    we solve this?
  19. © 2019, Amazon Web Services, Inc. or its Affiliates. Amazon

    SageMaker Data Scientist DevOps Amazon SageMaker
  20. © 2019, Amazon Web Services, Inc. or its Affiliates. Amazon

    Sagemaker Flow Fully managed hosting with auto- scaling One-click deployment Pre-built notebooks for common problems Built-in, high performance algorithms One-click training Hyperparameter optimization BUILD TRAIN DEPLOY MOST OF THE TIME THIS IS JUST PROTOTYPING
  21. © 2019, Amazon Web Services, Inc. or its Affiliates. Only

    one small part of a ML project ML Code Serving Infrastructure Feature Extraction Data Collection
  22. © 2019, Amazon Web Services, Inc. or its Affiliates. How

    do you … - Ensure code quality? Maintainability? - Test your ML code? - Version code? - Re-train model? - Version models? - Decide when to (re-)deploy? - Rollback? - Audit? - Security? - …
  23. Treat ML code as “normal” software code!

  24. © 2019, Amazon Web Services, Inc. or its Affiliates. Why

    Sagemaker is great?
  25. Build, Train and Deploy your Machine Learning Models Amazon SageMaker

    Models are abstracted into docker containers Fast & accurate data labeling Built-in, high performance algorithms & notebooks B U I L D 1 One-click training and tuning T R A I N Model tuning and optimization 2 Fully managed hosting with auto-scaling and elastic inference One-click Deployment of model or Inference Pipeline D E P L O Y 3
  26. Amazon SageMaker: algorithms A. Train and deploy a built-in algorithm

    B. Train and deploy your own code in a framework container C. Train and deploy your own container D. Deploy a pre-trained model E. NEW – Train and Deploy a Marketplace Algorithm F. NEW – Deploy a Marketplace Model Package
  27. Amazon SageMaker: interface A. Console B. AWS Command Line Interface

    (CLI): aws sagemaker create-endpoint --endpoint-name <value> --endpoint-config-name <value> C. SageMaker Python SDK model.deploy(initial_instance_count=1, instance_type='ml.m4.xlarge') D. Boto3 (python) SageMaker client client.create_endpoint(EndpointName='string', EndpointConfigName='string') E. Spark SDK
  28. In summary:

  29. © 2019, Amazon Web Services, Inc. or its Affiliates. All

    rights reserved. Architecture Patterns From MVP to Production
  30. © 2019, Amazon Web Services, Inc. or its Affiliates. Architecture

    – Let’s start simple for a POC SageMaker EndPoints Raw Data Lake Training flow Inference flow SageMaker Training Platform User Interface End User Source C 4. Build Source B Source A Sagemaker Jupyter Notebooks Model Repository DDB S3
  31. © 2019, Amazon Web Services, Inc. or its Affiliates. Architecture

    – Let’s look at the data pipeline SageMaker EndPoints Raw Data Lake Training flow Inference flow User Interface End User Source C 4. Build Source B Source A Model Repository DDB S3 ETL Glue EMR Feature Store Catalog Analytics Data DDB S3 Model Input Store Metadata Data DDB S3 ETL SageMaker Training Platform
  32. © 2019, Amazon Web Services, Inc. or its Affiliates. Inference

    Architecture – Inference SageMaker EndPoints Raw Data Lake Training flow Inference flow API Gateway Lambda Optimization compute User Interface End User Source C 4. Build Source B Source A Model Repository DDB S3 ETL Glue EMR Feature Store Catalog Analytics Data DDB S3 Model Input Store Metadata Data DDB S3 ETL SageMaker Training Platform
  33. © 2019, Amazon Web Services, Inc. or its Affiliates. Inference

    Architecture – Training Pipeline SageMaker EndPoints Raw Data Lake Training flow Inference flow API Gateway Lambda Optimization compute User Interface End User Source C 4. Build Source B Source A Model Repository DDB S3 ETL Glue EMR Feature Store Catalog Analytics Data DDB S3 Model Input Store Metadata Data DDB S3 ETL SageMaker Training Platform Evaluate Train ECR Config Store S3
  34. © 2019, Amazon Web Services, Inc. or its Affiliates. Inference

    Architecture – CI for ML Code SageMaker EndPoints Raw Data Lake API Gateway Lambda Optimization compute User Interface End User Source C 4. Build Source B Source A Model Repository DDB S3 ETL Glue EMR Feature Store Catalog Analytics Data DDB S3 Model Input Store Metadata Data DDB S3 ETL SageMaker Training Platform Evaluate Train ECR Config Store S3 CodePipeline Git Code Build Test Training flow Inference flow Build Flow
  35. © 2019, Amazon Web Services, Inc. or its Affiliates. Inference

    Architecture – 4 Blocks SageMaker EndPoints Raw Data Lake API Gateway Lambda Optimization compute User Interface End User Source C 4. Build Source B Source A Model Repository DDB S3 ETL Glue EMR Feature Store Catalog Analytics Data DDB S3 Model Input Store Metadata Data DDB S3 ETL SageMaker Training Platform Evaluate Train Pre Inference Post ECR DDB Config Store S3 CodePipeline Git Code Build Test Training flow Inference flow Build Flow Data Pipeline Inference Training Pipeline Build Pipeline
  36. © 2019, Amazon Web Services, Inc. or its Affiliates. Takeaways

    https://aws.amazon.com/fr/blogs/machine-learning/automated-and-continuous- deployment-of-amazon-sagemaker-models-with-aws-step-functions/ Treat ML code as normal software code!
  37. © 2019, Amazon Web Services, Inc. or its Affiliates. All

    rights reserved. © 2019, Amazon Web Services, Inc. or its Affiliates. Thank You!