Mise en place d'un CI/CD pour vos modèles de machine learning

La mise en production de models de Machine learning réponds aux mêmes contraintes que la mise en production d'applications classique. Dans cette sessions venez découvrir comment vos data scientists pourront mettre en production leur algorithms de machine learning avec des outils comme code comme code build, step functions et sagemaker.

Transcript

1. © 2019, Amazon Web Services, Inc. or its Affiliates. Marc Cabocel Startup Solutions Architect November 28th, 2019 Industrialisation du Machine Learning Mise en place d’une CI/CD pour vos modèles de machine learning
2. © 2019, Amazon Web Services, Inc. or its Affiliates. Customer Goals Business Outcomes I N N O V A T I O N M O D E R N I Z A T I O N & M I G R A T I O N Successful enterprises follow a common path F O U N D A T I O N
3. © 2019, Amazon Web Services, Inc. or its Affiliates. Challenges
4. © 2019, Amazon Web Services, Inc. or its Affiliates. Definition of DevOps Strong focus on automation and monitoring at all steps of software construction. Software Operations Software Development
5. © 2019, Amazon Web Services, Inc. or its Affiliates. Definition of MLOps Strong focus on automation and monitoring at all steps of ML System construction. ML System Operations ML System Development
6. © 2019, Amazon Web Services, Inc. or its Affiliates. Specific problem areas Development Deployment Operation ML System Construction Process Common anti-patterns in …
7. © 2019, Amazon Web Services, Inc. or its Affiliates. Specific problem areas Development Deployment Operation ML System Construction Process Common anti-patterns in …
8. © 2019, Amazon Web Services, Inc. or its Affiliates. Development Anti Pattern: Superhero-Dependence ML Researcher ML Full Stack Data Engineer DevOps Engineer Infrastructure Architect Product Manager Exec Partner POC Production
9. © 2019, Amazon Web Services, Inc. or its Affiliates. Development Anti Pattern: Superhero-Dependence ML Full Stack Scalability?
10. © 2019, Amazon Web Services, Inc. or its Affiliates. Scalable Team Development Anti Pattern: Superhero-Dependence ML Researcher Data Engineer Infra & Ops Engineer Product Manager Business Leader
11. © 2019, Amazon Web Services, Inc. or its Affiliates. Development Anti Pattern: Blackbox Problem ML Researcher Engineer POC Production
12. © 2019, Amazon Web Services, Inc. or its Affiliates. Development Anti Pattern: Blackbox Problem ML Researcher Engineer POC Production
13. © 2019, Amazon Web Services, Inc. or its Affiliates. Development Anti Pattern: Deep Embedded Failure „just changing one feature“ A change of • Feature • Hyper-parameter • Regularization • Learning Rate • Sampling • Thresholds can affect the whole result.
14. © 2019, Amazon Web Services, Inc. or its Affiliates. Semantically Interpretable Models Development Anti Pattern: Deep Embedded Failure Model I Model II Model III Ensemble Model Result: Better robustness and easier troubleshooting
15. © 2019, Amazon Web Services, Inc. or its Affiliates. Specific problem areas Development Deployment Operation ML System Construction Process Common anti-patterns in …
16. © 2019, Amazon Web Services, Inc. or its Affiliates. Deployment Anti Pattern: No Deployment Transparency
17. © 2019, Amazon Web Services, Inc. or its Affiliates. Deployment Anti Pattern: No ML Lifecycle Management ML Full Stack POC Production
18. © 2019, Amazon Web Services, Inc. or its Affiliates. Can we solve this?
19. © 2019, Amazon Web Services, Inc. or its Affiliates. Amazon SageMaker Data Scientist DevOps Amazon SageMaker
20. © 2019, Amazon Web Services, Inc. or its Affiliates. Amazon Sagemaker Flow Fully managed hosting with auto- scaling One-click deployment Pre-built notebooks for common problems Built-in, high performance algorithms One-click training Hyperparameter optimization BUILD TRAIN DEPLOY MOST OF THE TIME THIS IS JUST PROTOTYPING
21. © 2019, Amazon Web Services, Inc. or its Affiliates. Only one small part of a ML project ML Code Serving Infrastructure Feature Extraction Data Collection
22. © 2019, Amazon Web Services, Inc. or its Affiliates. How do you … - Ensure code quality? Maintainability? - Test your ML code? - Version code? - Re-train model? - Version models? - Decide when to (re-)deploy? - Rollback? - Audit? - Security? - …
23. Treat ML code as “normal” software code!
24. © 2019, Amazon Web Services, Inc. or its Affiliates. Why Sagemaker is great?
25. Build, Train and Deploy your Machine Learning Models Amazon SageMaker Models are abstracted into docker containers Fast & accurate data labeling Built-in, high performance algorithms & notebooks B U I L D 1 One-click training and tuning T R A I N Model tuning and optimization 2 Fully managed hosting with auto-scaling and elastic inference One-click Deployment of model or Inference Pipeline D E P L O Y 3
26. Amazon SageMaker: algorithms A. Train and deploy a built-in algorithm B. Train and deploy your own code in a framework container C. Train and deploy your own container D. Deploy a pre-trained model E. NEW – Train and Deploy a Marketplace Algorithm F. NEW – Deploy a Marketplace Model Package
27. Amazon SageMaker: interface A. Console B. AWS Command Line Interface (CLI): aws sagemaker create-endpoint --endpoint-name <value> --endpoint-config-name <value> C. SageMaker Python SDK model.deploy(initial_instance_count=1, instance_type='ml.m4.xlarge') D. Boto3 (python) SageMaker client client.create_endpoint(EndpointName='string', EndpointConfigName='string') E. Spark SDK
28. In summary:
29. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Architecture Patterns From MVP to Production
30. © 2019, Amazon Web Services, Inc. or its Affiliates. Architecture – Let’s start simple for a POC SageMaker EndPoints Raw Data Lake Training flow Inference flow SageMaker Training Platform User Interface End User Source C 4. Build Source B Source A Sagemaker Jupyter Notebooks Model Repository DDB S3
31. © 2019, Amazon Web Services, Inc. or its Affiliates. Architecture – Let’s look at the data pipeline SageMaker EndPoints Raw Data Lake Training flow Inference flow User Interface End User Source C 4. Build Source B Source A Model Repository DDB S3 ETL Glue EMR Feature Store Catalog Analytics Data DDB S3 Model Input Store Metadata Data DDB S3 ETL SageMaker Training Platform
32. © 2019, Amazon Web Services, Inc. or its Affiliates. Inference Architecture – Inference SageMaker EndPoints Raw Data Lake Training flow Inference flow API Gateway Lambda Optimization compute User Interface End User Source C 4. Build Source B Source A Model Repository DDB S3 ETL Glue EMR Feature Store Catalog Analytics Data DDB S3 Model Input Store Metadata Data DDB S3 ETL SageMaker Training Platform
33. © 2019, Amazon Web Services, Inc. or its Affiliates. Inference Architecture – Training Pipeline SageMaker EndPoints Raw Data Lake Training flow Inference flow API Gateway Lambda Optimization compute User Interface End User Source C 4. Build Source B Source A Model Repository DDB S3 ETL Glue EMR Feature Store Catalog Analytics Data DDB S3 Model Input Store Metadata Data DDB S3 ETL SageMaker Training Platform Evaluate Train ECR Config Store S3
34. © 2019, Amazon Web Services, Inc. or its Affiliates. Inference Architecture – CI for ML Code SageMaker EndPoints Raw Data Lake API Gateway Lambda Optimization compute User Interface End User Source C 4. Build Source B Source A Model Repository DDB S3 ETL Glue EMR Feature Store Catalog Analytics Data DDB S3 Model Input Store Metadata Data DDB S3 ETL SageMaker Training Platform Evaluate Train ECR Config Store S3 CodePipeline Git Code Build Test Training flow Inference flow Build Flow
35. © 2019, Amazon Web Services, Inc. or its Affiliates. Inference Architecture – 4 Blocks SageMaker EndPoints Raw Data Lake API Gateway Lambda Optimization compute User Interface End User Source C 4. Build Source B Source A Model Repository DDB S3 ETL Glue EMR Feature Store Catalog Analytics Data DDB S3 Model Input Store Metadata Data DDB S3 ETL SageMaker Training Platform Evaluate Train Pre Inference Post ECR DDB Config Store S3 CodePipeline Git Code Build Test Training flow Inference flow Build Flow Data Pipeline Inference Training Pipeline Build Pipeline
36. © 2019, Amazon Web Services, Inc. or its Affiliates. Takeaways https://aws.amazon.com/fr/blogs/machine-learning/automated-and-continuous- deployment-of-amazon-sagemaker-models-with-aws-step-functions/ Treat ML code as normal software code!
37. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. © 2019, Amazon Web Services, Inc. or its Affiliates. Thank You!