The Machine Learning Platform and continuous model deployment

Transcript

1. The Machine Learning Platform and continuous model deployment Loïc DIVAD
2. Loïc DIVAD Data Engineer @loicmdivad
3. CI/CD & Machine Learning The origin Sales prediction by equipment 4
4. Software engineering process CI/CD & Machine Learning “the right tool for the right job” VCS CI REPO 5
5. Software engineering process CI/CD & Machine Learning “the right tool for the right job” VCS CI REPO EXPLO 6
6. CI/CD & Machine Learning Model, quésaco ? Training App y Modèle 7
7. CI/CD & Machine Learning Model, quésaco ? Training App y ỹ Modèle 8 Predicting APP Modèle
8. CI/CD & Machine Learning Model training Creation of a serializable model VCS CI REPO EXPLO /apps/<myapp>/model_v1 9
9. CI/CD & Machine Learning Model training on new datasets “With more data comes great generalization power” VCS CI REPO EXPLO /apps/<myapp>/model_v1 /apps/<myapp>/model_v2 /apps/<myapp>/model_v3 … /apps/<myapp>/model_v? ▼ The ideas of Production model and Current model need to be dissociate ▼ We need to track the evolutions between this two models ▼ We need to track the training datasets used 10
10. CI/CD & Machine Learning Model training on new features Develop, Deploy, Fit … Repeat VCS CI REPO EXPLO /apps/<myapp>/model_v1 /apps/<myapp>/model_v2 /apps/<myapp>/model_v3 … /apps/<myapp>/model_v? VCS CI REPO EXPLO /apps/<myapp>/model_v41 /apps/<myapp>/model_v42 /apps/<myapp>/model_v43 … /apps/<myapp>/model_v4? Sprint N+1 11
11. 12 CI/CD & Machine Learning
12. Continuous Model Integration DIY 13
13. Distribution Features extraction Prediction Source Ingestion Source Ingestion Source Ingestion Model Training The Machine Learning Pipeline 14 DIY: Machine Learning Platform
14. Distribution Features extraction Prediction Source Ingestion Source Ingestion Source Ingestion Model Training Release The Machine Learning Pipeline 15 DIY: Machine Learning Platform
15. Continuous release DIY: Machine Learning Platform ỹ model_id @app-dt01-<ts> @app-dt01-<ts> ▼ Each models need to be versioned ▼ Tags like latest and current allow us to compare new models with the one online ▼ Predictions comes with the model version App Modèle 16 @app-dt02-<ts>
16. Distribution Features extraction Prediction Source Ingestion Source Ingestion Source Ingestion Model Training Release Test The Machine Learning Pipeline 17 DIY: Machine Learning Platform
17. An automated evaluation DIY: Machine Learning Platform y ỹ model_id @app-dt02-<ts> @app-dt02-<ts> @app-dt02-<ts> L ( y , ỹ ) ↦ Score new score 18 model n-2 model n-1
18. Distribution Features extraction Prediction Source Ingestion Source Ingestion Source Ingestion Model Training Release Report Test The Machine Learning Pipeline 19 DIY: Machine Learning Platform
19. Development Release Integration Deployment Keep code and model separated Train each models on production datasets Keep model history and the corresponding datasets used Benchmarks, features and hyperparamètres extraction Set the new model online Conclusion
20. Conclusion Machine Learning Platform The machine learning plateforme: your next CI/CD tool 21 VCS CI REPO EXPLO
21. MERCI 22
23. ANNEXES ? 24
24. Because they know better 25 The Machine Learning Platform and continuous model deployment References ▼ Meet Michelangelo: Uber’s Machine Learning Platform ▼ Serving Machine Learning Models ▼ TFX: A TensorFlow-Based Machine Learning Platform ▼ Hidden Technical Debt in Machine Learning Systems
25. Distribution Features extraction Prediction Source Ingestion Source Ingestion Source Ingestion Distribution Model Training Release Report Test The Machine Learning Pipeline 26 DIY: Machine Learning Platform
26. DIY: Machine Learning Platform Machine Learning Platform La machine learning plateforme, prochain outils de CI/CD 27 VCS CI REPO EXPLO
27. DIY: Machine Learning Platform Projet Bold Eagle The machine learning platform: your next CI/CD tool 28 VCS CI REPO EXPLO Services API
28. Serving 29 +10ms 5s - 1mins 1 day RCP Servers : Tensorflow Serving Prediction Servers: AWS Lambda, Cloud AI Structured Streaming - Kafka Based Prediction Batch Predictions
29. Serving: a real time example 30 Model serving architecture by Boris Lublinsky - from Serving Machine Learning Models Data Source Model Source Model Storage Current Model Processing Prediction Stream Processor
30. THE VERY END 31