Driving MLOps on Azure: A Strategic Blueprint for Scaling ML for African Enterprise Success

Transcript

2. Driving MLOps on Azure: A Strategic Blueprint for Scaling ML for African Enterprise Success Sam Ayo @officialsamayo
3. Meet Sam Ayo - AI/ML Engineer, Data Scientist and Head of Engineering - Extensive experience in software, data, and AI consulting - Providing insights into AI's transformative power Sam Ayo @officialsamayo
4. Any sufficiently advanced technology is indistinguishable from magic. Arthur C. Clark
5. Agenda • Introduction to MLOps • Elements of an MLOps solution • Orchestrating Azure CI/CD Workflow • Principles for scalable MLOps architecture
6. Introduction to MLOps
7. What is ML? Model = Algorithm + Training Data MLOps = ML model + Software System mlops=MLOps(“lets get started”)
8. • MLOps is the extension of DevOps to ML as a first class citizen • MLOps is the collaboration of infrastructure and tooling to productionize ML Machine Leaning Operations(MLOps) is the practice that combines software engineering, devOps and machine learning to design, develop, deploy and manage production-level machine learning models. • The happy marriage of AI and the traditional DevOps model mlops.whatis()
9. The goal of MLOps is to reduce technical friction to get the model from an idea into production in the shortest possible time with as little risk as possible.
11. According to the MITSIoan and BCG 2019 survey, 7/10 companies report little or no impact with the use of AI. 40% of organizations with significant investments in Al report no benefits. FACT Only 22% of companies using ML have successfully deployed an ML model into production. 87% of data science projects never make it into production. The main challenges people face when developing ML capabilities are scale, version control, model reproducibility, and aligning stakeholders. Reality is: • Al is a source of opportunities and advantages • Implementing Al is a risk • Implementing Al correctly is difficult
12. What MLOps should look like
13. What it actually looks like
15. Elements of an MLOps solution
16. MLOps levels: • Level 0 – Manual process • Level 1 – ML Pipeline automation • Level2 – CICD pipeline automation • Level3 – Full CICD pipeline automation and retraining mlops.elements()
17. Technical Concepts • Iterative-Incremental Dev • Automation • CT/Cl/CD • Versioning • Testing • Reproducibility • Monitoring • Source/version control • Experiment tracking • Test & build services • Automatic deployment services • Model/code registry • Feature store • ML metadata store • Model monitoring • Model & data performance assessment Technical Components
18. MLOps Setup Tools Description Experiment design/development Jupyter Notebook(Python, Pandas) Experiment tracking Comet, MLFlow Source/Version Control Git, DVC, Github Test & Build Services PyTest, Make Model & Dataset Registry Blob Storage, PostgreSQL Feature Store Feast, PostgreSQL, Blob Storage Model Serving FastAPI Model Monitoring Evidently AI
19. Orchestrating MLOps with CI/CD Workflow on Azure
20. mlops.azureml()
22. Principles for scalable MLOps architecture
23. Scalable ML is not for the weak.
24. The Critical Questions • How will the Predictions be served? • How will the model be served? • How will ML meet the software system? mlops.principles()
25. Integrating ML? • Batch inference • Real-time inference • Streaming inference • Edge inference 1. Serving model Predictions
26. Integrating ML? • One-to-one • One-to-all 2. Serving model weights
27. Integrating ML? • Experimentation is at the heart of the Machine Learning profession. • We progress because we experiment and it begins in a notebook. 3. Design model experiments
28. • Document each cell • Create descriptive notebook iterations Notebook Practices
29. • Create sectionally headlined workflows • Linear flow of execution • Set Parameters on top of the notebook Notebook Practices
30. • Create sectionally headlined workflows • Linear flow of execution • Set Parameters on top of the notebook Notebook Practices
31. Integrating ML? • Monolithic integration • single service integration • Microservice integration 3. Model meets the software system
32. Monolithic integration Single service integration Microservice integration The ML service code base is integrated within the rest of the backend code base. The ML service code base is deployed on a single server, with elastic load balancing for scaling. The ML service code base is deployed such that components get their own services. The entire system process is slowed down by the ML service, the model size and computation requirements usually add additional load on the backend servers. Usually considered if the inference process is very light to run. The model size can be complex without putting load pressure on the rest of the infrastructure. This is typically the easiest way to deploy a model while ensuring scalability, maintainability and reliability. This is a relief system for the entire codebase. It ensures the different components of the ML system can be reused for different purposes. For example, the ML inference manager at RadioAdSpread. www.radioadspread.com
33. Python is not enough!
34. Python is not enough!
35. Build Responsibly!
36. Sam Ayo @officialsamayo THANK YOU!