There is no such thing as MLOps

After the huge hype and investments in data science and machine learning in the last decade, the word "MLOps" is on the lips of every data executive and - therefore - recruiter, trying to sell it as the new shiny thing. But do we really know what it's actually about? Since the 90s, DevOps specialists have always been a key part of every solid engineering team, taking care of all the messy and obscure aspects of delivering real things to real people; MLOps isn't much different, working both on cultural and technical sides of delivering data products at scale. Why do we need a different naming and skillset? In this talk, Massimo Belloni (Lead Machine Learning Engineer at Bumble) will give an overview of the current MLOps space, advices and best practices on building an engineering driven Data Science team and a lot of random opinions.

Transcript

1. MLOps Summit London - 9/10th Nov 2022 There is no such thing as MLOps Massimo Belloni - Lead Machine Learning Engineer
2. Massimo Belloni Data Science Manager @Bumble Lead Machine Learning Engineer Focused on Trust & Safety Overall strategy for MLOps platform Interests in philosophy of science, consciousness, Strong vs Weak AI ◦ Coding consciousness ◦ Interpretability and trust
4. Contents What is MLOps? MLOps: a matter of skills ML Maturity Levels Things at Bumble Inc
5. MLOps got hot! There is no such thing as MLOps 5 Source: Google Trends
6. What is MLOps? MLOps or ML Ops is a set of practices that aims to deploy and maintain machine learning models in production reliably and efficiently.The word is a compound of "machine learning" and the continuous development practice of DevOps in the software field. Machine learning models are tested and developed in isolated experimental systems. When an algorithm is ready to be launched, MLOps is practiced between Data Scientists, DevOps, and Machine Learning engineers to transition the algorithm to production systems. Theory Source: Wikipedia There is no such thing as MLOps 6
7. DevOps is a popular practice in developing and operating large-scale software systems. This practice provides benefits such as shortening the development cycles, increasing deployment velocity, and dependable releases. DevOps A definition from Google Source: Google Cloud Architecture Center There is no such thing as MLOps 7
8. Infrastructure can be messy! MLOps in practice is evangelisation and problem solving! A good chunk of the actual work of a Machine Learning Engineer is to evangelise fellow Data Scientists on approaches and best practices (code quality, Git, general set up) In practice, deploying a Machine Learning model to production doesn’t end when the inference server is in production! Engineering work has decades of literature around its own efficiency, ML research doesn’t! Data Scientists aren’t always engineers Getting things done is hard 8
9. Solutions Architecture Data Engineering Monitoring Coding Best practices Legacy Systems Understanding Pragmatism There is no such thing as MLOps 9
10. Data Scientist Machine Learning Engineer Sr. Software Engineer People of MLOps Some usual career paths There is no such thing as MLOps 10 Junior MLE Data Engineer
11. Usual teams’ journeys Maturity Levels
12. Automated Training Automated runs, manual deployments No MLOps DevOps but no MLOps Heavy reliance on other teams (eg. data engineering) The Microsoft Azure Model There is no such thing as MLOps 12 Source: MLOps Maturity Model Automated Model Deployment Entire process is automatic and traceable Full MLOps Full monitoring and zero downtime
13. Level 2 Self serving and democratized access to infrastructure. No ad-hoc pipelines’ design. Level 1 Automated pipeline, from training to serving Level 0 Manual processes There is no such thing as MLOps 13 The Google Cloud Model
14. Some takeaways There is never any mention to the ability to build effective models that the business needs and ship them to production! Optimistic No mentions to what happens when the model is in production. How the team reacts to drift? How easy is to iterate? Experimenting offline in a reliable and replicable fashion comes before and after automatic pipelines, and it’s the key to a team’s success! Ability to build models isn’t covered! Experiments replicability is key! No reaction to change and drifts 14
15. Full maturity Ability to monitor and react to changes in an automatic fashion. No external dependencies. Maturity Level -1 Never deployed anything! Foundational MLE Replicable training, reliable serving Data access Features storage, experiment replicability, time to production. Service performance Ability to serve multiple frameworks, efficient monitoring, CPU vs GPU inference. A realistic model A mix of different visions There is no such thing as MLOps 15
16. Full maturity Data access Service performance Foundational capabilities There is no such thing as MLOps 16 • Easy access to computing resources (no race conditions) • Easy to share and “reproducible” Jupyter Notebooks • Multi-framework and (almost) 0 code serving • Out of the box monitoring
17. Full maturity Foundational MLE Replicable training, reliable serving Towards full maturity There is no such thing as MLOps 17 • Experiment tracking and replicability • Infrastructural work towards feature stores and training/serving features • Reliable and efficient monitoring • Quick access to input/output for drift detection et al
18. Wrapping up Good Machine Learning Engineers are not unicorns 🦄 Embrace different backgrounds (and career paths!) for high performing teams Every context is different 🏗 There are no secret recipes for success! As every engineering field, MLE requires deep contextual knowledge Slow and steady iterations for success 🚀 Rome wasn’t built in a day, why an MLOps platform should? Steady progress (and strategy!) can do wonders. What is MLOps? There is no such thing as MLOps 18
19. Questions?
20. Thank You