@useautomation | DevOpsDays Austin A Tale of Two Ops: How MLOps can learn from DevOps Andre Elizondo Solutions Architect @ WhyLabs 

@useautomation | DevOpsDays Austin Who am I? ● Seattle, WA ● Recovering Sysadmin, SRE, Evangelist ○ Chef, Adobe, Datadog, Big Fish Games, Lacework, etc. ● >10 yrs part of the DevOps community 

@useautomation | DevOpsDays Austin 

@useautomation | DevOpsDays Austin 

@useautomation | DevOpsDays Austin 

@useautomation | DevOpsDays Austin https://mattturck.com/landscape/mad2023.pdf 

@useautomation | DevOpsDays Austin https://mattturck.com/landscape/mad2023.pdf 

@useautomation | DevOpsDays Austin Machine Learning is the new OpenStack 

@useautomation | DevOpsDays Austin 

@useautomation | DevOpsDays Austin We’re here 

@useautomation | DevOpsDays Austin We’re here We’ll get here soon 

@useautomation | DevOpsDays Austin Machine Learning is the new shadow IT 

@useautomation | DevOpsDays Austin We’re at risk of another pig over the fence 

@useautomation | DevOpsDays Austin We’re at risk of another pig over the fence 

@useautomation | DevOpsDays Austin What is MLOps? ● Applying DevOps practices & culture in Machine Learning ● A process that involves multiple teams/silos https://ml-ops.org/content/mlops-principles 

@useautomation | DevOpsDays Austin What is MLOps? ● Applying DevOps practices & culture in Machine Learning ● A process that involves multiple teams/silos ● Not AIOps https://ml-ops.org/content/mlops-principles 

@useautomation | DevOpsDays Austin DevOps handles… ● Compute ● Networking ● Storage ● Release ● Service Reliability ● Security 

@useautomation | DevOpsDays Austin DevOps handles… ● Compute ● Networking ● Storage ● Release ● Service Reliability ● Security ● ML Reliability 

@useautomation | DevOpsDays Austin How is it similar to DevOps? ● Shared concepts, but different ○ CI/CD ○ Observability ○ Automation ○ Containers ● Huge silos between teams ○ Data Engineering ○ Data Scientists ○ ML Engineers ○ Product Managers ○ DevOps/SRE/Operations https://ml-ops.org/content/mlops-principles 

@useautomation | DevOpsDays Austin CI/CD in MLOps ● Deploying your model ○ Testing is different ○ Scaling is different(ish) ○ Packaging is more or less the same ○ Continuous delivery is possible but harder ● ML Data Pipelines ○ Training ○ Feature ○ Inference https://ml-ops.org/content/mlops-principles 

@useautomation | DevOpsDays Austin CI/CD in MLOps ● Deploying your model ○ Testing is different ○ Scaling is different(ish) ○ Packaging is more or less the same ○ Continuous delivery is possible but harder ● ML Data Pipelines ○ Training ○ Feature ○ Inference https://ml-ops.org/content/mlops-principles 

@useautomation | DevOpsDays Austin Observability in MLOps ● Performance is important ○ Some similar metrics, some different ones ○ Threshold baselines are different ● Availability is important ○ Service availability isn’t enough ● External dependencies need to be monitored upstream ● Sometimes batch, sometimes real time https://www.oreilly.com/library/view/reliable-machine-learning/9781098106218/ 

@useautomation | DevOpsDays Austin Automation in MLOps ● Response workflows ○ Retraining ○ Roll-back ● Infrastructure as code ○ Terraform ● Monitoring as code https://ml-ops.org/content/mlops-principles 

@useautomation | DevOpsDays Austin Containers in MLOps ● Dependency isolation ● Yes, it’s still kubernetes. ○ With all of it’s usual complaints. ● Sometimes controlled directly, most times through a platform ○ Kubeflow ○ Sagemaker ○ AzureML ○ Vertex ● Scaled for model serving, training, and pipelines 

@useautomation | DevOpsDays Austin What is unique about MLOps? ● Machine learning systems tend to be: ○ Fragile to changes in data ○ Harder to test ○ Harder to scale ● Complex to measure ○ What is good vs what is bad? ○ You may not know if something is good or bad for a while ● Models get worse over time, not better ● Data Scientists <3 Jupyter notebooks https://cloud.google.com/architecture/mlops-continuous-delivery-and-automation-pipelines-in-machine-learning 

@useautomation | DevOpsDays Austin Why should you be excited about MLOps? ● There’s a TON of new innovation happening ● There is a desperate need for operating experience ● MLOps is where DevOps was ~8-10 years ago ● Open source development is happening fast ● ML is here to stay 

@useautomation | DevOpsDays Austin We have a LOT of knowledge to share 

@useautomation | DevOpsDays Austin What should you do? ● Find out what models you’re running (or planning to run) in production ● Get involved, share knowledge and experiences ● Start experimenting with open source models & examples ● Talk about this with your team and think about how you can avoid surprises 

@useautomation | DevOpsDays Austin Thank you