Deploying Machine Learning Systems to Production Challenges & Solutions using MLOps Adarsh Shah Engineering Leader, Coach, Hands-on Architect Independent Consultant @shahadarsh shahadarsh.com 

shahadarsh.com @shahadarsh Traditional Software Development vs Machine Learning 

shahadarsh.com @shahadarsh Machine Learning Workflow Data Acquisition Data Preparation Model Development Model Training Model Serving Accuracy Evaluation Code
 Changes Retraining Data Management Experimentation Production Deployment 

shahadarsh.com @shahadarsh Fun Fact about Model training Everybody has. They just don’t realize it. Have you ever helped train a ML model? 

shahadarsh.com @shahadarsh Hidden Technical Debt in ML Systems From the paper Hidden Technical Debt in Machine Learning Systems 

shahadarsh.com @shahadarsh Challenges Unique to ML 

shahadarsh.com @shahadarsh #1: Data Management Data Location Large Datasets Security Compliance 

shahadarsh.com @shahadarsh #2: Constant Research and Experimentation Code Quality Experimentation Notebooks Tracking experiments 

shahadarsh.com @shahadarsh #3: Training Process Retraining Training Time Reproducibility 

shahadarsh.com @shahadarsh #4: Infrastructure Requirements Edge devices GPU & 
 high density cores Costs Elasticity 

shahadarsh.com @shahadarsh #5: Testing Model Accuracy Data Validation Model
 Bias & Fairness 

shahadarsh.com @shahadarsh #6: Dependency Hell Dependency Hell ARM architecture 

shahadarsh.com @shahadarsh MLOps 

shahadarsh.com @shahadarsh MLOps MLOps = Machine Learning + DevOps People Process + Technology + 

shahadarsh.com @shahadarsh Roles ML Researcher ML Engineer Data Engineer MLOps Engineer 

shahadarsh.com @shahadarsh Team Structure considerations Cross functional Team Separate Data Science Team ML Platform Engineering Team 

shahadarsh.com @shahadarsh Data pipeline Data Source A Data Acquisition A Data Preparation A Training 
 Dataset Data Validation A Data Source B Data Source N Data Acquisition B Data Acquisition N Data Preparation B Data Preparation N Data Validation B Data Validation N Input Training Process Input 

shahadarsh.com @shahadarsh Training Pipeline Training
 Code Continuous 
 Integration Training Data Data Pipeline Pre-trained
 Weights Validation Artifact Repository Push image Training Environment Cloud, On-Prem 
 or Edge location Infra provisioning automation GPU support Monitoring/ Logging/Alerting UI or 
 command Schedule Training Bias & Fairness Testing Build & Version
 Model 

shahadarsh.com @shahadarsh Deployment Pipeline GitOps Monitoring/ Logging/Alerting Artifact Repository Pull Model
 Image Model 
 Training Retrain Model
 (if accuracy 
 below acceptable %) Push to Master Infra provisioning automation GPU support Model Serving 
 Environment Cloud, On-Prem 
 or Edge location Deploy Model Evaluate Model Accuracy Periodic 

shahadarsh.com @shahadarsh Platforms available 

shahadarsh.com @shahadarsh Platforms 

shahadarsh.com @shahadarsh Kubeflow 

shahadarsh.com @shahadarsh To sum it up • Machine Learning Workflow • Traditional Software Development vs Machine Learning • Unique ML Challenges • Data Management • Constant Research and Experimentation • Training Process • Infrastructure Requirements • Testing • Dependency Hell • MLOps • Roles & Team Structure Considerations • Data, Training & Deployment Pipeline • Platforms Available 

Questions Adarsh Shah Engineering Leader, Coach, Hands-on Architect Independent Consultant @shahadarsh shahadarsh.com