Machine learning projects fail in production

Transcript

1. learn.machinelearning THIS IS WHY MACHINE LEARNING PROJECTS FAIL IN PRODUCTION
2. Consider a non-ML project like a calculator, which can still correctly add and multiply numbers a month, a year, or 10 years later. This is no longer the case when you are deploying machine learning (ML) models because your ML system does interact with the real world and the real world is changing rapidly over time. ML projects != non-ML projects learn.machinelearning
3. Not Enough Expertise A Disconnect Between Data Science and Traditional Software Development The data in your model will always be slightly wrong. Projects Are Too Complex No Confidence in the Models Built Issues with Integrating to existing systems Taking an ML model from desktop POC to running in production implies a massive, continuous effort. It’s Difficult to Update Models Lack of Leadership etc.... Reasons why ML projects never make it into production learn.machinelearning
4. Development of the machine learning-based applications is fundamentally different from the development of the traditional software. The complete development pipeline includes three levels of change: Data, ML Model, and Code. This means that in machine learning-based systems, the trigger for a build might be the combination of a code change, data change, or model change. Changing Anything Changes Everything learn.machinelearning Source - ml-ops.org
5. The performance of ML models in production degenerate over time because of changes in the real-life data that has not been seen during the model training. Take an example like ad classification models in which preferences change over time. Problem - 1 Model Drift learn.machinelearning
6. Since ML models are built on data, they are sensitive to the semantics, amount and completeness of incoming data. Problem - 2 Data quality learn.machinelearning
7. when transferring ML models to new business customers, these models, which have been pre-trained on different user demographics, data, might not work correctly according to quality metrics. Problem - 3 Environment changes learn.machinelearning
8. Since ML/AI is expanding into new applications and shaping new industries, building successful ML projects remains a challenging task. As shown, there is a need to establish effective practices and processes around designing, building, and deploying ML models into production - MLOps. Solving above problems learn.machinelearning
9. THANK YOU learn.machinelearning