Machine learning projects fail in production

Transcript

1. learn.machinelearning
   THIS IS WHY
   MACHINE LEARNING
   PROJECTS
   FAIL IN
   PRODUCTION
   

   View Slide

2. Consider a non-ML project like a calculator, which can still correctly add
   and multiply numbers a month, a year, or 10 years later. This is no longer
   the case when you are deploying machine learning (ML) models because
   your ML system does interact with the real world and the real world is
   changing rapidly over time.
   ML projects != non-ML projects
   learn.machinelearning
   

   View Slide

3. Not Enough Expertise
   A Disconnect Between Data Science and Traditional Software Development
   The data in your model will always be slightly wrong.
   Projects Are Too Complex
   No Confidence in the Models Built
   Issues with Integrating to existing systems
   Taking an ML model from desktop POC to running in production implies a
   massive, continuous effort.
   It’s Difficult to Update Models
   Lack of Leadership
   etc....
   Reasons why ML projects never
   make it into production
   learn.machinelearning
   

   View Slide

4. Development of the machine learning-based applications is fundamentally
   different from the development of the traditional software. The complete
   development pipeline includes three levels of change: Data, ML Model, and
   Code. This means that in machine learning-based systems, the trigger for a
   build might be the combination of a code change, data change, or model
   change.
   Changing Anything Changes Everything
   learn.machinelearning Source - ml-ops.org
   

   View Slide

5. The performance of ML models in production degenerate over time
   because of changes in the real-life data that has not been seen during the
   model training. Take an example like ad classification models in which
   preferences change over time.
   Problem - 1 Model Drift
   learn.machinelearning
   

   View Slide

6. Since ML models are built on data, they are sensitive to the semantics,
   amount and completeness of incoming data.
   Problem - 2 Data quality
   learn.machinelearning
   

   View Slide

7. when transferring ML models to new business customers, these models,
   which have been pre-trained on different user demographics, data, might
   not work correctly according to quality metrics.
   Problem - 3 Environment changes
   learn.machinelearning
   

   View Slide

8. Since ML/AI is expanding into new applications and shaping new industries,
   building successful ML projects remains a challenging task. As shown,
   there is a need to establish effective practices and processes around
   designing, building, and deploying ML models into production - MLOps.
   Solving above problems
   learn.machinelearning
   

   View Slide

9. THANK YOU
   learn.machinelearning
   

   View Slide