Machine Learning Experimentation DVC and VS Code Presented by: Gift Ojeabulu

Who am I? - Co-founder and community lead for Data Community Africa/DatafestAfrica - Content & Community Support Advocate at Iterative.ai - AWS ML Community Builder & MLOps Lagos Community Lead. - DevNetwork Advisory Board Member (Developer Advocate) - Facilitator of the African Data Community Newsletter. - Technical Writer, Public Speaker, Social Media Content creator, Machine Learning Thought Leader (Global AI Hub),Open-source & Community Advocate. Gift Ojeabulu Iterative.ai @GiftOjeabulu_

An Overture - What is MLOps - What is DVC? - CRISP-DM VS ASUM-DM Methodology - Experiment tracking tools. - VS Code Extensions for Machine learning.

Lesson Outline ⬥ What is experiment tracking? ⬥ Why is experiment tracking important in machine learning? ⬥ Experimenting workflow with DVC and VS Code ⬥ Project Setup and DVCLive ⬥ Conclusion

What is Experiment Tracking?

⬥Experiment tracking refers to the practice of systematically recording and managing information about the various experiments and iterations conducted during the development and training of machine learning models. ⬥It is a crucial aspect of managing the machine learning lifecycle and ensuring reproducibility, collaboration, and effective model management. What is Experiment tracking?

Why Experiment Tracking?

⬥ Experiment tracking is a best practice in machine learning because it promotes transparency, collaboration, and reproducibility. It helps data scientists and teams manage the complexity of machine learning projects, make informed decisions, and deliver more robust and reliable models. Why Experiment tracking?

Here are some specific examples of how experiment tracking can be used in machine learning: - A data scientist is developing a new model to predict customer churn. They track their experiments to compare different model architectures and hyperparameters. They also track their results to identify the best performing model (compare different experiment) - A team of data scientists is working on a project to develop a model to detect fraud. They use experiment tracking to share their results with each other and to collaborate on the development of the model (collaborate with others) - An ML Engineer is monitoring a model that is in production. They use experiment tracking to track the performance of the model over time and to identify any potential problems (Track progress overtime)

Machine learning without experiment tracking is like sailing without navigation. You might reach your destination, but you'll never know how you got there or how to improve the journey.

Experimenting Workflow with DVC and VS Code

Experimenting Workflow ⬥ Git: branching and tagging experiments a. create a separate branch & tag for each experiment b. keep failed experiments ⬥ DVC: run pipelines and keep states a. use dvc exp run b. clean up DVC cache for not important experiments ⬥ VS Code: visualize metrics and collaborate a. metrics visualization b. experiments comparison c. collaboration

Project Setup

Pip or conda DVC init DVC get data DVC add data

What Experiment tracking looks like with DVC on VS Code

No content

⬥ DVCLive is an auto-experiment tracking library that integrates seamlessly with DVC and the DVC VSCode extension. It is framework-specific, meaning it has separate tracking callbacks for many popular ML and DL frameworks

DVC Live

What have we learned?

Conclusion

In the world of MLOps, DVC is the secret sauce that makes everything taste better. It keeps our experiments reproducible, our data organized, and our models accountable

Links for further learning ◆ Community article on experiment tracking https://pub.towardsai.net/how-to-track-ml-experiments-with-dvc-inside-vsco de-to-boost-your-productivity-a654ace60bab ◆ Article - https://towardsdatascience.com/turn-vs-code-into-a-one-stop-shop-for-ml-exp eriments-49c97c47db27 ◆ Article - https://towardsdatascience.com/the-minimalists-guide-to-experiment-trackin g-with-dvc-f07e4636bdbb ◆ Youtube - https://www.youtube.com/@dvcorg8370/videos ◆ Newsletter - https://www.linkedin.com/pulse/dvc-august-23-community-updates-iterative -ai ◆ Discord - https://discord.gg/N5YzBBuFms

Webinar Alert!!!

/gift-ojabu /Gift Ojabu GiftOjeabulu_

Thank you! Gift Ojeabulu iterative.ai @GiftOjeabulu_