Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Sign up for free
Menu
Search
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Pricing
Search
Sign in
Sign up for free
ML Models and Dataset Versioning
Search
Kurian Benoy
October 13, 2019
Programming
530
0
Share
Embed
Copy iframe code
Copy JS code
Copy link
Start on current slide
ML Models and Dataset Versioning
Kurian Benoy
October 13, 2019
Transcript
34 slides
ML MODELS AND DATASET VERSIONING Kurian Benoy
$ WHOAMI Open source contributor FOSSASIA OpenTechNights Winner Kaggle Expert
in Kernels
$ WHOAMI Open source contributor FOSSASIA OpenTechNights Winner Kaggle Expert
Final Year BTech student @MEC
OUTLINE Start up Adventures Challenges Model and Dataset versioning How
I discovered DVC? Use case: Versioning dogs and Cats Conclusion
Startup Adventures
CHALLENGE 1: ML IS SLOW
CHALLENGE 2: WORKING WITH ML PROJECTS Most software products take
a few seconds to execute. $ git clone project-repo $ pip install -r requirements.txt
None
CHALLENGE 3: METRIC DRIVEN
CHALLENGE 4: NOT ABLE TO USE GIT git not suitable
for projects > 1GB git clone becomes slow
MODEL VERSIONING
TRACKING EXPERIMENTS TRACKING METRICS
Why Model Versioning? > To keep track of experiments >
Choose the best ideas >> EXPERIMENTS = CODE + OUTPUTS Models are outputs
DATASET VERSIONING
None
4 TB/day
None
Why Dataset management? > Moving Datasets around > Datasets evolve,
so versioning required >> EXPERIMENTS = CODE + DATA + OUTPUTS Source code, Datasets
HOW I DISCOVERED DVC
DATA VERSION CONTROL(DVC)
> Experiment and Dataset tracking > Open-source(3500+ stars) > Build
to adopt the best practises of ML > Works well with git > Language and framework agnostic
VERSIONING CATS & DOGS
DEMO TIME
DVC WORKFLOW
Tracking data 1 Tracking 1000 cats and dogs 2 Add
1000 more labelled images of cats & dogs
SWITCHING VERSIONS
CONCLUSION
"Data science as different from software as software was different
from hardware." Nick Elprin, CEO, DominoLabs.
Think about your processes(ML projects)
Think about your processes Try to version control for your
projects
Try it out in your ML project!
THANK YOU Twitter: kurianbenoy2 Email :
[email protected]
Speaker Deck: bit.ly/mlversion19
APPENDIX
Other Tools for versioning ML Flow - Tracking Models, Metrics
Git-LFS - Tracking Large files Jovian - JupyterNB based tracking Neptune.Ml Hangar Py - Versioning Tensor Data
More Decks by Kurian Benoy
See All by Kurian Benoy
Mental Models for the four years that matter
kurianbenoy
0
37
How I ended up maintaining a python package with 1M+ downloads so far?
kurianbenoy
0
32
MTech Final Project - Presentation Slides
kurianbenoy
0
93
Project Review Report 5 - MTech Project
kurianbenoy
1
60
Joy of Programming
kurianbenoy
0
77
Expert Interaction on ML
kurianbenoy
0
120
Project Review Report 4 - Robust Speech Recognition in Malayalam
kurianbenoy
0
170
Final project report - Phase 1
kurianbenoy
0
130
Project Review Slides
kurianbenoy
0
75
Other Decks in Programming
See All in Programming
はじめてのPLATEAU
nokonoko1203
1
660
Claude Codeのおかげでやりたいことがやれるようになったんだよォー!
sms_tech
0
140
Kotlin Fest 2026: Kotlin コンパイラに Contribute してみよう! - Kotlin/Wasm で深い再帰を書けるようにした話
ternbusty
1
330
LLM 101
dalinaum
0
510
React Native経験者が Flutterを30⽇間触って驚いたこと
3saito
0
200
flutter_test のその先へ — Flutter アプリの LLM 機能に「評価」という品質保証を
kosukesaigusa
0
200
ビジネスロジックを堅牢に~Scala 3のドメインプリミティブ~
j5ik2o
0
200
ローカルLLMの基本と近況 2611 / Local LLM basics and recent situation 2611
kishida
3
580
DroidKnights 2026 Android A2UI Jetpack Compose 기반의 안전한 AI UI 렌더링 - 박제창
itsmedreamwalker
1
250
Vue.jsで作る空間解析アプリとCapacitorプラグインのData Bridge
hal_spidernight
1
180
조합이폭발하는디자인시스템이걸눈으로도테스트하라고요
giljongyu
0
780
Goコンパイラを作ってGoをもっと理解する / Build a Go compiler to better understand Go
ikura_hamu
0
130
Featured
See All Featured
The untapped power of vector embeddings
frankvandijk
2
2k
Building Experiences: Design Systems, User Experience, and Full Site Editing
marktimemedia
1
670
Scaling GitHub
holman
464
140k
Bridging the Design Gap: How Collaborative Modelling removes blockers to flow between stakeholders and teams @FastFlow conf
baasie
0
770
A Guide to Academic Writing Using Generative AI - A Workshop
ks91
PRO
1
570
Optimizing for Happiness
mojombo
378
72k
The Cult of Friendly URLs
andyhume
79
7.1k
Chasing Engaging Ingredients in Design
codingconduct
0
400
The Illustrated Children's Guide to Kubernetes
chrisshort
51
53k
Game over? The fight for quality and originality in the time of robots
wayneb77
1
330
Leveraging Curiosity to Care for An Aging Population
cassininazir
1
570
Fashionably flexible responsive web design (full day workshop)
malarkey
409
67k