Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Sign up for free
Menu
Search
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Pricing
Search
Sign in
Sign up for free
Practical DevOps for the busy data scientist
Search
Tania Allard
October 09, 2019
Programming
1k
1
Share
Embed
Copy iframe code
Copy JS code
Copy link
Start on current slide
Practical DevOps for the busy data scientist
Tania Allard
October 09, 2019
Transcript
45 slides
Practical DevOps for the busy data Scientist
bit.ly/PyConDE-mlops Slides
What you’ll learn 01 02 Why MLOps/ DevOps ? Who
is responsible? 03 04 Getting started Getting from A to B
About Me
Software engineering Algorithm Data Answers @ixek bit.ly/PyConDE-mlops
Machine learning Answers Data Algorithm @ixek bit.ly/PyConDE-mlops
Machine learning Answers Data Model @ixek bit.ly/PyConDE-mlops @ixek bit.ly/PyConDE-mlops
Machine learning Answers Data Model Answers Predictions @ixek bit.ly/PyConDE-mlops
The data cycle Magic? R&D Generation @ixek bit.ly/PyConDE-mlops
Anyone? @ixek bit.ly/PyConDE-mlops
A common scenario @ixek bit.ly/PyConDE-mlops
@ixek bit.ly/PyConDE-mlops
If you had one wish? @ixek bit.ly/PyConDE-mlops
Replacing the magic ML Ops and robust pipelines R&D Generation
@ixek bit.ly/PyConDE-mlops
How skills are perceived @ixek bit.ly/PyConDE-mlops
Better @ixek bit.ly/PyConDE-mlops
How they really are @ixek bit.ly/PyConDE-mlops
DevOps is the union of people, process, and products to
enable continuous delivery of value into production - Donovan Brown What is devops @ixek bit.ly/PyConDE-mlops
MlOps Aims to reduce the end-to-end cycle time and friction
of data analytics/science from the origin of ideas to the creation of data artifacts. What is devops @ixek bit.ly/PyConDE-mlops
But I do not work in a big company with
many ML engineers @ixek bit.ly/PyConDE-mlops
Build your own MLOps Platform @ixek bit.ly/PyConDE-mlops
None
None
Practical steps @ixek bit.ly/PyConDE-mlops
We have the notebooks in source control @ixek bit.ly/PyConDE-mlops
Your saviour Source control • Code and comments only (not
Jupyter output) • Plus every part of the pipeline • And Infrastructure and dependencies • And maybe a subset of data @ixek bit.ly/PyConDE-mlops
Everything should be in source control!! Except your training data
which should be a known, shared data source Do not touch the raw data! Not even with a stick Your saviour @ixek bit.ly/PyConDE-mlops
Deterministic environments @ixek bit.ly/PyConDE-mlops
Whatever that environment is @ixek bit.ly/PyConDE-mlops
Your laptop is not a production environment… so ensure reproducibility
@ixek bit.ly/PyConDE-mlops
@ixek bit.ly/PyConDE-mlops
Use pipelines for repeatability and reproducibility @ixek bit.ly/PyConDE-mlops
ml.azure.com
@ixek bit.ly/PyConDE-mlops
@ixek bit.ly/PyConDE-mlops
Automate wisely @ixek bit.ly/PyConDE-mlops
Adopt automation • Orchestration for Continuous Integration and Continuous Delivery
• Gates, tasks, and processes for quality • Integration with other services • Triggers on code and non-code events @ixek bit.ly/PyConDE-mlops
Complete pipeline @ixek bit.ly/PyConDE-mlops
Kubeflow example https://www.kubeflow.org/docs/azure/azureendtoend/ @ixek bit.ly/PyConDE-mlops
Build pipeline- https://azure.microsoft.com/en-us/services/devops/https://azure.microsoft.com/e n-us/services/devops/
Code event trigger @ixek bit.ly/PyConDE-mlops
Release / deploy @ixek bit.ly/PyConDE-mlops
In brief Deterministic environments Use pipelines Continuous integration and delivery
Source control (done right) Code, infrastructure, everything! Ensure production readiness For repeatable workflows Detect errors early and seamless deployments @ixek bit.ly/PyConDE-mlops
Want to learn more? • ml.azure.com • https://azure.microsoft.com/en-us/services/devops/ • https://docs.microsoft.com/en-us/azure/machine-learning/ser
vice/concept-ml-pipelines @ixek bit.ly/PyConDE-mlops
Come talk to us! @ ixek
[email protected]
More Decks by Tania Allard
See All by Tania Allard
Keeping Research Software Relevant for Tomorrow
trallard
0
100
2024_pydata_lndn.pdf
trallard
1
360
The RSE hiring and career progression pipelines: Top tips to navigate them efficiently
trallard
0
430
Mentored Sprints - 2023
trallard
0
360
Mentored Sprints 2022 - kickoff
trallard
3
410
Como participar en el mercado emergente del codigo abierto
trallard
4
420
El presente y futuro del computo cientifico con Python
trallard
0
390
Foss for fun and profit
trallard
3
450
Open source for fun and profit: rethinking the long road of sustainability.
trallard
0
300
Other Decks in Programming
See All in Programming
動画プレイヤーはなぜカクつくのか 〜Flutterのレンダリングとネイティブの境界を解剖する〜
naruogram
0
340
複数ドメインを一つのアプリに統合するFlutterの設計と運用_FlutterKaigi2026登壇資料(須永高弘)
takahirosunaga
0
190
Materialが消えた日 〜真っ白なFlutterキャンバスにRenderObjectで絵を描く〜
umigishiaoi
0
170
Augmenting AI with the Power of Jakarta EE
ivargrimstad
0
360
Spec-driven Development with AI
simas
PRO
0
2.3k
Apple Intelligenceとオンデバイス時代の到来
yahir0
0
250
そのデザイントークン、Flutterでも同じ見た目になりますか?
nakamizu
0
400
AIがコードを書いてくれる時代、我々は何を確かめるのか
yosuke_furukawa
PRO
13
6k
TanStack Vue Query と Pinia Colada から学ぶ非同期状態の型設計
aoshi
0
680
Spec-Driven Development - Why Vaadin Is the Perfect Fit
simas
PRO
0
250
Beyond Chatbots: Agentic UI with Open Standards @W-JAX 2026, München
manfredsteyer
PRO
0
110
Engineering Biometric UX Across Android Versions | DroidKnights 2026
ryunen344
1
210
Featured
See All Featured
Introduction to Domain-Driven Design and Collaborative software design
baasie
1
1.1k
Git: the NoSQL Database
bkeepers
PRO
433
67k
Ten Tips & Tricks for a 🌱 transition
stuffmc
1
280
A better future with KSS
kneath
240
18k
Why You Should Never Use an ORM
jnunemaker
PRO
61
10k
Tell your own story through comics
letsgokoyo
1
1.1k
Navigating the Design Leadership Dip - Product Design Week Design Leaders+ Conference 2024
apolaine
2
520
The Anti-SEO Checklist Checklist. Pubcon Cyber Week
ryanjones
0
300
Effective software design: The role of men in debugging patriarchy in IT @ Voxxed Days AMS
baasie
1
610
Highjacked: Video Game Concept Design
rkendrick25
PRO
2
580
Optimizing for Happiness
mojombo
378
71k
Groundhog Day: Seeking Process in Gaming for Health
codingconduct
0
490