Tania Allard, PhD @ixek
Developer Advocate @Microsoft
Practical DevOps for
the busy Data
Scientist
http://bit.ly/MancML-trallard
Slide 2
Slide 2 text
2
A bit of background never hurt anyone
About us
Slide 3
Slide 3 text
3
@ixek
Slide 4
Slide 4 text
4
@ixek
Slide 5
Slide 5 text
5
Top top
view…
@ixek
Stable model/application
ready to be productised
R&D - develop, iterate fast,
usually local or cloud
Magic
Is it live??
Slide 6
Slide 6 text
6
How I
would like
everything
to work….
@ixek
It works…. now send it over to
production
R&D - develop, iterate fast,
usually local or cloud
Push code, tag, tag data*
Worry free deployment!
Wait and relax
Slide 7
Slide 7 text
7
@ixek
Slide 8
Slide 8 text
8
Slide 9
Slide 9 text
9
@ixek
DevOps / DataOps /
MLOps
Slide 10
Slide 10 text
10
DevOps is the union of people, process, and
products to enable continuous delivery of value
into production
What is
DevOps
anyway?
@ixek
Slide 11
Slide 11 text
11
Sort of DevOps applied to data-intensive
applications.
Requires close collaboration between engineers,
data scientists, architects, data engineers and
Ops.
How does it
fit for DS?
@ixek
Slide 12
Slide 12 text
12
@ixek
Aims to reduce the end-to-end cycle time of data
analytics/science from the origin of ideas to the
creation of data artifacts.
Slide 13
Slide 13 text
13
Slide 14
Slide 14 text
14
Slide 15
Slide 15 text
15
7 steps to DS
Slide 16
Slide 16 text
16
Keep everything in source control - but allow for
experimentation
Slide 17
Slide 17 text
17
Slide 18
Slide 18 text
18
Standardize and define your environments in
code (conda, pipfiles, Docker)
Slide 19
Slide 19 text
19
Use canonical data sources - always know what
data you are using (where it comes and goes)
Slide 20
Slide 20 text
20
Slide 21
Slide 21 text
21
Automate wisely
Slide 22
Slide 22 text
22
https://xkcd.com/1205/
Slide 23
Slide 23 text
23
Slide 24
Slide 24 text
24
Use pipelines for repeatability and explainability
Slide 25
Slide 25 text
25
Deploy portable models
Slide 26
Slide 26 text
26
Slide 27
Slide 27 text
27
Test continuously and monitor production: shift
left