Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Practical Devops for the busy data scientist: Alice in DataOps land

Practical Devops for the busy data scientist: Alice in DataOps land

Everyone uses the buzzword DevOps. Pretty much every company has hired or is about to hire a DevOps engineer or specialist. But how does this translates to Machine learning? What is more, how does this relate to your models' development, training, and deployment?

In this talk, we will debunk some of the DevOps terms and myths. The talk will include Machine learning examples and terms will be explained in a suitable way for data scientists/machine learners/ anyone willing to understand from scratch what is DevOps and how to use it.


Tania Allard

July 16, 2019

More Decks by Tania Allard

Other Decks in Technology


  1. Tania Allard, PhD @ixek Developer Advocate @Microsoft Google Developer expert

    ML -Tensorflow Practical DevOps for the busy Data Scientist OSCON 2019 http://bit.ly/OSCON-MLOps
  2. 2 Story time…. @ixek Down the rabbit hole

  3. 3 @ixek

  4. 4 A common story... @ixek Model / application to be

    productised R&D - develop, iterate fast, usually local or cloud Magic Is it live??
  5. 5 @ixek

  6. 6 Replacing the magic @ixek Model/app ready to productise R&D

    - develop, iterate fast, usually local or cloud MLOPs, automation, controlled deployment Worry free deployment! Wait and relax
  7. 7 @ixek How skills are perceived

  8. 8 @ixek

  9. 9 @ixek

  10. 10 DevOps is the union of people, process, and products

    to enable continuous delivery of value into production What is DevOps anyway? @ixek
  11. 11 Sort of DevOps applied to data-intensive applications. Requires close

    collaboration between engineers, data scientists, architects, data engineers and Ops. How does it fit for DS? @ixek
  12. 12 Story time…. The advice… getting started with MLOps

  13. 13 @ixek MlOps Aims to reduce the end-to-end cycle time

    of data analytics/science from the origin of ideas to the creation of data artifacts.
  14. 14 @ixek What to automate? Establish checkpoints Find the low

    hanging fruits How stable and robust are my processes? Devise a long term strategy What can I readily improve? Where am I? Can I count? Getting started
  15. 15 It’s all madness @ixek

  16. None
  17. None
  18. 18 @ixek Practical steps

  19. 19 Keep everything in source control (data, code, infrastructure) -

    but allow for experimentation @ixek
  20. 20 @ixek

  21. 21 @ixek

  22. 22 Standardize and define your environments in code (conda, pipfiles,

    Docker) @ixek
  23. 23 Use canonical data sources - always know what data

    you are using (where it comes and goes) @ixek
  24. 24 @ixek

  25. 25 Automate wisely @ixek

  26. 26 What and when to automate? @ixek • What should

    we automate? • Define success and failure metrics • Go from simple to complex tasks • Evaluate and monitor
  27. 27 https://xkcd.com/1205/

  28. 28 @ixek

  29. 29 Use pipelines for repeatability and explainability @ixek

  30. None
  31. None
  32. 32 Deploy portable models @ixek

  33. 33 @ixek

  34. 34 Test continuously and monitor production: push left @ixek

  35. 35 @ixek

  36. 36 Summary @ixek 1. DataOps help create value and improve

    end-to-end ML 2. Start by identifying the low-hanging fruits and defining automation success 3. Choose the right tooling and processes 4. Leverage people and processes 5. Implement wisely
  37. 37 Thank you @ixek http://bit.ly/OSCON-MLOps