Upgrade to Pro — share decks privately, control downloads, hide ads and more …

DevOps for Data Science

DevOps for Data Science

Slides of the talk presented at ODSC India 2018.


Anand Chitipothu

August 31, 2018


  1. DevOps for Data Science Experiences from building a cloud-based data

    science pla6orm ... Anand Chi)pothu rorodata
  2. Who is Speaking? Anand Chi)pothu @anandology Co-founder and pla.orm architect

    of @rorodata Worked at Internet Archive & Strand Life Sciences Advanced programming courses at @pipalacademy
  3. DevOps for Data Science

  4. Managing Data Science in Produc1on is Hard! • The tools

    and prac0ces are not very mature • Everyone ends up building their own solu0ons • Building own solu0ons require careful system architecture and complex devops
  5. The Goal: The Effec-ve Data Science Team • The data

    science team is self-sufficient to build end-to-end ML applica8ons • Not steep learning curve
  6. How? ???

  7. Inspira(on from Origami

  8. Flapping Bird - Crease Pa1erns

  9. Take Aways • Focus on the experience • Right-level of

  10. None
  11. None
  12. Example: Heroku To deploy a web applica.on: $ git push

    heroku master
  13. Case Studies • Lauching notebooks • Deploy ML models as

    an APIs
  14. Case Study 1: Launching Notebooks

  15. Launching Notebooks - Challenges • Switching between different compute needs

    • Installing required so9ware dependencies • Data storage • GPU support
  16. Abstrac(ons • Project • Run-me • Instance Size

  17. Project Manages all the notebooks, data and the specified so4ware

  18. Run$me Base so'ware setup for the project. • python3-tensorflow •

    python3-keras • python3-pytorch
  19. Instance Size Different instance sizes to pick from: • S1

    - 1 CPU core, 1 GB RAM • S2 - 1 CPU core, 3.5 GB RAM • M1 - 2 CPU core, 15 GB RAM • X1 - 64 CPU core, 1024 GB RAM • G1 - 4 CPU cores, 60GB RAM, 1 K100 GPU
  20. How to specify addi/onal dependencies? • runtime.txt: specifies the run.me

    • environment.yml: conda environment file with python dependencies • requirements.txt: python dependencies to be installed from pip • apt.txt: system packages that need to be installed • postBuild: script for custom needs Wri$ng a Dockerfile is too low-level.
  21. Behind the Scenes • Two docker images are built for

    each project - One for CPU and another for GPU • Run@mes are also built using the same approach • Manages compute instances • Pools the compute resources to op@mize resource consump@on • Uses a network file system to persist data and notebooks • Automa@c endpoint and HTTPS management
  22. Sample Usage (1/4)

  23. Sample Usage (2/4)

  24. Sample Usage (3/4)

  25. Sample Usage (4/4)

  26. Discussion

  27. Case Study 2: Deploying Machine Learning Models

  28. Challenges • Designing and documen/ng APIs • Running the service

    and configuring URL endpoints • Scale to meet the usage • Client library to use the API • Authen/ca/on • Tracking the usage and performance
  29. Tradi&onal Approach

  30. Observa(ons Running a Python func/on as an API is hard!

    It doesn't have to be!!
  31. Right-level of Abstrac3on! Step 1: Write your func1on # sq.py

    def square(n): """compute square of a number. """ return n*n
  32. Step 2: Run it as an API $ firefly sq.square ... Firefly is the open-source library that we built to solve that problem.
  33. Step 3: Use it >>> import firefly >>> api =

    firefly.Client("") >>> api.square(n=4) 16 Out-of the box client library to access the API.
  34. What about ML models? Write your predict func-on and run

    it as API. # face_detection.py import joblib model = joblib.load("model.pkl") def predict(image): ...
  35. Integra(on Write a config file in the project specifying what

    services to run. services: - name: api function: face_detection.predict size: S2
  36. Need Javascript support? services: - name: api function: face_detection.predict size:

    S2 cors_allow_origins: *
  37. Need to scale up? services: - name: api function: face_detection.predict

    size: S2 cors_allow_origins: * scale: 4
  38. The Push Bu)on The deploy command submits the code to

    pla3orm and it starts the requires services in that project. $ roro deploy ...
  39. Behind the Scenes • It builts the docker images •

    Starts the specified services • Provides URL endpoints with HTTPS
  40. Discussion

  41. Good Design is Invisible!

  42. Summary • Making the data science team self-sufficient is key

    to their produc9vity • Op9mize for developer experience • Right-level of abstrac9on is the key!
  43. Thank You! Anand Chitipothu @anandology https://rorodata.com/