DevOps for Data Science

Slides of the talk presented at ODSC India 2018.

Transcript

1. DevOps for Data Science Experiences from building a cloud-based data science pla6orm ... Anand Chi)pothu rorodata
2. Who is Speaking? Anand Chi)pothu @anandology Co-founder and pla.orm architect of @rorodata Worked at Internet Archive & Strand Life Sciences Advanced programming courses at @pipalacademy
3. DevOps for Data Science
4. Managing Data Science in Produc1on is Hard! • The tools and prac0ces are not very mature • Everyone ends up building their own solu0ons • Building own solu0ons require careful system architecture and complex devops
5. The Goal: The Effec-ve Data Science Team • The data science team is self-sufficient to build end-to-end ML applica8ons • Not steep learning curve
6. How? ???
7. Inspira(on from Origami
8. Flapping Bird - Crease Pa1erns
9. Take Aways • Focus on the experience • Right-level of abstrac8on
12. Example: Heroku To deploy a web applica.on: $ git push heroku master
13. Case Studies • Lauching notebooks • Deploy ML models as an APIs
14. Case Study 1: Launching Notebooks
15. Launching Notebooks - Challenges • Switching between different compute needs • Installing required so9ware dependencies • Data storage • GPU support
16. Abstrac(ons • Project • Run-me • Instance Size
17. Project Manages all the notebooks, data and the specified so4ware dependencies.
18. Run$me Base so'ware setup for the project. • python3-tensorflow • python3-keras • python3-pytorch
19. Instance Size Different instance sizes to pick from: • S1 - 1 CPU core, 1 GB RAM • S2 - 1 CPU core, 3.5 GB RAM • M1 - 2 CPU core, 15 GB RAM • X1 - 64 CPU core, 1024 GB RAM • G1 - 4 CPU cores, 60GB RAM, 1 K100 GPU
20. How to specify addi/onal dependencies? • runtime.txt: specifies the run.me • environment.yml: conda environment file with python dependencies • requirements.txt: python dependencies to be installed from pip • apt.txt: system packages that need to be installed • postBuild: script for custom needs Wri$ng a Dockerfile is too low-level.
21. Behind the Scenes • Two docker images are built for each project - One for CPU and another for GPU • Run@mes are also built using the same approach • Manages compute instances • Pools the compute resources to op@mize resource consump@on • Uses a network file system to persist data and notebooks • Automa@c endpoint and HTTPS management
22. Sample Usage (1/4)
23. Sample Usage (2/4)
24. Sample Usage (3/4)
25. Sample Usage (4/4)
26. Discussion
27. Case Study 2: Deploying Machine Learning Models
28. Challenges • Designing and documen/ng APIs • Running the service and configuring URL endpoints • Scale to meet the usage • Client library to use the API • Authen/ca/on • Tracking the usage and performance
29. Tradi&onal Approach
30. Observa(ons Running a Python func/on as an API is hard! It doesn't have to be!!
31. Right-level of Abstrac3on! Step 1: Write your func1on # sq.py def square(n): """compute square of a number. """ return n*n
32. Step 2: Run it as an API $ firefly sq.square http://127.0.0.1:8000/ ... Firefly is the open-source library that we built to solve that problem.
33. Step 3: Use it >>> import firefly >>> api = firefly.Client("http://127.0.0.1:8000/") >>> api.square(n=4) 16 Out-of the box client library to access the API.
34. What about ML models? Write your predict func-on and run it as API. # face_detection.py import joblib model = joblib.load("model.pkl") def predict(image): ...
35. Integra(on Write a config file in the project specifying what services to run. services: - name: api function: face_detection.predict size: S2
36. Need Javascript support? services: - name: api function: face_detection.predict size: S2 cors_allow_origins: *
37. Need to scale up? services: - name: api function: face_detection.predict size: S2 cors_allow_origins: * scale: 4
38. The Push Bu)on The deploy command submits the code to pla3orm and it starts the requires services in that project. $ roro deploy ...
39. Behind the Scenes • It builts the docker images • Starts the specified services • Provides URL endpoints with HTTPS
40. Discussion
41. Good Design is Invisible!
42. Summary • Making the data science team self-sufficient is key to their produc9vity • Op9mize for developer experience • Right-level of abstrac9on is the key!
43. Thank You! Anand Chitipothu @anandology https://rorodata.com/