Kubeflow: portable and scalable machine learning using Jupyterhub and Kubernetes [PyData Delhi 2018]

ML solutions in production start from data ingestion and extend upto the
actual deployment step. We want this workflow to be scalable, portable
and simple. Containers and kubernetes are great at the former two but not
the latter if you aren't a devops practitioner. We'll explore how you can
leverage the Kubeflow project to deploy best-of-breed open-source systems
for ML to diverse infrastructures.

Transcript

1. Kubeflow: scalable and portable ML Akash Tandon, Data engineering@SocialCops Github: @analyticalmonk Twitter: @AkashTandon
2. Agenda - Need of DevOps for ML and Data Science (DataOps) - Containers and Kubernetes for ML - Opportunities and challenges - Kubeflow: composable, portable and scalable ML - Components - Low bar, high ceiling - Issues and roadmap - Summary and demo
3. Current ML workflow What you think
4. Current ML workflow The reality Source: https://papers.nips.cc/paper/5656-hidden-technical-debt-in-machine-learning-systems.pdf
5. DataOps - DevOps in Data Science and ML DataOps is an automated, process-oriented methodology, used by analytic and data teams to improve the quality and reduce the cycle time of data analytics. DataOps manifesto: http://dataopsmanifesto.org
6. DataOps - DevOps in Data Science and ML
7. We need tools that are great at DevOps
8. Enter containers and Kubernetes
9. Containers • Containers allow you to easily package an application's code, configurations, and dependencies into easy to use building blocks. • These building blocks deliver environmental consistency, operational efficiency, developer productivity, and version control. • To put it simply, your code runs in any environment!
10. But managing multiple containers can be a pain. That’s where K8s steps in.
11. Kubernetes • Kubernetes is an orchestration manager for containers. • It orchestrates computing, network and storage. • Simply put, it makes your life easier when working with containers.
12. Sample K8s manifest
13. But there’s a catch.
14. Steep DevOps learning curve • Containers • Kubernetes primitives • Persistent storage • APIs • Cloud platforms • and it goes on...
15. DevOps practitioners don’t know enough Data Science. Data Scientists don’t know enough DevOps. And we don’t want them to!
16. How do we get DevOps goodness Without driving data teams crazy?!
17. Enter Kubeflow
18. Kubeflow • ML toolkit for Kubernetes • Open-source and community-driven • Support for multiple ML frameworks • End-to-end workflows which can be shared, scaled and deployed Source: https://github.com/kubeflow/kubeflow/issues/187
19. Low bar, high ceiling • Low bar: allow data science practitioners to get up and running on Kubernetes cluster even without DevOps know-how. • High ceiling: allow sysdmins and DevOps practitioners to modify defaults and extend the framework as needed.
20. Components • Jupyterhub (collaboration and interactivity) • K8s- native tensorflow controller (model building) • K8s- native tensorflow serving deployment (model deployment) • Ambassador (reverse proxy) • Current and upcoming components for model tuning, model building and much more... • Out-of-the-box setup for putting all of this together!
21. Jupyterhub
22. Tensorflow - Open source numerical computing and ML - Developed by Google, open-sourced in 2015 - Huge community and ecosystem - Support for multiple ML models - Tf-serving (model deployment), tensorboard (training visualization), etc. - Supports distributed training and deployment of models
23. Why Kubeflow? Based on current functionality you should consider using Kubeflow if: • You want to train/serve TensorFlow models in different environments (e.g. local, on prem, and cloud) • You want to use Jupyter notebooks to manage TensorFlow training jobs • You want to launch training jobs that use resources – such as additional CPUs or GPUs – that aren’t available on your personal computer • You want to combine TensorFlow with other processes ◦ For example, you may want to use tensorflow/agents to run simulations to generate data for training reinforcement learning models. Refer https://www.kubeflow.org/docs/started/getting-started/ for more info.
24. Demo - Kubeflow tutorial using a sequence-to-sequence model - Based on Hamel Husain’s wonderful post: How to create data products that are magical using sequence-to-sequence models - Github repo: https://github.com/kubeflow/examples/tree/master/github_issue_summarization - Let’s get started!
25. Demo
26. Road ahead - Get the entry (bar)rier lower - Multi-tenancy on Kubernetes - Support for different ML libraries/packages - PyTorch - Caffe2 - Mxnet - v1.0 to be launched by December 2018
27. Find out more - Official website: https://www.kubeflow.org/ - Github: https://github.com/kubeflow/kubeflow - Katacoda tutorials: https://www.katacoda.com/kubeflow/
28. Reach out at Email: akashtndn.acm@gmail.com, akash@socialcops.com Twitter: @AkashTandon Github: @analyticalmonk
29. is hiring! https://socialcops.com/careers/
30. That’s all, folks! Questions?!