Kubeflow: Tus herramientas de ML en Kubernetes

Las herramientas de ML están cada vez más presentes y nos ayudan a resolver más problemas en nuestro día a día. Por ello necesitamos ser capaces de montar infraestructuras que simplifiquen el flujo de trabajo de científicos de datos y admiradores de estas tecnologías.

En esta charla veremos cómo Kubeflow nos permite ejecutar y desplegar diferentes de esas herramientas de una forma fácil y rápida en un cluster de Kubernetes. De esta forma podremos aprovechar la portabilidad y diversidad de hardware que Kubernetes nos provee y utilizarlo para entrenar y servir nuestros modelos.

Transcript

1. Kubeflow Escalando tus modelos de TensorFlow con Kubernetes Laura Morillo-Velarde Iván Cañaveral
2. Laura Morillo-Velarde • Tech Lead at seedtag • GDG Madrid Organizer / WTM Lead • GDE Google Cloud • 11 years working with different technologies • Twitter: @Laura_Morillo
3. Iván Cañaveral • Data scientist at seedtag • Math lover • :) poner foto de verdad
4. ML Code Development Production Serving Training at scale Data transformation Data analysis Monitoring Logging
5. ML Code Development Production Serving Training at scale Data transformation Data analysis Monitoring Logging
6. Kubernetes
7. Cluster Kubernetes Node 1 (VM or physical machine) Node 2 (VM or physical machine) Node 3 (VM or physical machine)
8. KubeFlow
9. Project to make deployment of machine learning on Kubernetes simple, portable and scalable. Based on ksonnet. Kubeflow
10. Ksonnet
11. A framework for extensible Kubernetes configurations. Ksonnet
12. Ksonnet
13. Kubeflow + ksonnet TensorFlow Training Prototype (kubeflow) TensorFlow Serving Prototype (kubeflow) JupyterHub Prototype (kubeflow)
14. TensorFlow
15. TensorFlow Machine Learning framework, primarily conceived for deep neural network models. It provides several high level APIs to build complex and powered-up estimators fairly accessible to developers. It also packs a variety of tools for serving, logging and tracking the performance of the models.
16. training structure distributed strategies distributed tensorflow TFjob
17. [training] distributed basics
18. [training] clusters, jobs, tasks Cluster
19. [training] clusters, jobs, tasks task 0 task 1 task 2 task 3 Cluster
20. Vestibulum congue tempus [training] clusters, jobs, tasks Job 2 Job 1 task 0 task 1 task 2 task 3 Cluster
21. [training] parameter servers and workers Workers Lorem ipsum dolor sit dolor amet, consectetur nec adipiscing elit, sed do ipsum eiusmod tempor. Donec facilisis lacus eget sit nec lorem mauris. Parameter servers Lorem ipsum dolor sit dolor amet, consectetur nec adipiscing elit, sed do ipsum eiusmod tempor. Donec facilisis lacus eget sit nec lorem mauris.
22. [training] cluster spec and servers cluster_spec = tf.train.ClusterSpec({ "ps": [ "127.0.0.1:2221", # /job:ps/task:0 ], "worker": [ "127.0.0.1:2223", # /job:worker/task:0 "127.0.0.1:2224", # /job:worker/task:1 ]}) The cluster specification dictionary maps job names to lists of network addresses. A ClusterSpec represents the set of processes that participate in a distributed TensorFlow computation.
23. [training] cluster spec and servers server = tf.train.Server( cluster_spec, job_name="worker", task_index=0 ) A Server object contains a set of local devices, a set of connections to other tasks in its ClusterSpec, and a Session that can use these to perform a distributed computation. Each server is a member of a specific named job and has a task index within that job. A server can communicate with any other server in the cluster.
24. [training] cluster spec and servers
25. [training] distributing the work
26. [training] distributing the work
27. [training] distributing the work
28. [training] distributing the work
29. [training] distributing the work
30. [training] distributing the work
31. [training] distributing the work
32. [training] distributing the work parameter server
33. [training] distributing the work parameter server
34. [training] distributing the work parameter server
35. [training] distributing the work
36. [training] distributing the work
37. [training] distributed tensorflow
38. [training] distributed training with tensorflow distributed TensorFlow architecture m Lorem ipsum dolor w2 Lorem ipsum dolor ps Lorem ipsum dolor ps Lorem ipsum dolor w1 Lorem ipsum dolor Lorem ipsum dolor
39. [training] distributed training with tensorflow
40. [training] kubeflow
41. [training] TFJob apiVersion: kubeflow.org/v1alpha2 kind: TFJob metadata: name: my-tfjob spec: tfReplicaSpecs: Worker: replicas: 2 template: spec: containers: - image: repo/image name: tensorflow restartPolicy: OnFailure kind: TFJob tfReplicaSpecs: TfReplicaType WORKER, PS replicas: number of replicas for this job template: as usual
42. [training] TF_CONFIG { "master":[ "distributed-mnist-master-5oz2-0:2222" ], "ps":[ "distributed-mnist-ps-5oz2-0:2222" ], "worker":[ "distributed-mnist-worker-5oz2-0:2222", "distributed-mnist-worker-5oz2-1:2222" ] } cluster_spec = ...
43. [training] up and running
44. serving TensorFlow Serving Docker Kubernetes/Kubeflow
45. [serving] tf serving
46. [serving] tfserving def myModel(x,y): compute things return value
47. [serving] tfserving def myModel(x,y): compute things return value
48. [serving] tfserving def myModel(x,y): compute things return value
49. [serving] docker and local server TensorFlow Serving TensorFlow Serving is a flexible, high-performance serving system for machine learning models, designed for production environments. TensorFlow Serving makes it easy to deploy new algorithms and experiments, while keeping the same server architecture and APIs. TensorFlow Serving provides out-of-the-box integration with TensorFlow models, but can be easily extended to serve other types of models and data.
50. [serving] $ ks generate tf-serving dino-serving --name=trex $ ks param set dino-serving modelPath gs://demo-dino $ ks param set dino-serving deployHttpProxy true
51. [serving] $ ks param set dino-serving serviceType LoadBalancer
52. [serving] now in kubernetes/kubeflow
53. Resources: • https://www.kubeflow.org/ • https://github.com/kubeflow/kubeflow • https://www.tensorflow.org/extend/architecture
55. Thanks! Laura Morillo-Velarde: @Laura_Morillo laura.morillovelarde@gmail.com