Portable Machine Learning Deployments in Cloud

One of the most common hurdles with developing data science/machine learning models is to design end-to-end pipelines that can operate at scale and in real-time. Data scientists and engineers are often expected to learn, develop and maintain the infrastructure for their experiments. This process takes time away from focussing on training and developing the models.

What if there was a way of abstracting away the non Machine Learning related tasks while still retaining control? This talk will discuss the merits of using Kubeflow. Kubeflow is an open source Kubernetes based platform. With the help of Kubeflow, users can:
- Develop Machine Learning models easily and make repeatable, portable deployments on a diverse infrastructure e.g. laptop to production cluster.
- Scale infrastructure based on the demand.

This talk will also present the current use cases of Kubeflow and how teams from other industries have been utilising the cloud to scale their machine learning operations.

Transcript

12. GPT3 DALLE CLIP OpenAI
13. !!" Postgres SQL with properties # Employee(id, name, department_id) # Department(id, name, address) # Salary_Payments(id, employee_id, amount, date) !!" A query that lists names of departments which employed > 10 employees in the last 3 months
14. !!" Postgres SQL with properties # Employee(id, name, department_id) # Department(id, name, address) # Salary_Payments(id, employee_id, amount, date) !!" A query that lists names of departments which employed > 10 employees in the last 3 months GPT3
15. GPT3
16. DISTINCT depart.name FROM department JOIN employee ON department.id = employee.department_id JOIN salary_payments ON employee.id = salary_payments.employee_id WHERE salary_payments.date > (CURRENT_DATA - INTERVAL ‘3 months’) GROUP BY department.name HAVING COUNT(employee.id)>10; GPT3
17. !!" Convert movie titles into emoji Back to the Future: !"#$ Batman: !% Transformers: &' Star Wars: GPT3
18. GPT3
20. Machine Learning on Kub rnetes Machine Learning on Kub rnetes soulmaniqbal
21. Salman Iqbal Instructor MLOps Engineer soulmaniqbal soulman iqbal
22. soulman iqbal
23. Thank you KCDC! Thank you KCDC! Jeff Strauss @jeffreystrauss Heather Downing @quorralyne
25. 37,903 37,903
26. MONOLITH
30. AUTH API WORKER FRONTEND
32. CentOS
33. CentOS
34. CentOS
35. CentOS CentOS CentOS CentOS
36. CentOS CentOS CentOS CentOS CentOS CentOS CentOS CentOS CentOS CentOS CentOS CentOS CentOS CentOS CentOS CentOS
37. Containers
42. 9 Application 1 10 Application 2 SPRING BOOT SPRING BOOT
43. Containers
45. PY TXT +
46. PY TXT + +
47. PY TXT + + CONTAINER
50. GPU
51. GPU ?
54. ?
69. Machine Learning
70. Recommendation Antispam Churn rate
71. Recommendation Antispam Churn rate Recommendation v1
72. Recommendation Antispam Churn rate Recommendation v1 Recommendation v2
73. Recommendation Antispam Churn rate Recommendation v1 Recommendation v2 Recommendation v3
74. Recommendation Antispam Churn rate Recommendation v1 Recommendation v2 Recommendation v3 Recommendation v4
75. Recommendation Antispam Churn rate v1 Recommendation v1 Recommendation v2 Recommendation v3 Recommendation v4 Antispam Antispam v2 Antispam v3 Antispam v4
76. Churn rate v1 Churn rate v2 Churn rate v3 Churn rate v4 Recommendation Antispam Churn rate v1 Recommendation v1 Recommendation v2 Recommendation v3 Recommendation v4 Antispam Antispam v2 Antispam v3 Antispam v4
77. Churn rate v1 Churn rate v2 Churn rate v3 Churn rate v1 Recommendation Antispam Churn rate v1 Recommendation v1 Recommendation v2 Recommendation v3 Recommendation v4 Antispam Antispam v2 Antispam v3 Antispam v4 Monday
78. Churn rate v1 Churn rate v2 Churn rate v3 Churn rate v1 Recommendation Antispam Churn rate v1 Recommendation v1 Recommendation v2 Recommendation v3 Recommendation v4 Antispam Antispam v2 Antispam v3 Antispam v4 Churn rate v1 Churn rate v2 Churn rate v3 Churn rate v1 Recommendation Antispam Churn rate v1 Recommendation v1 Recommendation v2 Recommendation v3 Recommendation v4 Antispam Antispam v2 Antispam v3 Antispam v4 Monday Tuesda y
79. Churn rate v1 Churn rate v2 Churn rate v3 Churn rate v1 Recommendation Antispam Churn rate v1 Recommendation v1 Recommendation v2 Recommendation v3 Recommendation v4 Antispam Antispam v2 Antispam v3 Antispam v4 Churn rate v1 Churn rate v2 Churn rate v3 Churn rate v1 Recommendation Antispam Churn rate v1 Recommendation v1 Recommendation v2 Recommendation v3 Recommendation v4 Antispam Antispam v2 Antispam v3 Antispam v4 Churn rate v1 Churn rate v2 Churn rate v3 Churn rate v1 Recommendation Antispam Churn rate v1 Recommendation v1 Recommendation v2 Recommendation v3 Recommendation v4 Antispam Antispam v2 Antispam v3 Antispam v4 Monday Tuesda y Wednesday
80. Churn rate v1 Churn rate v2 Churn rate v3 Churn rate v1 Recommendation Antispam Churn rate v1 Recommendation v1 Recommendation v2 Recommendation v3 Recommendation v4 Antispam Antispam v2 Antispam v3 Antispam v4 Churn rate v1 Churn rate v2 Churn rate v3 Churn rate v1 Recommendation Antispam Churn rate v1 Recommendation v1 Recommendation v2 Recommendation v3 Recommendation v4 Antispam Antispam v2 Antispam v3 Antispam v4 Churn rate v1 Churn rate v2 Churn rate v3 Churn rate v1 Recommendation Antispam Churn rate v1 Recommendation v1 Recommendation v2 Recommendation v3 Recommendation v4 Antispam Antispam v2 Antispam v3 Antispam v4 Churn rate v1 Churn rate v2 Churn rate v3 Churn rate v1 Recommendation Antispam Churn rate v1 Recommendation v1 Recommendation v2 Recommendation v3 Recommendation v4 Antispam Antispam v2 Antispam v3 Antispam v4 Monday Tuesda y Wednesday Thursda y
81. Churn rate v1 Churn rate v2 Churn rate v3 Churn rate v1 Recommendation Antispam Churn rate v1 Recommendation v1 Recommendation v2 Recommendation v3 Recommendation v4 Antispam Antispam v2 Antispam v3 Antispam v4 Churn rate v1 Churn rate v2 Churn rate v3 Churn rate v1 Recommendation Antispam Churn rate v1 Recommendation v1 Recommendation v2 Recommendation v3 Recommendation v4 Antispam Antispam v2 Antispam v3 Antispam v4 Churn rate v1 Churn rate v2 Churn rate v3 Churn rate v1 Recommendation Antispam Churn rate v1 Recommendation v1 Recommendation v2 Recommendation v3 Recommendation v4 Antispam Antispam v2 Antispam v3 Antispam v4 Churn rate v1 Churn rate v2 Churn rate v3 Churn rate v1 Recommendation Antispam Churn rate v1 Recommendation v1 Recommendation v2 Recommendation v3 Recommendation v4 Antispam Antispam v2 Antispam v3 Antispam v4 Churn rate v1 Churn rate v2 Churn rate v3 Churn rate v1 Recommendation Antispam Churn rate v1 Recommendation v1 Recommendation v2 Recommendation v3 Recommendation v4 Antispam Antispam v2 Antispam v3 Antispam v4 Monday Tuesda y Wednesday Thursda y Friday
82. Churn rate v1 Churn rate v2 Churn rate v3 Churn rate v1 Recommendation Antispam Churn rate v1 Recommendation v1 Recommendation v2 Recommendation v3 Recommendation v4 Antispam Antispam v2 Antispam v3 Antispam v4 Churn rate v1 Churn rate v2 Churn rate v3 Churn rate v1 Recommendation Antispam Churn rate v1 Recommendation v1 Recommendation v2 Recommendation v3 Recommendation v4 Antispam Antispam v2 Antispam v3 Antispam v4 Churn rate v1 Churn rate v2 Churn rate v3 Churn rate v1 Recommendation Antispam Churn rate v1 Recommendation v1 Recommendation v2 Recommendation v3 Recommendation v4 Antispam Antispam v2 Antispam v3 Antispam v4 Churn rate v1 Churn rate v2 Churn rate v3 Churn rate v1 Recommendation Antispam Churn rate v1 Recommendation v1 Recommendation v2 Recommendation v3 Recommendation v4 Antispam Antispam v2 Antispam v3 Antispam v4 Churn rate v1 Churn rate v2 Churn rate v3 Churn rate v1 Recommendation Antispam Churn rate v1 Recommendation v1 Recommendation v2 Recommendation v3 Recommendation v4 Antispam Antispam v2 Antispam v3 Antispam v4 Churn rate v1 Churn rate v2 Churn rate v3 Churn rate v1 Recommendation Antispam Churn rate v1 Recommendation v1 Recommendation v2 Recommendation v3 Recommendation v4 Antispam Antispam v2 Antispam v3 Antispam v4 Monday Tuesda y Wednesday Thursda y Friday Saturday
83. Churn rate v1 Churn rate v2 Churn rate v3 Churn rate v1 Recommendation Antispam Churn rate v1 Recommendation v1 Recommendation v2 Recommendation v3 Recommendation v4 Antispam Antispam v2 Antispam v3 Antispam v4 Churn rate v1 Churn rate v2 Churn rate v3 Churn rate v1 Recommendation Antispam Churn rate v1 Recommendation v1 Recommendation v2 Recommendation v3 Recommendation v4 Antispam Antispam v2 Antispam v3 Antispam v4 Churn rate v1 Churn rate v2 Churn rate v3 Churn rate v1 Recommendation Antispam Churn rate v1 Recommendation v1 Recommendation v2 Recommendation v3 Recommendation v4 Antispam Antispam v2 Antispam v3 Antispam v4 Churn rate v1 Churn rate v2 Churn rate v3 Churn rate v1 Recommendation Antispam Churn rate v1 Recommendation v1 Recommendation v2 Recommendation v3 Recommendation v4 Antispam Antispam v2 Antispam v3 Antispam v4 Churn rate v1 Churn rate v2 Churn rate v3 Churn rate v1 Recommendation Antispam Churn rate v1 Recommendation v1 Recommendation v2 Recommendation v3 Recommendation v4 Antispam Antispam v2 Antispam v3 Antispam v4 Churn rate v1 Churn rate v2 Churn rate v3 Churn rate v1 Recommendation Antispam Churn rate v1 Recommendation v1 Recommendation v2 Recommendation v3 Recommendation v4 Antispam Antispam v2 Antispam v3 Antispam v4 Churn rate v1 Churn rate v2 Churn rate v3 Churn rate v1 Recommendation Antispam Churn rate v1 Recommendation v1 Recommendation v2 Recommendation v3 Recommendation v4 Antispam Antispam v2 Antispam v3 Antispam v4 Monday Tuesda y Wednesday Thursda y Sunday Friday Saturday
85. Training Data
86. Training Data Test Data
87. Training Data Test Data Model
88. Training Data Test Data Model Evaluate
89. Training Data Test Data Model Evaluate
90. Training Data Test Data Model Evaluate Data collection
91. Training Data Test Data Model Evaluate Data collection Feature extraction
92. Training Data Test Data Model Evaluate Data collection Feature extraction Data verification
93. Training Data Test Data Model Evaluate Data collection Configuration Feature extraction Data verification
94. Training Data Test Data Model Evaluate Data collection Configuration Feature extraction Data verification Machine resource management
95. Training Data Test Data Model Evaluate Data collection Configuration Feature extraction Data verification Serving Infrastructure Machine resource management
96. Training Data Test Data Model Evaluate Data collection Configuration Feature extraction Data verification Serving Infrastructure Monitoring Machine resource management
97. Training Data Test Data Model Evaluate Data collection Configuration Feature extraction Data verification Serving Infrastructure Monitoring Analysis tools Machine resource management
99. ML  K8s
100. Self-Healing
101. Ingress Incoming traffic Pod 1 Pod 2 Pod 3 Pod 4 Pod 5 Service
102. Ingress Incoming traffic Pod 1 Pod 2 Pod 3 Pod 4 Pod 5 Service
103. Ingress Incoming traffic Pod 1 Pod 3 Pod 4 Pod 5 Service
104. Ingress Incoming traffic Pod 1 Pod 6 Pod 3 Pod 4 Pod 5 Service
106. Smart Scheduling
107. LOW PRIORITY
108. LOW PRIORITY
109. LOW PRIORITY
110. GPU LOW PRIORITY
111. GPU LOW PRIORITY
112. LOW GPU LOW PRIORITY
113. GPU LOW PRIORITY LOW
114. Cluster Autoscaler
125. Pod Autoscaler
126. Deployment
127. Deployment Horizontal Pod Autoscaler
128. Deployment Query Horizontal Pod Autoscaler
129. Deployment Query Calculate Horizontal Pod Autoscaler
130. Deployment Query Calculate Adjust Horizontal Pod Autoscaler
131. Deployment Query Calculate Adjust Horizontal Pod Autoscaler
132. Deployment Query Calculate Adjust Horizontal Pod Autoscaler
133. ML  K8s
139. T-shirt Model
140. T-shirt
141. T-shirt
142. T-shirt
144. https://www.semanticscholar.org/paper/An-intelligent-ETL-workflow-framework-based-on-data-Tu-Guo/e206246e928fc0f78dd8bf6ea0571b17b9626325/figure/1
146. https://towardsdatascience.com/pipeline-oriented-data-analytics-with-spark-ml-c664befe1c2d
164. Kubeflow Pipelines
165. Kubeflow Pipelines Multi-Step ML workflow Python SDK Productionise
166. Kubeflow Pipelines Multi-Step ML workflow Python SDK Productionise
167. Kubeflow Pipelines Multi-Step ML workflow Python SDK Productionise
169. WRITE CODE WRITE DSL KFP COMPILE DSL KFP UPLOAD PIPELINE TO KF RUN PIPELINE CREATE DOCKER IMAGE
170. Kubernetes Cluster
172. T-shirt
173. T-shirt
174. T-shirt
176. Model Serving
177. Ankle boot
178. BACKEND API
179. BACKEND API
180. Kubernetes Models
181. Kubernetes Istio Models
182. Kubernetes Istio Knative Models
183. Kubernetes KFServing Istio Knative Models
184. Kubernetes Operators
185. Resources Pod
186. Resources Pod Deployment
187. Resources Pod Job Deployment CronJob ReplicaSet Service Ingress ConfigMap Secret
188. Resources Pod Job Endpoint Deployment CronJob ServiceAccount ReplicaSet Role Service Ingress RoleBinding ClusterRole ConfigMap Secret
189. Resources Pod Job Endpoint Namespace Binding StorageClass Volume Deployment CronJob ServiceAccount ReplicaSet Role ResourceQuota Service Ingress RoleBinding ClusterRoleBinding ClusterRole ConfigMap Secret PersistentVolume
190. https://engineering.salesforce.com/project-agumbe-share-objects-across-namespaces-in-kubernetes-1fc2e1ddb3eb
191. ~$ cat postgres.yaml apiVersion: kubedb.com/v1alpha1 kind: Postgres metadata: name: p1 namespace: demo spec: version: 9.6.5 replicas: 1 doNotPause: true ~$ _
192. ~$ cat postgres.yaml apiVersion: kubedb.com/v1alpha1 kind: Postgres metadata: name: p1 namespace: demo spec: version: 9.6.5 replicas: 1 doNotPause: true ~$ _
193. ~$ cat postgres.yaml apiVersion: kubedb.com/v1alpha1 kind: Postgres metadata: name: p1 namespace: demo spec: version: 9.6.5 replicas: 1 doNotPause: true ~$ _
195. mxnet/python mxnet/r-lang mxnet/julia mxnet/scala mxnet/perl Docker
196. SCHEDULER
197. SCHEDULER SERVER SERVER
198. SCHEDULER SERVER SERVER
199. SCHEDULER WORKER WORKER SERVER SERVER
200. SCHEDULER WORKER WORKER WORKER WORKER SERVER SERVER
201. SCHEDULER WORKER WORKER WORKER WORKER SERVER SERVER
202. SCHEDULER WORKER WORKER WORKER WORKER SERVER SERVER
206. When not to
207. Cloud Services Complexity Auto-scaling Cost Say no to k8s
210. Summary
212. soulman iqbal soulmaniqbal
213. Thank you!