Kubeflow - Cloud Native ML Toolkit

Kubeflow is an open source project deticated to mkaing ML workflows on cloud portable and scalabe. In this presentation, Kubeflow archiecture is presented with use cases.

Transcript

1. Kubeflow Cloud-Native ML Toolbox
2. salman iqbal @soulmaniqbal
7. Datascience Devops
8. Datascience Devops — building on
9. Datascience Devops — building things on machine learning
10. Agenda - Kubernetes - Kubeflow - Demo
12. 13 layer policy network
13. 13 layer policy network 30 million positions
14. 13 layer policy network 30 million positions 1,920 cpus & 280 gpus
15. challenges
16. 1. compile once run anywhere
17. 1. compile once run anywhere
18. 1. works on my machine
20. ◻ big data ◻ scale with team and model ◻ cost for money ◻ quick fix
21. ❌ big data ◻ scale with team and model ◻ cost for money ◻ quick fix
22. ❌ big data ❌ scale with team and model ◻ cost for money ◻ quick fix
23. ❌ big data ❌ scale with team and model ❌ cost for money ◻ quick fix
24. ❌ big data ❌ scale with team and model ❌ cost for money ✅ quick fix
25. 2. complexity
26. Training Data
27. Training Data Test Data
28. Training Data Test Data Model
29. Training Data Test Data Model Evaluate
30. Training Data Test Data Model Evaluate
31. Training Data Test Data Model Evaluate Data collection
32. Training Data Test Data Model Evaluate Data collection Feature extraction
33. Training Data Test Data Model Evaluate Data collection Feature extraction Data verification
34. Training Data Test Data Model Evaluate Data collection Configuration Feature extraction Data verification
35. Training Data Test Data Model Evaluate Data collection Configuration Feature extraction Data verification Machine resource management
36. Training Data Test Data Model Evaluate Data collection Configuration Feature extraction Data verification Serving Infrastructure Machine resource management
37. Training Data Test Data Model Evaluate Data collection Configuration Feature extraction Data verification Serving Infrastructure Monitoring Machine resource management
38. Training Data Test Data Model Evaluate Data collection Configuration Feature extraction Data verification Serving Infrastructure Monitoring Analysis tools Machine resource management
39. ◻ learn development ◻ learn ops & infra ◻ learn data science
40. ✅ learn development ◻ learn ops & infra ◻ learn data science
41. ✅ learn development ✅ learn ops & infra ◻ learn data science
42. ✅ learn development ✅ learn ops & infra ✅ learn data science
43. data scientist — expectation
44. 3. training hundreds of models
45. Recommendation Antispam Churn rate
46. Recommendation Antispam Churn rate Recommendation v1
47. Recommendation Antispam Churn rate Recommendation v1 Recommendation v2
48. Recommendation Antispam Churn rate Recommendation v1 Recommendation v2 Recommendation v3
49. Recommendation Antispam Churn rate Recommendation v1 Recommendation v2 Recommendation v3 Recommendation v4
50. Recommendation Antispam Churn rate v1 Recommendation v1 Recommendation v2 Recommendation v3 Recommendation v4 Antispam Antispam v2 Antispam v3 Antispam v4
51. Churn rate v1 Churn rate v2 Churn rate v3 Churn rate v4 Recommendation Antispam Churn rate v1 Recommendation v1 Recommendation v2 Recommendation v3 Recommendation v4 Antispam Antispam v2 Antispam v3 Antispam v4
52. Churn rate v1 Churn rate v2 Churn rate v3 Churn rate v1 Recommendation Antispam Churn rate v1 Recommendation v1 Recommendation v2 Recommendation v3 Recommendation v4 Antispam Antispam v2 Antispam v3 Antispam v4 Monday
53. Churn rate v1 Churn rate v2 Churn rate v3 Churn rate v1 Recommendation Antispam Churn rate v1 Recommendation v1 Recommendation v2 Recommendation v3 Recommendation v4 Antispam Antispam v2 Antispam v3 Antispam v4 Churn rate v1 Churn rate v2 Churn rate v3 Churn rate v1 Recommendation Antispam Churn rate v1 Recommendation v1 Recommendation v2 Recommendation v3 Recommendation v4 Antispam Antispam v2 Antispam v3 Antispam v4 Monday Tuesday
54. Churn rate v1 Churn rate v2 Churn rate v3 Churn rate v1 Recommendation Antispam Churn rate v1 Recommendation v1 Recommendation v2 Recommendation v3 Recommendation v4 Antispam Antispam v2 Antispam v3 Antispam v4 Churn rate v1 Churn rate v2 Churn rate v3 Churn rate v1 Recommendation Antispam Churn rate v1 Recommendation v1 Recommendation v2 Recommendation v3 Recommendation v4 Antispam Antispam v2 Antispam v3 Antispam v4 Churn rate v1 Churn rate v2 Churn rate v3 Churn rate v1 Recommendation Antispam Churn rate v1 Recommendation v1 Recommendation v2 Recommendation v3 Recommendation v4 Antispam Antispam v2 Antispam v3 Antispam v4 Monday Tuesday Wednesday
55. Churn rate v1 Churn rate v2 Churn rate v3 Churn rate v1 Recommendation Antispam Churn rate v1 Recommendation v1 Recommendation v2 Recommendation v3 Recommendation v4 Antispam Antispam v2 Antispam v3 Antispam v4 Churn rate v1 Churn rate v2 Churn rate v3 Churn rate v1 Recommendation Antispam Churn rate v1 Recommendation v1 Recommendation v2 Recommendation v3 Recommendation v4 Antispam Antispam v2 Antispam v3 Antispam v4 Churn rate v1 Churn rate v2 Churn rate v3 Churn rate v1 Recommendation Antispam Churn rate v1 Recommendation v1 Recommendation v2 Recommendation v3 Recommendation v4 Antispam Antispam v2 Antispam v3 Antispam v4 Churn rate v1 Churn rate v2 Churn rate v3 Churn rate v1 Recommendation Antispam Churn rate v1 Recommendation v1 Recommendation v2 Recommendation v3 Recommendation v4 Antispam Antispam v2 Antispam v3 Antispam v4 Monday Tuesday Wednesday Thursday
56. Churn rate v1 Churn rate v2 Churn rate v3 Churn rate v1 Recommendation Antispam Churn rate v1 Recommendation v1 Recommendation v2 Recommendation v3 Recommendation v4 Antispam Antispam v2 Antispam v3 Antispam v4 Churn rate v1 Churn rate v2 Churn rate v3 Churn rate v1 Recommendation Antispam Churn rate v1 Recommendation v1 Recommendation v2 Recommendation v3 Recommendation v4 Antispam Antispam v2 Antispam v3 Antispam v4 Churn rate v1 Churn rate v2 Churn rate v3 Churn rate v1 Recommendation Antispam Churn rate v1 Recommendation v1 Recommendation v2 Recommendation v3 Recommendation v4 Antispam Antispam v2 Antispam v3 Antispam v4 Churn rate v1 Churn rate v2 Churn rate v3 Churn rate v1 Recommendation Antispam Churn rate v1 Recommendation v1 Recommendation v2 Recommendation v3 Recommendation v4 Antispam Antispam v2 Antispam v3 Antispam v4 Churn rate v1 Churn rate v2 Churn rate v3 Churn rate v1 Recommendation Antispam Churn rate v1 Recommendation v1 Recommendation v2 Recommendation v3 Recommendation v4 Antispam Antispam v2 Antispam v3 Antispam v4 Monday Tuesday Wednesday Thursday Friday
57. Churn rate v1 Churn rate v2 Churn rate v3 Churn rate v1 Recommendation Antispam Churn rate v1 Recommendation v1 Recommendation v2 Recommendation v3 Recommendation v4 Antispam Antispam v2 Antispam v3 Antispam v4 Churn rate v1 Churn rate v2 Churn rate v3 Churn rate v1 Recommendation Antispam Churn rate v1 Recommendation v1 Recommendation v2 Recommendation v3 Recommendation v4 Antispam Antispam v2 Antispam v3 Antispam v4 Churn rate v1 Churn rate v2 Churn rate v3 Churn rate v1 Recommendation Antispam Churn rate v1 Recommendation v1 Recommendation v2 Recommendation v3 Recommendation v4 Antispam Antispam v2 Antispam v3 Antispam v4 Churn rate v1 Churn rate v2 Churn rate v3 Churn rate v1 Recommendation Antispam Churn rate v1 Recommendation v1 Recommendation v2 Recommendation v3 Recommendation v4 Antispam Antispam v2 Antispam v3 Antispam v4 Churn rate v1 Churn rate v2 Churn rate v3 Churn rate v1 Recommendation Antispam Churn rate v1 Recommendation v1 Recommendation v2 Recommendation v3 Recommendation v4 Antispam Antispam v2 Antispam v3 Antispam v4 Churn rate v1 Churn rate v2 Churn rate v3 Churn rate v1 Recommendation Antispam Churn rate v1 Recommendation v1 Recommendation v2 Recommendation v3 Recommendation v4 Antispam Antispam v2 Antispam v3 Antispam v4 Monday Tuesday Wednesday Thursday Friday Saturday
58. Churn rate v1 Churn rate v2 Churn rate v3 Churn rate v1 Recommendation Antispam Churn rate v1 Recommendation v1 Recommendation v2 Recommendation v3 Recommendation v4 Antispam Antispam v2 Antispam v3 Antispam v4 Churn rate v1 Churn rate v2 Churn rate v3 Churn rate v1 Recommendation Antispam Churn rate v1 Recommendation v1 Recommendation v2 Recommendation v3 Recommendation v4 Antispam Antispam v2 Antispam v3 Antispam v4 Churn rate v1 Churn rate v2 Churn rate v3 Churn rate v1 Recommendation Antispam Churn rate v1 Recommendation v1 Recommendation v2 Recommendation v3 Recommendation v4 Antispam Antispam v2 Antispam v3 Antispam v4 Churn rate v1 Churn rate v2 Churn rate v3 Churn rate v1 Recommendation Antispam Churn rate v1 Recommendation v1 Recommendation v2 Recommendation v3 Recommendation v4 Antispam Antispam v2 Antispam v3 Antispam v4 Churn rate v1 Churn rate v2 Churn rate v3 Churn rate v1 Recommendation Antispam Churn rate v1 Recommendation v1 Recommendation v2 Recommendation v3 Recommendation v4 Antispam Antispam v2 Antispam v3 Antispam v4 Churn rate v1 Churn rate v2 Churn rate v3 Churn rate v1 Recommendation Antispam Churn rate v1 Recommendation v1 Recommendation v2 Recommendation v3 Recommendation v4 Antispam Antispam v2 Antispam v3 Antispam v4 Churn rate v1 Churn rate v2 Churn rate v3 Churn rate v1 Recommendation Antispam Churn rate v1 Recommendation v1 Recommendation v2 Recommendation v3 Recommendation v4 Antispam Antispam v2 Antispam v3 Antispam v4 Monday Tuesday Wednesday Thursday Sunday Friday Saturday
59. ! manual automation
60. slow!
61. slow! error prone!!
62. slow! error prone!! boring!!!
63. !
64. what can you do?
65. what if…
66. you could use ?
67. rent kit by the hour
68. rent kit by the hour bigger machines
69. rent kit by the hour bigger machines scale with the team
70. what if…
71. ☁ ☁ ☁ ☁ ☁ ☁ ☁ ☁ ☁ ☁ ☁ ☁ ☁ ☁ ☁ ☁ ☁ ☁ ☁ ☁ ☁ ☁ ☁ ☁ ☁ ☁ ☁ ☁ ☁ ☁ ☁ ☁ ☁ ☁ ☁ ☁ ☁ ☁ ☁ ☁
72. rent kit by the hour
73. rent kit by the hour rent fleet of machine by the hour
74. rent fleet of machine by the hour bigger machines
75. rent fleet of machine by the hour bigger machines mixed nodes w/ cpus , gpus , tpus
76. rent fleet of machine by the hour mixed nodes w/ cpus , gpus , tpus scale with the team
77. rent fleet of machine by the hour mixed nodes w/ cpus , gpus , tpus scale with the team
78. rent fleet of machine by the hour mixed nodes w/ cpus , gpus , tpus scale with the company
79. what if…
80. this is ready to be used?
82. what is it?
96. also
97. LOW PRIORITY
98. LOW PRIORITY
99. LOW PRIORITY
100. GPU LOW PRIORITY
101. GPU LOW PRIORITY
102. LOW GPU LOW PRIORITY
103. GPU LOW PRIORITY LOW
104. still single job…
112. how can i jobs?
114. Kubernetes
115. Kubernetes Kubernetes
116. Kubernetes Kubernetes Jupyter
117. Kubernetes Kubernetes Jupyter Pipelines
118. Kubernetes Kubernetes Jupyter Pipelines Operators
119. Kubernetes Kubernetes Jupyter Pipelines Operators Katib
120. Kubernetes Kubernetes Jupyter Pipelines Operators Katib Katib Metadata
121. Kubernetes Kubernetes Jupyter Pipelines Operators Katib Katib Metadata KFServing
122. Kubernetes Kubernetes Jupyter Pipelines Operators Katib Metadata KFServing
123. Kubernetes Kubernetes Jupyter Pipelines Operators Katib Metadata KFServing
124. Kubernetes Kubernetes Jupyter Pipelines Operators Katib Metadata KFServing
125. Kubernetes Kubernetes Jupyter Pipelines Operators Katib Metadata KFServing
126. meanwhile
135. does k8s run ! python ?
136. nope!
137. container orchestrator
138. ! container
141. PY JSON +
142. PY JSON + +
143. PY JSON + + CONTAINER
144. language agnostic
146. Java 1.8 Node.js 10 Python 2.7 Python 3.8 .NET Core Lua
154. !
155. if you were to…
156. 1. write a tf model
157. 1. write a tf model 2. package it up
158. 1. write a tf model 2. package it up 3. deploy to kubeflow
159. machine learning in the
160. recognising clothing items
164. model
172. T-shirt
173. Prediction
175. Ankle boot
176. Process
181. Pipelines
182. 1. Multi-step ML workflow
183. 1. Multi-step ML workflow 2. Python SDK
184. 1. Multi-step ML workflow 2. Python SDK 3. Productionise
186. demo
187. ◻ create LKE cluster ◻ install kubeflow ◻ write mnist model
188. ✅ create LKE cluster ◻ install kubeflow ◻ write mnist model
189. ✅ create LKE cluster ✅ install kubeflow ◻ write mnist model
190. ✅ create LKE cluster ✅ install kubeflow ◻ write mnist model
191. Pipelines on GPUs/TPUs
193. There's more
194. Model Serving
195. Ankle boot
196. BACKEND API
197. BACKEND API
198. Models
199. Kubernetes Models
200. Kubernetes Istio Models
201. Kubernetes Istio Knative Models
202. Kubernetes KFServing Istio Knative Models
203. 2. hyperparameter tuning
204. T-shirt
205. T-shirt
206. T-shirt
208. machine learning at scale with k8s
210. 6000 instances of apache solr ~ 1000 servers
214. training ml with 2500 nodes
215. horizontal pod autoscaler
223. cluster autoscaler
227. 256 P100 128000 preemptible vcpus
228. 256 P100 $400/hr 128000 preemptible vcpus $1280/hr
229. 256 P100 $400/hr 128000 preemptible vcpus $1280/hr
231. 100 Users
232. 100 Users 18,000 Expermients
233. 100 Users 18,000 Expermients 7x Experiments
236. 320,000 cores
237. 320,000 cores 4,300 projects
238. 320,000 cores 4,300 projects 250 petabytes
239. 3,300 users
240. 3,300 users 10,000 hypervisors
241. 3,300 users 10,000 hypervisors 210 kubernetes clusters
242. what's next?
243. 1. Create Cluster 2. Deploy Kubeflow 3. Experiment Piplines
244. Kubeflow on Linode · linode.com Code: LEARNK8S100 · bit.ly/kubeflow-on-linode · kubeflow.org
245. Ebooks 12 courses Interactive challenges CKAD preparation Certificate of completion Online material
246. Ebooks 12 courses Interactive challenges CKAD preparation Certificate of completion Online material Linode exclusive 20% discount https://learnk8s.io/linode LIMITED OFFER
247. what if I have a question?
248. learnk8s-slack-invite.herokuapp.com
249. Thank you
255. Thank you all!
256. - Sign up to JenkinsX Lab - Linode Credits - Learnk8s Courses - Survey
257. stay in touch! · @soulmaniqbal · @learnk8s · Newsletter