Python in the land of serverless

Pycon Colombia 2018
One year ago I joined a team that favours Serverless, since then I’ve been building and maintaining lots of services using Serverless. With a pinch of Skepticism, I sailed through some of the challenges and tooling, I want to share with the community the pains and glory of it.

Transcript

1. Python in the Land of Serverless David Przybilla dav009 dav009
2. " " last time in here was ~9 years ago
3. NLP Data Engineering
4. Gov spending dataset How to access datasets
5. ColombiaDev Encuesta de Salarios
6. ..Ops + Golang + Python..
8. Tokyo
9. cool problems to solve
10. check our booth
11. ..a bit of context…
12. Roughly 1.5 years ago…
13. I joined HDE
14. first week..started looking at projects..
15. first week..started looking at projects.. Github repos…
16. first week..started looking at projects.. Github repos… New project I had to work on…
17. turns out some projects were built on “serverless”
18. so in this talk I will guide you through:
19. some of the tools
20. some of the use cases
21. some common mistakes
22. …my opinion…
23. my first task: log stream processing
25. near realtime
26. If you have been doing data pipelines, probably you would go for:
27. If you have been doing data pipelines, probably you would go for:
28. If you have been doing data pipelines, probably you would go for:
29. a lot to configure a lot to manage
 

30. a lot to configure a lot to manage
 
 zookeeper..
 yarn cluster.. driver instance.. JVM.. HDFS..
31. a lot to configure a lot to manage
 
 zookeeper..
 yarn cluster.. driver instance.. JVM.. HDFS..
32. you have to make sure those parts are running smoothly
33. you have to make sure those parts are running smoothly
34. I met Jeff
35. I met Jeff
36. a coworker ops engineer
37. He loves Serverless
38. And he has good reasons for it…
39. he does not want to do any server management
40. So Jeff suggested to use serverless to solve our stream processing problem
41. I raised my eyebrow in doubt:
42. “Serverless”??
43. a quite controversial & confusing term..
44. “Serverless” 1. Fully managed services, managed on your behalf Databases (DynamoDB).. Storage (s3).. Queues (kafka as a service, sqs…) heroku…
45. “Serverless” 2. Function as a Service (FaaS) AWS lambda Azure functions Kubeless Google Cloud functions.. Nuclio ..openwhisk…. …many more..…..
46. Function as a service
47. Upload your code and it will run…
48. Upload your code and it will run… you don’t care on top of what is running..
49. Upload your code and it will run… you don’t care on top of what is running.. you only focus on your business logic
50. (i) zero administration - Focus on a single function
51. zero administration - Focus on a single function - Managed by provider

52. zero administration - Focus on a single function - Managed by provider
 
 - You gain peace of mind

53. zero administration - Focus on a single function - Managed by provider
 
 - You gain peace of mind
 - cost: more integration with your vendor(vendor lock-in)
54. (ii) You are billed by number of invocations
55. so how does this looks like in python?
56. def handler(event, context): # do something pass
57. def handler(event, context): # do something url = event[‘url’] scrape(url)
58. def handler(event, context): # do something database = # magic username = event[‘username’] database.find(username)
59. so how do I deploy my function?
60. in order run your project on FaaS: 0. define your function 1. package your function 2. upload your package 3. call your function tools address those steps
61. (tools) 0. define function (infrastructure / glue)
62. (tools) 0. define function (infrastructure / glue) - runtime (python, golang, js..)
63. (tools) 0. define function (infrastructure / glue) - memory (128mb, 500?..) - runtime (python, golang, js..)
64. (tools) 0. define function (infrastructure / glue) - memory (128mb, 500?..) - runtime (python, golang, js..) - access to resources
65. (tools) 0. define function (infrastructure / glue) mouse & clicking - memory (128mb, 500?..) - runtime (python, golang, js..) - access to resources
66. (tools) 0. define function (infrastructure / glue) mouse & clicking - memory (128mb, 500?..) - runtime (python, golang, js..) - access to resources
67. (tools) 0. define function (infrastructure / glue) mouse & clicking - memory (128mb, 500?..) - runtime (python, golang, js..) - access to resources
68. 1. package your function
69. 1. package your function your function code
70. + dependencies 1. package your function your function code
71. “deploying it” 2. upload package
72. 3. call your function
73. 3. call your function “what triggers it?”
74. 3. call your function “what triggers it?” function is called whenever:
75. 3. call your function “what triggers it?” url gets hit.. an object is uploaded to s3.. a record is added to your database.. while queue is not empty.. function is called whenever:
76. 3. call your function Deeply entangled on your cloud provider services
77. (infrastructure /glue) 3. call your function Deeply entangled on your cloud provider services mouse & clicking
78. tooling address those steps
79. deploying: infrastructure + code
80. back to my story..
81. back to my story.. first week.. started looking at repos..
82. One of them is an API
83. something.com/endpointA something.com/endpointB something.com/endpointC
84. my team was an early adopter of this technology
85. they stitched projects with the tools available back then
86. request to urlA
87. request to urlA
88. def a(..) request to urlA
89. def a(..) def b(..) request to urlA request to urlB
90. def a(..) def b(..) def c(..) request to urlA request to urlB request to urlC
91. deploy: makefiles
92. deploy: makefiles glue:
93. 3 functions to deploy
94. 3 functions to deploy 3 functions to package
95. 3 functions to deploy 3 functions to package lots of glue : - many pieces to move - to worry about - hard to test
96. changing this kind of trigger(url) on terraform is painful
97. changing this kind of trigger(url) on terraform is painful running terraform is scary you might destroy other infrastructure
98. if for whatever reason you want to move to non-serverless.. it is harder..
99. if for whatever reason you want to move to non-serverless.. it is harder.. why?
100. because all that glue that triggers the functions you have to implement that glue in a different way
101. 3 different lambdas.. also make people structure their project like 3 independent projects..
102. with tools available today you can avoid this..
103. any request to something.com
104. wsgi wrapper (like a flask app) any request to something.com
105. wsgi wrapper (like a flask app) any request to something.com wsgi wrapper?
106. wsgi wrapper (like a flask app) any request to something.com translates requests arriving to def handler(..) into wsgi requests wsgi wrapper?
107. wsgi wrapper (like a flask app) any request to something.com translates requests arriving to def handler(..) into wsgi requests wsgi wrapper? it means we can use tools like flask, django ..
108. you can use all tools already available and all that come with them testing is easier
109. you can use all tools already available and all that come with them testing is easier you can make your api with the same tools, and have serverless “for free”
110. you can use all tools already available and all that come with them testing is easier you can make your api with the same tools, and have serverless “for free”
111. minimal implementation of that wrapper (python3) is available at: https://github.com/slank/awsgi
112. so if you have a dummy flask app
113. from flask import ( Flask, jsonify, ) app = Flask(__name__) @app.route(‘/endpointB’) def endpoint_b(): # … return jsonify(status=200, message='OK')
114. to make it serverless you just need to add this function:
115. def handler(event, context): return awsgi.response(app, event, context) to make it server less you just need to add this function:
116. def handler(event, context): return awsgi.response(app, event, context) to make it server less you just need to add this function: wsgi app
117. - can run it locally
118. - can run it locally - easy to test routing
119. - less glue - can run it locally - easy to test routing
120. API building serverless tools:
121. API building server less tools: serverless chalice zappa … there are more..many..
122. https://github.com/serverless/serverless https://serverless.com/ Serverless
123. serverless
124. Good: - many plugins - got funding - particularly good when you are building apis - it is not sluggish (for the given use case) - provide you easy way to handle environments (stg, prd) - great community serverless
125. Bad: no plan building: serverless
126. Bad: no plan building: serverless deploying: infrastructure + code
127. Bad: no plan building: serverless it does not tell you what will change before deploying deploying: infrastructure + code
128. Bad: serverless changing something in config file can leak infrastructure It is dangerous to leave infrastructure leaking behind
129. How to use it? serverless
130. How to use it? serverless 1. define glue in .yml file
131. How to use it? serverless 2. do `sls deploy` 1. define glue in .yml file
132. serverless .yml file
133. serverless .yml file • I want a function with this memory.. • I want a function with this name… • I want it to get called when X and Y happens
134. serverless `sls deploy` • Package + upload
135. serverless serverless has a lot fo plugins that you can add to your .yml file. don’t use it to manage infrastructure.
136. serverless serverless “applications” > serverless install --url <service-github-url> > sls deploy code + glue + infrastructure i.e: serverless service to get a slack bot via FaaS
137. Chalice https://github.com/aws/chalice - comes with “wsgi wrapper” - purely focused on AWS - aimed at API particular case > chalice new-project my_sample_project > chalice deploy
138. How to use it? Zappa 1. define glue in .json file
139. How to use it? Zappa 2. some of the glue code is defined as python decorators 1. define glue in .json file
140. @task def make_pie(): """ This takes a long time! """ ingredients = get_ingredients() pie = bake(ingredients) deliver(pie) @task def make_soup(): ingredients = get_ingredients() soup = bake(ingredients) deliver(soup) @task
141. How to use it? Zappa it has some cool decorators it lacks plugins/addons good if you are building APIs if you have lots of cron/async calls
142. Another use case..
143. you have a lot of data to process in batch
144. you have a lot of data to process in batch text.. numbers..
145. compute to TB datasets across hundreds of functions clean wikipedia etl data preprocessing count frequencies
146. map & reduce..right?
147. map & reduce..right? get me a spark cluster use pyspark ..done
148. this is a great use case for FaaS
149. this is a great use case for FaaS pywren https://github.com/pywren/pywren
150. benchmark of
 - 80 GB/sec read 
 - 60 GB/sec write pywren
151. No knowledge of AWS required pywren
152. No knowledge of AWS required pywren No large (expensive) cluster up
153. No knowledge of AWS required pywren No large (expensive) cluster up Using vanilla python
154. > pywren-setup pywren
155. > pywren-setup pywren essentially : 1. takes a python function (creates a FaaS)
156. > pywren-setup pywren essentially : 1. takes a python function (creates a FaaS) 2. takes data and uploads it to s3
157. > pywren-setup pywren essentially : 1. takes a python function (creates a FaaS) 2. takes data and uploads it to s3 3. runs your python function in parallel on the data uploaded to s3
158. def add_one(x): return x + 1 pywren
159. def add_one(x): return x + 1 [0, 1…9] pywren
160. def clean_text(text): # clean text return cleaned_text pywren
161. def clean_text(text): # clean text return cleaned_text pywren wikidump (100G)
162. pywren def add_one(x): return x + 1
163. pywren def add_one(x): return x + 1 creates lambda
164. pywren def add_one(x): return x + 1 creates lambda [0, 1…9]
165. pywren def add_one(x): return x + 1 creates lambda [0, 1…9] uploads data to part1
166. pywren def add_one(x): return x + 1 creates lambda [0, 1…9] uploads data to part1 part2
167. pywren def add_one(x): return x + 1 creates lambda [0, 1…9] uploads data to part1 part2 part3
168. pywren def add_one(x): return x + 1 [0, 1…9]
169. pywren def add_one(x): return x + 1 [0, 1…9] add_one(0)
170. pywren def add_one(x): return x + 1 [0, 1…9] add_one(0) add_one(1)
171. pywren def add_one(x): return x + 1 [0, 1…9] add_one(0) add_one(1) add_one(2) …..
172. pywren def add_one(x): return x + 1 [0, 1…9] add_one(0) add_one(1) add_one(2) …..
173. pywren def add_one(x): return x + 1 [0, 1…9] add_one(0) add_one(1) add_one(2) …..
174. pywren def add_one(x): return x + 1 [0, 1…9] add_one(0) add_one(1) add_one(2) …..
175. pywren how does it look like ? import pywren number_list = np.arange(10) # [0,1,2…9] data
176. pywren how does it look like ? import pywren number_list = np.arange(10) # [0,1,2…9] data # pywren magic wrenexec = pywren.default_executor() futures = wrenexec.map(addone, number_list)
177. pywren how does it look like ? import pywren number_list = np.arange(10) # [0,1,2…9] data # pywren magic wrenexec = pywren.default_executor() futures = wrenexec.map(addone, number_list) # f.result() blocks until s3 file result is available print [f.result() for f in futures]
178. pywren how does it look like ? # f.result() blocks until s3 file result is available print [f.result() for f in futures] > python sample.py import pywren number_list = np.arange(10) # [0,1,2…9] data # pywren magic wrenexec = pywren.default_executor() futures = wrenexec.map(addone, number_list)
179. pywren good for: - ETL tasks.. - Scraping.. - Data crunching in general..
180. similar to pywren https://github.com/qubole/spark-on-lambda spark-on-lambda
181. similar to pywren https://github.com/qubole/spark-on-lambda spark-on-lambda looks experimental though
182. similar to pywren https://github.com/qubole/spark-on-lambda spark-on-lambda looks experimental though same as spark.. just executors are lambda functions
183. Scanning 1 TB of Data 1000 Lambda executors took 47s cost turns out to be $1.18. spark-on-lambda
184. Scanning 1 TB of Data 1000 Lambda executors took 47s cost turns out to be $1.18. spark-on-lambda on regular spark 50 r3.Xlarge instances.. 2 or 3 mins just to setup + start
185. remarks..
186. no multiprocessing.. module
187. common mistakes
188. structure your project as any other python project
189. structure your project as any other python project don’t think of it as FaaS
190. if your project has many FaaS
191. if your project has many FaaS it is a single project, with different entry points
192. immutability
193. FaaS is built on top of containers
194. containers for the same function sometimes gets reused..
195. db_connection = connect(something) def handler(a,b): db_connection.query(something(a))
196. > db_connection = connect(something) def handler(a,b): db_connection.query(something(a)) 1st run
197. db_connection = connect(something) > def handler(a,b): db_connection.query(something(a)) 1st run
198. db_connection = connect(something) > def handler(a,b): db_connection.query(something(a)) 2nd run
199. as in any project
200. as in any project mutable globals are not desirable
201. list_of_users = [‘admin’] def handler(a,b): list_of_users = list_of_users + a[‘user’] do_something(list_of_users)
202. list_of_users = [‘admin’] def handler(a,b): list_of_users = list_of_users + a[‘user’] do_something(list_of_users)
203. list_of_users = [‘admin’] def handler(a,b): list_of_users = list_of_users + a[‘user’] do_something(list_of_users) [‘admin’, user1] intended input to do_something [‘admin’, user1] do_something actual input to
204. list_of_users = [‘admin’] def handler(a,b): list_of_users = list_of_users + a[‘user’] do_something(list_of_users) [‘admin’, user1] intended input to do_something [‘admin’, user1] do_something actual input to [‘admin’, user2] do_something intended input to
205. list_of_users = [‘admin’] def handler(a,b): list_of_users = list_of_users + a[‘user’] do_something(list_of_users) [‘admin’, user1] intended input to do_something [‘admin’, user1] do_something actual input to [‘admin’, user2] do_something [‘admin’, user1, user2] do_something actual input to intended input to
206. list_of_users = [‘admin’] def handler(a,b): list_of_users = list_of_users + a[‘user’] do_something(list_of_users) [‘admin’, user1] intended input to do_something [‘admin’, user1] do_something actual input to [‘admin’, user2] do_something [‘admin’, user1, user2] do_something actual input to intended input to
207. don’t mutate
208. ..security..
209. Dependencies.. please update them
210. delete old functions
211. you don’t pay if you don’t use them so you don’t get reminded in your bill
212. if you don’t use them, delete them
213. sounds easy..but..in a large organisation: - you got no idea who is the owner
214. sounds easy..but..in a large organisation: - you got no idea who is the owner - is it safe to delete?
215. straight up your permissions only give needed permissions
216. DDoS are wallet attacks
217. to sum up
218. lots of new tools are yet to come…
219. serverless provides a peace of mind.: it will be running it won’t be down but you agree on going full on using your cloud provider features
220. There is a lot of glue.. lots of events here and there….
221. serverless is [in my opinion] cheaper not simpler
222. data crunching : yes handling small events: yes
223. total complexity of your system might grow
225. this is your extra complexity: glue, wiring
227. glue is hard to test
228. this is the hard part.. integration testing
229. does the event triggers the expected behaviour?
230. it is getting more popular..
231. many surveys seems to indicate FaaS adoption is as fast as that of containers
232. I guess new tools will help tame this complexity
233. I guess now you are wondering. should I use serverless or not?
234. If you find yourself in any of these situations:
235. you have a developer complaining about having to spin up infrastructure before they can get something done
236. maybe you are a data scientist annoyed by how difficult is to run your experiment
237. then maybe is worth trying it
238. by the way, yes..
239. we built our stream processing system on top of FaaS
240. we built our stream processing system on top of FaaS
241. Gracias! dav009 dav009