Background Processing, Serverless Style

Background processing is a critical component of many applications. The serverless programming model offers an alternative to traditional job systems that can reduce overhead while increasing productivity and happiness. We'll look at some typical background processing scenarios and see how to modify them to run as serverless functions. You'll see the advantages and trade-offs, as well as some situations in which you might not want to go serverless. We'll also talk about the serverless ecosystem, and you'll walk away with the knowledge and tools you need to experiment on your own.

Transcript

1. Background Processing, Serverless Style Ben Bleything (he/him) @bleything
2. Obligatory Intro Slide • @bleything on the internet • Ruby old-timer • infrastructure, ops, architecture • developer advocate at Google • I Have Seen Some Stuff
3. This is not a sales pitch. You probably don’t need this stuff. I think it’s cool, but in an abstract way. Only you can decide if it’s right for you.
4. Background Processing
5. Quick Survey!
6. Serverless
7. Jason Watkins @jasonwatkinspdx Function as a Service works fine for me.
8. Aja @the_thagomizer Pay per unit compute. You pay, some tiny amount, each time you use compute and you don't pay when nothing is computing.
9. Sandeep Dinesh @SandeepDinesh Anything that charges only by usage and requires no manual scaling or provisioning
10. Jack C @enth other repliers seem not to have covered: serverless heavily implies a near-zero ops workload. no chef. no terraform. no dockerfile. etc
11. Functions as a Service
12. Webhook Handler
13. Webhook Handler
14. FaaS Invocation via HTTP API Gateway HTTP
15. Serverless Webhook Handler API Gateway HTTP
16. Okay, but why?
17. Text Processing
18. Text Processing Database
19. Text Processing Database Elasticsearch
20. Text Processing Database Elasticsearch
21. Text Processing Database Elasticsearch
22. Text Processing, Serverlessly
23. FaaS Invocation via Message Queue Pub/Sub
24. Serverless Text Processing Database Pub/Sub index spam filter analytics
25. Okay, but why?
26. Serverless Text Processing Database Pub/Sub index spam filter analytics
27. Serverless Text Processing Database index spam filter analytics
28. Okay, but why?
29. Video Processing
30. Video Processing Object Store Metadata Extraction
31. Video Processing Transcode Extract Metadata Upload to Storage Notify User
32. Video Processing Transcode Upload to Storage Extract Metadata Upload Original Notify User Transcode Upload to Storage
33. Serverless Video Processing
34. FaaS Invocation via Service Event event details Cloud Storage
35. Serverless Video Processing Cloud Storage object finalized transcode metadata notify
36. Okay, but why?
37. Things To Know
38. Further Exploration • https://aws.amazon.com/serverless/ • https://azure.microsoft.com/en-us/overview/serverless-computing/ • https://cloud.google.com/serverless/
39. Further Exploration • AWS Lambda • Azure Functions • Google Cloud Functions • Apache OpenWhisk • Fn • fission.io
40. Thanks! github.com/bleything twitter.com/bleything Keynote Theme - http://sanographix.github.io/azusa-colors/ ben@bleything.net bleything.net