Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Background Processing, Serverless Style

Background Processing, Serverless Style

Background processing is a critical component of many applications. The serverless programming model offers an alternative to traditional job systems that can reduce overhead while increasing productivity and happiness. We'll look at some typical background processing scenarios and see how to modify them to run as serverless functions. You'll see the advantages and trade-offs, as well as some situations in which you might not want to go serverless. We'll also talk about the serverless ecosystem, and you'll walk away with the knowledge and tools you need to experiment on your own.


Ben Bleything

May 02, 2019


  1. Background Processing, Serverless Style Ben Bleything (he/him) @bleything

  2. Obligatory Intro Slide • @bleything on the internet • Ruby

    old-timer • infrastructure, ops, architecture • developer advocate at Google • I Have Seen Some Stuff
  3. This is not a sales pitch. You probably don’t need

    this stuff. I think it’s cool, but in an abstract way. Only you can decide if it’s right for you.
  4. Background Processing

  5. Quick Survey!

  6. Serverless

  7. Jason Watkins @jasonwatkinspdx Function as a Service works fine for

  8. Aja @the_thagomizer Pay per unit compute. You pay, some tiny

    amount, each time you use compute and you don't pay when nothing is computing.
  9. Sandeep Dinesh @SandeepDinesh Anything that charges only by usage and

    requires no manual scaling or provisioning
  10. Jack C @enth other repliers seem not to have covered:

    serverless heavily implies a near-zero ops workload. no chef. no terraform. no dockerfile. etc
  11. Functions as a Service

  12. Webhook Handler

  13. Webhook Handler

  14. FaaS Invocation via HTTP API Gateway HTTP

  15. Serverless Webhook Handler API Gateway HTTP

  16. Okay, but why?

  17. Text Processing

  18. Text Processing Database

  19. Text Processing Database Elasticsearch

  20. Text Processing Database Elasticsearch

  21. Text Processing Database Elasticsearch

  22. Text Processing, Serverlessly

  23. FaaS Invocation via Message Queue Pub/Sub

  24. Serverless Text Processing Database Pub/Sub index spam filter analytics

  25. Okay, but why?

  26. Serverless Text Processing Database Pub/Sub index spam filter analytics

  27. Serverless Text Processing Database index spam filter analytics

  28. Okay, but why?

  29. Video Processing

  30. Video Processing Object Store Metadata Extraction

  31. Video Processing Transcode Extract Metadata Upload to Storage Notify User

  32. Video Processing Transcode Upload to Storage Extract Metadata Upload Original

    Notify User Transcode Upload to Storage
  33. Serverless Video Processing

  34. FaaS Invocation via Service Event event details Cloud Storage

  35. Serverless Video Processing Cloud Storage object finalized transcode metadata notify

  36. Okay, but why?

  37. Things To Know

  38. Further Exploration • https://aws.amazon.com/serverless/ • https://azure.microsoft.com/en-us/overview/serverless-computing/ • https://cloud.google.com/serverless/

  39. Further Exploration • AWS Lambda • Azure Functions • Google

    Cloud Functions • Apache OpenWhisk • Fn • fission.io
  40. Thanks! github.com/bleything twitter.com/bleything Keynote Theme - http://sanographix.github.io/azusa-colors/ ben@bleything.net bleything.net