Workers and the Rise of Reactive Infrastructure

By Paul Burt!

In this presentation, I argue that Functional Programming is great for infrastructure. But, the Worker pattern is more familiar to OO folks, while offering many of the same advantages.

SLIDE NOTES:
2)
I’m a Developer Evangelist at iron.io.
We make easy to use, high throughput, infrastructure solutions.
4)
Functional Programming and Reactive Infrastructure are trendy.
For good reason, they require fewer mental gymnastics than traditional approaches.
7)
https://medium.com/javascript-scene/the-two-pillars-of-javascript-pt-2-functional-programming-a63aa53a41a4
8)
http://www.reactivemanifesto.org/
9)
Image CC BY 2.0
tracy the astonishing https://www.flickr.com/photos/tracy_the_astonishing/3398886840
12)
Image CC BY 2.0 https://upload.wikimedia.org/wikipedia/commons/e/e0/The_Grumpy_Old_Punks.jpg
14)
Quote from https://en.wikipedia.org/wiki/Idempotence#Examples
15)
The response is returning a webpage. This may change between requests. The “outcome” is the resource returns a page.
17)
Fish is treats other fish just like any other piece of food.
Functions should be treated just like any other variable.
CC BY 2.0 source https://www.flickr.com/photos/38446022@N00/2874171158
18)
One thing to note, this code is odd to write. But, it’s easy to read Hungry(bear). It gets even easier with piping syntax.
20)
One function does work.
The output is passed to the next one.
CC BY 2.0 from Alden Jewell https://www.flickr.com/photos/autohistorian/16617915558
22)
Image CC BY 2.0 https://upload.wikimedia.org/wikipedia/commons/4/4a/Todd_Huffman_-_Lattice_(by).jpg
25)
Like this banana slicer that does one thing, and does it well...
26)
Write code purpose built for one task. Chain tasks together. Yay composability!
28)
CC BY 2.0 Beate Meier https://www.flickr.com/photos/beate_meier/8337014543
31)
CC BY 2.0 khrawlings https://www.flickr.com/photos/khrawlings/3622143862/
32)
http://bravenewgeek.com/you-cannot-have-exactly-once-delivery/
33) This Greek structure was shoddily repurposed, just like many try to repurpose yesterday's infrastructure.
Image CC BY 2.0 https://upload.wikimedia.org/wikipedia/commons/1/11/Vilnius_Cathedral_Facade.jpg
34)
Image from http://www.reactivemanifesto.org/
39)
FP offers a declarative approach.
This simplifies reading, reasoning, and debugging code.
Writing is harder, which makes it hard to get a big team to adopt.
41)
They’re fast,
fail gracefully, message driven,
and are easy to scale
42)
Devs already embrace OOP.
Just need to make code more modular. Embrace the Unix philosophy.
Workers are a natural way to get many benefits of FP, without the learning curve.

Transcript

1. Workers and the Rise of Reactive Infrastructure “Hey bruh, how many nines are in your SLA?” [sic]
2. Who is this guy?
3. Workers? Reactive whats?
4. FP + Reactive Infra
6. FP + Reactive Workers
9. Functional Programming “The only way we’ll be able to keep up with the exponentially increasing complexity will be to decrease the complexity of understanding programs.”
10. Functional Programming • Immutable & Idempotent • First Class Functions • Lists => Streams
11. Functional Programming • Immutable & Idempotent • First Class Functions • Lists => Streams
12. Functional Programming • Immutable & Idempotent • First Class Functions • Lists We should Change. No.
13. Functional Programming • Immutable & Idempotent • First Class Functions • Lists => Streams Function add(a, b) {
 return a + b
 } add(2, 3) // 5
 add(2, 3) // 5 Same input produces 
 the same output State is
 wholly inside
 the function.
 (no side effects)
14. Functional Programming • Immutable & Idempotent • First Class Functions • Lists Function add(a, b) {
 return a + b
 } add(2, 3) // 5
 add(2, 3) // 5 Same input produces 
 the same output Idempotence 
 
 the ability of a system to 
 produce the same outcome, 
 even if an event or message is received more than once.
15. Functional Programming • Immutable & Idempotent • First Class Functions • Lists Function add(a, b) {
 return a + b
 } add(2, 3) // 5
 add(2, 3) // 5 Same input produces 
 the same output Idempotence 
 
 Caveat!
 The response might be different, even if the “outcome” is the same. eg
 HTTP GET requests
16. Functional Programming • Immutable & Idempotent • First Class Functions • Lists => Streams
17. First Class Functions? A fish, is a fish, is a fish.
18. Functional Programming • Immutable & Idempotent • First Class Functions • Lists => Streams var bear = function(err) {
 if (err) throw err
 console.log(“Was hungry.”)
 } function hungry(func) {
 console.log(
 “Has now eaten.”
 )
 func(null)
 }
 
 hungry(bear) Function stored 
 like a var
 (first class) Pass functions
 as an arg
 (higher order)
19. Functional Programming • Immutable & Idempotent • First Class Functions • Lists => Streams
20. Streams Kind of like an 
 assembly line.
21. Functional Programming • Immutable & Idempotent • First Class Functions • Lists => Streams // Finally, a concrete example var nums = [1, 4, 9, 16]
 
 nums.map(Math.sqrt) // [1, 2, 3, 4]
 .filter(function(n) { 
 return n > 2 
 }) // [3, 4]
 .reduce(function(pre, cur) { 
 return pre + cur 
 }) // 7
 Functions
 operating
 on the stream
22. Workers Unix philosophy
 + at scale.
 Do one thing, 
 do it well
 + do a lot of it.
23. Workers • Do one thing, Do it well • Do it at Scale • Great designers steal
24. Workers • Do one thing, Do it well • Do it at Scale • Great designers steal
26. Workers • Do one thing, Do it well • Do it at Scale • Great designers steal > find ./ironio/**/ > curl iron.io > grep -E ‘meta' ironio.html # composability is key > curl iron.io | grep 'html' Workers are chainable
 in the same way.
27. Workers • Do one thing, Do it well • Do it at Scale • Great designers steal
28. Workers • Do one thing, Do it well • Do it at Scale • Great designers steal Scaling? Use 
 message queues
29. Workers • Do one thing, Do it well • Do it at Scale • Great designers steal # use CLI to queue jobs > iron worker queue 
 --payload-file hello.payload.json 
 --wait projectName # need speed?
 # just add workers to 
 # consume faster Payload files:
 may be serialized data,
 or a URL to a video
 (anything to be processed)
30. Workers • Do one thing, Do it well • Do it at Scale • Great designers steal
31. Workers • Do one thing, Do it well • Do it at Scale • Great designers steal
32. Workers • Do one thing, Do it well • Do it at Scale • Great designers steal Steal idempotence from FP. Why? “Within the context of a distributed system, you cannot have exactly-once message delivery.“
33. Reactive Manifesto Today's demands are simply not met by yesterday’s software architectures.
34. Reactive Manifesto
35. Reactive Manifesto Fast!
 Responds to actions quickly.
36. Reactive Manifesto Fails gracefully
 (isolation + replication)
37. Reactive Manifesto + Non-blocking
 + Real time
 + Loosely coupled (NOT event emmitters)
 Messages are:

38. Reactive Manifesto Scales simply
 & horizontally
39. FP + Reactive
 = a neat trend
40. FP + Reactive
 = a neat trend Big learning
 curve sad face :(
41. Workers What about Workers?
42. Workers
43. OOP FP + Reactive
 = a neat trend Use workers & . . . Yay! :D Build
 modular programs! Embrace the
 Unix
 Philosophy
44. Let’s take a look at a Worker
 iron.io/worker