Concurrency’s Next Frontier: Actor Systems in Clojure

Wunderlist, with millions of active users, continues to grow, bending its realtime sync technology to the limits. This pushed them to rethink and experiment with different architectural approaches. As a result Akka’s Actor System become a key component of theirs realtime architecture. However Actor Systems are not popular in the Clojure Ecosystem. In this talk Andrei will cover what they learned from using Actor Systems, what are their advantages and how to use them in Clojure.

Transcript

1. Concurrency’s Next Fron0er Actor Systems in Clojure #clojuredconf 2016.02.20 @andreiursan 1
2. WHOAMI (def me {:name "Andrei Ursan" :twitter "@andreiursan" :work "Software Engineer, @Microsoft" :builds "Wunderlist" :tags #{:polyglot :microservices :scalability}}) @andreiursan 2
3. Agenda • How Wunderlist uses Actors. • How to use Actors in Clojure. @andreiursan 3
4. Wunderlist @andreiursan 4
5. What’s behind the client? @andreiursan 5
6. @andreiursan 6
7. @andreiursan 7
8. @andreiursan 8
9. Akka • a toolkit for distributed applica.ons • emphasises actor based concurrency • actors are async • part of Scala’s standard library @andreiursan 9
10. Wunderlist and Actors • For each socket connec.on an actor is created. @andreiursan 10
11. Wunderlist and Actors • For each socket connec.on an actor is created. • The WebSocketActor creates: • an actor for h9p requests @andreiursan 11
12. Wunderlist and Actors • For each socket connec.on an actor is created. • The WebSocketActor creates: • an actor for h9p requests • an actor that listens for user events @andreiursan 12
13. Wunderlist and Actors • For each socket connec.on an actor is created. • The WebSocketActor creates: • an actor for h9p requests • an actor that listens for user events • User events are distributed via RabbitMQ @andreiursan 13
14. Clojure the Language of the System But, Actor Systems? @andreiursan 14
15. Actors in Clojure • Akka wrappers • not complete • not maintained @andreiursan 15
16. Actors in Clojure • Akka wrappers • not complete • not maintained • puniverse/pulsar • Idioma4c Clojure wrapper around quasar @andreiursan 16
17. @andreiursan 17
18. @andreiursan 18
19. Quasar > Pulsar @andreiursan 19
20. Pulsar • like erlang • lightweight processes (a.k.a. fibers) • actors can block • gen-servers • supervisors • core.async like channels @andreiursan 20
21. Popularity Akka vs Quasar porter.io/quasar, porter.io/akka @andreiursan 21
22. Pulsar Actors @andreiursan 22
23. (defactor actor-name [] body) @andreiursan 23
24. (defactor actor-name [] (receive)) @andreiursan 24
25. (defactor actor-name [] (loop [] (receive) (recur))) @andreiursan 25
26. (defactor actor-name [] (loop [] (rcv @mailbox) (recur))) @andreiursan 26
27. (defactor actor-name [] (loop [] (receive :pattern-one (do (action-one) (recur)) [:pattern-two msg] (action-two msg) :pattern-three (action-three)))) @andreiursan 27
28. (defactor actor-name [] (loop [] (receive :pattern-one (receive [:pattern-two msg] (action-one-two msg)) :pattern-three (action-three)))) @andreiursan 28
29. (defactor inc-actor [n] (loop [n n] (receive :inc (recur (inc n)) [:inc number] (recur (+ n number)) :print (println n)))) ; @andreiursan 29
30. (defactor inc-actor [n] (loop [n n] (receive :inc (recur (inc n)) [:inc number] (recur (+ n number)) :print (println n)))) (register! :my-inc (spawn inc-actor 5)) (! :my-inc :inc) (! :my-inc [:inc 4]) (! :my-inc :print) ; => 10 @andreiursan 30
31. (require '[co.paralleluniverse.pulsar.actors :refer :all]) • defactor defines a template for an actor • receive returns the next message from the mailbox • spawn launches an actor • ! sends a message to an actor • self evaluated in an actor, returns its address @andreiursan 31
32. Link Checker @andreiursan 32
33. (defactor get-link [parent link]) (defactor check-links [links] (doseq [link links] (spawn get-link @self link)) ; … @andreiursan 33
34. (require `[co.paralleluniverse.fiber.httpkit.client :as client]) (defactor get-link [parent link] (let [response (client/get link)] (! parent [:link-status {link (:status response)}]))) (defactor check-links [links] (doseq [link links] (spawn get-link @self link)) ; … @andreiursan 34
35. (require `[co.paralleluniverse.fiber.httpkit.client :as client]) (defactor get-link [parent link] (let [response (client/get link)] (! parent [:link-status {link (:status response)}]))) (defactor check-links [links] (doseq [link links] (spawn get-link @self link)) (loop [data {}] (receive [:link-status link-status] (do (recur (merge data link-status))) @andreiursan 35
36. (defactor get-link [parent link] (let [response (client/get link)] (! parent [:link-status {link (:status response)}]))) (defactor check-links [links] (doseq [link links] (spawn get-link @self link)) (loop [data {}] (receive [:link-status link-status (do (recur (merge data link-status)))] :print (do (pprint data) (recur data)) :stop (println "Good Bye!")))) (def links ["http://bobkonf.de/" "http://clojured.de"]) (register! :checker (spawn check-links links)) (! :checker :print) ;=> {"http://bobkonf.de/" 200, "http://clojured.de" 200])) @andreiursan 36
37. Supervisors @andreiursan 37
38. (defactor dangerous-dave [] (receive) (throw (Exception. "Boom! Boom!"))) (def sup (spawn (supervisor :one-for-one #()))) ; :escalate :one-for-one :all-for-one or :rest-for-one @andreiursan 38
39. (defactor dangerous-dave [] (receive) (throw (Exception. "Boom! Boom!"))) (def sup (spawn (supervisor :one-for-one #()))) (add-child! sup "dave-id" :permanent 3 1 :sec 10 dangerous-dave) ; [sup id mode max-restarts duration unit ; shutdown-deadline-millis actor-fn & actor-args] ; mode: :permanent :transient or :temporary. @andreiursan 39
40. (defactor dangerous-dave [] (receive) (throw (Exception. "Boom! Boom!"))) (def sup (spawn (supervisor :one-for-one #()))) (add-child! sup "dave-id" :permanent 3 1 :sec 10 dangerous-dave) (def dave (get-child sup "dave-id")) (! dave :foo) ; You cannot kill me! @andreiursan 40
41. Tes$ng Actors4 4 github.com/.../pulsar/actors_test.clj @andreiursan 41
42. (fact "Test actor matching receive" (let [actor (spawn (actor [] (receive :abc "yes!" :else "oy")))] (! actor :abc) (join actor)) => "yes!") @andreiursan 42
43. (fact "When matching receive and timeout then run :after clause" (let [actor (spawn (actor [] (receive [:foo] nil :else (println "got it!") :after 30 :timeout)))] (Thread/sleep 150) (! actor 1) (join actor)) => :timeout) @andreiursan 43
44. Final Thoughts • actor model can be a powerful tool • simple resource access • fits well for event-driven/async problems • tool diversity for the Clojure Ecosystem is GOOD. @andreiursan 44
45. History • 1973, Carl Hewi, et al.: A universal modular ACTOR formalism for ar<ficial intelligence • 1986, Gul Agha et. al.: Actors • 1995, Ericson: first commercial use of Erlang/OTP • 2006, Erlang/OTP SMP (adds Symmetrical Mul<Processing) • 2009, Jonas Bonér: crea<on of Akka @andreiursan 45
46. Resources • The Actor Model (video) • What are fibers and why should you care? • Farewell to Asynchronous Code • Pulsar Docs • Web Actors: Comsat • Distributed Actors in Java and Clojure • Galaxy part 1, part 2, part 3 @andreiursan 46
47. Big thanks to Fabio! @"udone, github/circlespainter @andreiursan 47
48. Thank you! @andreiursan 48
49. ? @andreiursan 49