The State of Concurrency in Clojure

Transcript

1. The State of Concurrency in Clojure atoms, agents, refs, and core.async
2. “Shared mutable state is the root of all evil.”
3. State, Identity, and Value • Values don't change (if they did, they'd be different values) • e.g., 72, {–, ‖, —, ―}, "cat", [2, 3, 5, 7] • An Identity is an entity that is associated with different values over time • State is a value at a point in time • Identities give us continuity across state changes
4. Imperative programming and State • "Stop the World" programming model • Assumes observing and changing data happens immediately (that computation halts until an operation is completed) • In OOP, Complects Identity and State • No way to obtain state independent of identity without copying • Objects not treated as values • Unsustainable single-threaded premise
5. Mutable values + Identities = Complexity favorite_langs_joe = Set.new ["Clojure", "Swift"] favorite_langs_bob = favorite_langs_joe favorite_langs_joe << "J" favorite_langs_joe # #<Set: {"Clojure", "Swift", "J"}> favorite_langs_bob # #<Set: {"Clojure", "Swift", "J"}>
6. Imperative programming with multi-threading • Attempts to maintain the illusion that the world stops as data is observed or mutated by multiple participants, which is both expensive and complex: • Requires coordination with mutexes/locks • Changes must be propagated to shared memory to be visible to other threads • Writers must block readers / readers must block writers to observe and operate on stable state • Complexity rapidly increases with more threads • Dead-lock, Live-lock, Race-conditions
7. Clojure • Functional programming language • Lisp dialect • Focus on Simplicity • Decomplects State and Identity • Managed refs • Designed around Immutability.
8. Immutable Data • New values are functions of old values • An Identity's state at any point in time is an immutable value • Values can always be observed and new values calculated from old values without coordination • Values will never change "in hand" • Values can be safely shared between threads
9. Atomic references to values • Models Identity by way of references to immutable values • Dereferencing a reference type gives you its (immutable) value • Changes to an identity's state are controlled/coordinated by the system
10. Clojure Reference Containers Model Usage Functions Atoms Synchronized, Independent updates pure Refs and STM Synchronized, Coordinated updates pure Agents Asynchronous, Independent updates any Vars Thread-local updates any
11. Clojure Syntax in a few slides
12. Types symbol blah keyword :thing string "hello, Lambda Lounge!" integer 12345 float 3.141 exponent notation 2e5 regular expression #"^hello,\s\w+" booleans true false undefined nil
13. Persistent, Immutable Data-structures list '(1 2 3 "badger") vector [1 2 3 "squirrel!"] map {:name "Bob" :age 34} set #{"cat" "dog" "snake"}
14. Expressions (filter odd? (map inc [7 8 9 10 11 12])) => [9 11 13] (->> [7 8 9 10 11 12] (map inc) (filter odd?))
15. Special Forms (def symbol init?) define symbol (if test then else?) conditional, yields `then` or `else` (do exprs*) evaluate expressions, return last (let [bindings* ] exprs*) lexical context (quote form) yields an unevaluated form (unquote form) unquotes a quoted form (var symbol) the var object named by a symbol (fn name? ([params*] cond-map? exprs*) +) defines a function (loop [bindings* ] exprs*) like let, but provides a recursion target (recur exprs*) evaluates expressions and rebinds at recur point (throw expr) evaluates expr and throws result (try expr* catch-clause* finally- clause?) try/catch/finally semantics . new set! javascript interop
16. (defn make-adder [inc-by] (fn [n] (+ inc-by n))) (def add3 (make-adder 3)) (add3 9) => 4
17. Atoms • Manage shared, independent state • Ideal for updates that commute • Synchronous, independent updates • State changes by applying pure functions to the current state's value to produce the next state's value • Operations: • swap!, reset!, compare-and-set!, deref, add-watch, set-validator!
18. deref / @ Returns a reference type's current state (deref ref) @ref (def myatom (atom {:name "Joe"})) (deref myatom) => {:name "Joe"} @myatom => {:name "Joe"}
19. swap! Atomically swaps the value of atom `a` to be: (apply f current-value-of-atom args). Returns the value that was swapped in. (swap! a f) (swap! a f & args) (def mycounter (atom 0)) (swap! mycounter inc) => 1 (def names (atom [])) (swap names conj "John") => ["John"]
20. atom a (f @a …) a₁ (g @a …) a₂ Time a ₁ !₌ a ₂ (g @a …) a₃ retry (swap! a f …) (swap! a g …) (deref a) a₂
21. Atoms Demo
22. Agents • Bound to a single storage location, managing updates reactively, serially • Ideal for guarding a contested resource • Asynchronous, independent updates • State changes by asynchronously applying pure functions to the current state's value to produce the next state's value • Agents update are applied in the order received by a thread from the agent thread pool • Operations: • send, send-off, deref, reset-agent, agent-error, shutdown-agents, add-watch, set- validator!
23. send Dispatch an action to an agent. Returns the agent immediately. Subsequently, in a thread from a thread pool, the state of the agent will be set to the value of: (apply action-fn state-of-agent args) (send a f & args) (defn log-msg [msg user] {:timestamp (java.util.Date.) :msg msg :user user}) (def command-log (agent [])) (send command-log conj (log-msg "Fire ze missiles!" "Joe")) @command-log => [{:timestamp #inst"2016-02-15T01:23:49.745-00:00", :msg "Fire ze missiles!", :user "Joe"}]
25. Agent Demo
26. Refs and Software Transactional Memory • Synchronous, coordinated update • Ideal for coordinating changes to multiple states • Updates in transactions; Transactions "snapshot" of data + in-transaction-value • actions on refs are atomic, consistent, and isolated from other transactions • Software Transactional memory implements a form of MVCC with adaptive history queues for snapshot isolation • Operations: • alter, ref-set, commute, ensure
27. alter, dosync Sets the in-transaction-value of ref to: (apply fun in-transaction-value-of-ref args) and returns the in-transaction-value of ref. (alter ref func & args) (def in-ref (ref [:a :b :c :d])) (def out-ref (ref [])) (dosync (alter out-ref conj (peek @in-ref)) (alter in-ref pop)) @in-ref => [:b :c :d] @out-ref => [:a]
29. Refs/STM Demo
30. core.async • Communicating Sequential Processes (C.A.R. Hoare) • Asynchronous blocks of code communicating over "channels" • Channels are queues that can be combined, split, filtered, mapped over, etc. • Channels can exert "backpressure" and have different kinds of buffers of different sizes (sliding, dropping, etc.) • async blocks execute in a thread pool • Think Go, but richer API and works in ClojureScript (in browsers!)
31. core.async (def msg-chan (chan 10)) (go-loop [] (when-let [msg (<! msg-chan)] (expensive-long-running-operation msg) (println (str "processed message: " (:id msg))) (recur))) (>!! msg {:id 1 :value 72})
32. core.async (>!! c val) puts a val onto a channel. nil values are not allowed. Will block if no buffer space is available. Returns nil. (<!! c) takes a val from a channel. Will return nil if closed. Will block if nothing is available.
33. core.async Demo