Yet another introduction to core.async

A presentation given at the first #karasumaclj meetup in Kyoto

http://e6a302c89833f490f111a94ebc.doorkeeper.jp/events/17230

Transcript

1. Yet another introduction to core.async Takahrio Noda (@tnoda) 2014-11-15 Sat
2. Outline Queue Blocking/Non-blocking Secret Alien Technology
3. Producers and consumers p1 c1 c2 c3 c4 c5 p2 p3 p4 p5
4. Queue p1 Queue p2 p3 p4 p5 c1 c2 c3 c4 c5
5. java.util.concurrent.LinkedBlockingQueue .put LinkedBlockingQueue .take Example (Clojure) user> (import ’java.util.concurrent.LinkedBlockingQueue) java.util.concurrent.LinkedBlockingQueue user> (def q (LinkedBlockingQueue.)) #’user/q
6. .put/.take Example (Clojure) user> (.put q 1) nil user> (.put q "foo") nil user> (.take q) 1 user> (.take q) "foo"
7. clojure.core.async/chan put! chan take! Example (require) user> (require ’[clojure.core.async :refer [chan put! take!] :as async]) nil user> (def c (chan)) #’user/c
8. put!/take! (1/2) Example (put! and take!) user> (put! c 1) true user> (put! c 2) true user> (take! c println) 1 nil user> (take! c println) 2 nil
9. put!/take! (2/2) Example (Asynchronous) user> (take! c #(println "Got first:" %)) nil user> (take! c #(println "Got second:" %)) nil user> (put! c 11) true user> (put! c 12) true *nrepl-server* ; ; ; ; ; ; ; Got first: 11 ; ; Got second: 12
10. close! (1/2) put! X chan take! Example (require) user> (require ’[clojure.core.async :refer [chan put! take! close!] :as async]) nil user> (def c (chan)) #’user/c
11. close! (2/2) Example (close!) user> (put! c 1) true user> (close! c) nil user> (put! c 2) false user> (take! c println) 1 nil user> (take! c println) nil nil
12. Overflow Example (Pending puts) user> (def c (chan)) #’user/c user> (dorun (repeatedly #(put! c 42))) AssertionError Assert failed: No more than 1024 pending puts are allowed on a single channel. Consider using a windowed buffer. (< (.size puts) impl/MAX-QUEUE-SIZE) clojure.core.async.impl.channels.ManyToManyChannel (channels.clj:150)
13. dropping-buffer Example (dropping-buffer) user> (require ’[clojure.core.async :refer [chan put! take! close! dropping-buffer] :as async]) nil user> (def c (chan (dropping-buffer 3))) #’user/c user> (dotimes [n 10000] (put! c n)) nil user> (dotimes [_ 3] (take! c #(print % "| "))) 0 | 1 | 2 | nil
14. sliding-buffer Example (sliding-buffer) user> (require ’[clojure.core.async :refer [chan put! take! close! sliding-buffer] :as async]) nil user> (def c (chan (sliding-buffer 3))) #’user/c user> (dotimes [n 10000] (put! c n)) nil user> (dotimes [_ 3] (take! c #(print % "| "))) 9997 | 9998 | 9999 | nil
15. default chan Example (default chan) user> (require ’[clojure.core.async :refer [chan put! take! close!] :as async]) nil user> (def c (chan)) #’user/c user> (dotimes [n 1024] (put! c n)) nil user> (put! c 1024) AssertionError Assert failed: ...
16. fixed-size buffer (size == 0) Example (buffer) user> (require ’[clojure.core.async :refer [chan put! take! close! buffer] :as async]) nil user> (def c (chan (buffer 0))) #’user/c user> (dotimes [n 1024] (put! c n)) nil user> (put! c 1024) AssertionError Assert failed: ...
17. fixed-size buffer (size == 10) Example (buffer) user> (def c (chan (buffer 10))) #’user/c user> (dotimes [n 1024] (put! c n)) nil user> (dotimes [n 1024] (put! c n (println n))) 0 1 2 . . . 10 AssertionError Assert failed: ...
18. Parking puts put! take! Parking (up to 1024) Buffer Channel
19. Checkpoint 1 chan puts! take! close! buffer No more than 1024 pending puts are allowed on a single channel. dropping-buffer sliding-buffer buffer default (size == 0)
20. Return value LinkedBlockingQueue .put LinkedBlockingQueue .take value chan put! chan take! nil
21. >!! and <!! Example (Blocking-put and blocking-take) user> (put! c 1) true user> (put! c 2) true user> (<!! c) 1 user> (<!! c) 2 user> (>!! c 3) ; will be blocked
22. Return values nil (callback) put! / take! Value >!! / <!! Channel chan ??? ??? ???
23. Thread Example (future) user> (def f #(-> (apply *’ (range 1 100000)) (rem 1000000009))) #’user/f user> (def c (chan)) #’user/c user> (future (>!! c (f))) #<core$future_call$reify__6320@47e6951e: :pending> user> (<!! c) 722570113N
24. thread Example (thread) user> (<!! (thread (f))) 722570113N
25. timeout Example (Thread/sleep and future) user> (def c (let [ch (chan)] (future (Thread/sleep 3000) (close! ch)) ch)) #’user/c user> (<!! c) nil Example (timeout) user> (<!! (timeout 3000)) nil
26. Checkpoint 2 Blocking/Non-blocking Blocking Return value put!, take! No nil (callback) >!!, <!! Yes value Functions that return a channel chan thread timeout ???
27. Promise >!! promise >!! promise <!! promise <!! promise OS thread OS thread OS thread OS thread
28. Secret Alien Technology >! ??? >! ??? <! ??? <! ??? Thread Pool OS thread OS thread
29. go go block (go ; do >! or <! here ) Example (return value) user> (def f #(-> (apply *’ (range 1 100000)) (rem 1000000009))) #’user/f user> (go (f)) #<ManyToManyChannel clojure.core.async.impl.channels.ManyToManyC user> (<!! (go (f))) 722570113N
30. 10k go-blocks demo
31. Go blocks Example (Quil) (def c (chan (sliding-buffer 10000))) (defn -main [& args] (dotimes [x size] (dotimes [y size] (go-loop [] (>! c [x y (rand-int 255)]) (<! (timeout (rand-int 1000))) (recur)))) (q/defsketch core-async-demo :size [(* size scale) (* size scale)] :draw draw))
32. go/go-loop Example (go + loop) (go (loop [x a y b] ; do something (recur c d))) Example (go-loop) (go-loop [x a y b] ; do something (recur c d))
33. Go blocks (recap) Example (Quil) (def c (chan (sliding-buffer 10000))) (defn -main [& args] (dotimes [x size] (dotimes [y size] (go-loop [] (>! c [x y (rand-int 255)]) (<! (timeout (rand-int 1000))) (recur)))) (q/defsketch core-async-demo :size [(* size scale) (* size scale)] :draw draw))
34. alt!! Channels c1 <!! c2 <!! alt!! c1 alt!! c2
35. alt! alt! c1 alt! c2 alt!! c1 alt!! c2
36. Checkpoint 3 Go blocks go, go-loop >!, <! Mixing alt!!, alt!
37. core.async Channel chan, thread, timeout, go Blocking, Non-blocking >!!, <!!, put!, take!, >!, <! Mixing alt!!, alt!