Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Yet another introduction to core.async

Yet another introduction to core.async

A presentation given at the first #karasumaclj meetup in Kyoto

http://e6a302c89833f490f111a94ebc.doorkeeper.jp/events/17230

7759960ea09f516856e55113170442d8?s=128

Takahiro Noda

November 15, 2014
Tweet

Transcript

  1. Yet another introduction to core.async Takahrio Noda (@tnoda) 2014-11-15 Sat

  2. Outline Queue Blocking/Non-blocking Secret Alien Technology

  3. Producers and consumers p1 c1 c2 c3 c4 c5 p2

    p3 p4 p5
  4. Queue p1 Queue p2 p3 p4 p5 c1 c2 c3

    c4 c5
  5. java.util.concurrent.LinkedBlockingQueue .put LinkedBlockingQueue .take Example (Clojure) user> (import ’java.util.concurrent.LinkedBlockingQueue) java.util.concurrent.LinkedBlockingQueue

    user> (def q (LinkedBlockingQueue.)) #’user/q
  6. .put/.take Example (Clojure) user> (.put q 1) nil user> (.put

    q "foo") nil user> (.take q) 1 user> (.take q) "foo"
  7. clojure.core.async/chan put! chan take! Example (require) user> (require ’[clojure.core.async :refer

    [chan put! take!] :as async]) nil user> (def c (chan)) #’user/c
  8. put!/take! (1/2) Example (put! and take!) user> (put! c 1)

    true user> (put! c 2) true user> (take! c println) 1 nil user> (take! c println) 2 nil
  9. put!/take! (2/2) Example (Asynchronous) user> (take! c #(println "Got first:"

    %)) nil user> (take! c #(println "Got second:" %)) nil user> (put! c 11) true user> (put! c 12) true *nrepl-server* ; ; ; ; ; ; ; Got first: 11 ; ; Got second: 12
  10. close! (1/2) put! X chan take! Example (require) user> (require

    ’[clojure.core.async :refer [chan put! take! close!] :as async]) nil user> (def c (chan)) #’user/c
  11. close! (2/2) Example (close!) user> (put! c 1) true user>

    (close! c) nil user> (put! c 2) false user> (take! c println) 1 nil user> (take! c println) nil nil
  12. Overflow Example (Pending puts) user> (def c (chan)) #’user/c user>

    (dorun (repeatedly #(put! c 42))) AssertionError Assert failed: No more than 1024 pending puts are allowed on a single channel. Consider using a windowed buffer. (< (.size puts) impl/MAX-QUEUE-SIZE) clojure.core.async.impl.channels.ManyToManyChannel (channels.clj:150)
  13. dropping-buffer Example (dropping-buffer) user> (require ’[clojure.core.async :refer [chan put! take!

    close! dropping-buffer] :as async]) nil user> (def c (chan (dropping-buffer 3))) #’user/c user> (dotimes [n 10000] (put! c n)) nil user> (dotimes [_ 3] (take! c #(print % "| "))) 0 | 1 | 2 | nil
  14. sliding-buffer Example (sliding-buffer) user> (require ’[clojure.core.async :refer [chan put! take!

    close! sliding-buffer] :as async]) nil user> (def c (chan (sliding-buffer 3))) #’user/c user> (dotimes [n 10000] (put! c n)) nil user> (dotimes [_ 3] (take! c #(print % "| "))) 9997 | 9998 | 9999 | nil
  15. default chan Example (default chan) user> (require ’[clojure.core.async :refer [chan

    put! take! close!] :as async]) nil user> (def c (chan)) #’user/c user> (dotimes [n 1024] (put! c n)) nil user> (put! c 1024) AssertionError Assert failed: ...
  16. fixed-size buffer (size == 0) Example (buffer) user> (require ’[clojure.core.async

    :refer [chan put! take! close! buffer] :as async]) nil user> (def c (chan (buffer 0))) #’user/c user> (dotimes [n 1024] (put! c n)) nil user> (put! c 1024) AssertionError Assert failed: ...
  17. fixed-size buffer (size == 10) Example (buffer) user> (def c

    (chan (buffer 10))) #’user/c user> (dotimes [n 1024] (put! c n)) nil user> (dotimes [n 1024] (put! c n (println n))) 0 1 2 . . . 10 AssertionError Assert failed: ...
  18. Parking puts put! take! Parking (up to 1024) Buffer Channel

  19. Checkpoint 1 chan puts! take! close! buffer No more than

    1024 pending puts are allowed on a single channel. dropping-buffer sliding-buffer buffer default (size == 0)
  20. Return value LinkedBlockingQueue .put LinkedBlockingQueue .take value chan put! chan

    take! nil
  21. >!! and <!! Example (Blocking-put and blocking-take) user> (put! c

    1) true user> (put! c 2) true user> (<!! c) 1 user> (<!! c) 2 user> (>!! c 3) ; will be blocked
  22. Return values nil (callback) put! / take! Value >!! /

    <!! Channel chan ??? ??? ???
  23. Thread Example (future) user> (def f #(-> (apply *’ (range

    1 100000)) (rem 1000000009))) #’user/f user> (def c (chan)) #’user/c user> (future (>!! c (f))) #<core$future_call$reify__6320@47e6951e: :pending> user> (<!! c) 722570113N
  24. thread Example (thread) user> (<!! (thread (f))) 722570113N

  25. timeout Example (Thread/sleep and future) user> (def c (let [ch

    (chan)] (future (Thread/sleep 3000) (close! ch)) ch)) #’user/c user> (<!! c) nil Example (timeout) user> (<!! (timeout 3000)) nil
  26. Checkpoint 2 Blocking/Non-blocking Blocking Return value put!, take! No nil

    (callback) >!!, <!! Yes value Functions that return a channel chan thread timeout ???
  27. Promise >!! promise >!! promise <!! promise <!! promise OS

    thread OS thread OS thread OS thread
  28. Secret Alien Technology >! ??? >! ??? <! ??? <!

    ??? Thread Pool OS thread OS thread
  29. go go block (go ; do >! or <! here

    ) Example (return value) user> (def f #(-> (apply *’ (range 1 100000)) (rem 1000000009))) #’user/f user> (go (f)) #<ManyToManyChannel clojure.core.async.impl.channels.ManyToManyC user> (<!! (go (f))) 722570113N
  30. 10k go-blocks demo

  31. Go blocks Example (Quil) (def c (chan (sliding-buffer 10000))) (defn

    -main [& args] (dotimes [x size] (dotimes [y size] (go-loop [] (>! c [x y (rand-int 255)]) (<! (timeout (rand-int 1000))) (recur)))) (q/defsketch core-async-demo :size [(* size scale) (* size scale)] :draw draw))
  32. go/go-loop Example (go + loop) (go (loop [x a y

    b] ; do something (recur c d))) Example (go-loop) (go-loop [x a y b] ; do something (recur c d))
  33. Go blocks (recap) Example (Quil) (def c (chan (sliding-buffer 10000)))

    (defn -main [& args] (dotimes [x size] (dotimes [y size] (go-loop [] (>! c [x y (rand-int 255)]) (<! (timeout (rand-int 1000))) (recur)))) (q/defsketch core-async-demo :size [(* size scale) (* size scale)] :draw draw))
  34. alt!! Channels c1 <!! c2 <!! alt!! c1 alt!! c2

  35. alt! alt! c1 alt! c2 alt!! c1 alt!! c2

  36. Checkpoint 3 Go blocks go, go-loop >!, <! Mixing alt!!,

    alt!
  37. core.async Channel chan, thread, timeout, go Blocking, Non-blocking >!!, <!!,

    put!, take!, >!, <! Mixing alt!!, alt!