Ben Mabey Communicating Sequential Processes in Clojure core.async @bmabey Lambda Lounge Utah, April 8 2014

Everything is a Remix … but especially this presentation.

Everything is a Remix … but especially this presentation. https://speakerdeck.com/hlship/mastering-time-with-clojure-core-dot-async Mastering Time with Clojure core.async by Howard M. Lewis Ship http://www.infoq.com/presentations/Death-by-Accidental-Complexity Death by Accidental Complexity by Ulf Wiger https://github.com/stuarthalloway/presentations/wiki/Core-Async core.async by Stuart Halloway

Event loop w/Callbacks Promises/Futures/Tasks Rx (push based FRP) Coroutines & Generators Actors Communicating Sequential Processes (CSP) async keyword

core.async is an efficient implementation of CSP that drastically reduces the accidental complexity usually associated with concurrent programming.

core.async is an efficient implementation of CSP that drastically reduces the accidental complexity usually associated with concurrent programming.

Embarrassingly Parallel …emparallel? aka brain dead concurrency

(defn request-handler! [request]! (let [data1 (do-query-1 request)! data2 (do-query-2 request)! data3 (do-query-3 request)]! (send-response data1 data2 data3))) Embarrassingly Parallel …emparallel? aka brain dead concurrency

1 3 2

1 3 2

1 3 2

Channel Writer Writer Writer Reader Reader

Channel Writer Writer Writer Reader Reader 1 2 3

Channel Writer Writer Writer Reader Reader 1 2 3 Any value can be placed in channel, except for nil

Channel Writer Writer Writer Reader Reader 1 2 3

Channel Writer Writer Writer Reader Reader 1 2 3

Channel Writer Writer Writer Reader Reader 1 2 3

Channel Writer Writer Writer Reader Reader 3

Channel Writer Writer Writer Reader Reader

Channel Reader Reader (close! channel)

Channel Reader Reader (close! channel) nil nil

Channel Writer Writer Writer

Channel Writer Writer Writer 1 2 3 4

Channel Writer Reader Reader Writer Writer 1 2 3 4

Channel Writer Reader Reader Writer Writer 1 2 3 4

No content

(defn request-handler! [request]! (let [data1 (do-query-1 request)! data2 (do-query-2 request)! data3 (do-query-3 request)]! (send-response data1 data2 data3)))

(defn request-handler [request] (let [query1 (thread (do-query-1 request)) query2 (thread (do-query-2 request)) query3 (thread (do-query-3 request)) data1 (

(defn request-handler [request] (let [query1 (thread (do-query-1 request)) query2 (thread (do-query-2 request)) query3 (thread (do-query-3 request)) data1 (

(thread (do-query-1 request)) (

1 3 2

core.async is an efficient implementation of CSP that drastically reduces the accidental complexity usually associated with concurrent programming.

No content

No content

(defn request-handler [request] (let [query1 (thread (do-query-1 request)) query2 (thread (do-query-2 request)) query3 (thread (do-query-3 request)) data1 (

(defn request-handler [request] (let [result (go (let [query1 (thread (do-query-1 request)) query2 (thread (do-query-2 request)) query3 (thread (do-query-3 request)) data1 (

(go (let […
 data1 (

(go (let […
 data1 (

1 3 2

1 3 2

(defn request-handler [request] (go (let [query1 (thread (do-query-1 request)) query2 (thread (do-query-2 request)) query3 (thread (do-query-3 request)) data1 (

query1 (thread (do-query-1 request)) query2 (thread (do-query-2 request)) query3 (thread (do-query-3 request)) 0 data1 (

No content

No content

(defn google [query]! (let [results-chan (chan)! timeout-chan (timeout 100)]! (go (>! results-chan (! results-chan (! results-chan (>! (alt! [results-chan timeout-chan] ([v] v))! (conj results))))))) Shared timeout!

Event loop w/Callbacks Promises/Futures/Tasks Rx (push based FRP) Coroutines & Generators Actors Communicating Sequential Processes (CSP) async keyword

No content

var  tweets,  answers,  kaggleProfile;
  
 twitter.getTweetsFor("bmabey",  function  (result)  {
    tweets  =  result;
    somethingFinished();
 });
 stackOverflow.getAnswersFor("bmabey",  function  (result)   {
    answers  =  result;
    somethingFinished();
 });
 kaggle.getProfileOf("bmabey",  function  (result)  {
    kaggleProfile  =  result;
    somethingFinished();
 }); var  finishedSoFar  =  0;
  
 function  somethingFinished()  {
    if  (++finishedSoFar  ===  3)  {
        ui.show(tweets,  answers,  kaggleProfile);
    }
 } http://www.slideshare.net/domenicdenicola/callbacks-promises-and-coroutines-oh-my-the-evolution-of-asynchronicity-in-javascript

var  tweets,  answers,  kaggleProfile;
  
 twitter.getTweetsFor("bmabey",  function  (result)  {
    tweets  =  result;
    somethingFinished();
 });
 stackOverflow.getAnswersFor("bmabey",  function  (result)   {
    answers  =  result;
    somethingFinished();
 });
 kaggle.getProfileOf("bmabey",  function  (result)  {
    kaggleProfile  =  result;
    somethingFinished();
 }); var  finishedSoFar  =  0;
  
 function  somethingFinished()  {
    if  (++finishedSoFar  ===  3)  {
        ui.show(tweets,  answers,  kaggleProfile);
    }
 } http://www.slideshare.net/domenicdenicola/callbacks-promises-and-coroutines-oh-my-the-evolution-of-asynchronicity-in-javascript

var  tweets,  answers,  kaggleProfile;
  
 twitter.getTweetsFor("bmabey",  function  (result)  {
    tweets  =  result;
    somethingFinished();
 });
 stackOverflow.getAnswersFor("bmabey",  function  (result)   {
    answers  =  result;
    somethingFinished();
 });
 kaggle.getProfileOf("bmabey",  function  (result)  {
    kaggleProfile  =  result;
    somethingFinished();
 }); var  finishedSoFar  =  0;
  
 function  somethingFinished()  {
    if  (++finishedSoFar  ===  3)  {
        ui.show(tweets,  answers,  kaggleProfile);
    }
 } - Continuation Passing Style - Destroys readability - Forces shared mutable state - Can’t use normal language constructs for error handling and looping. http://www.slideshare.net/domenicdenicola/callbacks-promises-and-coroutines-oh-my-the-evolution-of-asynchronicity-in-javascript

(go! (let [results (chan 3)]! (twitter/get-tweets "bmabey" c)! (stack-overflow/get-answers "bmabey" c)! (kaggle/get-profile "bmabey" c)! (send-async-response (async/into [] results)))

No content

Q.all([
    twitter.getTweetsFor("bmabey"),
    stackOverflow.getAnswersFor("bmabey"),
    kaggle.getProfileOf("bmabey")   ]).then(function  (results)  {
    console.log(results[0],  results[1],  results[2]);
 }); http://www.slideshare.net/domenicdenicola/callbacks-promises-and-coroutines-oh-my-the-evolution-of-asynchronicity-in-javascript

Q.all([
    twitter.getTweetsFor("bmabey"),
    stackOverflow.getAnswersFor("bmabey"),
    kaggle.getProfileOf("bmabey")   ]).then(function  (results)  {
    console.log(results[0],  results[1],  results[2]);
 }); http://www.slideshare.net/domenicdenicola/callbacks-promises-and-coroutines-oh-my-the-evolution-of-asynchronicity-in-javascript + Promises are values + Composable + Better error handling - Continuation Passing Style! - Still can’t use language constructs. - Mutable state still flying around. - By design, are not for streams! - Have to create a new promise for each event.. every time! !

C# Async async void Go() {! _button.IsEnabled = false;! string[] urls = "clojure.org www.albahari.com/nutshell/ golang.org".Split();! int totalLength = 0;! foreach (string url in urls)! {! var uri = new Uri ("http://" + url);! byte[] data = await new WebClient().DownloadDataTaskAsync (uri); _results.Text += "Length of " + url + " is " + data.Length +! totalLength += data.Length;! }! _results.Text += "Total length: " + totalLength;! }

C# Async async void Go() {! _button.IsEnabled = false;! string[] urls = "clojure.org www.albahari.com/nutshell/ golang.org".Split();! int totalLength = 0;! foreach (string url in urls)! {! var uri = new Uri ("http://" + url);! byte[] data = await new WebClient().DownloadDataTaskAsync (uri); _results.Text += "Length of " + url + " is " + data.Length +! totalLength += data.Length;! }! _results.Text += "Total length: " + totalLength;! } + No more Continuation Passing Style! - Still can’t use for streams.

Async Adoption 2007 - F# adds introduces async workflows 2010? - C# adds async 2012-13? - Scala library adds async via macros 2013 - Python announced it will be adding it 2014 - Javascript, ES7 proposal for async

No content

(defn batch-until-pause! "Batches values from the `in` channel into a single a vec of values up until no! messages have appeared in `timeout-pause` milliseconds. `max-batch-time` is the! number of milliseconds to wait overall before sending a batch."! ([timeout-pause max-batch-time in]! (batch-until-pause timeout-pause max-batch-time in (chan)))! ([timeout-pause max-batch-time in out]! (go! (loop [batch []! batch-timeout (timeout max-batch-time)]! (alt!! ;; batch timed out, send what we have now and start a new batch timeout! batch-timeout (do! (>! out batch)! (recur [] (timeout max-batch-time)))! in ([v]! (if v! ;; we got a value, add it to the batch! (recur (conj batch v) batch-timeout)! ;; we got nil so the 'in' channel was closed, send our batch! (>! out batch)))! ! ;; a timeout happened, send the batch and start a new one! (timeout timeout-pause) (do! (>! out batch)! (send-batch!)! (recur [] batch-timeout))))! (close! out))! out))

What about Rx?

What about Rx? + Very composable + Handles streams - Push-based semantics only cover ~80% use cases - Have to learn all the combinators - Writing a combinator can be difficult. - Mutable state with call backs again! ! Awesome abstraction, but in the end it is more complex and less powerful than CSP.

No content

No content

We’ve covered brain dead concurrency… what about mind blowing concurrency?

No content

No content

(defn start-phone-process [phone]! (let [...]! (go-loop []! (alt!! off-hook (

(defn start-phone-process [phone]! (let [...]! (go-loop []! (alt!! off-hook (

No content

No content

No content

No content

No content

Differences with Actors…

Macro Limitations

Thank you! ! ! BenMabey.com ! github.com/bmabey ! @bmabey !