Devoxx Antwerpen 2015 Concurrency Models on the JVM Lutz Hühnken @lutzhuehnken https://github.com/lutzh/concurrency_talk

#devoxx 2 Before we start: Why Concurrency is hard (demonstrated with printed books)

#devoxx 3

#devoxx 4

#devoxx 5

#devoxx 6

#devoxx 7 Provocative claim (for you to discuss over drinks later): We need a new standard concurrency model for Java!

#devoxx 8 Thread Problems Problem 1: Efficiency (resp. the lack thereof)

#devoxx 9 CPU Threads Work

#devoxx 10 1 2 3 … 10.000 4-core CPU Thousands of Threads Work (a lot)

#devoxx 11 We want to move towards this:

#devoxx 12 From Thread Process Process Worker Task To Also see https://en.wikipedia.org/wiki/Reactor_pattern

#devoxx Thinking about blocking I/O When one thread corresponds to one task 13 Note: This is a snapshot in time of the system, not a sequence of events

#devoxx Never block! When one thread is a worker, working on multiple tasks 14 Note: This is a snapshot in time of the system, not a sequence of events

#devoxx Solution for the efficiency problem: • Sub-thread level concurrency • Asynchronous I/O
 • This is something the approaches we will look at have in common
 • Note: This is also something that all Reactive Systems* have in common
 * http://www.reactivemanifesto.org 15

#devoxx 16 Thread Problems Problem 1: Efficiency (resp. the lack thereof) Problem 2: Programming Model

#devoxx 17 They discard the most essential and appealing properties of sequential computation: understandability, predictability, and determinism. Threads, as a model of computation, are wildly nondeterministic, and the job of the programmer becomes one of pruning that nondeterminism.

#devoxx 18 Source: John Rose, Java VM Architect, JFokus, Stockholm, February 2015

#devoxx Digression: Callback Hell fs.readdir(source, function(err, files) { if (err) { console.log('Error finding files: ' + err) } else { files.forEach(function(filename, fileIndex) { console.log(filename) gm(source + filename).size(function(err, values) { if (err) { console.log('Error identifying file size: ' + err) } else { console.log(filename + ' : ' + values) aspect = (values.width / values.height) widths.forEach(function(width, widthIndex) { height = Math.round(width / aspect) console.log('resizing ' + filename + 'to ' + height + 'x' + height) this.resize(width, height).write(destination + 'w' + width + '_' + filename, function(err) { if (err) console.log('Error writing file: ' + err) }) }.bind(this)) } }) }) } }) 19

#devoxx I admit: It could be written in a more readable way • Scala Futures / for-expressions • async / await (also Scala) • JavaScript Promises (then.. then.. then)
 • All this is syntactic sugar for callbacks - which is good • But let’s explore other (better) approaches for composition 20

#devoxx 21 What are we interested in? Composition State Management Integration, I/O (both async and sync) Predictability

#devoxx 22 Coroutines (Green Threads, User Mode Threads, IOC Threads, Fibers) Quasar Fibers http://www.paralleluniverse.co/quasar/ Channels Clojure core.async https://github.com/clojure/core.async/ Event Loop vert.x http://vertx.io Actor Model Akka http://akka.io Concurrency Models

Fibers

#devoxx 24 new Fiber() { @Override protected V run() throws SuspendExecution, InterruptedException { // code here } }.start(); looks like a thread, walks like a thread..

#devoxx 25 class RetrieveInfo extends Fiber { ... @Override public Void run() { final CloseableHttpClient client = FiberHttpClientBuilder.create().build(); try { String response = client.execute(new HttpGet(MessageFormat.format(url, searchTerm)), new BasicResponseHandler()); addResult(key, StringUtils.abbreviate(response, 1000)); } catch (IOException iox) { ... Use like blocking Need to aggregate results.. „fiber blocking“ async http client

#devoxx 26 private final Map store = new ConcurrentHashMap<>(); public void define(final String searchTerm) { for (String currentKey : sources.keySet()) { new RetrieveInfo(currentKey, sources.get(currentKey), searchTerm).start(); } } /** * This method is working on shared mutual state - this is what we look to avoid! */ void addResult(final String key, final String result) { store.put(key, result); ... This method is working on shared mutual state - this is what we look to shared mutable state Start retrieval of definitions in background

#devoxx Fibers • Major advantage: Imperative Programming, just like with threads
 • Major disadvantage: Imperative Programming, just like with threads 27

#devoxx 28 class FooAsync extends FiberAsync implements FooCompletion { @Override public void success(String result) { asyncCompleted(result); } @Override public void failure(FooException exception) { asyncFailed(exception); } } String op() { new FooAsync() { protected void requestAsync() { Foo.asyncOp(this); } }.run(); } Turn callback APIs into fiber blocking APIs!

#devoxx Fibers Summary • Thread-like programming model • Nice „tricks“: Instrumentation, Suspendable, Thread Interop • Fibers alone are a bit low-level • (Almost) drop-in replacement for threads 29

Channels

#devoxx Channels • Theoretical Foundation:
 Communicating Sequential Processes (Tony Hoare 1978, https:// en.wikipedia.org/wiki/ Communicating_sequential_processes) • Very popular in the „Go“ community • Our example: Clojure core.async 31 Channels

#devoxx 32 (def echo-chan (chan)) (go (println (!! echo-chan „ketchup") ; => true ; => ketchup go block - like a fiber >!! blocks thread

#devoxx 33 (def echo-buffer (chan 2)) (>!! echo-buffer "ketchup") ; => true (>!! echo-buffer "ketchup") ; => true (>!! echo-buffer "ketchup") ; blocks buffered channel - async

#devoxx 34 (def responses-channel (chan 10)) (go (println (loop [values []] (if (= 3 (count values)) values (recur (conj values (! result (:body %))))) (async-get "http:m-w.com..." responses-channel) (async-get "http:wiktionary.." responses-channel) (async-get "http:urbandictionary.." responses-channel) callback puts result in channel channel for responses Aggregate results recursively

#devoxx Channels • Allows flow-oriented programming, very flexible composition (e.g. alts!) • Sanity through sequencing • go block / thread duality allows integration with both async and sync libraries • Channels as keepers of state? 35 Channels Summary

Event Loop

#devoxx Channels • Event loop model - think node.js • „multi event loop“, on the JVM • Polyglot - Java, Scala, Clojure, JRuby, JS.. • Deploy small pieces (Verticles) and have them send JSON to each other 37 vert.x

#devoxx 38 public class Receiver extends AbstractVerticle { @Override public void start() throws Exception { EventBus eb = vertx.eventBus(); eb.consumer("ping-address", message -> { System.out.println("Received message: " + message.body()); // Now send back reply message.reply("pong!"); }); System.out.println("Receiver ready!"); } } register on event bus extend Verticle connect

#devoxx 39 MessageConsumer consumer = eventBus.consumer("definer.task"); consumer.handler(message -> { httpClient.getNow(message.body().getString("host"), message.body().getString("path"), httpClientResponse -> { httpClientResponse.bodyHandler(buffer -> { eventBus.send("collector", new JsonObject() .put(...) callback based async http register send result as message

#devoxx 40 Vertx vertx = Vertx.vertx(); EventBus eventBus = vertx.eventBus(); DeploymentOptions options = new DeploymentOptions().setInstances(sources.size()); vertx.deployVerticle("de.huehnken.concurrency.Task", options, msg -> { for (String currentKey : sources.keySet()) { eventBus.send("definer.task", new JsonObject() .put(..) which one gets the message? deploy multiple

#devoxx 41 private final Map store = new HashMap<>(); @Override public void start(Future startFuture) { EventBus eventBus = vertx.eventBus(); MessageConsumer consumer = eventBus.consumer("collector"); consumer.handler(message -> { store.put(message.body().getString(„key"), message.body().getString("definition")); wait for results.. encapsultate state in Verticle messages are usually JSON

#devoxx Event Bus (vert.x) 42 Image from Jonas Bandi @jbandi

#devoxx Channels • Loose coupling - very • Sanity through events („single thread illusion“) • Hybrid thread model allows integration of synchronous APIs, off-loading work from event loop • State handling needs some care • Utility APIs are callback heavy, node inspired • Distributed! Unified model for local and distributed concurrency 43 vert.x Summary

Actors

#devoxx Channels • The actor model [..] is a [model] that treats "actors" as the universal primitives of concurrent computation: in response to a message that it receives, an actor can make local decisions, create more actors, send more messages, and determine how to respond to the next message received. https://en.wikipedia.org/wiki/Actor_model • Described by Carl Hewitt 1973 • Popular in the Erlang community 45 Akka

#devoxx Actors (Akka) 46

#devoxx Actors (Akka) 47

#devoxx Actors (Akka) 48

#devoxx 49 class Coordinator(searchTerm: String) extends Actor { var results = Map[String, String]() def receive = { case StartCommand => sources foreach { case (key, url) => context.actorOf(Task.props(key, self)) ! GetDefinition(url) } context.system.scheduler.scheduleOnce(5 seconds, self, Done) case Result(key, definition) => results += key -> definition if (results.size == sources.size) { self ! Done } case Done => spawn child actors, send command encapsulate state aggregate results

#devoxx 50 class Task(key: String, coordinator: ActorRef) extends Actor { def receive = { case GetDefinition(url) => http.singleRequest(HttpRequest(uri = url)) pipeTo self case HttpResponse(StatusCodes.OK, headers, entity, _) => coordinator ! Result(key, entity.dataBytes.utf8String) case HttpResponse(code, _, _, _) => // handle error send result to aggregator async call, translated to message

#devoxx Actors (Akka) 51

#devoxx Channels • Sanity through messages („single thread illusion“) • Messages (as opposed to events) • Dispatchers allows integration of synchronous APIs, off-loading work from event loop • pipeTo & ask to integrate with async APIs • Distributed! Unified model for local and distributed concurrency. • Supervision! Fault handling enforced. 52 Actors Summary

#devoxx 53 Programming model chart Tasks (sub- thread level) Asynchronous Messaging Distribution (unified model) Supervision Fibers ✔ Channels (core.async) ✔ ✔ Event Bus (vert.x) ✔ ✔ ✔ Aktoren (Akka) ✔ ✔ ✔ ✔

#devoxx • Concurrency is interesting
 • Threads are passé, we want a better abstraction on top of it • Alternatives are available today
 • If you only have time to look at one model, I recommend Akka 54 Summary

#devoxx 55 Closing the loop.. another book..

Thank You Lutz Hühnken @lutzhuehnken https://github.com/lutzh/concurrency_talk

Agents

#devoxx 58 (def x (agent 0)) (defn increment [c n] (+ c n)) (send x increment 5) ; @x -> 5 (send x increment 10) ; @x -> 15 „wrap“ a value in an agent send the code to the agent to modify value! a function

#devoxx Agents • The agent encapsulates state • Send code to the agent to change value • Interesting feature especially for functional programming • Async execution, but not really a concurrency model 59

#devoxx Beware of the benchmark! • Unfit: CPU bound • Also unfit: Anything using blocking I/O • It’s not about single request performance, it’s about scalability! 60

#devoxx But I just want to build a little REST service.. • If you don’t program on this level yourself: know you frameworks! • Look out for „reactive“ or „async“ • Play! • Vert.x • Akka HTTP • (Pivotal Reactor ? Rat pack ?) 61

#devoxx For completeness • Quasar also offers Channels, and even Actors • Akka also offers a Bus, and Agents. • A lot of this comes from outside of the JVM (Go Channels, node.js, Erlang Actors) 62

Thanks Again! Lutz Hühnken @lutzhuehnken https://github.com/lutzh/concurrency_talk

#devoxx Image Credits Cat on hot tin roof - from "McPhillamyActorBlog" - http://mcphillamy.com Agents - from "Matrix Wiki" - http://matrix.wikia.com/wiki/Agent Event Loop - http://www.freeimages.com/photo/big-looping-1522456 Channel - http://www.freeimages.com/photo/white-cliffs-of-dover-1256973 Fibers - http://www.freeimages.com/photo/fiber-optics-1503081 64