JVM Concurrency on Devoxx at Nov 12 2015

Until recently, concurrency in Java meant: java.util.concurrent and threads. Threads were originally envisioned as "lightweight processes" - starting a new process for concurrent operations meant to much overhead, and posed the problem of inter-process communication. Threads were supposed to be light and remove both disadvantages - less resource consumption for creation and scheduling, and shared memory. Today it seems this model has reached it's limits. The context switch between threads is not a good match for modern processor architectures, resource needs are still to high for fine-grained concurrency, and shared mutual state is a curse, not a blessing, leading to race conditions, locks, contention. To quote Oracle JVM architect John Rose: "Threads are passé". We will explore different approaches to concurrency below the thread level, and have a look at their advantages and disadvantages. Namely we will look at Quasar Fibres, Clojure Agents, vert.x Verticles and Akka Actors.

Transcript

1. Devoxx Antwerpen 2015 Concurrency Models on the JVM Lutz Hühnken @lutzhuehnken https://github.com/lutzh/concurrency_talk
2. #devoxx 2 Before we start: Why Concurrency is hard (demonstrated with printed books)
3. #devoxx 3
4. #devoxx 4
5. #devoxx 5
6. #devoxx 6
7. #devoxx 7 Provocative claim (for you to discuss over drinks later): We need a new standard concurrency model for Java!
8. #devoxx 8 Thread Problems Problem 1: Efficiency (resp. the lack thereof)
9. #devoxx 9 CPU Threads Work
10. #devoxx 10 1 2 3 … 10.000 4-core CPU Thousands of Threads Work (a lot)
11. #devoxx 11 We want to move towards this:
12. #devoxx 12 From Thread Process Process Worker Task To Also see https://en.wikipedia.org/wiki/Reactor_pattern
13. #devoxx Thinking about blocking I/O When one thread corresponds to one task 13 Note: This is a snapshot in time of the system, not a sequence of events
14. #devoxx Never block! When one thread is a worker, working on multiple tasks 14 Note: This is a snapshot in time of the system, not a sequence of events
15. #devoxx Solution for the efficiency problem: • Sub-thread level concurrency • Asynchronous I/O
 • This is something the approaches we will look at have in common
 • Note: This is also something that all Reactive Systems* have in common
 * http://www.reactivemanifesto.org 15
16. #devoxx 16 Thread Problems Problem 1: Efficiency (resp. the lack thereof) Problem 2: Programming Model
17. #devoxx 17 They discard the most essential and appealing properties of sequential computation: understandability, predictability, and determinism. Threads, as a model of computation, are wildly nondeterministic, and the job of the programmer becomes one of pruning that nondeterminism.
18. #devoxx 18 Source: John Rose, Java VM Architect, JFokus, Stockholm, February 2015
19. #devoxx Digression: Callback Hell fs.readdir(source, function(err, files) { if (err) { console.log('Error finding files: ' + err) } else { files.forEach(function(filename, fileIndex) { console.log(filename) gm(source + filename).size(function(err, values) { if (err) { console.log('Error identifying file size: ' + err) } else { console.log(filename + ' : ' + values) aspect = (values.width / values.height) widths.forEach(function(width, widthIndex) { height = Math.round(width / aspect) console.log('resizing ' + filename + 'to ' + height + 'x' + height) this.resize(width, height).write(destination + 'w' + width + '_' + filename, function(err) { if (err) console.log('Error writing file: ' + err) }) }.bind(this)) } }) }) } }) 19
20. #devoxx I admit: It could be written in a more readable way • Scala Futures / for-expressions • async / await (also Scala) • JavaScript Promises (then.. then.. then)
 • All this is syntactic sugar for callbacks - which is good • But let’s explore other (better) approaches for composition 20
21. #devoxx 21 What are we interested in? Composition State Management Integration, I/O (both async and sync) Predictability
22. #devoxx 22 Coroutines (Green Threads, User Mode Threads, IOC Threads, Fibers) Quasar Fibers http://www.paralleluniverse.co/quasar/ Channels Clojure core.async https://github.com/clojure/core.async/ Event Loop vert.x http://vertx.io Actor Model Akka http://akka.io Concurrency Models
23. Fibers
24. #devoxx 24 new Fiber<V>() { @Override protected V run() throws SuspendExecution, InterruptedException { // code here } }.start(); looks like a thread, walks like a thread..
25. #devoxx 25 class RetrieveInfo extends Fiber<Void> { ... @Override public Void run() { final CloseableHttpClient client = FiberHttpClientBuilder.create().build(); try { String response = client.execute(new HttpGet(MessageFormat.format(url, searchTerm)), new BasicResponseHandler()); addResult(key, StringUtils.abbreviate(response, 1000)); } catch (IOException iox) { ... Use like blocking Need to aggregate results.. „fiber blocking“ async http client
26. #devoxx 26 private final Map<String, String> store = new ConcurrentHashMap<>(); public void define(final String searchTerm) { for (String currentKey : sources.keySet()) { new RetrieveInfo(currentKey, sources.get(currentKey), searchTerm).start(); } } /** * This method is working on shared mutual state - this is what we look to avoid! */ void addResult(final String key, final String result) { store.put(key, result); ... This method is working on shared mutual state - this is what we look to shared mutable state Start retrieval of definitions in background
27. #devoxx Fibers • Major advantage: Imperative Programming, just like with threads
 • Major disadvantage: Imperative Programming, just like with threads 27
28. #devoxx 28 class FooAsync extends FiberAsync<String, FooException> implements FooCompletion { @Override public void success(String result) { asyncCompleted(result); } @Override public void failure(FooException exception) { asyncFailed(exception); } } String op() { new FooAsync() { protected void requestAsync() { Foo.asyncOp(this); } }.run(); } Turn callback APIs into fiber blocking APIs!
29. #devoxx Fibers Summary • Thread-like programming model • Nice „tricks“: Instrumentation, Suspendable, Thread Interop • Fibers alone are a bit low-level • (Almost) drop-in replacement for threads 29
30. Channels
31. #devoxx Channels • Theoretical Foundation:
 Communicating Sequential Processes (Tony Hoare 1978, https:// en.wikipedia.org/wiki/ Communicating_sequential_processes) • Very popular in the „Go“ community • Our example: Clojure core.async 31 Channels
32. #devoxx 32 (def echo-chan (chan)) (go (println (<! echo-chan))) (>!! echo-chan „ketchup") ; => true ; => ketchup go block - like a fiber >!! blocks thread <! blocks go block
33. #devoxx 33 (def echo-buffer (chan 2)) (>!! echo-buffer "ketchup") ; => true (>!! echo-buffer "ketchup") ; => true (>!! echo-buffer "ketchup") ; blocks buffered channel - async
34. #devoxx 34 (def responses-channel (chan 10)) (go (println (loop [values []] (if (= 3 (count values)) values (recur (conj values (<! responses-channel))))))) (defn async-get [url result] (http/get url #(go (>! result (:body %))))) (async-get "http:m-w.com..." responses-channel) (async-get "http:wiktionary.." responses-channel) (async-get "http:urbandictionary.." responses-channel) callback puts result in channel channel for responses Aggregate results recursively
35. #devoxx Channels • Allows flow-oriented programming, very flexible composition (e.g. alts!) • Sanity through sequencing • go block / thread duality allows integration with both async and sync libraries • Channels as keepers of state? 35 Channels Summary
36. Event Loop
37. #devoxx Channels • Event loop model - think node.js • „multi event loop“, on the JVM • Polyglot - Java, Scala, Clojure, JRuby, JS.. • Deploy small pieces (Verticles) and have them send JSON to each other 37 vert.x
38. #devoxx 38 public class Receiver extends AbstractVerticle { @Override public void start() throws Exception { EventBus eb = vertx.eventBus(); eb.consumer("ping-address", message -> { System.out.println("Received message: " + message.body()); // Now send back reply message.reply("pong!"); }); System.out.println("Receiver ready!"); } } register on event bus extend Verticle connect
39. #devoxx 39 MessageConsumer<JsonObject> consumer = eventBus.consumer("definer.task"); consumer.handler(message -> { httpClient.getNow(message.body().getString("host"), message.body().getString("path"), httpClientResponse -> { httpClientResponse.bodyHandler(buffer -> { eventBus.send("collector", new JsonObject() .put(...) callback based async http register send result as message
40. #devoxx 40 Vertx vertx = Vertx.vertx(); EventBus eventBus = vertx.eventBus(); DeploymentOptions options = new DeploymentOptions().setInstances(sources.size()); vertx.deployVerticle("de.huehnken.concurrency.Task", options, msg -> { for (String currentKey : sources.keySet()) { eventBus.send("definer.task", new JsonObject() .put(..) which one gets the message? deploy multiple
41. #devoxx 41 private final Map<String, String> store = new HashMap<>(); @Override public void start(Future<Void> startFuture) { EventBus eventBus = vertx.eventBus(); MessageConsumer<JsonObject> consumer = eventBus.consumer("collector"); consumer.handler(message -> { store.put(message.body().getString(„key"), message.body().getString("definition")); wait for results.. encapsultate state in Verticle messages are usually JSON
42. #devoxx Event Bus (vert.x) 42 Image from Jonas Bandi @jbandi
43. #devoxx Channels • Loose coupling - very • Sanity through events („single thread illusion“) • Hybrid thread model allows integration of synchronous APIs, off-loading work from event loop • State handling needs some care • Utility APIs are callback heavy, node inspired • Distributed! Unified model for local and distributed concurrency 43 vert.x Summary
44. Actors
45. #devoxx Channels • The actor model [..] is a [model] that treats "actors" as the universal primitives of concurrent computation: in response to a message that it receives, an actor can make local decisions, create more actors, send more messages, and determine how to respond to the next message received. https://en.wikipedia.org/wiki/Actor_model • Described by Carl Hewitt 1973 • Popular in the Erlang community 45 Akka
46. #devoxx Actors (Akka) 46
47. #devoxx Actors (Akka) 47
48. #devoxx Actors (Akka) 48
49. #devoxx 49 class Coordinator(searchTerm: String) extends Actor { var results = Map[String, String]() def receive = { case StartCommand => sources foreach { case (key, url) => context.actorOf(Task.props(key, self)) ! GetDefinition(url) } context.system.scheduler.scheduleOnce(5 seconds, self, Done) case Result(key, definition) => results += key -> definition if (results.size == sources.size) { self ! Done } case Done => spawn child actors, send command encapsulate state aggregate results
50. #devoxx 50 class Task(key: String, coordinator: ActorRef) extends Actor { def receive = { case GetDefinition(url) => http.singleRequest(HttpRequest(uri = url)) pipeTo self case HttpResponse(StatusCodes.OK, headers, entity, _) => coordinator ! Result(key, entity.dataBytes.utf8String) case HttpResponse(code, _, _, _) => // handle error send result to aggregator async call, translated to message
51. #devoxx Actors (Akka) 51
52. #devoxx Channels • Sanity through messages („single thread illusion“) • Messages (as opposed to events) • Dispatchers allows integration of synchronous APIs, off-loading work from event loop • pipeTo & ask to integrate with async APIs • Distributed! Unified model for local and distributed concurrency. • Supervision! Fault handling enforced. 52 Actors Summary
53. #devoxx 53 Programming model chart Tasks (sub- thread level) Asynchronous Messaging Distribution (unified model) Supervision Fibers ✔ Channels (core.async) ✔ ✔ Event Bus (vert.x) ✔ ✔ ✔ Aktoren (Akka) ✔ ✔ ✔ ✔
54. #devoxx • Concurrency is interesting
 • Threads are passé, we want a better abstraction on top of it • Alternatives are available today
 • If you only have time to look at one model, I recommend Akka 54 Summary
55. #devoxx 55 Closing the loop.. another book..
56. Thank You Lutz Hühnken @lutzhuehnken https://github.com/lutzh/concurrency_talk
57. Agents
58. #devoxx 58 (def x (agent 0)) (defn increment [c n] (+ c n)) (send x increment 5) ; @x -> 5 (send x increment 10) ; @x -> 15 „wrap“ a value in an agent send the code to the agent to modify value! a function
59. #devoxx Agents • The agent encapsulates state • Send code to the agent to change value • Interesting feature especially for functional programming • Async execution, but not really a concurrency model 59
60. #devoxx Beware of the benchmark! • Unfit: CPU bound • Also unfit: Anything using blocking I/O • It’s not about single request performance, it’s about scalability! 60
61. #devoxx But I just want to build a little REST service.. • If you don’t program on this level yourself: know you frameworks! • Look out for „reactive“ or „async“ • Play! • Vert.x • Akka HTTP • (Pivotal Reactor ? Rat pack ?) 61
62. #devoxx For completeness • Quasar also offers Channels, and even Actors • Akka also offers a Bus, and Agents. • A lot of this comes from outside of the JVM (Go Channels, node.js, Erlang Actors) 62
63. Thanks Again! Lutz Hühnken @lutzhuehnken https://github.com/lutzh/concurrency_talk
64. #devoxx Image Credits Cat on hot tin roof - from "McPhillamyActorBlog" - http://mcphillamy.com Agents - from "Matrix Wiki" - http://matrix.wikia.com/wiki/Agent Event Loop - http://www.freeimages.com/photo/big-looping-1522456 Channel - http://www.freeimages.com/photo/white-cliffs-of-dover-1256973 Fibers - http://www.freeimages.com/photo/fiber-optics-1503081 64