Upgrade to Pro — share decks privately, control downloads, hide ads and more …

JVM Concurrency on Devoxx at Nov 12 2015

JVM Concurrency on Devoxx at Nov 12 2015

Until recently, concurrency in Java meant: java.util.concurrent and threads. Threads were originally envisioned as "lightweight processes" - starting a new process for concurrent operations meant to much overhead, and posed the problem of inter-process communication. Threads were supposed to be light and remove both disadvantages - less resource consumption for creation and scheduling, and shared memory. Today it seems this model has reached it's limits. The context switch between threads is not a good match for modern processor architectures, resource needs are still to high for fine-grained concurrency, and shared mutual state is a curse, not a blessing, leading to race conditions, locks, contention. To quote Oracle JVM architect John Rose: "Threads are passé". We will explore different approaches to concurrency below the thread level, and have a look at their advantages and disadvantages. Namely we will look at Quasar Fibres, Clojure Agents, vert.x Verticles and Akka Actors.

Lutz Hühnken

November 12, 2015

More Decks by Lutz Hühnken

Other Decks in Programming


  1. Devoxx Antwerpen 2015 Concurrency Models on the JVM Lutz Hühnken

    @lutzhuehnken https://github.com/lutzh/concurrency_talk
  2. #devoxx 7 Provocative claim (for you to discuss over drinks

    later): We need a new standard concurrency model for Java!
  3. #devoxx 12 From Thread Process Process Worker Task To Also

    see https://en.wikipedia.org/wiki/Reactor_pattern
  4. #devoxx Thinking about blocking I/O When one thread corresponds to

    one task 13 Note: This is a snapshot in time of the system, not a sequence of events
  5. #devoxx Never block! When one thread is a worker, working

    on multiple tasks 14 Note: This is a snapshot in time of the system, not a sequence of events
  6. #devoxx Solution for the efficiency problem: • Sub-thread level concurrency

    • Asynchronous I/O
 • This is something the approaches we will look at have in common
 • Note: This is also something that all Reactive Systems* have in common
 * http://www.reactivemanifesto.org 15
  7. #devoxx 17 They discard the most essential and appealing properties

    of sequential computation: understandability, predictability, and determinism. Threads, as a model of computation, are wildly nondeterministic, and the job of the programmer becomes one of pruning that nondeterminism.
  8. #devoxx Digression: Callback Hell fs.readdir(source, function(err, files) { if (err)

    { console.log('Error finding files: ' + err) } else { files.forEach(function(filename, fileIndex) { console.log(filename) gm(source + filename).size(function(err, values) { if (err) { console.log('Error identifying file size: ' + err) } else { console.log(filename + ' : ' + values) aspect = (values.width / values.height) widths.forEach(function(width, widthIndex) { height = Math.round(width / aspect) console.log('resizing ' + filename + 'to ' + height + 'x' + height) this.resize(width, height).write(destination + 'w' + width + '_' + filename, function(err) { if (err) console.log('Error writing file: ' + err) }) }.bind(this)) } }) }) } }) 19
  9. #devoxx I admit: It could be written in a more

    readable way • Scala Futures / for-expressions • async / await (also Scala) • JavaScript Promises (then.. then.. then)
 • All this is syntactic sugar for callbacks - which is good • But let’s explore other (better) approaches for composition 20
  10. #devoxx 21 What are we interested in? Composition State Management

    Integration, I/O (both async and sync) Predictability
  11. #devoxx 22 Coroutines (Green Threads, User Mode Threads, IOC Threads,

    Fibers) Quasar Fibers http://www.paralleluniverse.co/quasar/ Channels Clojure core.async https://github.com/clojure/core.async/ Event Loop vert.x http://vertx.io Actor Model Akka http://akka.io Concurrency Models
  12. #devoxx 24 new Fiber<V>() { @Override protected V run() throws

    SuspendExecution, InterruptedException { // code here } }.start(); looks like a thread, walks like a thread..
  13. #devoxx 25 class RetrieveInfo extends Fiber<Void> { ... @Override public

    Void run() { final CloseableHttpClient client = FiberHttpClientBuilder.create().build(); try { String response = client.execute(new HttpGet(MessageFormat.format(url, searchTerm)), new BasicResponseHandler()); addResult(key, StringUtils.abbreviate(response, 1000)); } catch (IOException iox) { ... Use like blocking Need to aggregate results.. „fiber blocking“ async http client
  14. #devoxx 26 private final Map<String, String> store = new ConcurrentHashMap<>();

    public void define(final String searchTerm) { for (String currentKey : sources.keySet()) { new RetrieveInfo(currentKey, sources.get(currentKey), searchTerm).start(); } } /** * This method is working on shared mutual state - this is what we look to avoid! */ void addResult(final String key, final String result) { store.put(key, result); ... This method is working on shared mutual state - this is what we look to shared mutable state Start retrieval of definitions in background
  15. #devoxx Fibers • Major advantage: Imperative Programming, just like with

 • Major disadvantage: Imperative Programming, just like with threads 27
  16. #devoxx 28 class FooAsync extends FiberAsync<String, FooException> implements FooCompletion {

    @Override public void success(String result) { asyncCompleted(result); } @Override public void failure(FooException exception) { asyncFailed(exception); } } String op() { new FooAsync() { protected void requestAsync() { Foo.asyncOp(this); } }.run(); } Turn callback APIs into fiber blocking APIs!
  17. #devoxx Fibers Summary • Thread-like programming model • Nice „tricks“:

    Instrumentation, Suspendable, Thread Interop • Fibers alone are a bit low-level • (Almost) drop-in replacement for threads 29
  18. #devoxx Channels • Theoretical Foundation:
 Communicating Sequential Processes (Tony Hoare

    1978, https:// en.wikipedia.org/wiki/ Communicating_sequential_processes) • Very popular in the „Go“ community • Our example: Clojure core.async 31 Channels
  19. #devoxx 32 (def echo-chan (chan)) (go (println (<! echo-chan))) (>!!

    echo-chan „ketchup") ; => true ; => ketchup go block - like a fiber >!! blocks thread <! blocks go block
  20. #devoxx 33 (def echo-buffer (chan 2)) (>!! echo-buffer "ketchup") ;

    => true (>!! echo-buffer "ketchup") ; => true (>!! echo-buffer "ketchup") ; blocks buffered channel - async
  21. #devoxx 34 (def responses-channel (chan 10)) (go (println (loop [values

    []] (if (= 3 (count values)) values (recur (conj values (<! responses-channel))))))) (defn async-get [url result] (http/get url #(go (>! result (:body %))))) (async-get "http:m-w.com..." responses-channel) (async-get "http:wiktionary.." responses-channel) (async-get "http:urbandictionary.." responses-channel) callback puts result in channel channel for responses Aggregate results recursively
  22. #devoxx Channels • Allows flow-oriented programming, very flexible composition (e.g.

    alts!) • Sanity through sequencing • go block / thread duality allows integration with both async and sync libraries • Channels as keepers of state? 35 Channels Summary
  23. #devoxx Channels • Event loop model - think node.js •

    „multi event loop“, on the JVM • Polyglot - Java, Scala, Clojure, JRuby, JS.. • Deploy small pieces (Verticles) and have them send JSON to each other 37 vert.x
  24. #devoxx 38 public class Receiver extends AbstractVerticle { @Override public

    void start() throws Exception { EventBus eb = vertx.eventBus(); eb.consumer("ping-address", message -> { System.out.println("Received message: " + message.body()); // Now send back reply message.reply("pong!"); }); System.out.println("Receiver ready!"); } } register on event bus extend Verticle connect
  25. #devoxx 39 MessageConsumer<JsonObject> consumer = eventBus.consumer("definer.task"); consumer.handler(message -> { httpClient.getNow(message.body().getString("host"),

    message.body().getString("path"), httpClientResponse -> { httpClientResponse.bodyHandler(buffer -> { eventBus.send("collector", new JsonObject() .put(...) callback based async http register send result as message
  26. #devoxx 40 Vertx vertx = Vertx.vertx(); EventBus eventBus = vertx.eventBus();

    DeploymentOptions options = new DeploymentOptions().setInstances(sources.size()); vertx.deployVerticle("de.huehnken.concurrency.Task", options, msg -> { for (String currentKey : sources.keySet()) { eventBus.send("definer.task", new JsonObject() .put(..) which one gets the message? deploy multiple
  27. #devoxx 41 private final Map<String, String> store = new HashMap<>();

    @Override public void start(Future<Void> startFuture) { EventBus eventBus = vertx.eventBus(); MessageConsumer<JsonObject> consumer = eventBus.consumer("collector"); consumer.handler(message -> { store.put(message.body().getString(„key"), message.body().getString("definition")); wait for results.. encapsultate state in Verticle messages are usually JSON
  28. #devoxx Channels • Loose coupling - very • Sanity through

    events („single thread illusion“) • Hybrid thread model allows integration of synchronous APIs, off-loading work from event loop • State handling needs some care • Utility APIs are callback heavy, node inspired • Distributed! Unified model for local and distributed concurrency 43 vert.x Summary
  29. #devoxx Channels • The actor model [..] is a [model]

    that treats "actors" as the universal primitives of concurrent computation: in response to a message that it receives, an actor can make local decisions, create more actors, send more messages, and determine how to respond to the next message received. https://en.wikipedia.org/wiki/Actor_model • Described by Carl Hewitt 1973 • Popular in the Erlang community 45 Akka
  30. #devoxx 49 class Coordinator(searchTerm: String) extends Actor { var results

    = Map[String, String]() def receive = { case StartCommand => sources foreach { case (key, url) => context.actorOf(Task.props(key, self)) ! GetDefinition(url) } context.system.scheduler.scheduleOnce(5 seconds, self, Done) case Result(key, definition) => results += key -> definition if (results.size == sources.size) { self ! Done } case Done => spawn child actors, send command encapsulate state aggregate results
  31. #devoxx 50 class Task(key: String, coordinator: ActorRef) extends Actor {

    def receive = { case GetDefinition(url) => http.singleRequest(HttpRequest(uri = url)) pipeTo self case HttpResponse(StatusCodes.OK, headers, entity, _) => coordinator ! Result(key, entity.dataBytes.utf8String) case HttpResponse(code, _, _, _) => // handle error send result to aggregator async call, translated to message
  32. #devoxx Channels • Sanity through messages („single thread illusion“) •

    Messages (as opposed to events) • Dispatchers allows integration of synchronous APIs, off-loading work from event loop • pipeTo & ask to integrate with async APIs • Distributed! Unified model for local and distributed concurrency. • Supervision! Fault handling enforced. 52 Actors Summary
  33. #devoxx 53 Programming model chart Tasks (sub- thread level) Asynchronous

    Messaging Distribution (unified model) Supervision Fibers ✔ Channels (core.async) ✔ ✔ Event Bus (vert.x) ✔ ✔ ✔ Aktoren (Akka) ✔ ✔ ✔ ✔
  34. #devoxx • Concurrency is interesting
 • Threads are passé, we

    want a better abstraction on top of it • Alternatives are available today
 • If you only have time to look at one model, I recommend Akka 54 Summary
  35. #devoxx 58 (def x (agent 0)) (defn increment [c n]

    (+ c n)) (send x increment 5) ; @x -> 5 (send x increment 10) ; @x -> 15 „wrap“ a value in an agent send the code to the agent to modify value! a function
  36. #devoxx Agents • The agent encapsulates state • Send code

    to the agent to change value • Interesting feature especially for functional programming • Async execution, but not really a concurrency model 59
  37. #devoxx Beware of the benchmark! • Unfit: CPU bound •

    Also unfit: Anything using blocking I/O • It’s not about single request performance, it’s about scalability! 60
  38. #devoxx But I just want to build a little REST

    service.. • If you don’t program on this level yourself: know you frameworks! • Look out for „reactive“ or „async“ • Play! • Vert.x • Akka HTTP • (Pivotal Reactor ? Rat pack ?) 61
  39. #devoxx For completeness • Quasar also offers Channels, and even

    Actors • Akka also offers a Bus, and Agents. • A lot of this comes from outside of the JVM (Go Channels, node.js, Erlang Actors) 62
  40. #devoxx Image Credits Cat on hot tin roof - from

    "McPhillamyActorBlog" - http://mcphillamy.com Agents - from "Matrix Wiki" - http://matrix.wikia.com/wiki/Agent Event Loop - http://www.freeimages.com/photo/big-looping-1522456 Channel - http://www.freeimages.com/photo/white-cliffs-of-dover-1256973 Fibers - http://www.freeimages.com/photo/fiber-optics-1503081 64