Slide 1

Slide 1 text

by Mario Fusco [email protected] @mariofusco Comparing different concurrency models on the JVM

Slide 2

Slide 2 text

Moore's law The number of transistors on integrated circuits doubles approximately every two years Now achieved by increasing the number of cores idle

Slide 3

Slide 3 text

No content

Slide 4

Slide 4 text

No content

Slide 5

Slide 5 text

This is what typically happens in your computer

Slide 6

Slide 6 text

Concurrency & Parallelism Parallel programming Running multiple tasks at the same time Concurrent programming Managing concurrent requests Both are hard!

Slide 7

Slide 7 text

The native Java concurrency model Based on: They are sometimes plain evil … … and sometimes a necessary pain … … but always the wrong default Threads Semaphores Synchronization Locks

Slide 8

Slide 8 text

What do you think when I say parallelism? Threads And what do you think when I say threads? Locks What are they for? They prevent multiple threads to run in parallel Do you see the problem?

Slide 9

Slide 9 text

Summing attendants ages (Threads) class Blackboard { int sum = 0; int read() { return sum; } void write(int value) { sum = value; } } class Attendee implements Runnable { int age; Blackboard blackboard; public void run() { synchronized(blackboard) { int oldSum = blackboard.read(); int newSum = oldSum + age; blackboard.write(newSum); } } }

Slide 10

Slide 10 text

The Java Concurrency Bible You know you're in big troubles when you feel the need of taking this from your bookshelf

Slide 11

Slide 11 text

Don't call alien methods while holding a lock Threads – Good practices Acquire multiple locks in a fixed, global order ReentrantReadWriteLock rwl = new ReentrantReadWriteLock(); rwl.writeLock().tryLock(3L, TimeUnit.SECONDS); Use interruptible locks instead of intrinsic synchronization Avoid blocking using concurrent data structures and atomic variable when possible Use thread pools instead of creating threads directly Hold locks for the shortest possible amount of time Learn Java Concurrency API

Slide 12

Slide 12 text

How you designed it

Slide 13

Slide 13 text

What happens in reality

Slide 14

Slide 14 text

Threads and Locks – Pros & Cons + “Close to the metal” → can be very efficient when implemented correctly + Low abstraction level → widest range of applicability and high degree of control + Threads and lock can be implemented in existing imperative object-oriented languages with little effort - Low abstraction level → threads-and-locks programming is HARD and understanding the Java Memory Model even HARDER - Inter-threads communication done with shared mutable state leads to non-determinism - Threads are a scarce resource - It works only with shared-memory architectures → no support for distributed memory → cannot be used to solve problems that are too large to fit on a single system - Writing multithreaded programs is difficult but testing them is nearly impossible → a maintenance nightmare

Slide 15

Slide 15 text

Do not try and fix the deadlock, that's impossible. Instead, only try and realize the truth.... there is no deadlock. Then you will see it is not the deadlock that needs fixing, it is only yourself.

Slide 16

Slide 16 text

Threads & Locks Concurrent programming Assembler Programming = Patient: "Doctor, it hurts when I do this.” Doctor: "Then stop doing it."

Slide 17

Slide 17 text

What about Queues instead of Locks? In reality actors are just a more structured and powerful way of using queues. More on this later … + Better decoupling + Message passing instead of shared memory - High wake-up latency - No built-in failure recovery - Heavyweight - Unidirectional

Slide 18

Slide 18 text

The cause of the problem … Mutable state + Parallel processing = Non-determinism Functional Programming

Slide 19

Slide 19 text

Functional Programming

Slide 20

Slide 20 text

OOP makes code understandable by encapsulating moving parts FP makes code understandable by minimizing moving parts - Michael Feathers OOP vs FP

Slide 21

Slide 21 text

Mutability Parameter binding is about assigning names to things Mutating variables is about assigning things to names Does that second one sound weird? … well it's because it IS weird

Slide 22

Slide 22 text

Dangers of mutable state public class DateParser { private final DateFormat format = new SimpleDateFormat("yyyy-MM-dd"); public Date parse(String s) throws ParseException { return format.parse(s); } } Hidden Mutable State final == thread-safe ?

Slide 23

Slide 23 text

Dangers of mutable state public class Conference { private final List attendees = new LinkedList<>(); public synchronized void addAttendee(Attendee a) { attendees.add(a); } public synchronized Iterator getAttendeeIterator() { return attendees.iterator(); } } Escaped Mutable State synchronized == thread-safe ?

Slide 24

Slide 24 text

Summing attendants ages (Functional) class Blackboard { final int sum; Blackboard(int sum) { this.sum = sum; } } class Attendee { int age; Attendee next; public Blackboard addMyAge(Blackboard blackboard) { final Blackboard b = new Blackboard(blackboard.sum + age); return next == null ? b : next.addMyAge(b); } }

Slide 25

Slide 25 text

FP + Internal iteration = free parallelism public int sumAges(List attendees) { int total = 0; for (Attendee a : attendees) { total += a.getAge(); } return total; } public int sumAges(List attendees) { return attendees.stream() .map(Attendee::getAge) .reduce(0, Integer::sum); } External iteration Internal iteration

Slide 26

Slide 26 text

FP + Internal iteration = free parallelism public int sumAges(List attendees) { int total = 0; for (Attendee a : attendees) { total += a.getAge(); } return total; } public int sumAges(List attendees) { return attendees.stream() .map(Attendee::getAge) .reduce(0, Integer::sum); } External iteration Internal iteration public int sumAges(List attendees) { return attendees.parallelStream() .map(Attendee::getAge) .reduce(0, Integer::sum); } The best way to write parallel applications is NOT to have to think about parallelism

Slide 27

Slide 27 text

Parallel reduction – Divide and Conquer Use Java 7 Fork/Join framework under the hood, but expose an higher abstraction level

Slide 28

Slide 28 text

Using your own ForkJoinPool with parallel Streams public int sumAges(List attendees) { return new ForkJoinPool(2).submit(() -> attendees.parallelStream() .map(Attendee::getAge) .reduce(0, Integer::sum) ).join(); } CompletableFuture sum = CompletableFuture.supplyAsync(() -> attendees.parallelStream() .map(Attendee::getAge) .reduce(0, Integer::sum), new ForkJoinPool(2)); } Don't do this at home!!!

Slide 29

Slide 29 text

public static void sort(List list, Comparator super T> c) Essence of Functional Programming Data and behaviors are the same thing! Data Behaviors Collections.sort(persons, (p1, p2) -> p1.getAge() – p2.getAge())

Slide 30

Slide 30 text

Map/Reduce is a FP pattern public int sumAges(List attendees) { return attendees.stream() .map(Attendee::getAge) .reduce(0, Integer::sum); } Do these methods' names remember you something? Fast also because, when possible, Map/Reduce moves computation (functions) to the data and not the opposite.

Slide 31

Slide 31 text

Functions – Pros & Cons + Immutability definitively prevents any non-determinism + Declarative programming style improves readability → focus on the “what” not on the “how” + Parallelizing functional (side-effect free) code can be trivially easy in many cases + Better confidence that your program does what you think it does + Great support for distributed computation - “Functional thinking” can be unfamiliar for many OOP developers - Can be be less efficient than its imperative equivalent - In memory managed environment (like the JVM) put a bigger burden on the garbage collector - Less control → how the computational tasks are splitted and scheduled on threads is delegated to the library/framework - Great abstraction for parallelism not for concurrency

Slide 32

Slide 32 text

Actors

Slide 33

Slide 33 text

Summing attendants ages (Actors) class Blackboard extends UntypedActors { int sum = 0; public void onReceive(Object message) { if (message instanceof Integer) { sum += (Integer)message; } } } class Attendant { int age; Blackboard blackboard; public void sendAge() { blackboard.tell(age); } }

Slide 34

Slide 34 text

The way OOP is implemented in most common imperative languages is probably one of the biggest misunderstanding in the millenarian history of engineering This is Class Oriented Programming Actors are the real OOP (Message Passing)

Slide 35

Slide 35 text

I'm sorry that I coined the term "objects", because it gets many people to focus on the lesser idea. The big idea is "messaging". Alan Kay

Slide 36

Slide 36 text

Defensive programming Vs. Let it crash!

Slide 37

Slide 37 text

Throwing an exception in concurrent code will just simply blow up the thread that currently executes the code that threw it: 1. There is no way to find out what went wrong, apart from inspecting the stack trace 2. There is nothing you can do to recover from the problem and bring back your system to a normal functioning What’s wrong in trying to prevent errors? Supervised actors provide a clean error recovery strategy encouraging non-defensive programming

Slide 38

Slide 38 text

Actors – Pros & Cons + State is mutable but encapsulated → concurrency is implemented with message flow between actors + Built-in fault tolerance through supervision + Not a scarce resource as threads → can have multiple actors for each thread + Location transparency easily enables distributed programming + Actors map real-world domain model - Untyped messages don't play well with Java's lack of pattern matching - It's easy to violate state encapsulation → debugging can be hard - Message immutability is vital but cannot be enforced in Java - Actors are only useful if they produce side-effects - Composition can be awkward - Actors do not prevent deadlocks → it’s possible for two or more actors to wait on one another for messages

Slide 39

Slide 39 text

The state quadrants Mutable Immutable Shared Unshared Actors Functional Programming Threads Determinism Non-determinism

Slide 40

Slide 40 text

Software Transactional Memory

Slide 41

Slide 41 text

Software Transactional Memory An STM turns the Java heap into a transactional data set with begin/commit/rollback semantics. Very much like a regular database. It implements the first three letters in ACID; ACI: Atomic → all or none of the changes made during a transaction get applied Consistent → a transaction has a consistent view of reality Isolated → changes made by concurrent execution transactions are not visible to each other ➢ A transaction is automatically retried when it runs into some read or write conflict ➢ In this case a delay is used to prevent further contentions ➢ There shouldn’t be side-effects inside the transaction to avoid to repeat them

Slide 42

Slide 42 text

Summing attendants ages (STM) import org.multiverse.api.references.*; import static org.multiverse.api.StmUtils.*; public class Blackboard { private final TxnRef lastUpdate; private final TxnInteger sum = newTxnInteger(0); public Blackboard() { this.lastUpdate = newTxnRef(new Date()); } public void sumAge(Attendant attendant) { atomic(new Runnable() { public void run() { sum.inc(attendant.getAge()); lastUpdate.set(new Date()); } }); } }

Slide 43

Slide 43 text

STM – Pros & Cons + Eliminates a wide range of common problems related with explicit synchronization + Optimistic and non-blocking + Many developers are already used to think in transactional terms + It's possible to compose multiple transactional blocks nesting them in a higher level transaction - Write collision are proportional to threads contention level - Retries can be costly - Unpredictable performance - Transactional code has to be idempotent, no side-effects are allowed - No support for distribution

Slide 44

Slide 44 text

(Completable)Future The Future interface was introduced in Java 5 to model an asynch computation and then provide an handle to a result that will be made available at some point in the future. CompletableFuture introduced in Java 8 added fluent composability, callbacks and more. CompletableFuture .supplyAsync(() -> shop.getPrice(product)) .thenCombine(CompletableFuture.supplyAsync( () -> exchange.getRate(Money.EUR, Money.USD)), (price, rate) -> price * rate) .thenAccept(System.out::println); + Non-blocking composition + Freely sharable + Can recover from failure - Callback Hell - Debugging can be hard - Closing over mutable state

Slide 45

Slide 45 text

Reactive Programming + Non-blocking composition with richer semantic + Event centric → async in nature + Can recover from failure - Callback Hell - Push instead of pull → Inverted control flow - Fast producer/slow consumer → May require blocking Reactive programming consists in asynch processing and combining streams of events ordered in time. RxJava is a library, including a DSL, for composing asynch and event-based programs using observable sequences Observable stockFeed = Observable.interval(1, TimeUnit.SECONDS) .map(i -> StockServer.fetch(symbol)); stockFeed.subscribe(System.out::println);

Slide 46

Slide 46 text

Wrap up – There's No Silver Bullet One size does NOT fit all ➢ Concurrency and parallelism will be increasingly important in the near future ➢ Using threads & locks by default is (at best) premature optimization ➢ There are many different concurrency models with different characteristic ➢ Know them all and choose the one that best fit the problem at hand

Slide 47

Slide 47 text

Wrap up – The Zen of Concurrency Avoid shared mutability → if there is no clear way to avoid mutability, then favor isolated mutability Sequential programming idioms (e.g. external iteration) and tricks (e.g. reusing variables) are detrimental for parallelization Prototype different solutions with different concurrency models and discover their strengths and weaknesses Premature optimization is evil especially in concurrent programming → Make it right first and only AFTER make it faster Poly-paradigm programming is more effective than polyglot → you can experiment all those different concurrency models in plain Java Strive for immutability → Make fields and local variables final by default and make an exception only when strictly required

Slide 48

Slide 48 text

Suggested readings

Slide 49

Slide 49 text

Mario Fusco Red Hat – Senior Software Engineer [email protected] twitter: @mariofusco Q A Thanks … Questions?