Slide 1

Slide 1 text

A Tale of Two Cities: Blocking Code vs. Non- Blocking Code Bazlur Rahman (@bazlur_rahman)

Slide 2

Slide 2 text

Agenda

Slide 3

Slide 3 text

Let’s start with context & history

Slide 4

Slide 4 text

Java is made of Threads

Slide 5

Slide 5 text

Threads are in all layers of the Java Platform • Exception • Thread Locals • Debugger • Profiler

Slide 6

Slide 6 text

How do we create threads? var thread = new Thread(() -> { System.out.println("Hello from thread"); }); thread.start(); thread.join();

Slide 7

Slide 7 text

Java Threads == OS Threads • TID = the unique ID given by JVM • NID = Native ID, the unique id given by OS

Slide 8

Slide 8 text

Threads are expensive • Thread.start() consider inefficient • 2 MiB of memory (outside of the heap) • A wrapper of OS threads • Context switching: ~100µs • So 1 million Threads require 2 Terabytes of memory!

Slide 9

Slide 9 text

Concurrency and Throughput

Slide 10

Slide 10 text

Let’s counts threads

Slide 11

Slide 11 text

Native threads counts on my machine • MacBook Pro (16-inch, 2021) • macOS: Ventura 13.3 • Memory: 64 GB • Chips: Apple M1 Max

Slide 12

Slide 12 text

What problem do we have now?

Slide 13

Slide 13 text

Common Server Fanout

Slide 14

Slide 14 text

No content

Slide 15

Slide 15 text

An Example public Credit calculateCredit(Long personId) { var person = getPerson(personId); var assets = getAssets(person); var liabilities = getLiabilities(person); importantWork(); return getCredit(assets, liabilities); }

Slide 16

Slide 16 text

No content

Slide 17

Slide 17 text

Classical implementation var atomicReference = new AtomicReference(); new Thread(() -> { var person = getPerson(1L); atomicReference.set(person); }).start();

Slide 18

Slide 18 text

static Credit calculateCredit(Long personId) throws InterruptedException { var person = getPerson(personId); var assetRef = new AtomicReference(); var t1 = new Thread(() -> { var assets = getAssets(person); assetRef.set(assets); }); var liabilitiesRef = new AtomicReference(); var t2 = new Thread(() -> { var liabilities = getLiabilities(person); liabilitiesRef.set(liabilities); }); var t3 = new Thread(() -> importantWork()); t1.start(); t2.start(); t3.start(); t1.join(); t2.join(); Credit credit = calculateCredits(assetRef.get(), liabilitiesRef.get()); t3.join(); return credit; }

Slide 19

Slide 19 text

Let’s improve it a bit • Java 5 introduces Executors with • Future • Callable

Slide 20

Slide 20 text

static Credit calculateCredit(Long personId, ExecutorService threadPool) throws InterruptedException, ExecutionException { var person = getPerson(personId); Future assetsFuture = threadPool.submit(() -> getAssets(person)); Future liabilitiesFuture = threadPool.submit(() -> getLiabilities(person)); threadPool.submit(() -> importantWork()); return calculateCredits(assetsFuture.get(), liabilitiesFuture.get()); }

Slide 21

Slide 21 text

How traditional thread pool works

Slide 22

Slide 22 text

But not there yet

Slide 23

Slide 23 text

Let’s improve it more

Slide 24

Slide 24 text

Let’s introduce composability Java 8 introduce CompletableFuture

Slide 25

Slide 25 text

static Credit calculateCreditForPerson2(Long personId) throws InterruptedException, ExecutionException { return runAsync(() -> importantWork()) .thenCompose(aVoid -> supplyAsync(() -> getPerson(personId)) .thenCombineAsync(supplyAsync(() -> getAssets(getPerson(personId))), (person, assets) -> calculateCredits(assets, getLiabilities(person)))) .get(); }

Slide 26

Slide 26 text

CompletableFuture • It’s quite a big class with more than 3000 lines of code • It has more than 50 methods which we can readily use and pretty much serves our all purpose • A task can be combined, chained and composed

Slide 27

Slide 27 text

Reactive Java • RxJava • Akka • Eclipse Vert.x • Spring WebFlux • Slick

Slide 28

Slide 28 text

import io.reactivex.rxjava3.core.Single; Single calculateCredit (Long personId) { return Single.fromCallable(this::importantWork) .flatMap(aVoid -> Single.fromCallable(() -> getPerson(personId)) .flatMap(person -> Single.zip( Single.fromCallable(() -> getAssets(person)), Single.fromCallable(() -> getLiabilities(person)), this::calculateCredits ) ) ); }

Slide 29

Slide 29 text

Cons of Reactive framework • Learning curve • Debugging is difficult • The reading experience is terrible, cognitive load • Easy to over-complicated things • Don’t treat it as a hammer, and don’t hit every nail you find

Slide 30

Slide 30 text

Project Loom

Slide 31

Slide 31 text

Project loom (Cont.)

Slide 32

Slide 32 text

DEOM

Slide 33

Slide 33 text

No content

Slide 34

Slide 34 text

Benefits • Virtual threads don’t consume CPU while they are waiting/sleeping • Technically we can have millions of virtual threads • Threads are cheap • Request-per-thread style of server-side programming becomes manageable again as before. • Improved throughput • On Blocking I/O, virtual threads get automatically suspended • No longer need to maintain ThreadPool

Slide 35

Slide 35 text

Let’s Download 10K Images Flux.fromIterable(imageUrls) .flatMap(url -> downloadImage(httpClient, url)) .doOnNext(tuple -> saveImage(tuple.getT1(), tuple.getT2())) .doOnComplete(() -> System.out.println("Finished downloading and saving images.")) .blockLast();

Slide 36

Slide 36 text

Let’s Download 10K Images (2) for (String imageUrl : imageUrls) { executor.submit(() -> { byte[] imageBytes = downloadImage(httpClient, imageUrl); saveImage(imageUrl, imageBytes); System.out.println("Downloaded and saved: " + imageUrl); }); }

Slide 37

Slide 37 text

Framework Implementing Virtual Threads • Spring Boot • Helidon Nima • Quarkus • And many more

Slide 38

Slide 38 text

Think about a Bank with Fixed Number of Teller • In Java terms, a teller is like a Platform Thread • Generally, it would take time to process each customer, say an average of 5 minutes. • Sometimes, a customer would block the process, such as the teller needing to make a phone call to get some information • No work is performed while the teller is blocked waiting and consequently, the entire line is blocked • In Java terms, a teller is still like a Platform Thread but has the ability to park a customer • Generally, it still takes time to process each customer, say an average of 5 minutes • Sometimes, a customer would block the process, such as the teller needing some information before proceeding… • The teller sends a text message or emails to get the necessary information • The teller asks the customer to be seated, and as soon the information is available, they will be the next customer processed by the first available teller • The teller starts processing the next customer in line • This is analogous to a parked Virtual Thread, where the teller is like a Platform Thread, and the customer is like a Virtual Thread • Concurrency is increased by better policies and procedures in dealing with blocking operations

Slide 39

Slide 39 text

No content

Slide 40

Slide 40 text

No content

Slide 41

Slide 41 text

https://twitter.com/bazlur_rahman https://www.linkedin.com/in/bazlur/ https://foojay.io/today/author/bazlur-rahman/ https://bazlur.ca

Slide 42

Slide 42 text

Thank you