A tale of two cities: blocking code vs. non-blocking code

We don't have to exert much mental effort when writing blocking code, but this has drawbacks. Non-blocking code, on the other hand, could be used to increase application throughput. This is why non-blocking code is recommended in many scenarios. Over the last two decades, many approaches have been introduced to writing code that does not block.

In this session, we will look into those options that are available to us. To begin, I'll go over the evolution of the Java concurrency model since its inception with the vanilla thread. Then I will introduce Future/Callable, CompletableFuture, and briefly go over reactive programming, and finally conclude with the Project Loom that was added in Java 19.

Transcript

1. A Tale of Two Cities: Blocking Code vs. Non- Blocking Code Bazlur Rahman (@bazlur_rahman)
2. Agenda
3. Let’s start with context & history
4. Java is made of Threads
5. Threads are in all layers of the Java Platform • Exception • Thread Locals • Debugger • Profiler
6. How do we create threads? var thread = new Thread(() -> { System.out.println("Hello from thread"); }); thread.start(); thread.join();
7. Java Threads == OS Threads • TID = the unique ID given by JVM • NID = Native ID, the unique id given by OS
8. Threads are expensive • Thread.start() consider inefficient • 2 MiB of memory (outside of the heap) • A wrapper of OS threads • Context switching: ~100µs • So 1 million Threads require 2 Terabytes of memory!
9. Concurrency and Throughput
10. Let’s counts threads
11. Native threads counts on my machine • MacBook Pro (16-inch, 2021) • macOS: Ventura 13.3 • Memory: 64 GB • Chips: Apple M1 Max
12. What problem do we have now?
13. Common Server Fanout
15. An Example public Credit calculateCredit(Long personId) { var person = getPerson(personId); var assets = getAssets(person); var liabilities = getLiabilities(person); importantWork(); return getCredit(assets, liabilities); }
17. Classical implementation var atomicReference = new AtomicReference<Person>(); new Thread(() -> { var person = getPerson(1L); atomicReference.set(person); }).start();
18. static Credit calculateCredit(Long personId) throws InterruptedException { var person = getPerson(personId); var assetRef = new AtomicReference<Assets>(); var t1 = new Thread(() -> { var assets = getAssets(person); assetRef.set(assets); }); var liabilitiesRef = new AtomicReference<Liabilities>(); var t2 = new Thread(() -> { var liabilities = getLiabilities(person); liabilitiesRef.set(liabilities); }); var t3 = new Thread(() -> importantWork()); t1.start(); t2.start(); t3.start(); t1.join(); t2.join(); Credit credit = calculateCredits(assetRef.get(), liabilitiesRef.get()); t3.join(); return credit; }
19. Let’s improve it a bit • Java 5 introduces Executors with • Future • Callable
20. static Credit calculateCredit(Long personId, ExecutorService threadPool) throws InterruptedException, ExecutionException { var person = getPerson(personId); Future<Assets> assetsFuture = threadPool.submit(() -> getAssets(person)); Future<Liabilities> liabilitiesFuture = threadPool.submit(() -> getLiabilities(person)); threadPool.submit(() -> importantWork()); return calculateCredits(assetsFuture.get(), liabilitiesFuture.get()); }
21. How traditional thread pool works
22. But not there yet
23. Let’s improve it more
24. Let’s introduce composability Java 8 introduce CompletableFuture
25. static Credit calculateCreditForPerson2(Long personId) throws InterruptedException, ExecutionException { return runAsync(() -> importantWork()) .thenCompose(aVoid -> supplyAsync(() -> getPerson(personId)) .thenCombineAsync(supplyAsync(() -> getAssets(getPerson(personId))), (person, assets) -> calculateCredits(assets, getLiabilities(person)))) .get(); }
26. CompletableFuture • It’s quite a big class with more than 3000 lines of code • It has more than 50 methods which we can readily use and pretty much serves our all purpose • A task can be combined, chained and composed
27. Reactive Java • RxJava • Akka • Eclipse Vert.x • Spring WebFlux • Slick
28. import io.reactivex.rxjava3.core.Single; Single<Credit> calculateCredit (Long personId) { return Single.fromCallable(this::importantWork) .flatMap(aVoid -> Single.fromCallable(() -> getPerson(personId)) .flatMap(person -> Single.zip( Single.fromCallable(() -> getAssets(person)), Single.fromCallable(() -> getLiabilities(person)), this::calculateCredits ) ) ); }
29. Cons of Reactive framework • Learning curve • Debugging is difficult • The reading experience is terrible, cognitive load • Easy to over-complicated things • Don’t treat it as a hammer, and don’t hit every nail you find
30. Project Loom
31. Project loom (Cont.)
32. DEOM
34. Benefits • Virtual threads don’t consume CPU while they are waiting/sleeping • Technically we can have millions of virtual threads • Threads are cheap • Request-per-thread style of server-side programming becomes manageable again as before. • Improved throughput • On Blocking I/O, virtual threads get automatically suspended • No longer need to maintain ThreadPool
35. Let’s Download 10K Images Flux.fromIterable(imageUrls) .flatMap(url -> downloadImage(httpClient, url)) .doOnNext(tuple -> saveImage(tuple.getT1(), tuple.getT2())) .doOnComplete(() -> System.out.println("Finished downloading and saving images.")) .blockLast();
36. Let’s Download 10K Images (2) for (String imageUrl : imageUrls) { executor.submit(() -> { byte[] imageBytes = downloadImage(httpClient, imageUrl); saveImage(imageUrl, imageBytes); System.out.println("Downloaded and saved: " + imageUrl); }); }
37. Framework Implementing Virtual Threads • Spring Boot • Helidon Nima • Quarkus • And many more
38. Think about a Bank with Fixed Number of Teller • In Java terms, a teller is like a Platform Thread • Generally, it would take time to process each customer, say an average of 5 minutes. • Sometimes, a customer would block the process, such as the teller needing to make a phone call to get some information • No work is performed while the teller is blocked waiting and consequently, the entire line is blocked • In Java terms, a teller is still like a Platform Thread but has the ability to park a customer • Generally, it still takes time to process each customer, say an average of 5 minutes • Sometimes, a customer would block the process, such as the teller needing some information before proceeding… • The teller sends a text message or emails to get the necessary information • The teller asks the customer to be seated, and as soon the information is available, they will be the next customer processed by the first available teller • The teller starts processing the next customer in line • This is analogous to a parked Virtual Thread, where the teller is like a Platform Thread, and the customer is like a Virtual Thread • Concurrency is increased by better policies and procedures in dealing with blocking operations
41. https://twitter.com/bazlur_rahman https://www.linkedin.com/in/bazlur/ https://foojay.io/today/author/bazlur-rahman/ https://bazlur.ca
42. Thank you