Are Virtual Threads Going to Make Reactive Programming Irrelevant?

Slide 1

Slide 1 text

Are Virtual Threads Going to Make Reactive Programming Irrelevant? Loom, Structured Concurrency, Reactive Programming José Paumard Java Developer Advocate Java Platform Group

Slide 2

Slide 2 text

@josepaumard.bsky.social https://dev.java

Slide 3

Slide 3 text

Slide 4

Slide 4 text

Slide 5

Slide 5 text

Slide 6

Slide 6 text

Slide 7

Slide 7 text

Slide 8

Slide 8 text

Slide 9

Slide 9 text

Slide 10

Slide 10 text

Slide 11

Slide 11 text

Slide 12

Slide 12 text

Slide 13

Slide 13 text

Slide 14

Slide 14 text

2/5/2025 Copyright © 2024, Oracle and/or its affiliates 14 Virtual Threads and Reactive Programming are about solving the same problem Reactive Programming has been there for 10+ years At least you know how it is working! Virtual Threads & Structured Concurrency is not there yet Structured Concurrency is still a preview feature And it looks promising! What does it mean to be irrelevant?

Slide 15

Slide 15 text

2/5/2025 Copyright © 2024, Oracle and/or its affiliates 15 Concurrency is hard It is needed to achieve high performance (throughput) As of now, the paradigm to achieve high throughput is callback-based reactive programming … which comes with drawbacks Why do you need reactive programming? Loom is About Fixing Concurrency Issues

Slide 16

Slide 16 text

2/5/2025 Copyright © 2024, Oracle and/or its affiliates 16 Reading images and links Too unefficient to be used A Simple Example var image = someService.readImage(); var links = someService.readLinks(); var page = new Page(image, links);

Slide 17

Slide 17 text

2/5/2025 Copyright © 2024, Oracle and/or its affiliates 17 The 2004 solution (with lambdas and var) What is wrong with this code? Concurrency Issues ExecutorService es = ...; var f1 = es.submit(someService::readImages); var f2 = es.submit(someService::readLinks); var page = new Page(f1.get(1, TimeUnit.SECONDS), f2.get(1, TimeUnit.SECONDS));

Slide 18

Slide 18 text

2/5/2025 Copyright © 2024, Oracle and/or its affiliates 18 1) It blocks threads Blocking a platform thread is wrong, for many reasons (Here it is blocked for ~100ms) Concurrency Issues ExecutorService es = ...; var f1 = es.submit(someService::readImages); var f2 = es.submit(someService::readLinks); var page = new Page(f1.get(1, TimeUnit.SECONDS), f2.get(1, TimeUnit.SECONDS)); -> ES thread is blocked -> ES thread is blocked -> the main thread is blocked

Slide 19

Slide 19 text

2/5/2025 Copyright © 2024, Oracle and/or its affiliates 19 2) It can lead to a loose thread Concurrency Issues ExecutorService es = ...; var f1 = es.submit(someService::readImages); var f2 = es.submit(someService::readLinks); var page = new Page(f1.get(1, TimeUnit.SECONDS), f2.get(1, TimeUnit.SECONDS));

Slide 20

Slide 20 text

2/5/2025 Copyright © 2024, Oracle and/or its affiliates 20 2) It can lead to a loose thread Concurrency Issues ExecutorService es = ...; var f1 = es.submit(someService::readImages); var f2 = es.submit(someService::readLinks); var page = new Page(f1.get(1, TimeUnit.SECONDS), f2.get(1, TimeUnit.SECONDS)); -> if f1.get() crashes...

Slide 21

Slide 21 text

2/5/2025 Copyright © 2024, Oracle and/or its affiliates 21 2) It can lead to a loose thread If something goes wrong in readLinks, then the thread running it is never freed. You now have a loose thread! Concurrency Issues ExecutorService es = ...; var f1 = es.submit(someService::readImages); var f2 = es.submit(someService::readLinks); var page = new Page(f1.get(1, TimeUnit.SECONDS), f2.get(1, TimeUnit.SECONDS)); -> if f1.get() crashes... ... f2.get() is never called

Slide 22

Slide 22 text

2/5/2025 Copyright © 2024, Oracle and/or its affiliates 22 3) It’s hard to debug Stack traces do not give you the information you need. Concurrency Issues ExecutorService es = ...; var f1 = es.submit(someService::readImages); var f2 = es.submit(someService::readLinks); var page = new Page(f1.get(1, TimeUnit.SECONDS), f2.get(1, TimeUnit.SECONDS));

Slide 23

Slide 23 text

2/5/2025 Copyright © 2024, Oracle and/or its affiliates 23 3 issues to fix: - Blocking a platform thread is bad - Having a non-relevant stack trace is annoying - Having loose threads is annoying Concurrency Issues

Slide 24

Slide 24 text

2/5/2025 Copyright © 2024, Oracle and/or its affiliates 24 A java.lang.Thread is a wrapper on a kernel (or platform) thread It needs: - ~1ms to start - ~2MB of memory to store its stack - context switching costs ~0,1ms You can only have several thousands of them What is a Platform Thread?

Slide 25

Slide 25 text

2/5/2025 Copyright © 2024, Oracle and/or its affiliates 25 Handling one network request per thread Limits your application to ~4k concurrent requests If handling one request consumes 1s of CPU time Then your CPU can handle 1M requests per second The model one request per thread does not work with Platform Threads (anymore) Why is it Wrong to Block a P. Thread?

Slide 26

Slide 26 text

2/5/2025 Copyright © 2024, Oracle and/or its affiliates 26 There are two solutions: - The reactive programming solution: have a platform thread handle many requests - How many? ~1k per P. Thread What Solution?

Slide 27

Slide 27 text

2/5/2025 Copyright © 2024, Oracle and/or its affiliates 27 This is the classical way of writing it Let’s Rewrite the Previous Code ExecutorService es = ...; var f1 = es.submit(someService::readImages); var f2 = es.submit(someService::readLinks); var page = new Page(f1.get(1, TimeUnit.SECONDS), f2.get(1, TimeUnit.SECONDS));

Slide 28

Slide 28 text

2/5/2025 Copyright © 2024, Oracle and/or its affiliates 28 This is the CompletableFuture way of writing it Let’s Rewrite the Previous Code var cf1 = CompletableFuture .supplyAsync(someService::readImages) .whenComplete((images, error) -> { if (error != null) { // log something } }); var cf2 = // same with readLinks var cf3 = cf1.runAfterBoth(cf2, () -> { var page = new Page(cf1.get(), cf2.get()); // do something with page });

Slide 29

Slide 29 text

Slide 30

Slide 30 text

2/5/2025 Copyright © 2024, Oracle and/or its affiliates 30 Suppose something wrong happens in readImages() Your stack trace tells you who called readImages() What About Debug? 27: var images = someService.readImages(); Exception in thread "main" java.lang.IllegalStateException: Boom! at org.myapp.SomeService.readImages(SomeService.java:12) at org.myapp.MyApp.main(MyApp.java:27)

Slide 31

Slide 31 text

2/5/2025 Copyright © 2024, Oracle and/or its affiliates 31 Suppose something wrong happens in readImages() What About Debug? Exception in thread "main" java.util.concurrent.ExecutionException: java.lang.IllegalStateException: Boom! at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122) at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:191) at org.myapp.MyApp.main(MyApp.java:32) Caused by: java.lang.IllegalStateException: Boom! at org.myapp.SomeService.readImages(SomeService.java:12) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) at java.base/java.lang.Thread.run(Thread.java:1575) 29: var f1 = es.submit(someService::readImages); 30: var f2 = es.submit(someService::readLinks); 31: 32: var p = new Page(f1.get(), f2.get());

Slide 32

Slide 32 text

2/5/2025 Copyright © 2024, Oracle and/or its affiliates 32 Suppose something wrong happens in readImages() What About Debug? var cf1 = CompletableFuture.supplyAsync(SomeService::readImages) .whenComplete((images, error) -> { if (error != null) { error.printStackTrace(); error.getCause().printStackTrace(); } }); var cf2 = // the same with readLinks cf1.runAfterBoth(cf2, () -> { try { var page = new Page(cf1.get(), cf2.get()); // do something with page } catch (Exception e) { // do something with the error } });

Slide 33

Slide 33 text

2/5/2025 Copyright © 2024, Oracle and/or its affiliates 33 Suppose something wrong happens in readImages() What About Debug? java.util.concurrent.CompletionException: java.lang.IllegalStateException: Boom! at java.base/java.util.concurrent.CompletableFuture.wrapInCompletionException(CompletableFuture.java:323) at java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:359) at java.base/java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:364) at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1814) at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.exec(CompletableFuture.java:1804) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:507) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1458) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:2034) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:189) var cf1 = CompletableFuture.supplyAsync(SomeService::readImages) .whenComplete((images, error) -> { if (error != null) { error.printStackTrace(); error.getCause().printStackTrace(); } });

Slide 34

Slide 34 text

2/5/2025 Copyright © 2024, Oracle and/or its affiliates 34 Suppose something wrong happens in readImages() What About Debug? java.lang.IllegalStateException: Boom! at org.paumard.SomeService.readImages(SomeService.java:12) at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1812) at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.exec(CompletableFuture.java:1804) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:507) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1458) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:2034) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:189) var cf1 = CompletableFuture.supplyAsync(SomeService::readImages) .whenComplete((images, error) -> { if (error != null) { error.printStackTrace(); error.getCause().printStackTrace(); } });

Slide 35

Slide 35 text

2/5/2025 Copyright © 2024, Oracle and/or its affiliates 35 - Efficient - The programming model is hard to read / write / test - Impossible to debug - Impossible to profile - Don’t write blocking lambdas Async Programming

Slide 36

Slide 36 text

2/5/2025 Copyright © 2024, Oracle and/or its affiliates 36 There are two solutions: - The reactive programming solution: have a platform thread handle many requests - How many? ~1k per P. Thread - Build a model of thread lighter than a Platform Thread - How much lighter? ~1k lighter Being light is not enough! What Solution?

Slide 37

Slide 37 text

2/5/2025 Copyright © 2024, Oracle and/or its affiliates 37 - To be as efficient (same throughput) - To provide a simple programming model - To make debugging possible - To make profiling possible Virtual Threads Need…

Slide 38

Slide 38 text

2/5/2025 Copyright © 2024, Oracle and/or its affiliates 38 There are two solutions: - The async programming solution: have a platform thread handle many requests - How many? ~1k per P. Thread - Build a model of thread lighter than a Platform Thread - How much lighter? ~1k lighter What Solution?

Slide 39

Slide 39 text

Slide 40

Slide 40 text

Slide 41

Slide 41 text

Slide 42

Slide 42 text

Slide 43

Slide 43 text

Slide 44

Slide 44 text

Slide 45

Slide 45 text

Slide 46

Slide 46 text

2/5/2025 Copyright © 2024, Oracle and/or its affiliates 46 There are two solutions: - The async programming solution: have a platform thread handle many requests - How many? ~1k per P. Thread - Build a model of thread lighter than a Platform Thread - How much lighter? ~1k lighter - Blocking a virtual thread does not block the underlying Platform Thread What Solution?

Slide 47

Slide 47 text

Slide 48

Slide 48 text

Slide 49

Slide 49 text

2/5/2025 Copyright © 2024, Oracle and/or its affiliates 49 The code you write is the most basic code you can write Virtual threads handle the blocking API for you Writing blocking code is OK (Actually, this is what you should do) Virtual Threads Programming Model var images = someService.readImages();

Slide 50

Slide 50 text

2/5/2025 Copyright © 2024, Oracle and/or its affiliates 50 The code you write is the most basic code you can write Virtual threads handle the blocking API for you Writing blocking code is OK (Actually, this is what you should do) Virtual Threads Programming Model Callable fetchImages = () -> someService.readImages(); var f = Executors.newVirtualThreadPerTaskExecutor() .submit(fetchImages); System.out.println("f = " + f.get());

Slide 51

Slide 51 text

2/5/2025 Copyright © 2024, Oracle and/or its affiliates 51 And if something goes wrong: Virtual Threads Programming Model 78: Callable fetchImages = () -> someService.readImages(); 79: var f = Executors.newVirtualThreadPerTaskExecutor() 80: .submit(fetchImages); 81: System.out.println("f = " + f.get()); Exception in thread "main" java.util.concurrent.ExecutionException: java.lang.IllegalStateException: Boom! at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122) at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:191) at org.paumard.MyApp.main(MyApp.java:81) Caused by: java.lang.IllegalStateException: Boom! at org.paumard.SomeService.readImages(SomeService.java:12) at org.paumard.MyApp.lambda$main$0(MyApp.java:78) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317) at java.base/java.lang.VirtualThread.run(VirtualThread.java:329)

Slide 52

Slide 52 text

2/5/2025 Copyright © 2024, Oracle and/or its affiliates 52 When you need to run several task in parallel V. Threads and Structured Concurrency try (var scope = new StructuredTaskScope<>()) { } catch (InterruptedException e) { // do something with the error }

Slide 53

Slide 53 text

2/5/2025 Copyright © 2024, Oracle and/or its affiliates 53 When you need to run several task in parallel V. Threads and Structured Concurrency try (var scope = new StructuredTaskScope<>()) { var imagesSubtask = scope.fork(() -> someService.readImages()); var linksSubtask = scope.fork(() -> someService.readLinks()); } catch (InterruptedException e) { // do something with the error }

Slide 54

Slide 54 text

2/5/2025 Copyright © 2024, Oracle and/or its affiliates 54 When you need to run several task in parallel V. Threads and Structured Concurrency try (var scope = new StructuredTaskScope<>()) { var imagesSubtask = scope.fork(() -> someService.readImages()); var linksSubtask = scope.fork(() -> someService.readLinks()); scope.join(); } catch (InterruptedException e) { // do something with the error }

Slide 55

Slide 55 text

2/5/2025 Copyright © 2024, Oracle and/or its affiliates 55 When you need to run several task in parallel V. Threads and Structured Concurrency try (var scope = new StructuredTaskScope<>()) { var imagesSubtask = scope.fork(() -> someService.readImages()); var linksSubtask = scope.fork(() -> someService.readLinks()); scope.join(); var page = new Page(imagesSubtask.get(), linksSubtask.get()); // do something with page } catch (InterruptedException e) { // do something with the error }

Slide 56

Slide 56 text

2/5/2025 Copyright © 2024, Oracle and/or its affiliates 56 What if something goes wrong? V. Threads and Structured Concurrency try (var scope = new StructuredTaskScope<>()) { 62: var imagesSubtask = scope.fork(() -> someService.readImages()); var linksSubtask = scope.fork(() -> someService.readLinks()); scope.join(); if (imagesSubtask.state() == State.FAILED) { imagesSubtask.exception().printStackTrace(); } // ... }

Slide 57

Slide 57 text

2/5/2025 Copyright © 2024, Oracle and/or its affiliates 57 What if something goes wrong? (e.getCause() is null) V. Threads and Structured Concurrency java.lang.IllegalStateException: Boom! at org.myapp.SomeService.readImages(SomeService.java:12) at org.myapp.MyApp.lambda$main$0(MyApp.java:62) at java.base/java.util.concurrent. StructuredTaskScope$SubtaskImpl.run(StructuredTaskScope.java:893) at java.base/java.lang.VirtualThread.run(VirtualThread.java:329)

Slide 58

Slide 58 text

2/5/2025 Copyright © 2024, Oracle and/or its affiliates 58 So: - Blocking a platform thread is wrong - Fixed: Blocking a virtual thread is OK - Having a non-relevant stack trace is annoying - Fixed: Your stack trace takes you where your call is - Having loose threads is annoying - Fixed: No more loose threads thanks to AutoCloseable Concurrency Issues

Slide 59

Slide 59 text

2/5/2025 Copyright © 2024, Oracle and/or its affiliates 59 - Are as efficient - Provide a simple programming model - It’s your good old imperative model - Make debugging possible and easy - Make profiling possible Virtual Threads…

Slide 60

Slide 60 text

Slide 61

Slide 61 text

2/5/2025 Copyright © 2023, Oracle and/or its affiliates 61 So many things… - They are mutable - The VM cannot optimize them - They may be kept alive forever Virtual Threads support ThreadLocal variables! What is Wrong with ThreadLocal?

Slide 62

Slide 62 text

2/5/2025 Copyright © 2023, Oracle and/or its affiliates 62 ScopedValues are non-modifiable They are not bound to a particular thread They are bound to a single method call Welcome to ScopedValue! ScopedValue key = new ScopedValue.newInstance();

Slide 63

Slide 63 text

2/5/2025 Copyright © 2023, Oracle and/or its affiliates 63 ScopedValues are non-modifiable They are not bound to a particular thread They are bound to a single method call Welcome to ScopedValue! ScopedValue key = new ScopedValue.newInstance(); ScopedValue.where(key, "KEY_1") .run(() -> doSomethingSmart()));

Slide 64

Slide 64 text

2/5/2025 Copyright © 2023, Oracle and/or its affiliates 64 ScopedValues are non-modifiable They are not bound to a particular thread They are bound to a single method call Welcome to ScopedValue! ScopedValue key = ScopedValue.newInstance(); ScopedValue.where(key, "KEY_1") .run(() -> doSomethingSmart())); ScopedValue.where(key, "KEY_2") .run(() -> doSomethingSmart()) .run(() -> soSomethingSmarter());

Slide 65

Slide 65 text

2/5/2025 Copyright © 2023, Oracle and/or its affiliates 65 Welcome to ScopedValue! ScopedValues are non-modifiable They are not bound to a particular thread They are bound to a single method call void doSomethingSmart() { if (key.isBound()) { String value = key.get(); ... } else { throw new IllegalStateException("Key is not bound"); } }

Slide 66

Slide 66 text

2/5/2025 Copyright © 2023, Oracle and/or its affiliates 66 Scoped Values are not transmitted to threads reason: you don’t want loose scoped values They are transmitted to StructuredTaskScope reason: you can’t have loose scoped values ScopedValue

Slide 67

Slide 67 text

2/5/2025 Copyright © 2024, Oracle and/or its affiliates 67 Virtual Threads and Structured Concurrency fix the main problems of Asynchronous / Reactive Programming Without giving up on performance Scope Values fix the issues with ThreadLocal variables In a Nutshell

Slide 68

Slide 68 text

2/5/2025 Copyright © 2024, Oracle and/or its affiliates 68 JDK 23 JDK 24 Virtual Threads Final in 21 Final in 21 Structured Concurrency Preview ??? Scoped Values Preview ??? Loom: Current Plan for the JDK 23

Slide 69

Slide 69 text

Loom is great!