What the heck is Project Loom?

You may have heard the words Project Loom, Fibers, Structured concurrency, and Virtual threads from Java enthusiasts. But what exactly are these, and what is the current state of concurrency in Java? Can Java keep up with languages like Go and Rust when it comes to fearless concurrency? Let's be honest, concurrency in java is not among the easiest to master, and thread safety is even more difficult to get right. Can Project loom help to make it easier and less idiot-proof? In this session, I'll do a deep dive into Project loom and explain the different features in the work, look at the current state, and compare it with what we have today with some samples. We will also look into it from the aspect of what a normal Java developer can expect/benefit from it. So come join me for an exciting ride.

Transcript

1. @deepu105 @oktaDev What the heck is Project Loom? Deepu K Sasidharan @deepu105 | deepu.tech
2. @deepu105 @oktaDev Hi, I’m Deepu K Sasidharan ➔ JHipster co-lead developer ➔ Java Champion ➔ Creator of KDash, JDL Studio ➔ Developer Advocate @ Auth0 by Okta ➔ OSS aficionado, polyglot dev, author, speaker @deepu105@mastodon.online deepu.tech @deepu105 deepu05
3. @deepu105 @oktaDev Concurrency in Java https://deepu.tech/concurrency-in-modern-languages-java/
4. @deepu105 @oktaDev JDK Evolution JDK Green threads 1.0
5. @deepu105 @oktaDev JDK Evolution Platform threads 1.1 JDK Green threads 1.0
6. @deepu105 @oktaDev JDK Evolution Platform threads 1.1 JDK Green threads 1.0 Executor, mutex, concurrent collections, semaphore, barrier, latches and blocking queues 1.5 ForkJoinPool 1.7
7. @deepu105 @oktaDev JDK Evolution Platform threads 1.1 JDK Green threads 1.0 Executor, mutex, concurrent collections, semaphore, barrier, latches and blocking queues 1.5 ForkJoinPool 1.7 Streams, CompletableFuture and CompletionException 1.8 (8)
8. @deepu105 @oktaDev JDK Evolution Platform threads 1.1 JDK Green threads 1.0 Executor, mutex, concurrent collections, semaphore, barrier, latches and blocking queues 1.5 ForkJoinPool 1.7 Streams, CompletableFuture and CompletionException 1.8 (8) Virtual threads and Structured concurrency 19
9. @deepu105 @oktaDev Platform Threads • Platform threads == OS threads • Platforms threads are mapped 1:1 to OS threads
10. @deepu105 @oktaDev Thread-per-request model Request 1 Platform Thread 1 OS Thread 1 Request 2 Platform Thread 2 OS Thread 2 Request N Platform Thread N OS Thread N
11. @deepu105 @oktaDev Thread-per-request model Little’s law λ = L/W λ = Throughput (average rate of requests) L = Average concurrency (number of requests concurrently processed by the server) W = Latency (average duration of processing each request) Request 1 Platform Thread 1 OS Thread 1 Request 2 Platform Thread 2 OS Thread 2 Request N Platform Thread N OS Thread N
12. @deepu105 @oktaDev Parallel processing • Should handle data races and data corruption • Thread synchronization might be needed • Thread leaks and cancellation delays • Fragile • A lot of responsibility on the developer Task Subtask 1 Subtask 2 Subtask N Platform Thread 1 OS Thread 1 Platform Thread 2 OS Thread 2 Platform Thread N OS Thread N
13. @deepu105 @oktaDev Project Loom https://developer.okta.com/blog/2022/08/26/state-of-java-project-loom
14. @deepu105 @oktaDev Project Loom Project Loom aims to drastically reduce the effort of writing, maintaining, and observing high-throughput concurrent applications that make the best use of available hardware. — Ron Pressler (Tech lead, Project Loom)
15. @deepu105 @oktaDev Virtual threads a.k.a User mode threads a.k.a Coroutines
16. @deepu105 @oktaDev Green threads mapping Green thread 1 Green thread 2 Green thread N Green Threads (M:1) OS Thread 1
17. @deepu105 @oktaDev Platform threads mapping Green thread 1 Green thread 2 Green thread N Green Threads (M:1) Platform Thread 1 OS Thread 1 Platform Thread 2 OS Thread 2 Platform Thread N OS Thread N OS Thread 1 Platform Threads (1:1)
18. @deepu105 @oktaDev Virtual threads mapping Green thread 1 Green thread 2 Green thread N Green Threads (M:1) Platform Thread 1 OS Thread 1 Platform Thread 2 OS Thread 2 Platform Thread N OS Thread N OS Thread 1 Platform Threads (1:1) Virtual thread 1 Virtual thread 2 Virtual thread N Virtual Threads (M:N) OS Thread 1 OS Thread N
19. @deepu105 @oktaDev Goroutines go func() { println("Hello, Goroutines!") }()
20. @deepu105 @oktaDev Kotlin coroutines runBlocking { launch { println("Hello, Kotlin coroutines!") } }
21. @deepu105 @oktaDev Java virtual thread Thread.startVirtualThread(() -> { System.out.println("Hello, Project Loom!"); });
22. @deepu105 @oktaDev Virtual thread features • It is a Thread in code, runtime, debugger, and profiler • It’s a Java entity and not a wrapper around an OS thread • Creating and blocking them are cheap operations • They should not be pooled • Virtual threads use a work-stealing ForkJoinPool scheduler • Pluggable schedulers can be used for asynchronous programming • A virtual thread will have its own stack memory • The virtual threads API is very similar to platform threads and hence easier to adopt/migrate
23. @deepu105 @oktaDev Total number of platform threads var counter = new AtomicInteger(); while (true) { new Thread(() -> { int count = counter.incrementAndGet(); System.out.println("Thread count = " + count); LockSupport.park(); }).start(); }
24. @deepu105 @oktaDev Total number of virtual threads var counter = new AtomicInteger(); while (true) { Thread.startVirtualThread(() -> { int count = counter.incrementAndGet(); System.out.println("Thread count = " + count); LockSupport.park(); }); }
25. @deepu105 @oktaDev Task throughput for platform threads try (var executor = Executors.newThreadPerTaskExecutor(Executors.defaultThreadFactory())) { IntStream.range(0, 100_000).forEach(i -> executor.submit(() -> { Thread.sleep(Duration.ofSeconds(1)); System.out.println(i); return i; })); } # 'newThreadPerTaskExecutor' with 'defaultThreadFactory' 0:18.77 real, 18.15 s user, 7.19 s sys, 135% 3891pu, 0 amem, 743584 mmem # 'newCachedThreadPool' with 'defaultThreadFactory' 0:11.52 real, 13.21 s user, 4.91 s sys, 157% 6019pu, 0 amem, 2215972 mmem
26. @deepu105 @oktaDev Task throughput for virtual threads try (var executor = Executors.newVirtualThreadPerTaskExecutor()) { IntStream.range(0, 100_000).forEach(i -> executor.submit(() -> { Thread.sleep(Duration.ofSeconds(1)); System.out.println(i); return i; })); } 0:02.62 real, 6.83 s user, 1.46 s sys, 316% 14840pu, 0 amem, 350268 mmem
27. @deepu105 @oktaDev JMH Benchmarks # Throughput (more is better) Benchmark Mode Cnt Score Error Units LoomBenchmark.platformThreadPerTask thrpt 5 0.362 ± 0.079 ops/s LoomBenchmark.platformThreadPool thrpt 5 0.528 ± 0.067 ops/s LoomBenchmark.virtualThreadPerTask thrpt 5 1.843 ± 0.093 ops/s # Average time (less is better) Benchmark Mode Cnt Score Error Units LoomBenchmark.platformThreadPerTask avgt 5 5.600 ± 0.768 s/op LoomBenchmark.platformThreadPool avgt 5 3.887 ± 0.717 s/op LoomBenchmark.virtualThreadPerTask avgt 5 1.098 ± 0.020 s/op https://github.com/deepu105/java-loom-benchmarks
28. @deepu105 @oktaDev More benchmarks • An interesting benchmark using ApacheBench on GitHub by Elliot Barlas • A benchmark using Akka actors on Medium by Alexander Zakusylo • JMH benchmarks for I/O and non-I/O tasks on GitHub by Colin Cachia
29. @deepu105 @oktaDev Structured concurrency
30. @deepu105 @oktaDev Without structured concurrency void handleOrder() throws ExecutionException, InterruptedException { try (var esvc = new ScheduledThreadPoolExecutor(8)) { Future<Integer> inventory = esvc.submit(() -> updateInventory()); Future<Integer> order = esvc.submit(() -> updateOrder()); int theInventory = inventory.get(); // Join updateInventory int theOrder = order.get(); // Join updateOrder System.out.println("Inventory " + theInventory + " updated for order " + theOrder); } }
31. @deepu105 @oktaDev Without structured concurrency void handleOrder() throws ExecutionException, InterruptedException { try (var esvc = new ScheduledThreadPoolExecutor(8)) { Future<Integer> inventory = esvc.submit(() -> updateInventory()); // failed Future<Integer> order = esvc.submit(() -> updateOrder()); // runs in background int theInventory = inventory.get(); // Join updateInventory // fails int theOrder = order.get(); // Join updateOrder // unreachable System.out.println("Inventory " + theInventory + " updated for order " + theOrder); } }
32. @deepu105 @oktaDev Without structured concurrency void handleOrder() throws ExecutionException, InterruptedException { try (var esvc = new ScheduledThreadPoolExecutor(8)) { Future<Integer> inventory = esvc.submit(() -> updateInventory()); // expensive task Future<Integer> order = esvc.submit(() -> updateOrder()); // failed int theInventory = inventory.get(); // Join updateInventory // task blocked int theOrder = order.get(); // Join updateOrder // will fail System.out.println("Inventory " + theInventory + " updated for order " + theOrder); } }
33. @deepu105 @oktaDev Without structured concurrency void handleOrder() throws ExecutionException, InterruptedException { // interrupted try (var esvc = new ScheduledThreadPoolExecutor(8)) { Future<Integer> inventory = esvc.submit(() -> updateInventory()); // runs in bg Future<Integer> order = esvc.submit(() -> updateOrder()); // runs in bg int theInventory = inventory.get(); // Join updateInventory int theOrder = order.get(); // Join updateOrder System.out.println("Inventory " + theInventory + " updated for order " + theOrder); } }
34. @deepu105 @oktaDev Structured concurrency void handleOrder() throws ExecutionException, InterruptedException { try (var scope = new StructuredTaskScope.ShutdownOnFailure()) { Future<Integer> inventory = scope.fork(() -> updateInventory()); Future<Integer> order = scope.fork(() -> updateOrder()); scope.join(); // Join both forks scope.throwIfFailed(); // ... and propagate errors // Here, both forks have succeeded, so compose their results System.out.println("Inventory " + inventory.resultNow() + " updated for order " + order.resultNow()); } }
35. @deepu105 @oktaDev Structured concurrency void handleOrder() throws ExecutionException, InterruptedException { try (var scope = new StructuredTaskScope.ShutdownOnFailure()) { Future<Integer> inventory = scope.fork(() -> updateInventory()); // failed Future<Integer> order = scope.fork(() -> updateOrder()); // cancelled scope.join(); // Join both forks scope.throwIfFailed(); // ... and propagate errors // Here, both forks have succeeded, so compose their results System.out.println("Inventory " + inventory.resultNow() + " updated for order " + order.resultNow()); } }
36. @deepu105 @oktaDev State of Project Loom
37. @deepu105 @oktaDev Impact for regular developers • No breaking changes • Very low API surface and hence easy to adopt/migrate • Rely on underlying libraries to switch to virtual threads • Debugging virtual threads would need some getting used to • Can easily switch to virtual threads from thread pools • Structured concurrency could help to eliminate a lot of failsafe code • At the moment need to use preview and incubator modules • Some unlearning to do (no pooling, no reusing, no shared pool executors) • Proliferation of virtual threads in simple use cases.
38. @deepu105 @oktaDev Impact for libraries • Performance and throughput increases • Early adoption • Simpler codebase • Server software like tomcat, Undertow and Jetty will see improvements • Frameworks like Spring, Micronaut and Quarkus will see improvements • Libraries like RxJava and Akka might benefit from structured concurrency • Asynchronous and reactive programming will still be around but in many use cases virtual threads could replace them and give same benefits with less complexity
39. @deepu105 @oktaDev Early adoption • GraalVM ◦ Support added (https://github.com/oracle/graal/pull/4802) • Quarkus ◦ Support added (https://github.com/quarkusio/quarkus/pull/24942) • Micronaut ◦ Being discussed (https://github.com/micronaut-projects/micronaut-core/issues/7724) • Spring ◦ https://spring.io/blog/2022/10/11/embracing-virtual-threads
40. @deepu105 @oktaDev Caveats
41. @deepu105 @oktaDev Resources • https://www.infoq.com/articles/java-virtual-threads/ • https://inside.java/2020/08/07/loom-performance/ • http://cr.openjdk.java.net/~rpressler/loom/loom/sol1_part1.html • https://foojay.io/today/thinking-about-massive-throughput-meet-virtual-th reads/
42. @deepu105 @oktaDev Get the Slides
43. @deepu105 @oktaDev Thank You Deepu K Sasidharan @deepu105 | deepu.tech https://deepu.tech/tags#java https://developer.auth0.com