More action, more overview - From 4k to 1M Threads

Transcript

1. More action, more overview From 4k to 1M Threads
2. ©2024 Moderne, Inc. Merlin Bögershausen MBoegers Staff Software Engineer, Moderne Inc. https://mboegie.dev
3. ©2025 Moderne, Inc. Who sees a problem? Quick POLL POLL 2 for (int i = 0; i < 1_000_000; i++) { } new Thread(() -> processOrder()).start();
4. ©2024 Moderne, Inc. 18 The Money Slide 1 MB per Thread ~ 10.000 Threads max 1 ms creation time 1 KB per Thread 1.000.000+ Threads possible Created in 1 μs 1000 x 100 x 1000 x Platform Thread Virtual Thread Impact https://docs.oracle.com/en/java/javase/21/core/virtual-threads.html https://www.infoq.com/articles/java-virtual-threads-a-case-study/ https://github.com/ebarlas/java-httpserver-vthreads
5. ©2024 Moderne, Inc. 5 Rule Engine Example class Engine { interface Rule { void fire(); } void execute(List<Rule> rules, Map<String, String> data) {} } Rule Engine with 100.000 Rules executed in parallel GitHub GIST
6. ©2025 Moderne, Inc. 8 Virtual Threads
7. ©2024 Moderne, Inc. 7 Java Thread - Basics // Platform Thread = OS Thread = 1MB new Thread(() -> doWork()).start(); // Workaround: Thread Pools, ... Executors.newFixedThreadPool(100);
8. ©2024 Moderne, Inc. 8 Virtual Threads – Java 21 for (int i = 0; i < 1_000_000; i++) { // 1 Virtual Thread = 1KB Thread.startVirtualThread(() -> doWork()); } // Retrofit to Executors try (var executor = Executors.newVirtualThreadPerTaskExecutor()) { IntStream.range(0, 1_000_000) .forEach(i -> executor.submit(() -> doWork())); }
9. ©2024 Moderne, Inc. 10 Virtual Threads Architecture Virtual Thread Carrier Thread OS Thread 1 KB Heap ForkJoinPool 1 MB Stack Platform Thread OS Thread loose 1:1 1:1
10. ©2024 Moderne, Inc. 11 • I/O-bound operations DB, REST calls • High concurrency >1000 concurrent tasks • Request-per-thread model Great us if When to Use Virtual Threads • CPU-intensive calculations • Thread pools (NEVER pool Virtual Threads!) Avoid for
11. ©2025 Moderne, Inc. 12 Scoped Values - Thread-Safe Context
12. ©2024 Moderne, Inc. 13 Share an SSLContext
13. ©2024 Moderne, Inc. 14 Scoped Values API // JEP 506 - Final in Java 25! private static final ScopedValue<SSLContext> SSL_CTX = ScopedValue.newInstance(); private static final ScopedValue<Map<String, String>> CONTEXT = ScopedValue.newInstance(); // set and pass ScopedValue.where(SSL_CTX, sslContext) .where(CONTEXT, data) .run(() -> processWithContext());
14. ©2024 Moderne, Inc. 15 • Mutable data • Not memory Save • Copy per (virtual) Thread • Slower due to Implementation ThreadLocal What about ThreadLocal • Immutable Data • Memory Save • Designed for Virtual Threads • Efficient Implementation ScopedValue
15. ©2025 Moderne, Inc. 16 Structured Concurrency
16. ©2024 Moderne, Inc. 17 Manage multiple Task as Unit
17. ©2024 Moderne, Inc. 18 StructuredTaskScope API try (var scope = StructuredTaskScope.open()) { Subtask<String> task = scope.fork(() -> doWork()); scope.join(); // void return return task.get(); //Exception on failure }
18. ©2024 Moderne, Inc. 19 Concurrent Rule void fire() { try (var scope = StructuredTaskScope.open()) { Subtask<String> name = scope.fork(() -> readName()); Subtask<Address> address = scope.fork(() -> readAddress()); scope.join(); System.out.printf("%s's home is %s %d, %s%n", name.get(), address.get().street(), address.get().house(), address.get().city()); } catch (InterruptedException e) { throw new RuntimeException(e); } }
19. ©2025 Moderne, Inc. Concurrent Rule Engine and Migration analysis Live Code
20. ©2025 Moderne, Inc. 21 Best Practices
21. ©2024 Moderne, Inc. 22 Path to Migrate 1. Virtual Threads (Final seit Java 21) → migrate if I/O-bound workloads 2. Scoped Values (Final in Java 25) → ThreadLocal replace where immutable, use recipe 3. Structured Concurrency (Fifth Preview) → experiment and feedback; not in prod, yet 4. Stable Values (Preview in Java 25) → Check for lazy initialization
22. ©2024 Moderne, Inc. 23 Best Practices Thread.startVirtualThread() Executors.newVirtualThreadPerTaskExecutor() ScopedValues für immutable context A million Virtual Threads is fine! NEVER pool Virtual Threads No ThreadLocals with Virtual Threads Not for CPU-intensive Tasks Pro tip: Use a recipe to Understand surface
23. ©2024 Moderne, Inc. 24 Resources Code & Examples: github.com/MBoegers/87fa0928c46e037155dbc7d48aaf7553 github.com/MBoegers/openrewrite-misc-recipes Compile & Run: javac --release 25 --enable-preview *.java java --enable-preview Main JEPs: • JEP 444: Virtual Threads (Final) • JEP 506: Scoped Values (Final) • JEP 505: Structured Concurrency (Preview) • JEP 502: Stable Values (Preview) GitHub GIST
24. ©2024 Moderne, Inc. Merlin Bögershausen Thanks and see you around Please rate and give feedback