Demystifying Java Virtual Threads - Lessons learned from adding them to GlassFish

Do you want to understand how Java Virtual Threads really work and whether the myths you heard about are true or not?

In this presentation, we'll delve into how Java Virtual Threads work, dispelling myths and offering practical guidance on leveraging them effectively. Drawing from our experience with GlassFish and Grizzly, we'll discuss when and how to harness virtual threads to optimize performance and resource utilization in your applications. Along the way, we'll highlight common pitfalls to avoid.

We’ll debunk a few myths about virtual threads and explain why you shouldn’t be afraid to use them. Furthermore, we'll tackle the question of whether virtual threads pose a competition to reactive programming in Java or whether they can fit well together.

Transcript

1. Ondro Mihályi @OmniFishEE Demystifying Java Virtual Threads Lessons learned from adding them to GlassFish
2. Ondro Mihályi @OmniFishEE Who am I? Java Champion Czech JUG lead (CZJUG) Jakarta EE specifications committer Java developer, trainer & consultant
3. Ondro Mihályi @OmniFishEE Virtual threads are the next best thing in Java after Duke
4. Ondro Mihályi @OmniFishEE Virtual threads are the next best thing in Java after … Java Modules
5. Ondro Mihályi @OmniFishEE Virtual threads are the next best thing in Java after … Java Modules Java Lambdas!
6. Ondro Mihályi @OmniFishEE OK, Virtual Threads are cool. But why exactly?
7. Ondro Mihályi @OmniFishEE Platform threads First, let’s talk about why platform threads are not so cool
8. Ondro Mihályi @OmniFishEE What’s going on with platform threads Thread thread = new Thread(() -> { out.println("Hello from a thread"); }); thread.start(); // triggers a system call
9. Ondro Mihályi @OmniFishEE Issues with platform threads • Platform threads match OS threads • System calls required → slow: – Slow to create a new thread – Slow to switch thread context on a CPU core • Threads reserve off-heap memory
10. Ondro Mihályi @OmniFishEE Can we have unlimited nuber of threads? While (true) { Thread.ofPlatform().start(() -> { sleep(ofMinutes(10)); }); }
11. Ondro Mihályi @OmniFishEE Where’s the limit?
12. Ondro Mihályi @OmniFishEE Memory reserved by platform threads Thread 9 Thread 8 Thread 7 Thread 6 Thread 5 Thread 4 Thread 3 Thread 2 Thread 1 Used memory Reserved memory 1 MB
13. Ondro Mihályi @OmniFishEE Number of Platform threads is limited • By default, threads reserve 1MB • 1,000 threads → 1GB • 32,648 threads → > 32 GB
14. Ondro Mihályi @OmniFishEE Traditional answer – thread pools • Reuse same threads for multiple tasks – Avoid expensive creation • And limit their amount – Avoid getting out of memory
15. Ondro Mihályi @OmniFishEE Thread pools - lifecycle
16. Ondro Mihályi @OmniFishEE Issues with Thread pools Thread-locals may leak memory (GCed when removed or thread stops) Blocked threads block all tasks Thread-locals may leak state (when static and not cleared) Complicated tuning
17. Ondro Mihályi @OmniFishEE Anything better than thread pools? Thread-locals may leak memory (GCed when removed or thread stops) Blocked threads block all tasks Thread-locals may leak state (when static and not cleared) Complicated tuning
18. Ondro Mihályi @OmniFishEE Properties of Virtual Threads • Fast to create thread and switch context (no system call) • Millions of threads can run in parallel • No need for thread pools → simple • Stack saved in heap
19. Ondro Mihályi @OmniFishEE Virtual threads are fast Fast context switch • No system calls • More CPU time for work When blocked, create new threads • No system calls • No reserved RAM 15% Performance increase (22216.59 to 25895.12 req/sec)
20. Ondro Mihályi @OmniFishEE Unlimited nuber of virtual threads? While (true) { Thread.ofVirtual().start(() -> { sleep(ofMinutes(10)); }); }
21. Ondro Mihályi @OmniFishEE Plain Java objects How Virtual Threads work Store stacks in heap Executed on Platform Threads Yield to JVM when blocked. Cannot be paused if don’t yield
22. Ondro Mihályi @OmniFishEE Matching to carrier platform threads
23. Ondro Mihályi @OmniFishEE Execute HTTP requests on virtual threads • Enable custom executor service – A new thread for each request – Maximum parallel threads limited ➔ better throughput ➔ less memory needed Hands On: Virtual threads in GlassFish
24. Ondro Mihályi @OmniFishEE Background tasks on virtual threads • Coming in GlassFish 8 (Jakarta EE 11) @ManagedExecutorDefinition(virtual = true) executor.submit( () -> "Running in VT"); Hands On: Virtual threads in GlassFish
25. Ondro Mihályi @OmniFishEE • Thread locals don’t work • Need to avoid "synchronized" • Just use virtual threads and your done • Will kill reactive programming Myths about Virtual Threads
26. Ondro Mihályi @OmniFishEE FALSE: Each VT has its own values But be careful: • lot of VT → lot of thread locals → more memory • Threads not in pools, not reused – Thread locals not suitable for caching Myth 1: Thread locals don’t work
27. Ondro Mihályi @OmniFishEE FALSE. True only in rare cases Blocking operation in synchronized block Native method called in synch. block All still works, but less performant Better use ReentrantLock instead of synchronized Myth 2: Need to avoid "synchronized"
28. Ondro Mihályi @OmniFishEE MOSTLY TRUE All works fine, but things to consider: • Degraded performance in some cases – blocking in synchronized, cache in thread-locals • CPU heavy tasks not parallelized evenly RECOMMENDATION: Test performance Myth 3: Just use virtual threads and your done
29. Ondro Mihályi @OmniFishEE PARTIALLY TRUE • VTs are performant and scalable • But Reactive Programming is also about: – Responsivenes (to events, failures) – Back-pressure – Fine-grained control of thread utilization Myth 4: Will kill reactive programming
30. Ondro Mihályi @OmniFishEE Potential future is in mixing them • Reactive pipelines not necessary for reactive programming • New libraries may emerge for VT – Combine simplicity of VTs with advantages of reactive design Virtual threads + reactive programming
31. Ondro Mihályi @OmniFishEE Questions
32. Ondro Mihályi @OmniFishEE Thank you! Jakarta EE Consultancy (migrations, tuning) Jakarta EE Application Development GlassFish Server Support Jakarta EE Training omnifish.ee OmniFish - Jakarta EE Consulting & Support