Demystifying Java Virtual Threads - Lessons learned from adding them to GlassFish

Slide 1

Slide 1 text

Ondro Mihályi @OmniFishEE Demystifying Java Virtual Threads Lessons learned from adding them to GlassFish

Slide 2

Slide 2 text

Ondro Mihályi @OmniFishEE Who am I? Java Champion Czech JUG lead (CZJUG) Jakarta EE specifications committer Java developer, trainer & consultant

Slide 3

Slide 3 text

Ondro Mihályi @OmniFishEE Virtual threads are the next best thing in Java after Duke

Slide 4

Slide 4 text

Ondro Mihályi @OmniFishEE Virtual threads are the next best thing in Java after … Java Modules

Slide 5

Slide 5 text

Ondro Mihályi @OmniFishEE Virtual threads are the next best thing in Java after … Java Modules Java Lambdas!

Slide 6

Slide 6 text

Ondro Mihályi @OmniFishEE OK, Virtual Threads are cool. But why exactly?

Slide 7

Slide 7 text

Ondro Mihályi @OmniFishEE Platform threads First, let’s talk about why platform threads are not so cool

Slide 8

Slide 8 text

Ondro Mihályi @OmniFishEE What’s going on with platform threads Thread thread = new Thread(() -> { out.println("Hello from a thread"); }); thread.start(); // triggers a system call

Slide 9

Slide 9 text

Ondro Mihályi @OmniFishEE Issues with platform threads ● Platform threads match OS threads ● System calls required → slow: – Slow to create a new thread – Slow to switch thread context on a CPU core ● Threads reserve off-heap memory

Slide 10

Slide 10 text

Ondro Mihályi @OmniFishEE Can we have unlimited nuber of threads? While (true) { Thread.ofPlatform().start(() -> { sleep(ofMinutes(10)); }); }

Slide 11

Slide 11 text

Ondro Mihályi @OmniFishEE Where’s the limit?

Slide 12

Slide 12 text

Ondro Mihályi @OmniFishEE Memory reserved by platform threads Thread 9 Thread 8 Thread 7 Thread 6 Thread 5 Thread 4 Thread 3 Thread 2 Thread 1 Used memory Reserved memory 1 MB

Slide 13

Slide 13 text

Ondro Mihályi @OmniFishEE Number of Platform threads is limited ● By default, threads reserve 1MB ● 1,000 threads → 1GB ● 32,648 threads → > 32 GB

Slide 14

Slide 14 text

Ondro Mihályi @OmniFishEE Traditional answer – thread pools ● Reuse same threads for multiple tasks – Avoid expensive creation ● And limit their amount – Avoid getting out of memory

Slide 15

Slide 15 text

Ondro Mihályi @OmniFishEE Thread pools - lifecycle

Slide 16

Slide 16 text

Ondro Mihályi @OmniFishEE Issues with Thread pools Thread-locals may leak memory (GCed when removed or thread stops) Blocked threads block all tasks Thread-locals may leak state (when static and not cleared) Complicated tuning

Slide 17

Slide 17 text

Ondro Mihályi @OmniFishEE Anything better than thread pools? Thread-locals may leak memory (GCed when removed or thread stops) Blocked threads block all tasks Thread-locals may leak state (when static and not cleared) Complicated tuning

Slide 18

Slide 18 text

Ondro Mihályi @OmniFishEE Properties of Virtual Threads ● Fast to create thread and switch context (no system call) ● Millions of threads can run in parallel ● No need for thread pools → simple ● Stack saved in heap

Slide 19

Slide 19 text

Ondro Mihályi @OmniFishEE Virtual threads are fast Fast context switch ● No system calls ● More CPU time for work When blocked, create new threads ● No system calls ● No reserved RAM 15% Performance increase (22216.59 to 25895.12 req/sec)

Slide 20

Slide 20 text

Ondro Mihályi @OmniFishEE Unlimited nuber of virtual threads? While (true) { Thread.ofVirtual().start(() -> { sleep(ofMinutes(10)); }); }

Slide 21

Slide 21 text

Ondro Mihályi @OmniFishEE Plain Java objects How Virtual Threads work Store stacks in heap Executed on Platform Threads Yield to JVM when blocked. Cannot be paused if don’t yield

Slide 22

Slide 22 text

Ondro Mihályi @OmniFishEE Matching to carrier platform threads

Slide 23

Slide 23 text

Ondro Mihályi @OmniFishEE Execute HTTP requests on virtual threads ● Enable custom executor service – A new thread for each request – Maximum parallel threads limited ➔ better throughput ➔ less memory needed Hands On: Virtual threads in GlassFish

Slide 24

Slide 24 text

Ondro Mihályi @OmniFishEE Background tasks on virtual threads ● Coming in GlassFish 8 (Jakarta EE 11) @ManagedExecutorDefinition(virtual = true) executor.submit( () -> "Running in VT"); Hands On: Virtual threads in GlassFish

Slide 25

Slide 25 text

Ondro Mihályi @OmniFishEE ● Thread locals don’t work ● Need to avoid "synchronized" ● Just use virtual threads and your done ● Will kill reactive programming Myths about Virtual Threads

Slide 26

Slide 26 text

Ondro Mihályi @OmniFishEE FALSE: Each VT has its own values But be careful: ● lot of VT → lot of thread locals → more memory ● Threads not in pools, not reused – Thread locals not suitable for caching Myth 1: Thread locals don’t work

Slide 27

Slide 27 text

Ondro Mihályi @OmniFishEE FALSE. True only in rare cases Blocking operation in synchronized block Native method called in synch. block All still works, but less performant Better use ReentrantLock instead of synchronized Myth 2: Need to avoid "synchronized"

Slide 28

Slide 28 text

Ondro Mihályi @OmniFishEE MOSTLY TRUE All works fine, but things to consider: ● Degraded performance in some cases – blocking in synchronized, cache in thread-locals ● CPU heavy tasks not parallelized evenly RECOMMENDATION: Test performance Myth 3: Just use virtual threads and your done

Slide 29

Slide 29 text

Ondro Mihályi @OmniFishEE PARTIALLY TRUE ● VTs are performant and scalable ● But Reactive Programming is also about: – Responsivenes (to events, failures) – Back-pressure – Fine-grained control of thread utilization Myth 4: Will kill reactive programming

Slide 30

Slide 30 text

Ondro Mihályi @OmniFishEE Potential future is in mixing them ● Reactive pipelines not necessary for reactive programming ● New libraries may emerge for VT – Combine simplicity of VTs with advantages of reactive design Virtual threads + reactive programming

Slide 31

Slide 31 text

Ondro Mihályi @OmniFishEE Questions

Slide 32

Slide 32 text

Ondro Mihályi @OmniFishEE Thank you! Jakarta EE Consultancy (migrations, tuning) Jakarta EE Application Development GlassFish Server Support Jakarta EE Training omnifish.ee OmniFish - Jakarta EE Consulting & Support