Project Loom: Broadening Concurrency

Project Loom has landed in Java 19 as a preview feature. It promises to simplify concurrency the way garbage collection simplifies memory management and improves applications' scalability.

It's about time to closely examine the value proposition and how it is implemented to understand what Loom can provide for your application. We will dive into technical details to understand concurrency, look at examples of how they behave, and how we can debug virtual thread code. You will learn what to watch out for and how project Loom can help you to write simple, concurrent code to get the most out of your runtime environment.

Transcript

1. Project Loom
   Broadening Concurrency
   

   View Slide

2. Mark Paluch
   Spring Data Project Lead @ VMware
   R2DBC Spec Lead
   Lettuce Redis Driver Maintainer
   Paramedic & SAR Dogs
   

   View Slide

3. Sept 2020, https://paluch.biz/blog/182-experimenting-with-project-loom-eap-and-spring-webmvc.html
   „We’ve been there, already“
   

   View Slide

4. Tomcat + Virtual Threads Tomcat + Kernel Threads
   

   View Slide

5. What is Project Loom
   • Initiative to introduce virtual threads to the Java Platform. Virtual threads are
   lightweight threads that dramatically reduce the effort of writing, maintaining,
   and observing high-throughput concurrent applications.
   

   View Slide

6. What is Project Loom
   • Initiative to introduce Introduction of virtual threads and concurrency API to
   the Java Platform. Virtual threads are lightweight threads that eliminate I/O
   wait. dramatically reduce the effort of writing, maintaining, and observing
   high-throughput concurrent applications.
   

   View Slide

7. What Project Loom actually is
   • Introduction of virtual threads and concurrency API to the Java Platform.
   Virtual threads are lightweight threads that eliminate I/O wait.
   

   View Slide

8. Dramatically reduce the effort of writing,
   maintaining, and observing concurrent applications
   • True if you run on a Virtual Thread
   
   • Future.get(), Thread.sleep(), Lock.lock() are good friends again
   
   • No need for future chaining, using complex reactive or asynchronous API
   
   • Not so, if you accidentally run on a Kernel Thread
   

   View Slide

9. Overview
   

   View Slide

10. Platform Thread
    • Renamed from Kernel Thread
    
    • new Thread(…)
    

    View Slide

11. Virtual Thread
    • Package-private subtype of java.lang.Thread
    
    • Provides customized behavior
    
    • Subject to be executed on carrier Thread
    

    View Slide

12. Carrier Threads
    • Separate Fork/Join Pool of Platform Threads
    
    • Default size: Runtime.availableProcessors()
    
    • Configurable through system properties
    (jdk.virtualThreadScheduler.parallelism, maxPoolSize,
    minRunnable)
    

    View Slide

13. Virtual Threads API
    • Extended Thread API
    
    Thread.ofVirtual().start(() -> {…});
    Executors.newVirtualThreadPerTaskExecutor()
    Thread.currentThread().isVirtual()
    • Implemented almost entirely in Java
    

    View Slide

14. try(var e = Executors.newVirtualThreadPerTaskExecutor()){
    for (int j = 0; j < 100; j++) {
    e.execute(() -> Thread.sleep(100));
    }
    } // implicit wait
    

    View Slide

15. View Slide

16. Parking & Unparking
    • On each blocking operation
    
    • If blocked, capture stack and store it
    
    • Carrier thread free to perform other work (unpark next virtual thread)
    

    View Slide

17. View Slide

18. Blocking in synchronized code
    • Leads to Carrier Thread Pinning
    
    • Carrier Thread bound to virtual thread until unblock
    
    • synchronized is a JVM behavior, cannot be controlled from Java code
    
    • I/O, Locks, Future.get() in synchronized leads to Thread Pinning
    

    View Slide

19. Thread Type Selection
    • I/O-heavy, shared-resources with Java Locks, Concurrency Utils
    
    • Virtual Threads
    
    • Computation-heavy, synchronized workloads
    
    • Platform Threads
    

    View Slide

20. Observability, Tooling, Trouble Shooting
    • Java Flight Recorder Events
    
    • Virtual Thread start/end, pinned, submission failed
    
    • Debugging: IntelliJ announced tooling support for later this year
    
    • Pinned thread monitoring
    
    • -Djdk.tracePinnedThreads=full|short
    

    View Slide

21. Demo
    

    View Slide

22. Virtual Thread Usage
    

    View Slide

23. Usage of Virtual Thread API
    • In every piece that could block or wait in Java
    
    • Thread.sleep
    
    • Locks, ConcurrentHashMap
    
    • I/O (Socket)
    
    • Rewrite of several components as pre-requisite for Loom to avoid thread pinning
    

    View Slide

24. Usage Pattern
    • Kernel Threads („Platform Threads“)
    
    • Expensive
    
    • Thread Pooling
    
    • Virtual Threads
    
    • Cheap
    
    • New Virtual Thread per Runnable
    

    View Slide

25. Benchmarks
    

    View Slide

26. JMH Benchmarks
    Benchmark Mode Cnt Score Error Units
    VirtualThreadsBenchmark.createVirtualThreads thrpt 5 14488555,637 ± 1127513,241 ops/s
    VirtualThreadsBenchmark.runAndAwait thrpt 5 455494,235 ± 22033,286 ops/s
    PlatformThreadsBenchmark.createPlatformThreads thrpt 5 19094,736 ± 677,739 ops/s
    PlatformThreadsBenchmark.runAndAwait thrpt 5 15807,802 ± 7337,906 ops/s
    

    View Slide

27. JMH Benchmarks
    Benchmark Mode Cnt Score Error Units
    VirtualThreadsBenchmark.createVirtualThreads thrpt 5 14488555,637 ± 1127513,241 ops/s
    VirtualThreadsBenchmark.runAndAwait thrpt 5 455494,235 ± 22033,286 ops/s
    PlatformThreadsBenchmark.createPlatformThreads thrpt 5 19094,736 ± 677,739 ops/s
    PlatformThreadsBenchmark.runAndAwait thrpt 5 15807,802 ± 7337,906 ops/s
    ThreadPoolExecutorBenchmark.runAndAwait thrpt 5 461335,862 ± 54427,010 ops/s
    

    View Slide

28. Conclusion
    

    View Slide

29. What does that mean?
    • If the CPU availability is the limit: You cannot increase concurrency
    
    • If the Runnable wait is the limit: You can increase concurrency
    

    View Slide

30. Loom Overhead
    • Hard to say. What’s the typical application?
    

    View Slide

31. Scalability Boundaries
    • I/O in synchronized Code
    
    • Connection Pool Sizes
    
    • Databases
    
    • HTTP/JMS/… Connection Pools
    
    • File handles
    
    • number of open files
    

    View Slide

32. Pitfalls
    • System.out.println(…)
    • Attempts to obtain a lock on the I/O
    
    • Measurement and benchmark correctness („What do you measure?“)
    
    • Visible and invisible Thread Pinning
    

    View Slide

33. Client/Server Workloads
    • Workload to be served by virtual thread (carrier threads must be available)
    
    • System must have sufficient resources to serve the request (e.g. connection
    pools). Also a constraint for platform thread usage.
    
    • Request to server via I/O
    
    • During wait, virtual thread is parked
    
    • Client must not pin kernel thread (see carrier thread availability)
    

    View Slide

34. Computation Workloads
    • Incoming request/event to be served by virtual thread (carrier threads must be
    available)
    
    • System performs a computation based on in-memory resources
    
    • Response is returned
    

    View Slide

35. Call to Action
    • Investigate: What is limiting your application scalability?
    
    • Investigate: What workloads do you run?
    
    • Experiment: Run your application on Virtual Threads and benchmark it!
    
    • Observe: What libraries lead to thread pinning?
    

    View Slide

36. Maybe.
    „Will Loom help with concurrency and
    scalability in my app?“
    

    View Slide

37. Thank You.
    github.com/mp911de
    @mp911de
    paluch.biz
    

    View Slide