Upgrade to Pro — share decks privately, control downloads, hide ads and more …

What the heck is Project Loom?

What the heck is Project Loom?

You may have heard the words Project Loom, Fibers, Structured concurrency, and Virtual threads from Java enthusiasts. But what exactly are these, and what is the current state of concurrency in Java? Can Java keep up with languages like Go and Rust when it comes to fearless concurrency? Let's be honest, concurrency in java is not among the easiest to master, and thread safety is even more difficult to get right. Can Project loom help to make it easier and less idiot-proof? In this session, I'll do a deep dive into Project loom and explain the different features in the work, look at the current state, and compare it with what we have today with some samples. We will also look into it from the aspect of what a normal Java developer can expect/benefit from it. So come join me for an exciting ride.

Deepu K Sasidharan

December 16, 2022
Tweet

More Decks by Deepu K Sasidharan

Other Decks in Programming

Transcript

  1. @deepu105
    @oktaDev
    What the heck is
    Project Loom?
    Deepu K Sasidharan
    @deepu105 | deepu.tech

    View Slide

  2. @deepu105
    @oktaDev
    Hi, I’m Deepu K Sasidharan
    ➔ JHipster co-lead developer
    ➔ Java Champion
    ➔ Creator of KDash, JDL Studio
    ➔ Developer Advocate @ Auth0 by Okta
    ➔ OSS aficionado, polyglot dev, author, speaker
    @[email protected]
    deepu.tech
    @deepu105
    deepu05

    View Slide

  3. @deepu105
    @oktaDev
    Concurrency in Java
    https://deepu.tech/concurrency-in-modern-languages-java/

    View Slide

  4. @deepu105
    @oktaDev
    JDK Evolution
    JDK
    Green threads
    1.0

    View Slide

  5. @deepu105
    @oktaDev
    JDK Evolution
    Platform threads
    1.1
    JDK
    Green threads
    1.0

    View Slide

  6. @deepu105
    @oktaDev
    JDK Evolution
    Platform threads
    1.1
    JDK
    Green threads
    1.0
    Executor, mutex,
    concurrent
    collections,
    semaphore, barrier,
    latches and blocking
    queues
    1.5
    ForkJoinPool
    1.7

    View Slide

  7. @deepu105
    @oktaDev
    JDK Evolution
    Platform threads
    1.1
    JDK
    Green threads
    1.0
    Executor, mutex,
    concurrent
    collections,
    semaphore, barrier,
    latches and blocking
    queues
    1.5
    ForkJoinPool
    1.7
    Streams,
    CompletableFuture
    and
    CompletionException
    1.8 (8)

    View Slide

  8. @deepu105
    @oktaDev
    JDK Evolution
    Platform threads
    1.1
    JDK
    Green threads
    1.0
    Executor, mutex,
    concurrent
    collections,
    semaphore, barrier,
    latches and blocking
    queues
    1.5
    ForkJoinPool
    1.7
    Streams,
    CompletableFuture
    and
    CompletionException
    1.8 (8)
    Virtual threads and
    Structured
    concurrency
    19

    View Slide

  9. @deepu105
    @oktaDev
    Platform Threads
    ● Platform threads == OS threads
    ● Platforms threads are mapped 1:1 to OS threads

    View Slide

  10. @deepu105
    @oktaDev
    Thread-per-request model
    Request 1 Platform Thread 1 OS Thread 1
    Request 2 Platform Thread 2 OS Thread 2
    Request N Platform Thread N OS Thread N

    View Slide

  11. @deepu105
    @oktaDev
    Thread-per-request model
    Little’s law
    λ = L/W
    λ = Throughput (average rate of requests)
    L = Average concurrency (number of requests concurrently processed by the server)
    W = Latency (average duration of processing each request)
    Request 1 Platform Thread 1 OS Thread 1
    Request 2 Platform Thread 2 OS Thread 2
    Request N Platform Thread N OS Thread N

    View Slide

  12. @deepu105
    @oktaDev
    Parallel processing
    ● Should handle data races and data corruption
    ● Thread synchronization might be needed
    ● Thread leaks and cancellation delays
    ● Fragile
    ● A lot of responsibility on the developer
    Task
    Subtask 1
    Subtask 2
    Subtask N
    Platform Thread 1 OS Thread 1
    Platform Thread 2 OS Thread 2
    Platform Thread N OS Thread N

    View Slide

  13. @deepu105
    @oktaDev
    Project Loom
    https://developer.okta.com/blog/2022/08/26/state-of-java-project-loom

    View Slide

  14. @deepu105
    @oktaDev
    Project Loom
    Project Loom aims to drastically reduce the effort of writing, maintaining, and
    observing high-throughput concurrent applications that make the best use of
    available hardware.
    — Ron Pressler (Tech lead, Project Loom)

    View Slide

  15. @deepu105
    @oktaDev
    Virtual threads
    a.k.a User mode threads
    a.k.a Coroutines

    View Slide

  16. @deepu105
    @oktaDev
    Green threads mapping
    Green thread 1
    Green thread 2
    Green thread N
    Green Threads (M:1)
    OS Thread 1

    View Slide

  17. @deepu105
    @oktaDev
    Platform threads mapping
    Green thread 1
    Green thread 2
    Green thread N
    Green Threads (M:1)
    Platform Thread 1 OS Thread 1
    Platform Thread 2 OS Thread 2
    Platform Thread N OS Thread N
    OS Thread 1
    Platform Threads (1:1)

    View Slide

  18. @deepu105
    @oktaDev
    Virtual threads mapping
    Green thread 1
    Green thread 2
    Green thread N
    Green Threads (M:1)
    Platform Thread 1 OS Thread 1
    Platform Thread 2 OS Thread 2
    Platform Thread N OS Thread N
    OS Thread 1
    Platform Threads (1:1)
    Virtual thread 1
    Virtual thread 2
    Virtual thread N
    Virtual Threads (M:N)
    OS Thread 1
    OS Thread N

    View Slide

  19. @deepu105
    @oktaDev
    Goroutines
    go func() {
    println("Hello, Goroutines!")
    }()

    View Slide

  20. @deepu105
    @oktaDev
    Kotlin coroutines
    runBlocking {
    launch {
    println("Hello, Kotlin coroutines!")
    }
    }

    View Slide

  21. @deepu105
    @oktaDev
    Java virtual thread
    Thread.startVirtualThread(() -> {
    System.out.println("Hello, Project Loom!");
    });

    View Slide

  22. @deepu105
    @oktaDev
    Virtual thread features
    ● It is a Thread in code, runtime, debugger, and profiler
    ● It’s a Java entity and not a wrapper around an OS thread
    ● Creating and blocking them are cheap operations
    ● They should not be pooled
    ● Virtual threads use a work-stealing ForkJoinPool scheduler
    ● Pluggable schedulers can be used for asynchronous programming
    ● A virtual thread will have its own stack memory
    ● The virtual threads API is very similar to platform threads and hence easier to adopt/migrate

    View Slide

  23. @deepu105
    @oktaDev
    Total number of platform threads
    var counter = new AtomicInteger();
    while (true) {
    new Thread(() -> {
    int count = counter.incrementAndGet();
    System.out.println("Thread count = " + count);
    LockSupport.park();
    }).start();
    }

    View Slide

  24. @deepu105
    @oktaDev
    Total number of virtual threads
    var counter = new AtomicInteger();
    while (true) {
    Thread.startVirtualThread(() -> {
    int count = counter.incrementAndGet();
    System.out.println("Thread count = " + count);
    LockSupport.park();
    });
    }

    View Slide

  25. @deepu105
    @oktaDev
    Task throughput for platform threads
    try (var executor = Executors.newThreadPerTaskExecutor(Executors.defaultThreadFactory())) {
    IntStream.range(0, 100_000).forEach(i -> executor.submit(() -> {
    Thread.sleep(Duration.ofSeconds(1));
    System.out.println(i);
    return i;
    }));
    }
    # 'newThreadPerTaskExecutor' with 'defaultThreadFactory'
    0:18.77 real, 18.15 s user, 7.19 s sys, 135% 3891pu, 0 amem, 743584 mmem
    # 'newCachedThreadPool' with 'defaultThreadFactory'
    0:11.52 real, 13.21 s user, 4.91 s sys, 157% 6019pu, 0 amem, 2215972 mmem

    View Slide

  26. @deepu105
    @oktaDev
    Task throughput for virtual threads
    try (var executor = Executors.newVirtualThreadPerTaskExecutor()) {
    IntStream.range(0, 100_000).forEach(i -> executor.submit(() -> {
    Thread.sleep(Duration.ofSeconds(1));
    System.out.println(i);
    return i;
    }));
    }
    0:02.62 real, 6.83 s user, 1.46 s sys, 316% 14840pu, 0 amem, 350268 mmem

    View Slide

  27. @deepu105
    @oktaDev
    JMH Benchmarks
    # Throughput (more is better)
    Benchmark Mode Cnt Score Error Units
    LoomBenchmark.platformThreadPerTask thrpt 5 0.362 ± 0.079 ops/s
    LoomBenchmark.platformThreadPool thrpt 5 0.528 ± 0.067 ops/s
    LoomBenchmark.virtualThreadPerTask thrpt 5 1.843 ± 0.093 ops/s
    # Average time (less is better)
    Benchmark Mode Cnt Score Error Units
    LoomBenchmark.platformThreadPerTask avgt 5 5.600 ± 0.768 s/op
    LoomBenchmark.platformThreadPool avgt 5 3.887 ± 0.717 s/op
    LoomBenchmark.virtualThreadPerTask avgt 5 1.098 ± 0.020 s/op
    https://github.com/deepu105/java-loom-benchmarks

    View Slide

  28. @deepu105
    @oktaDev
    More benchmarks
    ● An interesting benchmark using ApacheBench on GitHub by Elliot Barlas
    ● A benchmark using Akka actors on Medium by Alexander Zakusylo
    ● JMH benchmarks for I/O and non-I/O tasks on GitHub by Colin Cachia

    View Slide

  29. @deepu105
    @oktaDev
    Structured concurrency

    View Slide

  30. @deepu105
    @oktaDev
    Without structured concurrency
    void handleOrder() throws ExecutionException, InterruptedException {
    try (var esvc = new ScheduledThreadPoolExecutor(8)) {
    Future inventory = esvc.submit(() -> updateInventory());
    Future order = esvc.submit(() -> updateOrder());
    int theInventory = inventory.get(); // Join updateInventory
    int theOrder = order.get(); // Join updateOrder
    System.out.println("Inventory " + theInventory + " updated for order " + theOrder);
    }
    }

    View Slide

  31. @deepu105
    @oktaDev
    Without structured concurrency
    void handleOrder() throws ExecutionException, InterruptedException {
    try (var esvc = new ScheduledThreadPoolExecutor(8)) {
    Future inventory = esvc.submit(() -> updateInventory()); // failed
    Future order = esvc.submit(() -> updateOrder()); // runs in background
    int theInventory = inventory.get(); // Join updateInventory // fails
    int theOrder = order.get(); // Join updateOrder // unreachable
    System.out.println("Inventory " + theInventory + " updated for order " + theOrder);
    }
    }

    View Slide

  32. @deepu105
    @oktaDev
    Without structured concurrency
    void handleOrder() throws ExecutionException, InterruptedException {
    try (var esvc = new ScheduledThreadPoolExecutor(8)) {
    Future inventory = esvc.submit(() -> updateInventory()); // expensive task
    Future order = esvc.submit(() -> updateOrder()); // failed
    int theInventory = inventory.get(); // Join updateInventory // task blocked
    int theOrder = order.get(); // Join updateOrder // will fail
    System.out.println("Inventory " + theInventory + " updated for order " + theOrder);
    }
    }

    View Slide

  33. @deepu105
    @oktaDev
    Without structured concurrency
    void handleOrder() throws ExecutionException, InterruptedException { // interrupted
    try (var esvc = new ScheduledThreadPoolExecutor(8)) {
    Future inventory = esvc.submit(() -> updateInventory()); // runs in bg
    Future order = esvc.submit(() -> updateOrder()); // runs in bg
    int theInventory = inventory.get(); // Join updateInventory
    int theOrder = order.get(); // Join updateOrder
    System.out.println("Inventory " + theInventory + " updated for order " + theOrder);
    }
    }

    View Slide

  34. @deepu105
    @oktaDev
    Structured concurrency
    void handleOrder() throws ExecutionException, InterruptedException {
    try (var scope = new StructuredTaskScope.ShutdownOnFailure()) {
    Future inventory = scope.fork(() -> updateInventory());
    Future order = scope.fork(() -> updateOrder());
    scope.join(); // Join both forks
    scope.throwIfFailed(); // ... and propagate errors
    // Here, both forks have succeeded, so compose their results
    System.out.println("Inventory " + inventory.resultNow() + " updated for order " +
    order.resultNow());
    }
    }

    View Slide

  35. @deepu105
    @oktaDev
    Structured concurrency
    void handleOrder() throws ExecutionException, InterruptedException {
    try (var scope = new StructuredTaskScope.ShutdownOnFailure()) {
    Future inventory = scope.fork(() -> updateInventory()); // failed
    Future order = scope.fork(() -> updateOrder()); // cancelled
    scope.join(); // Join both forks
    scope.throwIfFailed(); // ... and propagate errors
    // Here, both forks have succeeded, so compose their results
    System.out.println("Inventory " + inventory.resultNow() + " updated for order " +
    order.resultNow());
    }
    }

    View Slide

  36. @deepu105
    @oktaDev
    State of Project Loom

    View Slide

  37. @deepu105
    @oktaDev
    Impact for regular developers
    ● No breaking changes
    ● Very low API surface and hence easy to adopt/migrate
    ● Rely on underlying libraries to switch to virtual threads
    ● Debugging virtual threads would need some getting used to
    ● Can easily switch to virtual threads from thread pools
    ● Structured concurrency could help to eliminate a lot of failsafe code
    ● At the moment need to use preview and incubator modules
    ● Some unlearning to do (no pooling, no reusing, no shared pool executors)
    ● Proliferation of virtual threads in simple use cases.

    View Slide

  38. @deepu105
    @oktaDev
    Impact for libraries
    ● Performance and throughput increases
    ● Early adoption
    ● Simpler codebase
    ● Server software like tomcat, Undertow and Jetty will see improvements
    ● Frameworks like Spring, Micronaut and Quarkus will see improvements
    ● Libraries like RxJava and Akka might benefit from structured concurrency
    ● Asynchronous and reactive programming will still be around but in many use
    cases virtual threads could replace them and give same benefits with less
    complexity

    View Slide

  39. @deepu105
    @oktaDev
    Early adoption
    ● GraalVM
    ○ Support added (https://github.com/oracle/graal/pull/4802)
    ● Quarkus
    ○ Support added (https://github.com/quarkusio/quarkus/pull/24942)
    ● Micronaut
    ○ Being discussed
    (https://github.com/micronaut-projects/micronaut-core/issues/7724)
    ● Spring
    ○ https://spring.io/blog/2022/10/11/embracing-virtual-threads

    View Slide

  40. @deepu105
    @oktaDev
    Caveats

    View Slide

  41. @deepu105
    @oktaDev
    Resources
    ● https://www.infoq.com/articles/java-virtual-threads/
    ● https://inside.java/2020/08/07/loom-performance/
    ● http://cr.openjdk.java.net/~rpressler/loom/loom/sol1_part1.html
    ● https://foojay.io/today/thinking-about-massive-throughput-meet-virtual-th
    reads/

    View Slide

  42. @deepu105
    @oktaDev
    Get the Slides

    View Slide

  43. @deepu105
    @oktaDev
    Thank You
    Deepu K Sasidharan
    @deepu105 | deepu.tech
    https://deepu.tech/tags#java https://developer.auth0.com

    View Slide