Concurrency Made Easy: Structured Concurrency & Scoped Values in Action

The coordination of concurrent, potentially blocking subtasks quickly reaches its limits with classic approaches such as CompletableFuture and ExecutorService. For example, canceling subtasks, whether after error situations or if we only need the result of a subtask, quickly leads to confusing interdependencies between business logic and state handling. We then speak of “unstructured concurrency”.

With StructuredTaskScope, we now have an API for “structured concurrency” that allows us to start and end subtasks together, merge the results, and, if necessary, cancel subtasks cleanly.

Using practical examples, I will show you that only a few easy-to-understand lines of code are needed for most use cases thanks to predefined strategies such as “all subtasks must be successful” and “abort if one subtask is successful.”

You can also handle individual requirements cleanly with StructuredTaskScope – I’ll show you an example of how to implement a strategy that waits for a certain number of partial results and then returns the best result.

Transcript

1. Concurrency Made Easy: Structured Concurrency & Scoped Values in Action Sven Woltmann
2. Copyright © 2023, HappyCoders.eu What is Structured Concurrency?
3. Copyright © 2023, HappyCoders.eu What is Unstructured Concurrency?
4. Create invoice Get order Get customer Get invoice template Generate invoice Copyright © 2023, HappyCoders.eu Online Store: Invoice Creation
5. CompletableFuture<Invoice> createInvoiceAsync(int orderId, int customerId, String language) { CompletableFuture<Order> orderFuture = orderService.getOrderAsync(orderId); CompletableFuture<Customer> customerFuture = customerService.getCustomerAsync(customerId); CompletableFuture<InvoiceTemplate> templateFuture = invoiceTemplateService.getTemplateAsync(language); return CompletableFuture.allOf(orderFuture, customerFuture, templateFuture) .thenCompose( _ -> { Order order = orderFuture.resultNow(); Customer customer = customerFuture.resultNow(); InvoiceTemplate template = templateFuture.resultNow(); Invoice invoice = Invoice.generate(order, customer, template); return CompletableFuture.completedFuture(invoice); }); }
6. CompletableFuture<Invoice> createInvoiceAsync(int orderId, int customerId, String language) { CompletableFuture<Order> orderFuture = orderService.getOrderAsync(orderId); CompletableFuture<Customer> customerFuture = customerService.getCustomerAsync(customerId); CompletableFuture<InvoiceTemplate> templateFuture = invoiceTemplateService.getTemplateAsync(language); return CompletableFuture.allOf( orderFuture.exceptionally( e -> { customerFuture.cancel(true); templateFuture.cancel(true); if (e instanceof CancellationException) { return null; } else { throw new CompletionException(e); } }), customerFuture.exceptionally( e -> { orderFuture.cancel(true); templateFuture.cancel(true); if (e instanceof CancellationException) { return null; } else { throw new CompletionException(e); } }), templateFuture.exceptionally( e -> { customerFuture.cancel(true); orderFuture.cancel(true); if (e instanceof CancellationException) { return null; } else { throw new CompletionException(e); } })) .thenCompose( _ -> { Order order = orderFuture.resultNow(); Customer customer = customerFuture.resultNow(); InvoiceTemplate template = templateFuture.resultNow(); Invoice invoice = Invoice.generate(order, customer, template); return CompletableFuture.completedFuture(invoice); }); }
7. Copyright © 2023, HappyCoders.eu And with a “VirtualThreadPerTaskExecutor”?
8. Invoice createInvoice(int orderId, int customerId, String language) throws InterruptedException, ExecutionException { try (ExecutorService executor = Executors.newVirtualThreadPerTaskExecutor()) { Future<Order> orderFuture = executor.submit(() -> orderService.getOrder(orderId)); Future<Customer> customerFuture = executor.submit(() -> customerService.getCustomer(customerId)); Future<InvoiceTemplate> invoiceTemplateFuture = executor.submit(() -> invoiceTemplateService.getTemplate(language)); Order order = orderFuture.get(); Customer customer = customerFuture.get(); InvoiceTemplate invoiceTemplate = invoiceTemplateFuture.get(); return Invoice.generate(order, customer, invoiceTemplate); } }
9. Invoice createInvoice(int orderId, int customerId, String language) throws Throwable { Future<Order> orderFuture; Future<Customer> customerFuture; Future<InvoiceTemplate> invoiceTemplateFuture; try (ExecutorService executor = Executors.newVirtualThreadPerTaskExecutor()) { orderFuture = executor.submit(() -> { try { return orderService.getOrder(orderId); } catch (Exception e) { executor.shutdownNow(); throw e; }}); customerFuture = executor.submit(() -> { try { return customerService.getCustomer(customerId); } catch (Exception e) { executor.shutdownNow(); throw e; }}); invoiceTemplateFuture = executor.submit(() -> { try { return invoiceTemplateService.getTemplate(language); } catch (Exception e) { executor.shutdownNow(); throw e; }}); } if (orderFuture.state() == SUCCESS && customerFuture.state() == SUCCESS && invoiceTemplateFuture.state() == SUCCESS) { Order order = orderFuture.get(); Customer customer = customerFuture.get(); InvoiceTemplate invoiceTemplate = invoiceTemplateFuture.get(); return Invoice.generate(order, customer, invoiceTemplate); } else if (orderFuture.state() == FAILED && !(orderFuture.exceptionNow() instanceof InterruptedException)) { throw orderFuture.exceptionNow(); } else if (customerFuture.state() == FAILED && !(customerFuture.exceptionNow() instanceof InterruptedException)) { throw customerFuture.exceptionNow(); } else if (invoiceTemplateFuture.state() == FAILED) { throw invoiceTemplateFuture.exceptionNow(); } else if (orderFuture.state() == FAILED) { throw orderFuture.exceptionNow(); } else if (customerFuture.state() == FAILED) { throw customerFuture.exceptionNow(); } else { throw new IllegalStateException(); } }
10. API endpoint called Get product data Get availability in warehouse 1 Get availability in warehouse 2 Get availability in warehouse 3 Get availability from seller 1 Get availability from seller 2 Get availability from seller 3 Get delivery time from supplier 1 Get delivery time from supplier 2 Get delivery time from supplier 3 Available? Return product data, delivery time, sellers Yes No Copyright © 2023, HappyCoders.eu
11. Get order Get customer Get invoice template Copyright © 2023, HappyCoders.eu
12. Live Coding: StructuredTaskScope Copyright © 2023, HappyCoders.eu
13. Customer enters address Verify via address verification system A Verify via address verification system B Verify via address verification system C Copyright © 2023, HappyCoders.eu … Online Store: Delivery Address Verification
14. Live Coding Delivery Address Verification with StructuredTaskService Copyright © 2023, HappyCoders.eu
15. Custom StructuredTaskScope Copyright © 2023, HappyCoders.eu
16. API endpoint called Get product data Get delivery time from supplier Available? Return product data, delivery time Get availability in warehouse Yes No Copyright © 2023, HappyCoders.eu Online Store: Supplier Delivery Time Check
17. Get delivery time from supplier A Get delivery time from supplier B Get delivery time from supplier C Available? Return product data, delivery time Yes No Copyright © 2023, HappyCoders.eu API endpoint called Get product data Get availability in warehouse Online Store: Supplier Delivery Time Check
18. Live Coding Supplier Delivery Time Check with StructuredTaskService Copyright © 2023, HappyCoders.eu
19. Nested StructuredTaskScope Copyright © 2023, HappyCoders.eu
20. Copyright © 2023, HappyCoders.eu
21. Live Coding Nested StructuredTaskScope Copyright © 2023, HappyCoders.eu
22. Advantages of Structured Concurrency Copyright © 2023, HappyCoders.eu
23. Unstructured Copyright © 2023, HappyCoders.eu
24. Unstructured Copyright © 2023, HappyCoders.eu Structured
25. Unstructured Copyright © 2023, HappyCoders.eu Structured
26. Copyright © 2023, HappyCoders.eu Advantages of StructuredTaskScope - Clear entry and exit points - Errors in subtasks can be propagated to parent scope (or ignored) - All threads are terminated at the end of the scope • Language construct for Structured Concurrency • Easy to write, read, and maintain • Policies: ShutdownOnSuccess, ShutdownOnFailure, custom policies - Suceeded/failed subtasks can cancel others
27. Copyright © 2023, HappyCoders.eu Things to Consider • Make sure your tasks are interruptible • Make custom policies thread-safe
28. Scoped Values Copyright © 2023, HappyCoders.eu
29. What are Scoped Values? Copyright © 2023, HappyCoders.eu
30. Copyright © 2023, HappyCoders.eu Disadvantages of ThreadLocal • In memory until thread dies or ThreadLocal.remove() is called • Mutable • Stored in a HashMap → No clear scope, memory leaks → Complex code → High memory footprint • Copied to child threads when using InheritableThreadLocal → Even higher memory footprint
31. Live Coding: Scoped Values Copyright © 2023, HappyCoders.eu
32. Scoped Values Copyright © 2023, HappyCoders.eu = implicit method parameters
33. Scoped Values and Structured Concurrency Copyright © 2023, HappyCoders.eu
34. Copyright © 2023, HappyCoders.eu ScopedValue.where(USER, user).run(...);
35. Copyright © 2023, HappyCoders.eu getSupplierDeliveryTimes(...) getSupplierDeliveryTime(...) getSupplierDeliveryTime(...) getSupplierDeliveryTime(...)
36. Copyright © 2023, HappyCoders.eu user user user user user user user user user user user user ScopedValue.where(USER, user).run(...); ScopedValue .where(USER, null).run(...);
37. Copyright © 2023, HappyCoders.eu ScopedValue API ScopedValue.where(USER, user).run(runnable); ScopedValue.where(USER, user).call(callable); ScopedValue.where(USER, user).get(supplier); ScopedValue.where(USER, user) .where(SESSION, session) .run(runnable); ScopedValue.runWhere(USER, user, runnable); ScopedValue.callWhere(USER, user, callable); ScopedValue.getWhere(USER, user, supplier);
38. Copyright © 2023, HappyCoders.eu ScopedValue API User user = USER.get(); User user = USER.orElse(null); User user = USER.orElseThrow(UserNotLoggedInException::new); if (USER.isBound()) { User user = USER.get(); ... }
39. Copyright © 2023, HappyCoders.eu • No need to add a parameter to every method - Methods that don’t use it - Methods that must not see it Advantages of ScopedValues (over method parameters) - Methods you cannot change
40. Copyright © 2023, HappyCoders.eu • Only valid in the scope of the Runnable / Callable / Supplier • Immutable (only “rebindable”) • Stored in a Carrier → Clear scope, no memory leaks → Simpler code → Lower memory footprint • Not copied to child threads upfront Advantages of ScopedValues (over ThreadLocals) • But cached (only 16) • -Djava.lang.ScopedValue.cacheSize=2 / 4 / 8 / 16 (default) • -Djdk.preserveScopedValueCache=false (default: true)
41. Copyright © 2023, HappyCoders.eu The End
42. Copyright © 2023, HappyCoders.eu GitHub repositories: https://github.com/SvenWoltmann/structured-concurrency https://github.com/SvenWoltmann/scoped-values JEP 453: Structured Concurrency (Preview): https://openjdk.org/jeps/453 Scoped Values in Java: https://www.happycoders.eu/java/scoped-values/ https://twitter.com/svenwoltmann https://youtube.com/c/happycoders Links https://www.happycoders.eu/ https://github.com/SvenWoltmann https://linkedin.com/in/sven-woltmann/ JEP 446: Scoped Values (Preview): https://openjdk.org/jeps/446 Structured Concurrency in Java: https://www.happycoders.eu/java/structured-concurrency