Todd Ginsberg April 14, 2024 0 3 Loom is More Than Virtual Threads: Structured Concurrency and Scoped Values

Most of the talk about Project Loom as been about JEP 425: Virtual Threads. That's great because virtual threads are incredibly useful and will revolutionize how we write high-throughput applications. But did you know there are two more JEPs associated with Project Loom that are also very useful? First we'll learn about JEP 428: Structured Concurrency and how it will simplify multithreaded programming on the JVM. Next, we'll learn about JEP 429: Scoped Values which aims to solve some problems that ThreadLocals have had since their introduction way back in Java 1.2! Come to this talk and we'll learn why these projects are useful additions to the JVM, and write some code to illustrate where you'll be able to use them effectively.

Transcript

1. Loom Is More Than Virtual Threads! TriJUG 2024-04-15 Todd Ginsberg Structured Concurrency and Scoped Values @ToddGinsberg Lead Engineer - Payments Deutsche Bank
2. Hello! Todd Ginsberg Raleigh, NC TriJUG organizer 29 years of professional experience Currently: Director, Lead Engineer - Payments Deutsche Bank Cary, NC Photo Credit: Andrew Byala
3. Co-Author Charlie Golden Retriever 1 year, 3 months old Likes: - Everybody - Sticks - Playing Chase
4. @ToddGinsberg Agenda Project Loom Overview Virtual Threads Review Structured Concurrency Scoped Values Q & A
5. Project Loom
6. @ToddGinsberg Project Loom is to intended to explore, incubate and deliver Java VM features and APIs built on top of them for the purpose of supporting easy-to-use, high-throughput lightweight concurrency and new programming models on the Java platform. https://wiki.openjdk.org/display/loom/Main
7. @ToddGinsberg Project Loom is to intended to explore, incubate and deliver Java VM features and APIs built on top of them for the purpose of supporting easy-to-use, high-throughput lightweight concurrency and new programming models on the Java platform. https://wiki.openjdk.org/display/loom/Main
8. @ToddGinsberg Project Loom is to intended to explore, incubate and deliver Java VM features and APIs built on top of them for the purpose of supporting easy-to-use, high-throughput lightweight concurrency and new programming models on the Java platform. https://wiki.openjdk.org/display/loom/Main
9. @ToddGinsberg VIRTUAL THREADS
10. @ToddGinsberg READS
11. @ToddGinsberg Structured Concurrency Scoped Values
12. @ToddGinsberg Structured Concurrency Scoped Values
13. @ToddGinsberg Structured Concurrency Scoped Values
14. @ToddGinsberg Structured Concurrency Scoped Values
15. @ToddGinsberg VIRTUAL THREADS
16. @ToddGinsberg VIRTUAL THREADS
17. @ToddGinsberg Project Loom: Timeline 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 Loom
18. @ToddGinsberg Project Loom: Timeline 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 Loom Virtual Threads Released Preview Incubating
19. @ToddGinsberg Project Loom: Timeline 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 Loom Virtual Threads Structured Concurrency Released Preview Incubating
20. @ToddGinsberg Project Loom: Timeline 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 Loom Virtual Threads Structured Concurrency Scoped Values Released Preview Incubating
21. @ToddGinsberg Project Loom: Timeline 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 Loom Virtual Threads Structured Concurrency Scoped Values Released Preview Incubating ? ?
22. Review: Virtual Threads
23. @ToddGinsberg Platform Threads JVM Platform Thread Platform Thread Platform Thread Platform Thread
24. @ToddGinsberg Platform Threads Operating System JVM OS Thread OS Thread OS Thread OS Thread OS Thread OS Thread OS Thread Platform Thread Platform Thread Platform Thread Platform Thread
25. @ToddGinsberg Platform Thread Usage Platform Thread
26. @ToddGinsberg Platform Thread Usage Platform Thread
27. @ToddGinsberg Platform Thread Usage Platform Thread
28. @ToddGinsberg Platform Thread Usage Platform Thread
29. @ToddGinsberg Platform Thread Usage Platform Thread Not Optimal
30. @ToddGinsberg Java Threads: The Downside Memory hungry
31. @ToddGinsberg Java Threads: The Downside Memory hungry Slow creation time
32. @ToddGinsberg Java Threads: The Downside Memory hungry Slow creation time 0% useful when blocked
33. @ToddGinsberg Java Threads: The Downside Memory hungry Slow creation time Overhead for context switching 0% useful when blocked
34. @ToddGinsberg Java Threads: The Downside Memory hungry Limited number by OS Slow creation time Overhead for context switching 0% useful when blocked
35. @ToddGinsberg Platform Threads Operating System JVM OS Thread OS Thread OS Thread OS Thread OS Thread OS Thread OS Thread Platform Thread Platform Thread Platform Thread Platform Thread
36. @ToddGinsberg Platform Threads Operating System JVM OS Thread OS Thread OS Thread OS Thread OS Thread OS Thread OS Thread Platform Thread Platform Thread Platform Thread Platform Thread
37. @ToddGinsberg Virtual Threads Operating System JVM OS Thread OS Thread OS Thread OS Thread OS Thread OS Thread OS Thread Carrier Thread Carrier Thread Carrier Thread Carrier Thread
38. @ToddGinsberg Virtual Threads Operating System JVM OS Thread OS Thread OS Thread OS Thread OS Thread OS Thread OS Thread Carrier Thread Carrier Thread Carrier Thread Carrier Thread Virtual Thread Virtual Thread Virtual Thread Virtual Thread Virtual Thread Virtual Thread Virtual Thread Virtual Thread Virtual Thread Virtual Thread Virtual Thread Virtual Thread Virtual Thread Virtual Thread
39. @ToddGinsberg Virtual Threads Operating System JVM OS Thread OS Thread OS Thread OS Thread OS Thread OS Thread OS Thread Carrier Thread Carrier Thread Carrier Thread Carrier Thread Virtual Thread Virtual Thread Virtual Thread Virtual Thread Virtual Thread Virtual Thread Virtual Thread Virtual Thread Virtual Thread Virtual Thread Virtual Thread Virtual Thread Virtual Thread Virtual Thread
40. @ToddGinsberg Virtual Threads Operating System JVM OS Thread OS Thread OS Thread OS Thread OS Thread OS Thread OS Thread Carrier Thread Carrier Thread Carrier Thread Carrier Thread Virtual Thread Virtual Thread Virtual Thread Virtual Thread Virtual Thread Virtual Thread Virtual Thread Virtual Thread Virtual Thread Virtual Thread Virtual Thread Virtual Thread Virtual Thread Virtual Thread
41. @ToddGinsberg Virtual Threads Operating System JVM OS Thread OS Thread OS Thread OS Thread OS Thread OS Thread OS Thread Carrier Thread Carrier Thread Carrier Thread Carrier Thread Virtual Thread Virtual Thread Virtual Thread Virtual Thread Virtual Thread Virtual Thread Virtual Thread Virtual Thread Virtual Thread Virtual Thread Virtual Thread Virtual Thread Virtual Thread Virtual Thread
42. @ToddGinsberg Carrier Thread Usage 1 2 3 1 1 2 Carrier Virtual 1 Virtual 2 Virtual 3 Runnable Running Waiting 3
43. @ToddGinsberg Virtual Threads Operating System JVM OS Thread OS Thread OS Thread OS Thread OS Thread OS Thread OS Thread Carrier Thread Carrier Thread Carrier Thread Carrier Thread Virtual Thread Virtual Thread Virtual Thread Virtual Thread Virtual Thread Virtual Thread Virtual Thread Virtual Thread Virtual Thread Virtual Thread Virtual Thread Virtual Thread Virtual Thread Virtual Thread
44. @ToddGinsberg Virtual Threads Operating System JVM OS Thread OS Thread OS Thread OS Thread OS Thread OS Thread OS Thread Carrier Thread Carrier Thread Carrier Thread Carrier Thread Virtual Thread Virtual Thread Virtual Thread Virtual Thread Virtual Thread Virtual Thread Virtual Thread Virtual Thread Virtual Thread Virtual Thread Virtual Thread Virtual Thread Virtual Thread Virtual Thread Platform Thread
45. @ToddGinsberg Virtual Threads as Tasks Operating System JVM OS Thread OS Thread OS Thread OS Thread OS Thread OS Thread OS Thread Carrier Thread Carrier Thread Carrier Thread Carrier Thread Task Task Task Task Task Task Task Task Task Task Task Task Task Task Platform Thread
46. @ToddGinsberg Virtual Threads as Tasks Operating System JVM OS Thread OS Thread OS Thread OS Thread OS Thread OS Thread OS Thread Carrier Thread Carrier Thread Carrier Thread Carrier Thread Task Task Task Task Task Task Task Platform Thread Task Task Task Task Task Task Task Task Task Task Task Task Task Task Task Task Task Task Task Task Task
47. @ToddGinsberg What Virtual Threads Give Us Same API: java.lang.Thread
48. @ToddGinsberg What Virtual Threads Give Us Same API: java.lang.Thread Much lower memory requirements
49. @ToddGinsberg What Virtual Threads Give Us Same API: java.lang.Thread Much lower memory requirements Number limited by heap memory
50. @ToddGinsberg What Virtual Threads Give Us Same API: java.lang.Thread Much lower memory requirements Number limited by heap memory Much quicker creation time
51. @ToddGinsberg What Virtual Threads Give Us Same API: java.lang.Thread Much lower memory requirements Number limited by heap memory Much quicker creation time Make better use of system resources
52. Demo
53. @ToddGinsberg Project Loom is to intended to explore, incubate and deliver Java VM features and APIs built on top of them for the purpose of supporting easy-to-use, high-throughput lightweight concurrency and new programming models on the Java platform. https://wiki.openjdk.org/display/loom/Main
54. @ToddGinsberg Project Loom is to intended to explore, incubate and deliver Java VM features and APIs built on top of them for the purpose of supporting easy-to-use, high-throughput lightweight concurrency and new programming models on the Java platform. https://wiki.openjdk.org/display/loom/Main
55. @ToddGinsberg Better Throughput Little’s Law: concurrency = arrival rate * latency
56. @ToddGinsberg Better Throughput Little’s Law: concurrency = arrival rate * latency 10
57. @ToddGinsberg Better Throughput Little’s Law: concurrency = arrival rate * latency 10 .5
58. @ToddGinsberg Better Throughput Little’s Law: concurrency = arrival rate * latency 10 .5 5
59. @ToddGinsberg Better Throughput Little’s Law: concurrency = arrival rate * latency
60. @ToddGinsberg Better Throughput Little’s Law: concurrency = arrival rate * latency throughput
61. @ToddGinsberg Better Throughput Little’s Law: throughput = concurrency / latency
62. @ToddGinsberg Better Throughput Little’s Law: throughput = concurrency / latency
63. @ToddGinsberg Better Throughput Little’s Law: throughput = concurrency / latency
64. @ToddGinsberg Better Throughput Little’s Law: throughput = concurrency / latency Virtual Threads!
65. @ToddGinsberg Final Thoughts on Virtual Threads Do Not Pool Virtual Threads
66. @ToddGinsberg Final Thoughts on Virtual Threads Do Not Pool Virtual Threads
67. @ToddGinsberg Final Thoughts on Virtual Threads Do Not Pool Virtual Threads
68. @ToddGinsberg Final Thoughts on Virtual Threads Do Not Pool Virtual Threads Create
69. @ToddGinsberg Final Thoughts on Virtual Threads Do Not Pool Virtual Threads Create Use
70. @ToddGinsberg Final Thoughts on Virtual Threads Do Not Pool Virtual Threads Create Dispose Use
71. @ToddGinsberg Final Thoughts on Virtual Threads Do Not Pool Virtual Threads Be open to new ways of writing concurrent code
72. Structured Concurrency
73. Structured Concurrency Un-
74. Demo
75. @ToddGinsberg Three Problems With Unstructured Concurrency
76. @ToddGinsberg Three Problems With Unstructured Concurrency Relationships between task and subtask don’t really exist
77. @ToddGinsberg Three Problems With Unstructured Concurrency Relationships between task and subtask don’t really exist Observability is difficult
78. @ToddGinsberg Three Problems With Unstructured Concurrency Relationships between task and subtask don’t really exist Managing work is difficult, making the true purpose of code opaque Observability is difficult
79. Structured Concurrency
80. @ToddGinsberg Structured Concurrency: History • 2016 - Conceived by Martin Sústrik (golang) • 2017 - Nathaniel J. Smith “Nursery Pattern” in Python • Now shipping in Kotlin, Swift, and Java (Preview!)
81. @ToddGinsberg Structured Concurrency: What Is It?
82. @ToddGinsberg Structured Concurrency: What Is It? Subtasks have well-defined entry and exit points
83. @ToddGinsberg Structured Concurrency: What Is It? Subtasks have well-defined entry and exit points Strict nesting of task lifetimes, like they are presented in code
84. @ToddGinsberg Subtask Lifetime: Ideal Scenario public String doSomething() { String first = firstMethod(); String second = secondMethod(); return first + second; }
85. @ToddGinsberg Subtask Lifetime: Ideal Scenario public String doSomething() { String first = firstMethod(); String second = secondMethod(); return first + second; }
86. @ToddGinsberg Subtask Lifetime: Ideal Scenario at Example.firstMethod(Example.java:26) at Example.doSomething(Example.java:22) at Example.main(Example.java:42)
87. @ToddGinsberg Subtask Lifetime: Ideal Scenario at Example.firstMethod(Example.java:26) at Example.doSomething(Example.java:22) at Example.main(Example.java:42)
88. @ToddGinsberg Subtask Lifetime: Ideal Scenario at Example.firstMethod(Example.java:26) at Example.doSomething(Example.java:22) at Example.main(Example.java:42)
89. @ToddGinsberg Subtask Lifetime: Ideal Scenario public String doSomething() { String first = firstMethod(); String second = secondMethod(); return first + second; }
90. @ToddGinsberg Subtask Lifetime: Ideal Scenario public String doSomething() { String first = firstMethod(); String second = secondMethod(); return first + second; }
91. @ToddGinsberg Subtask Lifetime: Ideal Scenario public String doSomething() { String first = firstMethod(); String second = secondMethod(); return first + second; }
92. @ToddGinsberg Subtask Lifetime: Ideal Scenario public String doSomething() { String first = firstMethod(); String second = secondMethod(); return first + second; }
93. @ToddGinsberg Subtask Lifetime: Ideal Scenario public String doSomething() { String first = firstMethod(); String second = secondMethod(); return first + second; }
94. @ToddGinsberg Java’s Structured Concurrency: Goals https://openjdk.org/jeps/462
95. @ToddGinsberg Java’s Structured Concurrency: Goals Promote a style of concurrent programming that can eliminate common risks arising from cancellation and shutdown, such as thread leaks and cancellation delays https://openjdk.org/jeps/462
96. @ToddGinsberg Java’s Structured Concurrency: Goals Promote a style of concurrent programming that can eliminate common risks arising from cancellation and shutdown, such as thread leaks and cancellation delays https://openjdk.org/jeps/462
97. @ToddGinsberg Java’s Structured Concurrency: Goals Promote a style of concurrent programming that can eliminate common risks arising from cancellation and shutdown, such as thread leaks and cancellation delays Improve the observability of concurrent code https://openjdk.org/jeps/462
98. @ToddGinsberg StructuredTaskScope
99. @ToddGinsberg StructuredTaskScope Scope
100. @ToddGinsberg StructuredTaskScope Scope Task Task Task Task
101. @ToddGinsberg StructuredTaskScope Scope Task Task Task Task join()
102. @ToddGinsberg StructuredTaskScope Scope Task Task Task Task join()
103. @ToddGinsberg StructuredTaskScope Scope Task Task Task Task join() Interpret Results
104. @ToddGinsberg StructuredTaskScope Scope Task Task Task Task join() Interpret Results
105. @ToddGinsberg StructuredTaskScope Scope Task Task Task Task join() Throw Exception?
106. @ToddGinsberg This Feature Is In Preview java --enable-preview
107. Demo
108. @ToddGinsberg Remember: Structured Concurrency
109. @ToddGinsberg Remember: Structured Concurrency Explicit relationship between tasks and subtasks
110. @ToddGinsberg Remember: Structured Concurrency Explicit relationship between tasks and subtasks Observability is easier
111. @ToddGinsberg Remember: Structured Concurrency Explicit relationship between tasks and subtasks Managing work is easier (or automatic) making code intent clearer Observability is easier
112. Scoped Values
113. @ToddGinsberg Quick Refresher: ThreadLocal
114. @ToddGinsberg Quick Refresher: ThreadLocal Think of it as… Map<Thread,T>
115. @ToddGinsberg Quick Refresher: ThreadLocal Think of it as… static Map<Thread,T>
116. @ToddGinsberg Quick Refresher: ThreadLocal
117. @ToddGinsberg Quick Refresher: ThreadLocal ThreadLocal<String> accessLevel = new ThreadLocal<>();
118. @ToddGinsberg Quick Refresher: ThreadLocal ThreadLocal<String> accessLevel = ... void doSomething() { }
119. @ToddGinsberg Quick Refresher: ThreadLocal ThreadLocal<String> accessLevel = ... void doSomething() { accessLevel.set("GUEST"); }
120. @ToddGinsberg Quick Refresher: ThreadLocal ThreadLocal<String> accessLevel = ... void someOtherPlace() { }
121. @ToddGinsberg Quick Refresher: ThreadLocal ThreadLocal<String> accessLevel = ... void someOtherPlace() { switch(accessLevel.get()) { case "GUEST" -> guestLevelWork(); case "ADMIN" -> adminLevelWork(); } }
122. @ToddGinsberg Quick Refresher: ThreadLocal ThreadLocal<String> accessLevel = ... void someOtherPlace() { switch(accessLevel.get()) { case "GUEST" -> guestLevelWork(); case "ADMIN" -> adminLevelWork(); } }
123. @ToddGinsberg Three Problems with ThreadLocal
124. @ToddGinsberg Three Problems with ThreadLocal Unconstrained Mutability
125. @ToddGinsberg Problem 1: Unconstrained Mutability void quickHack() { var original = accessLevel.get(); accessLevel.set("ADMIN"); doTheThing(); accessLevel.set(original); }
126. @ToddGinsberg Problem 1: Unconstrained Mutability void quickHack() { var original = accessLevel.get(); accessLevel.set("ADMIN"); doTheThing(); accessLevel.set(original); }
127. @ToddGinsberg Problem 1: Unconstrained Mutability void quickHack() { var original = accessLevel.get(); accessLevel.set("ADMIN"); doTheThing(); accessLevel.set(original); }
128. @ToddGinsberg Three Problems with ThreadLocal Unconstrained Mutability Unbounded Lifetime
129. @ToddGinsberg Problem 2: Unbounded Lifecycle accessLevel.set("ADMIN"); doTheThing();
130. @ToddGinsberg Problem 2: Unbounded Lifecycle try { accessLevel.set("ADMIN"); doTheThing(); } finally { accessLevel.remove(); }
131. @ToddGinsberg Problem 2: Unbounded Lifecycle try { accessLevel.set("ADMIN"); doTheThing(); } finally { accessLevel.remove(); }
132. @ToddGinsberg Three Problems with ThreadLocal Unconstrained Mutability Unbounded Lifetime Expensive Inheritance
133. @ToddGinsberg Problem 3 : Expensive Inheritance ThreadLocal<GiantCache> cache = new ThreadLocal<>();
134. @ToddGinsberg Problem 3 : Expensive Inheritance ThreadLocal<GiantCache> cache = new InheritableThreadLocal<>();
135. @ToddGinsberg How Do Scoped Values Help?
136. @ToddGinsberg How Do Scoped Values Help? Unconstrained Mutability Values are Immutable
137. @ToddGinsberg How Do Scoped Values Help? Unconstrained Mutability Values are Immutable Unbounded Lifetime Defined Lifetime
138. @ToddGinsberg How Do Scoped Values Help? Unconstrained Mutability Values are Immutable Unbounded Lifetime Defined Lifetime Expensive Inheritance Cheap/Free Inheritance
139. @ToddGinsberg This Feature Is In Preview java --enable-preview
140. Demo
141. @ToddGinsberg Scoped Values: Remember
142. @ToddGinsberg Scoped Values: Remember Useful anywhere you need to pass data one way
143. @ToddGinsberg Scoped Values: Remember Useful anywhere you need to pass data one way Try to migrate to Scoped Values when you have structured sharing use cases
144. Remember…
145. @ToddGinsberg Preview Mode in Java 22 java --enable-preview
146. @ToddGinsberg Nothing Is Going Away It is not a goal to…
147. @ToddGinsberg Nothing Is Going Away “... remove the traditional implementation of threads, or to silently migrate existing applications to use virtual threads.” It is not a goal to… https://openjdk.org/jeps/444
148. @ToddGinsberg Nothing Is Going Away “... replace any of the concurrency constructs in the java.util.concurrent package, such as ExecutorService and Future.” It is not a goal to… https://openjdk.org/jeps/462
149. @ToddGinsberg Nothing Is Going Away “... require migration away from thread-local variables, or to deprecate the existing ThreadLocal API.” It is not a goal to… https://openjdk.org/jeps/464
150. @ToddGinsberg What Happens Next? https://openjdk.org/projects/jdk/23/ Follow the Java 23 Release Page
151. @ToddGinsberg How To Stay Informed? https://wiki.openjdk.org/display/loom/Main View the Project Loom Wiki
152. Questions? (And Maybe Answers!)
153. @ToddGinsberg @ToddGinsberg todd@ginsberg.com https://todd.ginsberg.com Thank You!