Reactive Async: Expressive Deterministic Concurrency

1. Reactive Async: Expressive Deterministic Concurrency Philipp Haller1, Simon Geries1,
 Michael Eichberg2, Guido Salvaneschi2 1 KTH Royal Institute of Technology, Sweden 2 TU Darmstadt, Germany
2. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Have we solved concurrent programming, yet? 2
3. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Have we solved concurrent programming, yet? The problem with concurrency: 2
4. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Have we solved concurrent programming, yet? The problem with concurrency: – It’s difficult! 2
5. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Have we solved concurrent programming, yet? The problem with concurrency: – It’s difficult! – Multiple hazards: 2
6. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Have we solved concurrent programming, yet? The problem with concurrency: – It’s difficult! – Multiple hazards: • Race conditions 2
7. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Have we solved concurrent programming, yet? The problem with concurrency: – It’s difficult! – Multiple hazards: • Race conditions • Deadlocks 2
8. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Have we solved concurrent programming, yet? The problem with concurrency: – It’s difficult! – Multiple hazards: • Race conditions • Deadlocks • Livelocks 2
9. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Have we solved concurrent programming, yet? The problem with concurrency: – It’s difficult! – Multiple hazards: • Race conditions • Deadlocks • Livelocks • Violation of fairness 2
10. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Non-Determinism 3
11. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Non-Determinism • At the root of several hazards 3
12. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Non-Determinism • At the root of several hazards • Example 1: 3 @volatile var x = 0 def m(): Unit = { Future { x = 1 } Future { x = 2 } .. // does not access x }
13. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Non-Determinism • At the root of several hazards • Example 1: 3 @volatile var x = 0 def m(): Unit = { Future { x = 1 } Future { x = 2 } .. // does not access x } What’s the value of x when an invocation of m returns?
14. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Reordering not always a problem! • Example 2: 4
15. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Reordering not always a problem! • Example 2: 4 import scala.collection.concurrent.TrieMap val set = new TrieMap[Int, Int] Future { set.put(1, 0) } set.put(2, 0)
16. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Reordering not always a problem! • Example 2: 4 import scala.collection.concurrent.TrieMap val set = new TrieMap[Int, Int] Future { set.put(1, 0) } set.put(2, 0) Eventually, set contains both 1 and 2, always
17. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Reordering not always a problem! • Example 2: 4 import scala.collection.concurrent.TrieMap val set = new TrieMap[Int, Int] Future { set.put(1, 0) } set.put(2, 0) Eventually, set contains both 1 and 2, always Bottom line: it depends on the datatype
18. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller … and its operations! • Example 3: 5
19. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller … and its operations! • Example 3: 5 val set = new TrieMap[Int, Int] Future { set.put(1, 0) } Future { if (set.contains(1)) { .. } } set.put(2, 0)
20. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller … and its operations! • Example 3: 5 val set = new TrieMap[Int, Int] Future { set.put(1, 0) } Future { if (set.contains(1)) { .. } } set.put(2, 0) Result depends on schedule!
21. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Important Questions 6
22. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Important Questions Q1:
 How do we know we have written a deterministic concurrent program? 6
23. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Important Questions Q1:
 How do we know we have written a deterministic concurrent program? – What about Heisenbugs? 6
24. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Important Questions Q1:
 How do we know we have written a deterministic concurrent program? – What about Heisenbugs? • No race in 1’000’000 runs, race in run 1’000’001 6
25. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Important Questions Q1:
 How do we know we have written a deterministic concurrent program? – What about Heisenbugs? • No race in 1’000’000 runs, race in run 1’000’001 6 Goal: simple criteria that guarantee determinism
26. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Important Questions 7
27. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Important Questions Q2:
 How do we ensure expressivity while providing determinism? 7
28. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Important Questions Q2:
 How do we ensure expressivity while providing determinism? – Draconian restrictions undesired! 7
29. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Important Questions Q2:
 How do we ensure expressivity while providing determinism? – Draconian restrictions undesired! 7 Goal: reconcile determinism and expressivity
30. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Approach 8
31. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Approach • Extend a future-style programming model with: 8
32. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Approach • Extend a future-style programming model with: – Lattice-based datatypes 8
33. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Approach • Extend a future-style programming model with: – Lattice-based datatypes – Quiescence 8
34. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Approach • Extend a future-style programming model with: – Lattice-based datatypes – Quiescence 8 Crucial for determinism!
35. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Approach • Extend a future-style programming model with: – Lattice-based datatypes – Quiescence – Resolution of cyclic dependencies 8 Crucial for determinism!
36. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Approach • Extend a future-style programming model with: – Lattice-based datatypes – Quiescence – Resolution of cyclic dependencies 8 Crucial for determinism! Increases expressivity!
37. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Approach • Extend a future-style programming model with: – Lattice-based datatypes – Quiescence – Resolution of cyclic dependencies • Evaluation of expressivity and performance – Case study: static analysis of JVM bytecode 8 Crucial for determinism! Increases expressivity!
38. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Programming Model 9
39. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Programming Model Programming model based on two core abstractions:
 cells and cell completers 9
40. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Programming Model Programming model based on two core abstractions:
 cells and cell completers – Cell = shared “variable” 9
41. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Programming Model Programming model based on two core abstractions:
 cells and cell completers – Cell = shared “variable” 9 Concurrently read and written
42. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Programming Model Programming model based on two core abstractions:
 cells and cell completers – Cell = shared “variable” – Cell[K,V]: read-only interface; read values of type V (akin to Future[V]) 9 Concurrently read and written
43. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Programming Model Programming model based on two core abstractions:
 cells and cell completers – Cell = shared “variable” – Cell[K,V]: read-only interface; read values of type V (akin to Future[V]) – CellCompleter[K,V] : write values of type V to its associated cell (akin to Promise[V]) 9 Concurrently read and written
44. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Programming Model Programming model based on two core abstractions:
 cells and cell completers – Cell = shared “variable” – Cell[K,V]: read-only interface; read values of type V (akin to Future[V]) – CellCompleter[K,V] : write values of type V to its associated cell (akin to Promise[V]) – V must have an instance of a lattice type class 9 Concurrently read and written
45. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Programming Model Programming model based on two core abstractions:
 cells and cell completers – Cell = shared “variable” – Cell[K,V]: read-only interface; read values of type V (akin to Future[V]) – CellCompleter[K,V] : write values of type V to its associated cell (akin to Promise[V]) – V must have an instance of a lattice type class 9 Monotonic updates Concurrently read and written
46. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Example 10
47. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Example • Given a social graph 10
48. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Example • Given a social graph – vertex = user 10
49. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Example • Given a social graph – vertex = user • Traverse graph and collect IDs of “interesting” users 10
50. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Example • Given a social graph – vertex = user • Traverse graph and collect IDs of “interesting” users • Graph large => concurrent traversal 10
51. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Solution 11
52. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Solution • Collect IDs of interesting users in a cell 11
53. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Solution • Collect IDs of interesting users in a cell • Cell contains set of integers 11
54. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Solution • Collect IDs of interesting users in a cell • Cell contains set of integers 11 OK: subsets of the set of all integers form a lattice
55. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Cell with User IDs (code simplified) 12
56. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Cell with User IDs (code simplified) 12 val userIDs = CellCompleter[Set[Int]]
57. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Cell with User IDs (code simplified) 12 implicit object IntSetLattice extends Lattice[Set[Int]] { val empty = Set() def join(left: Set[Int], right: Set[Int]) = left ++ right } val userIDs = CellCompleter[Set[Int]]
58. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Cell with User IDs (code simplified) 12 implicit object IntSetLattice extends Lattice[Set[Int]] { val empty = Set() def join(left: Set[Int], right: Set[Int]) = left ++ right } val userIDs = CellCompleter[Set[Int]] Bounded
 join-semilattice
59. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Cell with User IDs (code simplified) 12 implicit object IntSetLattice extends Lattice[Set[Int]] { val empty = Set() def join(left: Set[Int], right: Set[Int]) = left ++ right } // add a user ID userIDs.putNext(Set(theUserID)) val userIDs = CellCompleter[Set[Int]] Bounded
 join-semilattice
60. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Reading Results 13
61. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Reading Results • Problem: when reading a cell’s value, how do we know this value is not going to change any more? 13
62. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Reading Results • Problem: when reading a cell’s value, how do we know this value is not going to change any more? – There may still be ongoing concurrent activities 13
63. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Reading Results • Problem: when reading a cell’s value, how do we know this value is not going to change any more? – There may still be ongoing concurrent activities – Manual synchronization (e.g., latches) error-prone 13
64. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Reading Results • Problem: when reading a cell’s value, how do we know this value is not going to change any more? – There may still be ongoing concurrent activities – Manual synchronization (e.g., latches) error-prone • Solution: 13
65. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Reading Results • Problem: when reading a cell’s value, how do we know this value is not going to change any more? – There may still be ongoing concurrent activities – Manual synchronization (e.g., latches) error-prone • Solution: 13 Koyaanisqatsi
66. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Reading Results • Problem: when reading a cell’s value, how do we know this value is not going to change any more? – There may still be ongoing concurrent activities – Manual synchronization (e.g., latches) error-prone • Solution: 13 Koyaanisqatsi
67. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Reading Results • Problem: when reading a cell’s value, how do we know this value is not going to change any more? – There may still be ongoing concurrent activities – Manual synchronization (e.g., latches) error-prone • Solution: 13 Quiescence
68. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Quiescence 14
69. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Quiescence • Intuitively: situation when values of cells are guaranteed not to change any more 14
70. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Quiescence • Intuitively: situation when values of cells are guaranteed not to change any more • Technically: 14
71. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Quiescence • Intuitively: situation when values of cells are guaranteed not to change any more • Technically: – No concurrent activities ongoing or scheduled which could change values of cells 14
72. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Quiescence • Intuitively: situation when values of cells are guaranteed not to change any more • Technically: – No concurrent activities ongoing or scheduled which could change values of cells – Detected by the underlying thread pool 14
73. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Revisiting the Example 15
74. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Revisiting the Example 15 val pool = new HandlerPool val userIDs = CellCompleter[Set[Int]](pool)
75. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Revisiting the Example 15 // add a user ID userIDs.putNext(Set(theUserID)) .. val pool = new HandlerPool val userIDs = CellCompleter[Set[Int]](pool)
76. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Revisiting the Example 15 // add a user ID userIDs.putNext(Set(theUserID)) .. val pool = new HandlerPool val userIDs = CellCompleter[Set[Int]](pool) // register handler // upon quiescence: read result value of cell pool.onQuiescent(userIDs.cell) { collectedIDs => .. }
77. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Revisiting the Example 15 // add a user ID userIDs.putNext(Set(theUserID)) .. val pool = new HandlerPool val userIDs = CellCompleter[Set[Int]](pool) // register handler // upon quiescence: read result value of cell pool.onQuiescent(userIDs.cell) { collectedIDs => .. } Safe to read from cell when pool quiescent!
78. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Handler Pools 16
79. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Handler Pools Execution context with extensions: 16
80. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Handler Pools Execution context with extensions: – Event-driven quiescence API 16
81. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Handler Pools Execution context with extensions: – Event-driven quiescence API – Resolution of dependencies between cells 16
82. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Handler Pools Execution context with extensions: – Event-driven quiescence API – Resolution of dependencies between cells 16 Dependency resolution?
83. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Expressing Concurrent Dataflow 17
84. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Expressing Concurrent Dataflow • Cells provide non-blocking interface 17
85. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Expressing Concurrent Dataflow • Cells provide non-blocking interface – Low level: callbacks 17
86. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Expressing Concurrent Dataflow • Cells provide non-blocking interface – Low level: callbacks – High level: combinators 17
87. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Expressing Concurrent Dataflow • Cells provide non-blocking interface – Low level: callbacks – High level: combinators 17 Like futures
88. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Expressing Concurrent Dataflow • Cells provide non-blocking interface – Low level: callbacks – High level: combinators • Important purpose: express concurrent dataflow 17 Like futures
89. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Expressing Concurrent Dataflow • Cells provide non-blocking interface – Low level: callbacks – High level: combinators • Important purpose: express concurrent dataflow 17 Like futures fut.map(fun).filter(pred)
90. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Expressing Concurrent Dataflow • Cells provide non-blocking interface – Low level: callbacks – High level: combinators • Important purpose: express concurrent dataflow 17 Like futures fut.map(fun).filter(pred) Futures
91. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Expressing Concurrent Dataflow • Cells provide non-blocking interface – Low level: callbacks – High level: combinators • Important purpose: express concurrent dataflow 17 Like futures fut fun fut’ pred fut’’ fut.map(fun).filter(pred) Futures
92. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Expressing Concurrent Dataflow • Cells provide non-blocking interface – Low level: callbacks – High level: combinators • Important purpose: express concurrent dataflow • Problem: 17 Like futures fut fun fut’ pred fut’’ fut.map(fun).filter(pred) Futures
93. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Expressing Concurrent Dataflow • Cells provide non-blocking interface – Low level: callbacks – High level: combinators • Important purpose: express concurrent dataflow • Problem: 17 What if the dataflow graph is cyclic? Like futures fut fun fut’ pred fut’’ fut.map(fun).filter(pred) Futures
94. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Dataflow Graphs 18
95. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Dataflow Graphs Example: • Static analysis of JVM bytecode 18
96. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Dataflow Graphs Example: • Static analysis of JVM bytecode • Task:
 Determine purity of methods 18
97. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Dataflow Graphs Example: • Static analysis of JVM bytecode • Task:
 Determine purity of methods • Rules: 18
98. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Dataflow Graphs Example: • Static analysis of JVM bytecode • Task:
 Determine purity of methods • Rules: – A method is impure if it accesses a non-final field 18
99. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Dataflow Graphs Example: • Static analysis of JVM bytecode • Task:
 Determine purity of methods • Rules: – A method is impure if it accesses a non-final field – A method is impure if it calls an impure method 18
100. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Dataflow Graphs Example: • Static analysis of JVM bytecode • Task:
 Determine purity of methods • Rules: – A method is impure if it accesses a non-final field – A method is impure if it calls an impure method – … 18
101. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Dataflow Graphs Example: • Static analysis of JVM bytecode • Task:
 Determine purity of methods • Rules: – A method is impure if it accesses a non-final field – A method is impure if it calls an impure method – … 18 A
102. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Dataflow Graphs Example: • Static analysis of JVM bytecode • Task:
 Determine purity of methods • Rules: – A method is impure if it accesses a non-final field – A method is impure if it calls an impure method – … 18 A B
103. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Dataflow Graphs Example: • Static analysis of JVM bytecode • Task:
 Determine purity of methods • Rules: – A method is impure if it accesses a non-final field – A method is impure if it calls an impure method – … 18 A B “calls”
104. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Dataflow Graphs Example: • Static analysis of JVM bytecode • Task:
 Determine purity of methods • Rules: – A method is impure if it accesses a non-final field – A method is impure if it calls an impure method – … 18 A B C “calls”
105. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Dataflow Graphs Example: • Static analysis of JVM bytecode • Task:
 Determine purity of methods • Rules: – A method is impure if it accesses a non-final field – A method is impure if it calls an impure method – … 18 A B C “calls”
106. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Cycle Resolution 19
107. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Cycle Resolution • What if upon quiescence cells are empty and there is a dependency cycle? 19
108. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Cycle Resolution • What if upon quiescence cells are empty and there is a dependency cycle? 19 A B C
109. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Cycle Resolution • What if upon quiescence cells are empty and there is a dependency cycle? • Idea:
 Pluggable resolution policies 19 A B C
110. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Cycle Resolution • What if upon quiescence cells are empty and there is a dependency cycle? • Idea:
 Pluggable resolution policies • Example:
 Purity analysis should resolve all cells
 in cycle to “pure”, since could not show impurity 19 A B C
111. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Specifying Resolution Policies 20
112. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Specifying Resolution Policies • Role of the first type parameter of Cell[K,V] 20
113. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Specifying Resolution Policies • Role of the first type parameter of Cell[K,V] • Example: 20
114. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Specifying Resolution Policies • Role of the first type parameter of Cell[K,V] • Example: 20 object PurityKey extends Key[Purity] { def resolve[K <: Key[Purity]](cells: Seq[Cell[K, Purity]]) = cells.map(cell => (cell, Pure)) .. }
115. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Specifying Resolution Policies • Role of the first type parameter of Cell[K,V] • Example: 20 object PurityKey extends Key[Purity] { def resolve[K <: Key[Purity]](cells: Seq[Cell[K, Purity]]) = cells.map(cell => (cell, Pure)) .. } resolve all cells to Pure
116. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Specifying Resolution Policies • Role of the first type parameter of Cell[K,V] • Example: 20 object PurityKey extends Key[Purity] { def resolve[K <: Key[Purity]](cells: Seq[Cell[K, Purity]]) = cells.map(cell => (cell, Pure)) .. } closed strongly- connected component
117. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Purity Analysis: Set-up 21
118. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Purity Analysis: Set-up 21 for { classFile <- project.allProjectClassFiles method <- classFile.methods } { val methodCompleter = CellCompleter(pool, PurityKey) pool.execute { analyze(project, methodCompleter, classFile, method) } } val analysisDoneFuture = pool.quiescentResolveCells()
119. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Evaluation 22
120. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Evaluation • Static analysis of JVM bytecode using the OPAL framework (OPen extensible Analysis Library) 22
121. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Evaluation • Static analysis of JVM bytecode using the OPAL framework (OPen extensible Analysis Library) – New Scala-based, extensible bytecode toolkit 22
122. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Evaluation • Static analysis of JVM bytecode using the OPAL framework (OPen extensible Analysis Library) – New Scala-based, extensible bytecode toolkit – Fully concurrent 22
123. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Evaluation • Static analysis of JVM bytecode using the OPAL framework (OPen extensible Analysis Library) – New Scala-based, extensible bytecode toolkit – Fully concurrent 22 http://www.opal-project.de
124. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Evaluation • Static analysis of JVM bytecode using the OPAL framework (OPen extensible Analysis Library) – New Scala-based, extensible bytecode toolkit – Fully concurrent • Rewrote purity analysis and immutability analysis 22 http://www.opal-project.de
125. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Evaluation • Static analysis of JVM bytecode using the OPAL framework (OPen extensible Analysis Library) – New Scala-based, extensible bytecode toolkit – Fully concurrent • Rewrote purity analysis and immutability analysis • Ran analysis on JDK 8 update 45 (rt.jar) 22 http://www.opal-project.de
126. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Evaluation • Static analysis of JVM bytecode using the OPAL framework (OPen extensible Analysis Library) – New Scala-based, extensible bytecode toolkit – Fully concurrent • Rewrote purity analysis and immutability analysis • Ran analysis on JDK 8 update 45 (rt.jar) – 18’591 class files, 163’268 methods, 77’128 fields 22 http://www.opal-project.de
127. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Results: Immutability Analysis • RA about 10x faster than FPCF • RA = 294 LOC, FPCF = 424 LOC (1.44x) 23 FPCF (secs.) 1.0 1.5 2.0 2.5 Reactive-Async (secs.) 0.1 0.2 0.3 1 Thread 2 Threads 4 Threads 8 Threads 16 Threads 2.15 2.20 2.25 2.30 2.35 1.15 1.20 1.25 1.30 1.35 0.290 0.295 0.300 0.105 0.110 0.115
128. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Results: Immutability Analysis • RA about 10x faster than FPCF • RA = 294 LOC, FPCF = 424 LOC (1.44x) 23 FPCF (secs.) 1.0 1.5 2.0 2.5 Reactive-Async (secs.) 0.1 0.2 0.3 1 Thread 2 Threads 4 Threads 8 Threads 16 Threads 2.15 2.20 2.25 2.30 2.35 1.15 1.20 1.25 1.30 1.35 0.290 0.295 0.300 0.105 0.110 0.115 box plot: whiskers = min/max
 top/bottom of box = 
 1st and 3rd quartile band in box: median
129. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Results: Immutability Analysis • RA about 10x faster than FPCF • RA = 294 LOC, FPCF = 424 LOC (1.44x) 23 FPCF (secs.) 1.0 1.5 2.0 2.5 Reactive-Async (secs.) 0.1 0.2 0.3 1 Thread 2 Threads 4 Threads 8 Threads 16 Threads 2.15 2.20 2.25 2.30 2.35 1.15 1.20 1.25 1.30 1.35 0.290 0.295 0.300 0.105 0.110 0.115 box plot: whiskers = min/max
 top/bottom of box = 
 1st and 3rd quartile band in box: median FPCF = OPAL’s fixed point computation framework
130. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Results: Purity Analysis • Best configs. (FPCF: 4 thr, RA: 16 thr): RA 1.75x faster • RA = 113 LOC, FPCF = 166 LOC (1.47x) 24 FPCF (secs.) 0.15 0.20 0.25 0.30 0.35 Reactive-Async (secs.) 0.1 0.2 0.3 0.4 0.5 1 Thread 2 Threads 4 Threads 8 Threads 16 Threads 0.43 0.44 0.45 0.09 0.10 0.11 0.335 0.340 0.345 0.165 0.170 0.175 0.180 0.185 0.18 0.19 0.20
131. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Benchmarks: Monte Carlo Simulation • Compute Net Present Value • 14-16 threads: Scala futures 23-36% faster than RA 25 Sequential Reactive Async Scala Futures 2.11.7 Runtime (ms) 0 1000 2000 3000 4000 5000 6000 Number of Threads 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 400 500 600 700 14 15 16
132. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Benchmarks: Parallel Sum • Parallel sum of large collection of random ints • Performance competitive with Scala’s futures 26 Sequential Reactive Async Scala Futures 2.11.7 Runtime (ms) 0 50 100 150 200 250 300 Number of Threads 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
133. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Related Work • Deterministic-by-construction programming models – KPNs, concurrent revisions, FlowPools, LVars, etc. • Static vs. dynamic enforcement of determinism • Reactive programming models – Dynamic dependencies, determinism • See paper for discussion 27
134. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Reactive Async: Conclusion • New concurrent programming model – Lattices and quiescence for determinism – Resolution of cyclic dependencies • Promising experimental results • Future work: – Optimization – Determinism as a statically-checked effect 28
135. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Reactive Async: Conclusion • New concurrent programming model – Lattices and quiescence for determinism – Resolution of cyclic dependencies • Promising experimental results • Future work: – Optimization – Determinism as a statically-checked effect 28 https://github.com/phaller/reactive-async
136. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Reactive Async: Conclusion • New concurrent programming model – Lattices and quiescence for determinism – Resolution of cyclic dependencies • Promising experimental results • Future work: – Optimization – Determinism as a statically-checked effect 28 https://github.com/phaller/reactive-async Thank you!