Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Reactive Async: Expressive Deterministic Concurrency

3b84657fdb075382e3781310ca8a9a70?s=47 Philipp Haller
October 31, 2016
550

Reactive Async: Expressive Deterministic Concurrency

3b84657fdb075382e3781310ca8a9a70?s=128

Philipp Haller

October 31, 2016
Tweet

Transcript

  1. Reactive Async: Expressive Deterministic Concurrency Philipp Haller1, Simon Geries1,
 Michael

    Eichberg2, Guido Salvaneschi2 1 KTH Royal Institute of Technology, Sweden 2 TU Darmstadt, Germany
  2. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Have we

    solved concurrent programming, yet? 2
  3. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Have we

    solved concurrent programming, yet? The problem with concurrency: 2
  4. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Have we

    solved concurrent programming, yet? The problem with concurrency: – It’s difficult! 2
  5. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Have we

    solved concurrent programming, yet? The problem with concurrency: – It’s difficult! – Multiple hazards: 2
  6. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Have we

    solved concurrent programming, yet? The problem with concurrency: – It’s difficult! – Multiple hazards: • Race conditions 2
  7. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Have we

    solved concurrent programming, yet? The problem with concurrency: – It’s difficult! – Multiple hazards: • Race conditions • Deadlocks 2
  8. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Have we

    solved concurrent programming, yet? The problem with concurrency: – It’s difficult! – Multiple hazards: • Race conditions • Deadlocks • Livelocks 2
  9. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Have we

    solved concurrent programming, yet? The problem with concurrency: – It’s difficult! – Multiple hazards: • Race conditions • Deadlocks • Livelocks • Violation of fairness 2
  10. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Non-Determinism 3

  11. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Non-Determinism •

    At the root of several hazards 3
  12. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Non-Determinism •

    At the root of several hazards • Example 1: 3 @volatile var x = 0 def m(): Unit = { Future { x = 1 } Future { x = 2 } .. // does not access x }
  13. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Non-Determinism •

    At the root of several hazards • Example 1: 3 @volatile var x = 0 def m(): Unit = { Future { x = 1 } Future { x = 2 } .. // does not access x } What’s the value of x when an invocation of m returns?
  14. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Reordering not

    always a problem! • Example 2: 4
  15. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Reordering not

    always a problem! • Example 2: 4 import scala.collection.concurrent.TrieMap val set = new TrieMap[Int, Int] Future { set.put(1, 0) } set.put(2, 0)
  16. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Reordering not

    always a problem! • Example 2: 4 import scala.collection.concurrent.TrieMap val set = new TrieMap[Int, Int] Future { set.put(1, 0) } set.put(2, 0) Eventually, set contains both 1 and 2, always
  17. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Reordering not

    always a problem! • Example 2: 4 import scala.collection.concurrent.TrieMap val set = new TrieMap[Int, Int] Future { set.put(1, 0) } set.put(2, 0) Eventually, set contains both 1 and 2, always Bottom line: it depends on the datatype
  18. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller … and

    its operations! • Example 3: 5
  19. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller … and

    its operations! • Example 3: 5 val set = new TrieMap[Int, Int] Future { set.put(1, 0) } Future { if (set.contains(1)) { .. } } set.put(2, 0)
  20. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller … and

    its operations! • Example 3: 5 val set = new TrieMap[Int, Int] Future { set.put(1, 0) } Future { if (set.contains(1)) { .. } } set.put(2, 0) Result depends on schedule!
  21. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Important Questions

    6
  22. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Important Questions

    Q1:
 How do we know we have written a deterministic concurrent program? 6
  23. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Important Questions

    Q1:
 How do we know we have written a deterministic concurrent program? – What about Heisenbugs? 6
  24. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Important Questions

    Q1:
 How do we know we have written a deterministic concurrent program? – What about Heisenbugs? • No race in 1’000’000 runs, race in run 1’000’001 6
  25. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Important Questions

    Q1:
 How do we know we have written a deterministic concurrent program? – What about Heisenbugs? • No race in 1’000’000 runs, race in run 1’000’001 6 Goal: simple criteria that guarantee determinism
  26. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Important Questions

    7
  27. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Important Questions

    Q2:
 How do we ensure expressivity while providing determinism? 7
  28. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Important Questions

    Q2:
 How do we ensure expressivity while providing determinism? – Draconian restrictions undesired! 7
  29. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Important Questions

    Q2:
 How do we ensure expressivity while providing determinism? – Draconian restrictions undesired! 7 Goal: reconcile determinism and expressivity
  30. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Approach 8

  31. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Approach •

    Extend a future-style programming model with: 8
  32. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Approach •

    Extend a future-style programming model with: – Lattice-based datatypes 8
  33. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Approach •

    Extend a future-style programming model with: – Lattice-based datatypes – Quiescence 8
  34. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Approach •

    Extend a future-style programming model with: – Lattice-based datatypes – Quiescence 8 Crucial for determinism!
  35. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Approach •

    Extend a future-style programming model with: – Lattice-based datatypes – Quiescence – Resolution of cyclic dependencies 8 Crucial for determinism!
  36. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Approach •

    Extend a future-style programming model with: – Lattice-based datatypes – Quiescence – Resolution of cyclic dependencies 8 Crucial for determinism! Increases expressivity!
  37. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Approach •

    Extend a future-style programming model with: – Lattice-based datatypes – Quiescence – Resolution of cyclic dependencies • Evaluation of expressivity and performance – Case study: static analysis of JVM bytecode 8 Crucial for determinism! Increases expressivity!
  38. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Programming Model

    9
  39. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Programming Model

    Programming model based on two core abstractions:
 cells and cell completers 9
  40. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Programming Model

    Programming model based on two core abstractions:
 cells and cell completers – Cell = shared “variable” 9
  41. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Programming Model

    Programming model based on two core abstractions:
 cells and cell completers – Cell = shared “variable” 9 Concurrently read and written
  42. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Programming Model

    Programming model based on two core abstractions:
 cells and cell completers – Cell = shared “variable” – Cell[K,V]: read-only interface; read values of type V (akin to Future[V]) 9 Concurrently read and written
  43. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Programming Model

    Programming model based on two core abstractions:
 cells and cell completers – Cell = shared “variable” – Cell[K,V]: read-only interface; read values of type V (akin to Future[V]) – CellCompleter[K,V] : write values of type V to its associated cell (akin to Promise[V]) 9 Concurrently read and written
  44. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Programming Model

    Programming model based on two core abstractions:
 cells and cell completers – Cell = shared “variable” – Cell[K,V]: read-only interface; read values of type V (akin to Future[V]) – CellCompleter[K,V] : write values of type V to its associated cell (akin to Promise[V]) – V must have an instance of a lattice type class 9 Concurrently read and written
  45. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Programming Model

    Programming model based on two core abstractions:
 cells and cell completers – Cell = shared “variable” – Cell[K,V]: read-only interface; read values of type V (akin to Future[V]) – CellCompleter[K,V] : write values of type V to its associated cell (akin to Promise[V]) – V must have an instance of a lattice type class 9 Monotonic updates Concurrently read and written
  46. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Example 10

  47. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Example •

    Given a social graph 10
  48. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Example •

    Given a social graph – vertex = user 10
  49. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Example •

    Given a social graph – vertex = user • Traverse graph and collect IDs of “interesting” users 10
  50. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Example •

    Given a social graph – vertex = user • Traverse graph and collect IDs of “interesting” users • Graph large => concurrent traversal 10
  51. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Solution 11

  52. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Solution •

    Collect IDs of interesting users in a cell 11
  53. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Solution •

    Collect IDs of interesting users in a cell • Cell contains set of integers 11
  54. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Solution •

    Collect IDs of interesting users in a cell • Cell contains set of integers 11 OK: subsets of the set of all integers form a lattice
  55. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Cell with

    User IDs (code simplified) 12
  56. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Cell with

    User IDs (code simplified) 12 val userIDs = CellCompleter[Set[Int]]
  57. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Cell with

    User IDs (code simplified) 12 implicit object IntSetLattice extends Lattice[Set[Int]] { val empty = Set() def join(left: Set[Int], right: Set[Int]) = left ++ right } val userIDs = CellCompleter[Set[Int]]
  58. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Cell with

    User IDs (code simplified) 12 implicit object IntSetLattice extends Lattice[Set[Int]] { val empty = Set() def join(left: Set[Int], right: Set[Int]) = left ++ right } val userIDs = CellCompleter[Set[Int]] Bounded
 join-semilattice
  59. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Cell with

    User IDs (code simplified) 12 implicit object IntSetLattice extends Lattice[Set[Int]] { val empty = Set() def join(left: Set[Int], right: Set[Int]) = left ++ right } // add a user ID userIDs.putNext(Set(theUserID)) val userIDs = CellCompleter[Set[Int]] Bounded
 join-semilattice
  60. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Reading Results

    13
  61. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Reading Results

    • Problem: when reading a cell’s value, how do we know this value is not going to change any more? 13
  62. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Reading Results

    • Problem: when reading a cell’s value, how do we know this value is not going to change any more? – There may still be ongoing concurrent activities 13
  63. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Reading Results

    • Problem: when reading a cell’s value, how do we know this value is not going to change any more? – There may still be ongoing concurrent activities – Manual synchronization (e.g., latches) error-prone 13
  64. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Reading Results

    • Problem: when reading a cell’s value, how do we know this value is not going to change any more? – There may still be ongoing concurrent activities – Manual synchronization (e.g., latches) error-prone • Solution: 13
  65. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Reading Results

    • Problem: when reading a cell’s value, how do we know this value is not going to change any more? – There may still be ongoing concurrent activities – Manual synchronization (e.g., latches) error-prone • Solution: 13 Koyaanisqatsi
  66. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Reading Results

    • Problem: when reading a cell’s value, how do we know this value is not going to change any more? – There may still be ongoing concurrent activities – Manual synchronization (e.g., latches) error-prone • Solution: 13 Koyaanisqatsi
  67. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Reading Results

    • Problem: when reading a cell’s value, how do we know this value is not going to change any more? – There may still be ongoing concurrent activities – Manual synchronization (e.g., latches) error-prone • Solution: 13 Quiescence
  68. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Quiescence 14

  69. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Quiescence •

    Intuitively: situation when values of cells are guaranteed not to change any more 14
  70. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Quiescence •

    Intuitively: situation when values of cells are guaranteed not to change any more • Technically: 14
  71. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Quiescence •

    Intuitively: situation when values of cells are guaranteed not to change any more • Technically: – No concurrent activities ongoing or scheduled which could change values of cells 14
  72. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Quiescence •

    Intuitively: situation when values of cells are guaranteed not to change any more • Technically: – No concurrent activities ongoing or scheduled which could change values of cells – Detected by the underlying thread pool 14
  73. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Revisiting the

    Example 15
  74. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Revisiting the

    Example 15 val pool = new HandlerPool val userIDs = CellCompleter[Set[Int]](pool)
  75. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Revisiting the

    Example 15 // add a user ID userIDs.putNext(Set(theUserID)) .. val pool = new HandlerPool val userIDs = CellCompleter[Set[Int]](pool)
  76. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Revisiting the

    Example 15 // add a user ID userIDs.putNext(Set(theUserID)) .. val pool = new HandlerPool val userIDs = CellCompleter[Set[Int]](pool) // register handler // upon quiescence: read result value of cell pool.onQuiescent(userIDs.cell) { collectedIDs => .. }
  77. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Revisiting the

    Example 15 // add a user ID userIDs.putNext(Set(theUserID)) .. val pool = new HandlerPool val userIDs = CellCompleter[Set[Int]](pool) // register handler // upon quiescence: read result value of cell pool.onQuiescent(userIDs.cell) { collectedIDs => .. } Safe to read from cell when pool quiescent!
  78. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Handler Pools

    16
  79. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Handler Pools

    Execution context with extensions: 16
  80. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Handler Pools

    Execution context with extensions: – Event-driven quiescence API 16
  81. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Handler Pools

    Execution context with extensions: – Event-driven quiescence API – Resolution of dependencies between cells 16
  82. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Handler Pools

    Execution context with extensions: – Event-driven quiescence API – Resolution of dependencies between cells 16 Dependency resolution?
  83. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Expressing Concurrent

    Dataflow 17
  84. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Expressing Concurrent

    Dataflow • Cells provide non-blocking interface 17
  85. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Expressing Concurrent

    Dataflow • Cells provide non-blocking interface – Low level: callbacks 17
  86. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Expressing Concurrent

    Dataflow • Cells provide non-blocking interface – Low level: callbacks – High level: combinators 17
  87. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Expressing Concurrent

    Dataflow • Cells provide non-blocking interface – Low level: callbacks – High level: combinators 17 Like futures
  88. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Expressing Concurrent

    Dataflow • Cells provide non-blocking interface – Low level: callbacks – High level: combinators • Important purpose: express concurrent dataflow 17 Like futures
  89. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Expressing Concurrent

    Dataflow • Cells provide non-blocking interface – Low level: callbacks – High level: combinators • Important purpose: express concurrent dataflow 17 Like futures fut.map(fun).filter(pred)
  90. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Expressing Concurrent

    Dataflow • Cells provide non-blocking interface – Low level: callbacks – High level: combinators • Important purpose: express concurrent dataflow 17 Like futures fut.map(fun).filter(pred) Futures
  91. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Expressing Concurrent

    Dataflow • Cells provide non-blocking interface – Low level: callbacks – High level: combinators • Important purpose: express concurrent dataflow 17 Like futures fut fun fut’ pred fut’’ fut.map(fun).filter(pred) Futures
  92. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Expressing Concurrent

    Dataflow • Cells provide non-blocking interface – Low level: callbacks – High level: combinators • Important purpose: express concurrent dataflow • Problem: 17 Like futures fut fun fut’ pred fut’’ fut.map(fun).filter(pred) Futures
  93. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Expressing Concurrent

    Dataflow • Cells provide non-blocking interface – Low level: callbacks – High level: combinators • Important purpose: express concurrent dataflow • Problem: 17 What if the dataflow graph is cyclic? Like futures fut fun fut’ pred fut’’ fut.map(fun).filter(pred) Futures
  94. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Dataflow Graphs

    18
  95. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Dataflow Graphs

    Example: • Static analysis of JVM bytecode 18
  96. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Dataflow Graphs

    Example: • Static analysis of JVM bytecode • Task:
 Determine purity of methods 18
  97. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Dataflow Graphs

    Example: • Static analysis of JVM bytecode • Task:
 Determine purity of methods • Rules: 18
  98. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Dataflow Graphs

    Example: • Static analysis of JVM bytecode • Task:
 Determine purity of methods • Rules: – A method is impure if it accesses a non-final field 18
  99. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Dataflow Graphs

    Example: • Static analysis of JVM bytecode • Task:
 Determine purity of methods • Rules: – A method is impure if it accesses a non-final field – A method is impure if it calls an impure method 18
  100. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Dataflow Graphs

    Example: • Static analysis of JVM bytecode • Task:
 Determine purity of methods • Rules: – A method is impure if it accesses a non-final field – A method is impure if it calls an impure method – … 18
  101. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Dataflow Graphs

    Example: • Static analysis of JVM bytecode • Task:
 Determine purity of methods • Rules: – A method is impure if it accesses a non-final field – A method is impure if it calls an impure method – … 18 A
  102. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Dataflow Graphs

    Example: • Static analysis of JVM bytecode • Task:
 Determine purity of methods • Rules: – A method is impure if it accesses a non-final field – A method is impure if it calls an impure method – … 18 A B
  103. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Dataflow Graphs

    Example: • Static analysis of JVM bytecode • Task:
 Determine purity of methods • Rules: – A method is impure if it accesses a non-final field – A method is impure if it calls an impure method – … 18 A B “calls”
  104. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Dataflow Graphs

    Example: • Static analysis of JVM bytecode • Task:
 Determine purity of methods • Rules: – A method is impure if it accesses a non-final field – A method is impure if it calls an impure method – … 18 A B C “calls”
  105. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Dataflow Graphs

    Example: • Static analysis of JVM bytecode • Task:
 Determine purity of methods • Rules: – A method is impure if it accesses a non-final field – A method is impure if it calls an impure method – … 18 A B C “calls”
  106. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Cycle Resolution

    19
  107. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Cycle Resolution

    • What if upon quiescence cells are empty and there is a dependency cycle? 19
  108. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Cycle Resolution

    • What if upon quiescence cells are empty and there is a dependency cycle? 19 A B C
  109. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Cycle Resolution

    • What if upon quiescence cells are empty and there is a dependency cycle? • Idea:
 Pluggable resolution policies 19 A B C
  110. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Cycle Resolution

    • What if upon quiescence cells are empty and there is a dependency cycle? • Idea:
 Pluggable resolution policies • Example:
 Purity analysis should resolve all cells
 in cycle to “pure”, since could not show impurity 19 A B C
  111. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Specifying Resolution

    Policies 20
  112. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Specifying Resolution

    Policies • Role of the first type parameter of Cell[K,V] 20
  113. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Specifying Resolution

    Policies • Role of the first type parameter of Cell[K,V] • Example: 20
  114. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Specifying Resolution

    Policies • Role of the first type parameter of Cell[K,V] • Example: 20 object PurityKey extends Key[Purity] { def resolve[K <: Key[Purity]](cells: Seq[Cell[K, Purity]]) = cells.map(cell => (cell, Pure)) .. }
  115. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Specifying Resolution

    Policies • Role of the first type parameter of Cell[K,V] • Example: 20 object PurityKey extends Key[Purity] { def resolve[K <: Key[Purity]](cells: Seq[Cell[K, Purity]]) = cells.map(cell => (cell, Pure)) .. } resolve all cells to Pure
  116. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Specifying Resolution

    Policies • Role of the first type parameter of Cell[K,V] • Example: 20 object PurityKey extends Key[Purity] { def resolve[K <: Key[Purity]](cells: Seq[Cell[K, Purity]]) = cells.map(cell => (cell, Pure)) .. } closed strongly- connected component
  117. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Purity Analysis:

    Set-up 21
  118. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Purity Analysis:

    Set-up 21 for { classFile <- project.allProjectClassFiles method <- classFile.methods } { val methodCompleter = CellCompleter(pool, PurityKey) pool.execute { analyze(project, methodCompleter, classFile, method) } } val analysisDoneFuture = pool.quiescentResolveCells()
  119. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Evaluation 22

  120. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Evaluation •

    Static analysis of JVM bytecode using the OPAL framework (OPen extensible Analysis Library) 22
  121. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Evaluation •

    Static analysis of JVM bytecode using the OPAL framework (OPen extensible Analysis Library) – New Scala-based, extensible bytecode toolkit 22
  122. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Evaluation •

    Static analysis of JVM bytecode using the OPAL framework (OPen extensible Analysis Library) – New Scala-based, extensible bytecode toolkit – Fully concurrent 22
  123. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Evaluation •

    Static analysis of JVM bytecode using the OPAL framework (OPen extensible Analysis Library) – New Scala-based, extensible bytecode toolkit – Fully concurrent 22 http://www.opal-project.de
  124. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Evaluation •

    Static analysis of JVM bytecode using the OPAL framework (OPen extensible Analysis Library) – New Scala-based, extensible bytecode toolkit – Fully concurrent • Rewrote purity analysis and immutability analysis 22 http://www.opal-project.de
  125. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Evaluation •

    Static analysis of JVM bytecode using the OPAL framework (OPen extensible Analysis Library) – New Scala-based, extensible bytecode toolkit – Fully concurrent • Rewrote purity analysis and immutability analysis • Ran analysis on JDK 8 update 45 (rt.jar) 22 http://www.opal-project.de
  126. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Evaluation •

    Static analysis of JVM bytecode using the OPAL framework (OPen extensible Analysis Library) – New Scala-based, extensible bytecode toolkit – Fully concurrent • Rewrote purity analysis and immutability analysis • Ran analysis on JDK 8 update 45 (rt.jar) – 18’591 class files, 163’268 methods, 77’128 fields 22 http://www.opal-project.de
  127. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Results: Immutability

    Analysis • RA about 10x faster than FPCF • RA = 294 LOC, FPCF = 424 LOC (1.44x) 23 FPCF (secs.) 1.0 1.5 2.0 2.5 Reactive-Async (secs.) 0.1 0.2 0.3 1 Thread 2 Threads 4 Threads 8 Threads 16 Threads 2.15 2.20 2.25 2.30 2.35 1.15 1.20 1.25 1.30 1.35 0.290 0.295 0.300 0.105 0.110 0.115
  128. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Results: Immutability

    Analysis • RA about 10x faster than FPCF • RA = 294 LOC, FPCF = 424 LOC (1.44x) 23 FPCF (secs.) 1.0 1.5 2.0 2.5 Reactive-Async (secs.) 0.1 0.2 0.3 1 Thread 2 Threads 4 Threads 8 Threads 16 Threads 2.15 2.20 2.25 2.30 2.35 1.15 1.20 1.25 1.30 1.35 0.290 0.295 0.300 0.105 0.110 0.115 box plot: whiskers = min/max
 top/bottom of box = 
 1st and 3rd quartile band in box: median
  129. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Results: Immutability

    Analysis • RA about 10x faster than FPCF • RA = 294 LOC, FPCF = 424 LOC (1.44x) 23 FPCF (secs.) 1.0 1.5 2.0 2.5 Reactive-Async (secs.) 0.1 0.2 0.3 1 Thread 2 Threads 4 Threads 8 Threads 16 Threads 2.15 2.20 2.25 2.30 2.35 1.15 1.20 1.25 1.30 1.35 0.290 0.295 0.300 0.105 0.110 0.115 box plot: whiskers = min/max
 top/bottom of box = 
 1st and 3rd quartile band in box: median FPCF = OPAL’s fixed point computation framework
  130. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Results: Purity

    Analysis • Best configs. (FPCF: 4 thr, RA: 16 thr): RA 1.75x faster • RA = 113 LOC, FPCF = 166 LOC (1.47x) 24 FPCF (secs.) 0.15 0.20 0.25 0.30 0.35 Reactive-Async (secs.) 0.1 0.2 0.3 0.4 0.5 1 Thread 2 Threads 4 Threads 8 Threads 16 Threads 0.43 0.44 0.45 0.09 0.10 0.11 0.335 0.340 0.345 0.165 0.170 0.175 0.180 0.185 0.18 0.19 0.20
  131. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Benchmarks: Monte

    Carlo Simulation • Compute Net Present Value • 14-16 threads: Scala futures 23-36% faster than RA 25 Sequential Reactive Async Scala Futures 2.11.7 Runtime (ms) 0 1000 2000 3000 4000 5000 6000 Number of Threads 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 400 500 600 700 14 15 16
  132. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Benchmarks: Parallel

    Sum • Parallel sum of large collection of random ints • Performance competitive with Scala’s futures 26 Sequential Reactive Async Scala Futures 2.11.7 Runtime (ms) 0 50 100 150 200 250 300 Number of Threads 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
  133. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Related Work

    • Deterministic-by-construction programming models – KPNs, concurrent revisions, FlowPools, LVars, etc. • Static vs. dynamic enforcement of determinism • Reactive programming models – Dynamic dependencies, determinism • See paper for discussion 27
  134. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Reactive Async:

    Conclusion • New concurrent programming model – Lattices and quiescence for determinism – Resolution of cyclic dependencies • Promising experimental results • Future work: – Optimization – Determinism as a statically-checked effect 28
  135. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Reactive Async:

    Conclusion • New concurrent programming model – Lattices and quiescence for determinism – Resolution of cyclic dependencies • Promising experimental results • Future work: – Optimization – Determinism as a statically-checked effect 28 https://github.com/phaller/reactive-async
  136. Reactive Async: Expressive Deterministic Concurrency — Philipp Haller Reactive Async:

    Conclusion • New concurrent programming model – Lattices and quiescence for determinism – Resolution of cyclic dependencies • Promising experimental results • Future work: – Optimization – Determinism as a statically-checked effect 28 https://github.com/phaller/reactive-async Thank you!