Upgrade to Pro — share decks privately, control downloads, hide ads and more …

High-Level Concurrency Libraries: Challenges for Tool Support

Philipp Haller
October 27, 2015
34

High-Level Concurrency Libraries: Challenges for Tool Support

Philipp Haller

October 27, 2015
Tweet

Transcript

  1. High-Level Concurrency Libraries: Challenges for Tools Philipp Haller! KTH Royal

    Institute of Technology! Sweden! ! Eclipse Technology eXchange (ETX)! SPLASH, Pittsburgh, PA, USA, October 2015 1
  2. Haller High-Level Concurrency Libraries: Challenges for Tools Context • 2005–2015:

    author or co-author of a variety of concurrency libraries for Scala! • Some in wide industrial use, some experimental! • Common theme: how expressive are pure library approaches?! • High-level concurrency libraries ~= embedded domain-specific languages 2
  3. Haller High-Level Concurrency Libraries: Challenges for Tools Disclaimers • Disclaimer

    1: I am not an IDE expert! • A lot of work I am not aware of! • Disclaimer 2: my observations are biased toward certain languages and runtimes! • Statically-typed languages like Scala or Java! • Managed runtime environments à la JVM 3
  4. Haller High-Level Concurrency Libraries: Challenges for Tools Experience • Developed

    libraries:! • Scala Actors (2006)! • Scala Joins (2008)! • Scala Futures (2012)! • FlowPools (2012)! • Scala Async (2013)! • Exploration of Scala as a growable language 4
  5. Haller High-Level Concurrency Libraries: Challenges for Tools Experience • Developed

    libraries:! • Scala Actors (2006)! • Scala Joins (2008)! • Scala Futures (2012)! • FlowPools (2012)! • Scala Async (2013)! • Exploration of Scala as a growable language 4 Macro library
  6. Haller High-Level Concurrency Libraries: Challenges for Tools Experience • Developed

    libraries:! • Scala Actors (2006)! • Scala Joins (2008)! • Scala Futures (2012)! • FlowPools (2012)! • Scala Async (2013)! • Exploration of Scala as a growable language 4 Macro library Production use at Twitter, The Guardian and others
  7. Haller High-Level Concurrency Libraries: Challenges for Tools Experience • Developed

    libraries:! • Scala Actors (2006)! • Scala Joins (2008)! • Scala Futures (2012)! • FlowPools (2012)! • Scala Async (2013)! • Exploration of Scala as a growable language 4 Macro library Production use at Twitter, The Guardian and others Production use at LinkedIn, Gilt Groupe and others
  8. Haller High-Level Concurrency Libraries: Challenges for Tools Experience • Developed

    libraries:! • Scala Actors (2006)! • Scala Joins (2008)! • Scala Futures (2012)! • FlowPools (2012)! • Scala Async (2013)! • Exploration of Scala as a growable language 4 Production use at Twitter, The Guardian and others Production use at LinkedIn, Gilt Groupe and others Production use at Morgan Stanley, Gawker and others
  9. Haller High-Level Concurrency Libraries: Challenges for Tools Experience • Developed

    libraries:! • Scala Actors (2006)! • Scala Joins (2008)! • Scala Futures (2012)! • FlowPools (2012)! • Scala Async (2013)! • Exploration of Scala as a growable language 4 Guy Steele. “Growing a Language.” OOPSLA ’98 Production use at Twitter, The Guardian and others Production use at LinkedIn, Gilt Groupe and others Production use at Morgan Stanley, Gawker and others
  10. Haller High-Level Concurrency Libraries: Challenges for Tools Why Growable Languages

    for Concurrency? • Actors, agents, communicating event-loops! • CML! • Futures, promises! • Async/await, async-finish! • Reactive Extensions (Rx)! • Join-calculus! • STM! • … 5
  11. Haller High-Level Concurrency Libraries: Challenges for Tools Why Growable Languages

    for Concurrency? • Actors, agents, communicating event-loops! • CML! • Futures, promises! • Async/await, async-finish! • Reactive Extensions (Rx)! • Join-calculus! • STM! • … 5 Which one is going to “win”?
  12. Haller High-Level Concurrency Libraries: Challenges for Tools Why Growable Languages

    for Concurrency? (cont’d) Enables multiple high-level libraries embedded in the same host language! • Richer programming system! • Enables reusing the compilers, debuggers, and IDEs of the host language 6
  13. Haller High-Level Concurrency Libraries: Challenges for Tools Why Growable Languages

    for Concurrency? (cont’d) 7
  14. Haller High-Level Concurrency Libraries: Challenges for Tools Why Growable Languages

    for Concurrency? (cont’d) Simplifies research 7
  15. Haller High-Level Concurrency Libraries: Challenges for Tools Why Growable Languages

    for Concurrency? (cont’d) Simplifies research • Library extensions vs. language extensions 7
  16. Haller High-Level Concurrency Libraries: Challenges for Tools Why Growable Languages

    for Concurrency? (cont’d) Simplifies research • Library extensions vs. language extensions • Performance comparisons between different libraries more meaningful 7
  17. Haller High-Level Concurrency Libraries: Challenges for Tools Why Growable Languages

    for Concurrency? (cont’d) Simplifies research • Library extensions vs. language extensions • Performance comparisons between different libraries more meaningful • Experimental evaluation on real code 7
  18. Challenges for Tools 8

  19. Haller High-Level Concurrency Libraries: Challenges for Tools New API Designs

    9
  20. Haller High-Level Concurrency Libraries: Challenges for Tools New API Designs

    • Unique integration of features: Scala enables new API designs 9
  21. Haller High-Level Concurrency Libraries: Challenges for Tools New API Designs

    • Unique integration of features: Scala enables new API designs • Objects + functions, traits, path-dependent types 9
  22. Haller High-Level Concurrency Libraries: Challenges for Tools New API Designs

    • Unique integration of features: Scala enables new API designs • Objects + functions, traits, path-dependent types • Pattern matching, extractors 9
  23. Haller High-Level Concurrency Libraries: Challenges for Tools New API Designs

    • Unique integration of features: Scala enables new API designs • Objects + functions, traits, path-dependent types • Pattern matching, extractors • Implicits 9
  24. Haller High-Level Concurrency Libraries: Challenges for Tools New API Designs

    • Unique integration of features: Scala enables new API designs • Objects + functions, traits, path-dependent types • Pattern matching, extractors • Implicits • ... 9
  25. Haller High-Level Concurrency Libraries: Challenges for Tools New API Designs

    • Unique integration of features: Scala enables new API designs • Objects + functions, traits, path-dependent types • Pattern matching, extractors • Implicits • ... • Not all API designs well-supported by tools 9
  26. Haller High-Level Concurrency Libraries: Challenges for Tools New API Designs:

    Example 10
  27. Haller High-Level Concurrency Libraries: Challenges for Tools New API Designs:

    Example • Actors widely-used abstraction for concurrent programming in Scala 10
  28. Haller High-Level Concurrency Libraries: Challenges for Tools New API Designs:

    Example • Actors widely-used abstraction for concurrent programming in Scala • Actor = concurrent “process” that communicates via asynchronous messages 10
  29. Haller High-Level Concurrency Libraries: Challenges for Tools New API Designs:

    Example • Actors widely-used abstraction for concurrent programming in Scala • Actor = concurrent “process” that communicates via asynchronous messages • Asynchronous send, event-based receive 10
  30. Haller High-Level Concurrency Libraries: Challenges for Tools New API Designs:

    Example • Actors widely-used abstraction for concurrent programming in Scala • Actor = concurrent “process” that communicates via asynchronous messages • Asynchronous send, event-based receive • Low-level data races easy to avoid (by convention) 10
  31. Haller High-Level Concurrency Libraries: Challenges for Tools New API Designs:

    Example • Actors widely-used abstraction for concurrent programming in Scala • Actor = concurrent “process” that communicates via asynchronous messages • Asynchronous send, event-based receive • Low-level data races easy to avoid (by convention) • Debugging high-level messaging protocols can be difficult 10
  32. Haller High-Level Concurrency Libraries: Challenges for Tools Debugging Actor Programs

    11 class TaskCoordinator extends Actor { ! var worker: Option[ActorRef] = None ! def receive = { case Start(workerRef) => this.worker = Some(workerRef) ! case Do(tasks) => assert(worker.nonEmpty) ! .. } ! }
  33. Haller High-Level Concurrency Libraries: Challenges for Tools Debugging Actor Programs

    11 class TaskCoordinator extends Actor { ! var worker: Option[ActorRef] = None ! def receive = { case Start(workerRef) => this.worker = Some(workerRef) ! case Do(tasks) => assert(worker.nonEmpty) ! .. } ! } Fails if ‘Do’ sent without prior ‘Start’
  34. Haller High-Level Concurrency Libraries: Challenges for Tools Debugging Actor Programs

    11 class TaskCoordinator extends Actor { ! var worker: Option[ActorRef] = None ! def receive = { case Start(workerRef) => this.worker = Some(workerRef) ! case Do(tasks) => assert(worker.nonEmpty) ! .. } ! } Fails if ‘Do’ sent without prior ‘Start’ “At what point was ‘Do’ message sent that caused assertion failure?”
  35. Haller High-Level Concurrency Libraries: Challenges for Tools Problem • In

    general, sender and receiver are executed by two different worker threads, on two different stacks! • Messages transferred using a concurrent queue of the receiver (its “mailbox”)! • Flow of control cannot be traced using traditional stack-based debuggers 12
  36. Haller High-Level Concurrency Libraries: Challenges for Tools Solution: Stack Retention

    13
  37. Haller High-Level Concurrency Libraries: Challenges for Tools Solution: Stack Retention

    Idea: 13
  38. Haller High-Level Concurrency Libraries: Challenges for Tools Solution: Stack Retention

    Idea: • At the point when a message is sent: 13
  39. Haller High-Level Concurrency Libraries: Challenges for Tools Solution: Stack Retention

    Idea: • At the point when a message is sent: • Save the call stack as well as stack frame information (e.g., local variable values) 13
  40. Haller High-Level Concurrency Libraries: Challenges for Tools Solution: Stack Retention

    Idea: • At the point when a message is sent: • Save the call stack as well as stack frame information (e.g., local variable values) • Attach the call stack to the sent message object 13
  41. Haller High-Level Concurrency Libraries: Challenges for Tools Solution: Stack Retention

    Idea: • At the point when a message is sent: • Save the call stack as well as stack frame information (e.g., local variable values) • Attach the call stack to the sent message object • When a user-defined breakpoint is reached present the collected stack frames to the user (if they are relevant) 13
  42. Haller High-Level Concurrency Libraries: Challenges for Tools Solution: Stack Retention

    Idea: • At the point when a message is sent: • Save the call stack as well as stack frame information (e.g., local variable values) • Attach the call stack to the sent message object • When a user-defined breakpoint is reached present the collected stack frames to the user (if they are relevant) 13 Iulian Dragos. “Stack Retention in Debuggers for Concurrent Programs.” Scala Workshop ’13
  43. Haller High-Level Concurrency Libraries: Challenges for Tools Futures • Stack

    retention also works for futures! • Call stack is saved whenever a future is created 14 val fut = Future { // concurrent computation } ! val transformed = fut.map { x => transform(x) } ! transformed.onComplete { case Success(v) => .. case Failure(t) => .. }
  44. Haller High-Level Concurrency Libraries: Challenges for Tools Are We Done?

    15
  45. Haller High-Level Concurrency Libraries: Challenges for Tools Are We Done?

    • Actors and futures not the only concurrency abstractions that should be supported 15
  46. Haller High-Level Concurrency Libraries: Challenges for Tools Are We Done?

    • Actors and futures not the only concurrency abstractions that should be supported • Streams, data-flow, join patterns, etc. 15
  47. Haller High-Level Concurrency Libraries: Challenges for Tools Are We Done?

    • Actors and futures not the only concurrency abstractions that should be supported • Streams, data-flow, join patterns, etc. • Effort to implement support for a given concurrency abstraction is high: 15
  48. Haller High-Level Concurrency Libraries: Challenges for Tools Are We Done?

    • Actors and futures not the only concurrency abstractions that should be supported • Streams, data-flow, join patterns, etc. • Effort to implement support for a given concurrency abstraction is high: • First release of Scala Actors: 2006 15
  49. Haller High-Level Concurrency Libraries: Challenges for Tools Are We Done?

    • Actors and futures not the only concurrency abstractions that should be supported • Streams, data-flow, join patterns, etc. • Effort to implement support for a given concurrency abstraction is high: • First release of Scala Actors: 2006 • First use of Scala Actors at Twitter: 2008 15
  50. Haller High-Level Concurrency Libraries: Challenges for Tools Are We Done?

    • Actors and futures not the only concurrency abstractions that should be supported • Streams, data-flow, join patterns, etc. • Effort to implement support for a given concurrency abstraction is high: • First release of Scala Actors: 2006 • First use of Scala Actors at Twitter: 2008 • First release of Akka: 2009 15
  51. Haller High-Level Concurrency Libraries: Challenges for Tools Are We Done?

    • Actors and futures not the only concurrency abstractions that should be supported • Streams, data-flow, join patterns, etc. • Effort to implement support for a given concurrency abstraction is high: • First release of Scala Actors: 2006 • First use of Scala Actors at Twitter: 2008 • First release of Akka: 2009 • Iulian Dragos first presents Async debugger ideas: 2013 15 Iulian Dragos. “Stack Retention in Debuggers for Concurrent Programs.” Scala Workshop ’13
  52. Haller High-Level Concurrency Libraries: Challenges for Tools Are We Done?

    • Actors and futures not the only concurrency abstractions that should be supported • Streams, data-flow, join patterns, etc. • Effort to implement support for a given concurrency abstraction is high: • First release of Scala Actors: 2006 • First use of Scala Actors at Twitter: 2008 • First release of Akka: 2009 • Iulian Dragos first presents Async debugger ideas: 2013 • First release of Async debugger: 2015 15 Iulian Dragos. “Stack Retention in Debuggers for Concurrent Programs.” Scala Workshop ’13
  53. Haller High-Level Concurrency Libraries: Challenges for Tools Are We Done?

    • Actors and futures not the only concurrency abstractions that should be supported • Streams, data-flow, join patterns, etc. • Effort to implement support for a given concurrency abstraction is high: • First release of Scala Actors: 2006 • First use of Scala Actors at Twitter: 2008 • First release of Akka: 2009 • Iulian Dragos first presents Async debugger ideas: 2013 • First release of Async debugger: 2015 15 Iulian Dragos. “Stack Retention in Debuggers for Concurrent Programs.” Scala Workshop ’13 9 years!
  54. Haller High-Level Concurrency Libraries: Challenges for Tools Issues 16

  55. Haller High-Level Concurrency Libraries: Challenges for Tools Issues • Issue

    1: Concurrency libraries often provide new control-flow abstractions 16
  56. Haller High-Level Concurrency Libraries: Challenges for Tools Issues • Issue

    1: Concurrency libraries often provide new control-flow abstractions • Issue 2: Development of tool support decoupled from library implementation 16
  57. Haller High-Level Concurrency Libraries: Challenges for Tools Issues • Issue

    1: Concurrency libraries often provide new control-flow abstractions • Issue 2: Development of tool support decoupled from library implementation • Issue 3: High development effort means dedicated tool support only implemented for few libraries 16
  58. Haller High-Level Concurrency Libraries: Challenges for Tools Issue 1 17

  59. Haller High-Level Concurrency Libraries: Challenges for Tools Issue 1 •

    Concurrency libraries often provide new abstractions for managing concurrent control flow 17
  60. Haller High-Level Concurrency Libraries: Challenges for Tools Issue 1 •

    Concurrency libraries often provide new abstractions for managing concurrent control flow • Typically, control flow spans several threads 17
  61. Haller High-Level Concurrency Libraries: Challenges for Tools Issue 1 •

    Concurrency libraries often provide new abstractions for managing concurrent control flow • Typically, control flow spans several threads • Examples: actor multiplexing, future scheduling 17
  62. Haller High-Level Concurrency Libraries: Challenges for Tools Issue 1 •

    Concurrency libraries often provide new abstractions for managing concurrent control flow • Typically, control flow spans several threads • Examples: actor multiplexing, future scheduling • Control-flow abstractions not necessarily implemented in terms of abstractions known to tools 17
  63. Haller High-Level Concurrency Libraries: Challenges for Tools Issue 1 •

    Concurrency libraries often provide new abstractions for managing concurrent control flow • Typically, control flow spans several threads • Examples: actor multiplexing, future scheduling • Control-flow abstractions not necessarily implemented in terms of abstractions known to tools • Efficiency and scalability requirements usually preclude layering of abstractions 17
  64. Haller High-Level Concurrency Libraries: Challenges for Tools Issue 1: Example

    • Library defines new control-flow abstractions! • Example: synchronous channel using a Joins library 18
  65. Haller High-Level Concurrency Libraries: Challenges for Tools Issue 1: Example

    • Library defines new control-flow abstractions! • Example: synchronous channel using a Joins library 18 class SyncChannel extends Joins { object Read extends NullarySyncEvent[Int] object Write extends SyncEvent[Unit, Int] ! join { case Read() & Write(x) => Read reply x Write reply {} } } Philipp Haller and Tom Van Cutsem. “Implementing Joins using Extensible Pattern Matching.” Coordination ’08
  66. Haller High-Level Concurrency Libraries: Challenges for Tools Issue 1: Example

    • Library defines new control-flow abstractions! • Example: synchronous channel using a Joins library 18 class SyncChannel extends Joins { object Read extends NullarySyncEvent[Int] object Write extends SyncEvent[Unit, Int] ! join { case Read() & Write(x) => Read reply x Write reply {} } } Events not transferred using actors or futures Philipp Haller and Tom Van Cutsem. “Implementing Joins using Extensible Pattern Matching.” Coordination ’08
  67. Haller High-Level Concurrency Libraries: Challenges for Tools Issue 2 Development

    of tool support decoupled from library implementation! • Library implementer most familiar with mechanics, but cannot support tools through implementation! • Tool developers have to reverse engineer library implementations 19
  68. Haller High-Level Concurrency Libraries: Challenges for Tools Libraries with “Support

    for Tools” • Futures in Scala include usability features! • Little implementation effort makes big difference 20
  69. Haller High-Level Concurrency Libraries: Challenges for Tools Libraries with “Support

    for Tools” • Futures in Scala include usability features! • Little implementation effort makes big difference 20 Welcome to Scala version 2.11.6 (Java HotSpot(TM) ..). Type in expressions to have them evaluated. Type :help for more information. ! scala> import scala.concurrent._ import scala.concurrent._ ! scala> val fut = Future { 40 + 2 } ! ! ! ! !
  70. Haller High-Level Concurrency Libraries: Challenges for Tools <console>:10: error: Cannot

    find an implicit ExecutionContext. You might pass Libraries with “Support for Tools” • Futures in Scala include usability features! • Little implementation effort makes big difference 20 Welcome to Scala version 2.11.6 (Java HotSpot(TM) ..). Type in expressions to have them evaluated. Type :help for more information. ! scala> import scala.concurrent._ import scala.concurrent._ ! scala> val fut = Future { 40 + 2 } ! ! ! ! !
  71. Haller High-Level Concurrency Libraries: Challenges for Tools <console>:10: error: Cannot

    find an implicit ExecutionContext. You might pass Libraries with “Support for Tools” • Futures in Scala include usability features! • Little implementation effort makes big difference 20 Welcome to Scala version 2.11.6 (Java HotSpot(TM) ..). Type in expressions to have them evaluated. Type :help for more information. ! scala> import scala.concurrent._ import scala.concurrent._ ! scala> val fut = Future { 40 + 2 } ! ! ! ! ! ???
  72. Haller High-Level Concurrency Libraries: Challenges for Tools <console>:10: error: Cannot

    find an implicit ExecutionContext. You might pass Libraries with “Support for Tools” • Futures in Scala include usability features! • Little implementation effort makes big difference 20 Welcome to Scala version 2.11.6 (Java HotSpot(TM) ..). Type in expressions to have them evaluated. Type :help for more information. ! scala> import scala.concurrent._ import scala.concurrent._ ! scala> val fut = Future { 40 + 2 } ! ! ! ! ! But…
  73. Haller High-Level Concurrency Libraries: Challenges for Tools <console>:10: error: Cannot

    find an implicit ExecutionContext. You might pass Libraries with “Support for Tools” • Futures in Scala include usability features! • Little implementation effort makes big difference 20 Welcome to Scala version 2.11.6 (Java HotSpot(TM) ..). Type in expressions to have them evaluated. Type :help for more information. ! scala> import scala.concurrent._ import scala.concurrent._ ! scala> val fut = Future { 40 + 2 } ! ! ! ! ! an (implicit ec: ExecutionContext) parameter to your method or import scala.concurrent.ExecutionContext.Implicits.global. val fut = Future { 40 + 2 } ^ But…
  74. Haller High-Level Concurrency Libraries: Challenges for Tools Creating Futures 21

    val fut = Future { .. } // equivalent to: val fut = Future.apply({ .. })
  75. Haller High-Level Concurrency Libraries: Challenges for Tools Creating Futures 21

    object Future { ! /** Starts an asynchronous computation and returns a `Future` object.. * * @tparam T the type of the result * @param body the asynchronous computation * @param executor the execution context on which the future is run * @return the `Future` holding the result of the computation */ def apply[T](body: =>T)(implicit executor: ExecutionContext): Future[T] = .. val fut = Future { .. } // equivalent to: val fut = Future.apply({ .. })
  76. Haller High-Level Concurrency Libraries: Challenges for Tools Execution Contexts 22

    /** * An `ExecutionContext` can execute program logic asynchronously, * typically but not necessarily on a thread pool. * * .. */ @implicitNotFound("""Cannot find an implicit ExecutionContext. You might pass an (implicit ec: ExecutionContext) parameter to your method or import scala.concurrent.ExecutionContext.Implicits.global.""") ! trait ExecutionContext { ! def execute(runnable: Runnable): Unit ! .. }
  77. Haller High-Level Concurrency Libraries: Challenges for Tools Execution Contexts 22

    /** * An `ExecutionContext` can execute program logic asynchronously, * typically but not necessarily on a thread pool. * * .. */ @implicitNotFound("""Cannot find an implicit ExecutionContext. You might pass an (implicit ec: ExecutionContext) parameter to your method or import scala.concurrent.ExecutionContext.Implicits.global.""") ! trait ExecutionContext { ! def execute(runnable: Runnable): Unit ! .. }
  78. Haller High-Level Concurrency Libraries: Challenges for Tools Execution Contexts 22

    /** * An `ExecutionContext` can execute program logic asynchronously, * typically but not necessarily on a thread pool. * * .. */ @implicitNotFound("""Cannot find an implicit ExecutionContext. You might pass an (implicit ec: ExecutionContext) parameter to your method or import scala.concurrent.ExecutionContext.Implicits.global.""") ! trait ExecutionContext { ! def execute(runnable: Runnable): Unit ! .. } With suitable mark- up, tools could extract quick fixes automatically!
  79. Haller High-Level Concurrency Libraries: Challenges for Tools Going Further: A

    Proposal 23
  80. Haller High-Level Concurrency Libraries: Challenges for Tools Going Further: A

    Proposal • Library “explains” control flow in form understandable by tools 23
  81. Haller High-Level Concurrency Libraries: Challenges for Tools Going Further: A

    Proposal • Library “explains” control flow in form understandable by tools • Library implementers add annotations for: 23
  82. Haller High-Level Concurrency Libraries: Challenges for Tools Going Further: A

    Proposal • Library “explains” control flow in form understandable by tools • Library implementers add annotations for: • Saving context information 23
  83. Haller High-Level Concurrency Libraries: Challenges for Tools Going Further: A

    Proposal • Library “explains” control flow in form understandable by tools • Library implementers add annotations for: • Saving context information • Indicating which object to attach context information with 23
  84. Haller High-Level Concurrency Libraries: Challenges for Tools Going Further: A

    Proposal • Library “explains” control flow in form understandable by tools • Library implementers add annotations for: • Saving context information • Indicating which object to attach context information with • Async tools extract annotations and enable library- specific control-flow tracing 23
  85. Haller High-Level Concurrency Libraries: Challenges for Tools Example 1 24

  86. Haller High-Level Concurrency Libraries: Challenges for Tools Example 1 24

    object Future { ! /** Starts an asynchronous computation and returns a `Future` object.. * */ @asyncSaveContext def apply[T](body: =>T)(implicit executor: ExecutionContext): Future[T] = ..
  87. Haller High-Level Concurrency Libraries: Challenges for Tools Example 1 24

    object Future { ! /** Starts an asynchronous computation and returns a `Future` object.. * */ @asyncSaveContext def apply[T](body: =>T)(implicit executor: ExecutionContext): Future[T] = .. • Save context at invocation site of this method • Attach context information with object returned by this method
  88. Haller High-Level Concurrency Libraries: Challenges for Tools Example 2 25

  89. Haller High-Level Concurrency Libraries: Challenges for Tools Example 2 25

    abstract class ActorRef extends java.lang.Comparable[ActorRef] with Serializable { .. ! /** * Sends the specified message to this ActorRef, i.e. fire-­‐and-­‐forget * semantics, including the sender reference if possible. * * Pass [[akka.actor.ActorRef]] `noSender` or `null` as sender if there is * nobody to reply to */ @asyncSaveContext(msg) final def tell(msg: Any, sender: ActorRef): Unit = this.!(msg)(sender) ! ..
  90. Haller High-Level Concurrency Libraries: Challenges for Tools Example 2 25

    abstract class ActorRef extends java.lang.Comparable[ActorRef] with Serializable { .. ! /** * Sends the specified message to this ActorRef, i.e. fire-­‐and-­‐forget * semantics, including the sender reference if possible. * * Pass [[akka.actor.ActorRef]] `noSender` or `null` as sender if there is * nobody to reply to */ @asyncSaveContext(msg) final def tell(msg: Any, sender: ActorRef): Unit = this.!(msg)(sender) ! .. Akka actor framework
  91. Haller High-Level Concurrency Libraries: Challenges for Tools Example 2 25

    abstract class ActorRef extends java.lang.Comparable[ActorRef] with Serializable { .. ! /** * Sends the specified message to this ActorRef, i.e. fire-­‐and-­‐forget * semantics, including the sender reference if possible. * * Pass [[akka.actor.ActorRef]] `noSender` or `null` as sender if there is * nobody to reply to */ @asyncSaveContext(msg) final def tell(msg: Any, sender: ActorRef): Unit = this.!(msg)(sender) ! .. • Save context at invocation site of this method • Attach context information with argument `msg` Akka actor framework
  92. Haller High-Level Concurrency Libraries: Challenges for Tools Summary • Low

    implementation effort! • Leverages knowledge of library designers! • High payoff! • Still limited 26
  93. Haller High-Level Concurrency Libraries: Challenges for Tools More Challenges 27

  94. Haller High-Level Concurrency Libraries: Challenges for Tools More Challenges •

    “Step over”, “step into”, etc. 27
  95. Haller High-Level Concurrency Libraries: Challenges for Tools More Challenges •

    “Step over”, “step into”, etc. • Lost meaning when “steps” cross thread boundaries 27
  96. Haller High-Level Concurrency Libraries: Challenges for Tools More Challenges •

    “Step over”, “step into”, etc. • Lost meaning when “steps” cross thread boundaries • What’s the most useful interpretations of “step over/into/return” for a given library? 27
  97. Haller High-Level Concurrency Libraries: Challenges for Tools More Challenges •

    “Step over”, “step into”, etc. • Lost meaning when “steps” cross thread boundaries • What’s the most useful interpretations of “step over/into/return” for a given library? • Atomicity of library abstractions 27
  98. Haller High-Level Concurrency Libraries: Challenges for Tools More Challenges •

    “Step over”, “step into”, etc. • Lost meaning when “steps” cross thread boundaries • What’s the most useful interpretations of “step over/into/return” for a given library? • Atomicity of library abstractions • Atomic methods OK: can step over. Otherwise? 27
  99. Haller High-Level Concurrency Libraries: Challenges for Tools More Challenges •

    “Step over”, “step into”, etc. • Lost meaning when “steps” cross thread boundaries • What’s the most useful interpretations of “step over/into/return” for a given library? • Atomicity of library abstractions • Atomic methods OK: can step over. Otherwise? • Combining concurrency abstractions 27
  100. Haller High-Level Concurrency Libraries: Challenges for Tools Combining Libraries •

    Integration of multiple concurrency libraries natural (to some extent)! • Developers will do it anyway (see ECOOP ’13)! • Have to play nicely together! • We discovered many challenges! • Most of them have implications for tools 28
  101. Haller High-Level Concurrency Libraries: Challenges for Tools Example: Actors/Futures 29

    class ActorWithWorkers(workers: ...) extends Actor { ... ! def receive = { case Do(tasks) => val requests = (tasks zip workers).map { case (task, worker) => worker ? task } val allDone = Future.sequence(requests) allDone andThen { seq => sender ! seq.mkString(",") } } }
  102. Haller High-Level Concurrency Libraries: Challenges for Tools Example: Actors/Futures 29

    class ActorWithWorkers(workers: ...) extends Actor { ... ! def receive = { case Do(tasks) => val requests = (tasks zip workers).map { case (task, worker) => worker ? task } val allDone = Future.sequence(requests) allDone andThen { seq => sender ! seq.mkString(",") } } }
  103. Haller High-Level Concurrency Libraries: Challenges for Tools Fixed Version 30

    class ActorWithWorkers(workers: ...) extends Actor { ... ! def receive = { case Do(tasks) => val from = sender ! val requests = (tasks zip workers).map { case (task, worker) => worker ? task } val allDone = Future.sequence(requests) allDone andThen { seq => from ! seq.mkString(",") } } }
  104. Haller High-Level Concurrency Libraries: Challenges for Tools Fixed Version 30

    class ActorWithWorkers(workers: ...) extends Actor { ... ! def receive = { case Do(tasks) => val from = sender ! val requests = (tasks zip workers).map { case (task, worker) => worker ? task } val allDone = Future.sequence(requests) allDone andThen { seq => from ! seq.mkString(",") } } } Tool support?
  105. Haller High-Level Concurrency Libraries: Challenges for Tools Spores • Refinement

    of closures that demands an explicit environment! • Supports type-based constraints on captured variables 31
  106. Haller High-Level Concurrency Libraries: Challenges for Tools Spores • Refinement

    of closures that demands an explicit environment! • Supports type-based constraints on captured variables 31 case Do(tasks) => .. ! val allDone = Future.sequence(requests) allDone andThen spore { val from = sender seq => from ! seq.mkString(",") } Miller et al. Spores: A Type-Based Foundation for Closures in the Age of Concurrency and Distribution. ECOOP ‘14
  107. Haller High-Level Concurrency Libraries: Challenges for Tools Spores • Refinement

    of closures that demands an explicit environment! • Supports type-based constraints on captured variables 31 case Do(tasks) => .. ! val allDone = Future.sequence(requests) allDone andThen spore { val from = sender seq => from ! seq.mkString(",") } Miller et al. Spores: A Type-Based Foundation for Closures in the Age of Concurrency and Distribution. ECOOP ‘14 Using spores instead of closures disallows erroneous version!
  108. Haller High-Level Concurrency Libraries: Challenges for Tools Conclusion • Concurrency

    libraries present numerous challenges for development tools! • Both low-hanging fruit and major challenges! • Can we improve the interface between library and tool designers? 32