High-Level Concurrency Libraries: Challenges for Tool Support

High-Level Concurrency Libraries: Challenges for Tools Philipp Haller! KTH Royal
Institute of Technology! Sweden! ! Eclipse Technology eXchange (ETX)! SPLASH, Pittsburgh, PA, USA, October 2015 1

Haller High-Level Concurrency Libraries: Challenges for Tools Context • 2005–2015:
author or co-author of a variety of concurrency libraries for Scala! • Some in wide industrial use, some experimental! • Common theme: how expressive are pure library approaches?! • High-level concurrency libraries ~= embedded domain-speciﬁc languages 2

Haller High-Level Concurrency Libraries: Challenges for Tools Disclaimers • Disclaimer
1: I am not an IDE expert! • A lot of work I am not aware of! • Disclaimer 2: my observations are biased toward certain languages and runtimes! • Statically-typed languages like Scala or Java! • Managed runtime environments à la JVM 3

Haller High-Level Concurrency Libraries: Challenges for Tools Experience • Developed
libraries:! • Scala Actors (2006)! • Scala Joins (2008)! • Scala Futures (2012)! • FlowPools (2012)! • Scala Async (2013)! • Exploration of Scala as a growable language 4

libraries:! • Scala Actors (2006)! • Scala Joins (2008)! • Scala Futures (2012)! • FlowPools (2012)! • Scala Async (2013)! • Exploration of Scala as a growable language 4 Macro library

libraries:! • Scala Actors (2006)! • Scala Joins (2008)! • Scala Futures (2012)! • FlowPools (2012)! • Scala Async (2013)! • Exploration of Scala as a growable language 4 Macro library Production use at Twitter, The Guardian and others

libraries:! • Scala Actors (2006)! • Scala Joins (2008)! • Scala Futures (2012)! • FlowPools (2012)! • Scala Async (2013)! • Exploration of Scala as a growable language 4 Macro library Production use at Twitter, The Guardian and others Production use at LinkedIn, Gilt Groupe and others

libraries:! • Scala Actors (2006)! • Scala Joins (2008)! • Scala Futures (2012)! • FlowPools (2012)! • Scala Async (2013)! • Exploration of Scala as a growable language 4 Production use at Twitter, The Guardian and others Production use at LinkedIn, Gilt Groupe and others Production use at Morgan Stanley, Gawker and others

libraries:! • Scala Actors (2006)! • Scala Joins (2008)! • Scala Futures (2012)! • FlowPools (2012)! • Scala Async (2013)! • Exploration of Scala as a growable language 4 Guy Steele. “Growing a Language.” OOPSLA ’98 Production use at Twitter, The Guardian and others Production use at LinkedIn, Gilt Groupe and others Production use at Morgan Stanley, Gawker and others

Haller High-Level Concurrency Libraries: Challenges for Tools Why Growable Languages
for Concurrency? • Actors, agents, communicating event-loops! • CML! • Futures, promises! • Async/await, async-ﬁnish! • Reactive Extensions (Rx)! • Join-calculus! • STM! • … 5

for Concurrency? • Actors, agents, communicating event-loops! • CML! • Futures, promises! • Async/await, async-ﬁnish! • Reactive Extensions (Rx)! • Join-calculus! • STM! • … 5 Which one is going to “win”?

for Concurrency? (cont’d) Enables multiple high-level libraries embedded in the same host language! • Richer programming system! • Enables reusing the compilers, debuggers, and IDEs of the host language 6

for Concurrency? (cont’d) 7

for Concurrency? (cont’d) Simpliﬁes research 7

for Concurrency? (cont’d) Simpliﬁes research • Library extensions vs. language extensions 7

for Concurrency? (cont’d) Simpliﬁes research • Library extensions vs. language extensions • Performance comparisons between different libraries more meaningful 7

for Concurrency? (cont’d) Simpliﬁes research • Library extensions vs. language extensions • Performance comparisons between different libraries more meaningful • Experimental evaluation on real code 7

Challenges for Tools 8

Haller High-Level Concurrency Libraries: Challenges for Tools New API Designs
9

• Unique integration of features: Scala enables new API designs 9

• Unique integration of features: Scala enables new API designs • Objects + functions, traits, path-dependent types 9

• Unique integration of features: Scala enables new API designs • Objects + functions, traits, path-dependent types • Pattern matching, extractors 9

• Unique integration of features: Scala enables new API designs • Objects + functions, traits, path-dependent types • Pattern matching, extractors • Implicits 9

• Unique integration of features: Scala enables new API designs • Objects + functions, traits, path-dependent types • Pattern matching, extractors • Implicits • ... 9

• Unique integration of features: Scala enables new API designs • Objects + functions, traits, path-dependent types • Pattern matching, extractors • Implicits • ... • Not all API designs well-supported by tools 9

Haller High-Level Concurrency Libraries: Challenges for Tools New API Designs:
Example 10

Example • Actors widely-used abstraction for concurrent programming in Scala 10

Example • Actors widely-used abstraction for concurrent programming in Scala • Actor = concurrent “process” that communicates via asynchronous messages 10

Example • Actors widely-used abstraction for concurrent programming in Scala • Actor = concurrent “process” that communicates via asynchronous messages • Asynchronous send, event-based receive 10

Example • Actors widely-used abstraction for concurrent programming in Scala • Actor = concurrent “process” that communicates via asynchronous messages • Asynchronous send, event-based receive • Low-level data races easy to avoid (by convention) 10

Example • Actors widely-used abstraction for concurrent programming in Scala • Actor = concurrent “process” that communicates via asynchronous messages • Asynchronous send, event-based receive • Low-level data races easy to avoid (by convention) • Debugging high-level messaging protocols can be difﬁcult 10

Haller High-Level Concurrency Libraries: Challenges for Tools Debugging Actor Programs
11 class TaskCoordinator extends Actor { ! var worker: Option[ActorRef] = None ! def receive = { case Start(workerRef) => this.worker = Some(workerRef) ! case Do(tasks) => assert(worker.nonEmpty) ! .. } ! }

11 class TaskCoordinator extends Actor { ! var worker: Option[ActorRef] = None ! def receive = { case Start(workerRef) => this.worker = Some(workerRef) ! case Do(tasks) => assert(worker.nonEmpty) ! .. } ! } Fails if ‘Do’ sent without prior ‘Start’

11 class TaskCoordinator extends Actor { ! var worker: Option[ActorRef] = None ! def receive = { case Start(workerRef) => this.worker = Some(workerRef) ! case Do(tasks) => assert(worker.nonEmpty) ! .. } ! } Fails if ‘Do’ sent without prior ‘Start’ “At what point was ‘Do’ message sent that caused assertion failure?”

Haller High-Level Concurrency Libraries: Challenges for Tools Problem • In
general, sender and receiver are executed by two different worker threads, on two different stacks! • Messages transferred using a concurrent queue of the receiver (its “mailbox”)! • Flow of control cannot be traced using traditional stack-based debuggers 12

Haller High-Level Concurrency Libraries: Challenges for Tools Solution: Stack Retention
13

Idea: 13

Idea: • At the point when a message is sent: 13

Idea: • At the point when a message is sent: • Save the call stack as well as stack frame information (e.g., local variable values) 13

Idea: • At the point when a message is sent: • Save the call stack as well as stack frame information (e.g., local variable values) • Attach the call stack to the sent message object 13

Idea: • At the point when a message is sent: • Save the call stack as well as stack frame information (e.g., local variable values) • Attach the call stack to the sent message object • When a user-deﬁned breakpoint is reached present the collected stack frames to the user (if they are relevant) 13

Idea: • At the point when a message is sent: • Save the call stack as well as stack frame information (e.g., local variable values) • Attach the call stack to the sent message object • When a user-deﬁned breakpoint is reached present the collected stack frames to the user (if they are relevant) 13 Iulian Dragos. “Stack Retention in Debuggers for Concurrent Programs.” Scala Workshop ’13

Haller High-Level Concurrency Libraries: Challenges for Tools Futures • Stack
retention also works for futures! • Call stack is saved whenever a future is created 14 val fut = Future { // concurrent computation } ! val transformed = fut.map { x => transform(x) } ! transformed.onComplete { case Success(v) => .. case Failure(t) => .. }

Haller High-Level Concurrency Libraries: Challenges for Tools Are We Done?
15

• Actors and futures not the only concurrency abstractions that should be supported 15

• Actors and futures not the only concurrency abstractions that should be supported • Streams, data-ﬂow, join patterns, etc. 15

• Actors and futures not the only concurrency abstractions that should be supported • Streams, data-ﬂow, join patterns, etc. • Effort to implement support for a given concurrency abstraction is high: 15

• Actors and futures not the only concurrency abstractions that should be supported • Streams, data-ﬂow, join patterns, etc. • Effort to implement support for a given concurrency abstraction is high: • First release of Scala Actors: 2006 15

• Actors and futures not the only concurrency abstractions that should be supported • Streams, data-ﬂow, join patterns, etc. • Effort to implement support for a given concurrency abstraction is high: • First release of Scala Actors: 2006 • First use of Scala Actors at Twitter: 2008 15

• Actors and futures not the only concurrency abstractions that should be supported • Streams, data-ﬂow, join patterns, etc. • Effort to implement support for a given concurrency abstraction is high: • First release of Scala Actors: 2006 • First use of Scala Actors at Twitter: 2008 • First release of Akka: 2009 15

• Actors and futures not the only concurrency abstractions that should be supported • Streams, data-ﬂow, join patterns, etc. • Effort to implement support for a given concurrency abstraction is high: • First release of Scala Actors: 2006 • First use of Scala Actors at Twitter: 2008 • First release of Akka: 2009 • Iulian Dragos ﬁrst presents Async debugger ideas: 2013 15 Iulian Dragos. “Stack Retention in Debuggers for Concurrent Programs.” Scala Workshop ’13

• Actors and futures not the only concurrency abstractions that should be supported • Streams, data-ﬂow, join patterns, etc. • Effort to implement support for a given concurrency abstraction is high: • First release of Scala Actors: 2006 • First use of Scala Actors at Twitter: 2008 • First release of Akka: 2009 • Iulian Dragos ﬁrst presents Async debugger ideas: 2013 • First release of Async debugger: 2015 15 Iulian Dragos. “Stack Retention in Debuggers for Concurrent Programs.” Scala Workshop ’13

• Actors and futures not the only concurrency abstractions that should be supported • Streams, data-ﬂow, join patterns, etc. • Effort to implement support for a given concurrency abstraction is high: • First release of Scala Actors: 2006 • First use of Scala Actors at Twitter: 2008 • First release of Akka: 2009 • Iulian Dragos ﬁrst presents Async debugger ideas: 2013 • First release of Async debugger: 2015 15 Iulian Dragos. “Stack Retention in Debuggers for Concurrent Programs.” Scala Workshop ’13 9 years!

Haller High-Level Concurrency Libraries: Challenges for Tools Issues 16

Haller High-Level Concurrency Libraries: Challenges for Tools Issues • Issue
1: Concurrency libraries often provide new control-ﬂow abstractions 16

1: Concurrency libraries often provide new control-ﬂow abstractions • Issue 2: Development of tool support decoupled from library implementation 16

1: Concurrency libraries often provide new control-ﬂow abstractions • Issue 2: Development of tool support decoupled from library implementation • Issue 3: High development effort means dedicated tool support only implemented for few libraries 16

Haller High-Level Concurrency Libraries: Challenges for Tools Issue 1 17

Haller High-Level Concurrency Libraries: Challenges for Tools Issue 1 •
Concurrency libraries often provide new abstractions for managing concurrent control ﬂow 17

Concurrency libraries often provide new abstractions for managing concurrent control ﬂow • Typically, control ﬂow spans several threads 17

Concurrency libraries often provide new abstractions for managing concurrent control ﬂow • Typically, control ﬂow spans several threads • Examples: actor multiplexing, future scheduling 17

Concurrency libraries often provide new abstractions for managing concurrent control flow • Typically, control flow spans several threads • Examples: actor multiplexing, future scheduling • Control-flow abstractions not necessarily implemented in terms of abstractions known to tools 17

Concurrency libraries often provide new abstractions for managing concurrent control flow • Typically, control flow spans several threads • Examples: actor multiplexing, future scheduling • Control-flow abstractions not necessarily implemented in terms of abstractions known to tools • Efficiency and scalability requirements usually preclude layering of abstractions 17

Haller High-Level Concurrency Libraries: Challenges for Tools Issue 1: Example
• Library deﬁnes new control-ﬂow abstractions! • Example: synchronous channel using a Joins library 18

• Library deﬁnes new control-ﬂow abstractions! • Example: synchronous channel using a Joins library 18 class SyncChannel extends Joins { object Read extends NullarySyncEvent[Int] object Write extends SyncEvent[Unit, Int] ! join { case Read() & Write(x) => Read reply x Write reply {} } } Philipp Haller and Tom Van Cutsem. “Implementing Joins using Extensible Pattern Matching.” Coordination ’08

• Library deﬁnes new control-ﬂow abstractions! • Example: synchronous channel using a Joins library 18 class SyncChannel extends Joins { object Read extends NullarySyncEvent[Int] object Write extends SyncEvent[Unit, Int] ! join { case Read() & Write(x) => Read reply x Write reply {} } } Events not transferred using actors or futures Philipp Haller and Tom Van Cutsem. “Implementing Joins using Extensible Pattern Matching.” Coordination ’08

Haller High-Level Concurrency Libraries: Challenges for Tools Issue 2 Development
of tool support decoupled from library implementation! • Library implementer most familiar with mechanics, but cannot support tools through implementation! • Tool developers have to reverse engineer library implementations 19

Haller High-Level Concurrency Libraries: Challenges for Tools Libraries with “Support
for Tools” • Futures in Scala include usability features! • Little implementation effort makes big difference 20

Haller High-Level Concurrency Libraries: Challenges for Tools Libraries with “Support
for Tools” • Futures in Scala include usability features! • Little implementation effort makes big difference 20 Welcome to Scala version 2.11.6 (Java HotSpot(TM) ..). Type in expressions to have them evaluated. Type :help for more information. ! scala> import scala.concurrent._ import scala.concurrent._ ! scala> val fut = Future { 40 + 2 } ! ! ! ! !

Haller High-Level Concurrency Libraries: Challenges for Tools <console>:10: error: Cannot
find an implicit ExecutionContext. You might pass Libraries with “Support for Tools” • Futures in Scala include usability features! • Little implementation effort makes big difference 20 Welcome to Scala version 2.11.6 (Java HotSpot(TM) ..). Type in expressions to have them evaluated. Type :help for more information. ! scala> import scala.concurrent._ import scala.concurrent._ ! scala> val fut = Future { 40 + 2 } ! ! ! ! !

find an implicit ExecutionContext. You might pass Libraries with “Support for Tools” • Futures in Scala include usability features! • Little implementation effort makes big difference 20 Welcome to Scala version 2.11.6 (Java HotSpot(TM) ..). Type in expressions to have them evaluated. Type :help for more information. ! scala> import scala.concurrent._ import scala.concurrent._ ! scala> val fut = Future { 40 + 2 } ! ! ! ! ! ???

find an implicit ExecutionContext. You might pass Libraries with “Support for Tools” • Futures in Scala include usability features! • Little implementation effort makes big difference 20 Welcome to Scala version 2.11.6 (Java HotSpot(TM) ..). Type in expressions to have them evaluated. Type :help for more information. ! scala> import scala.concurrent._ import scala.concurrent._ ! scala> val fut = Future { 40 + 2 } ! ! ! ! ! But…

find an implicit ExecutionContext. You might pass Libraries with “Support for Tools” • Futures in Scala include usability features! • Little implementation effort makes big difference 20 Welcome to Scala version 2.11.6 (Java HotSpot(TM) ..). Type in expressions to have them evaluated. Type :help for more information. ! scala> import scala.concurrent._ import scala.concurrent._ ! scala> val fut = Future { 40 + 2 } ! ! ! ! ! an (implicit ec: ExecutionContext) parameter to your method or import scala.concurrent.ExecutionContext.Implicits.global. val fut = Future { 40 + 2 } ^ But…

Haller High-Level Concurrency Libraries: Challenges for Tools Creating Futures 21
val fut = Future { .. } // equivalent to: val fut = Future.apply({ .. })

Haller High-Level Concurrency Libraries: Challenges for Tools Creating Futures 21
object Future { ! /** Starts an asynchronous computation and returns a `Future` object.. * * @tparam T the type of the result * @param body the asynchronous computation * @param executor the execution context on which the future is run * @return the `Future` holding the result of the computation */ def apply[T](body: =>T)(implicit executor: ExecutionContext): Future[T] = .. val fut = Future { .. } // equivalent to: val fut = Future.apply({ .. })

Haller High-Level Concurrency Libraries: Challenges for Tools Execution Contexts 22
/** * An `ExecutionContext` can execute program logic asynchronously, * typically but not necessarily on a thread pool. * * .. */ @implicitNotFound("""Cannot find an implicit ExecutionContext. You might pass an (implicit ec: ExecutionContext) parameter to your method or import scala.concurrent.ExecutionContext.Implicits.global.""") ! trait ExecutionContext { ! def execute(runnable: Runnable): Unit ! .. }

Haller High-Level Concurrency Libraries: Challenges for Tools Execution Contexts 22
/** * An `ExecutionContext` can execute program logic asynchronously, * typically but not necessarily on a thread pool. * * .. */ @implicitNotFound("""Cannot find an implicit ExecutionContext. You might pass an (implicit ec: ExecutionContext) parameter to your method or import scala.concurrent.ExecutionContext.Implicits.global.""") ! trait ExecutionContext { ! def execute(runnable: Runnable): Unit ! .. } With suitable mark- up, tools could extract quick ﬁxes automatically!

Haller High-Level Concurrency Libraries: Challenges for Tools Going Further: A
Proposal 23

Proposal • Library “explains” control ﬂow in form understandable by tools 23

Proposal • Library “explains” control ﬂow in form understandable by tools • Library implementers add annotations for: 23

Proposal • Library “explains” control ﬂow in form understandable by tools • Library implementers add annotations for: • Saving context information 23

Proposal • Library “explains” control ﬂow in form understandable by tools • Library implementers add annotations for: • Saving context information • Indicating which object to attach context information with 23

Proposal • Library “explains” control flow in form understandable by tools • Library implementers add annotations for: • Saving context information • Indicating which object to attach context information with • Async tools extract annotations and enable library- specific control-flow tracing 23

Haller High-Level Concurrency Libraries: Challenges for Tools Example 1 24

object Future { ! /** Starts an asynchronous computation and returns a `Future` object.. * */ @asyncSaveContext def apply[T](body: =>T)(implicit executor: ExecutionContext): Future[T] = ..

object Future { ! /** Starts an asynchronous computation and returns a `Future` object.. * */ @asyncSaveContext def apply[T](body: =>T)(implicit executor: ExecutionContext): Future[T] = .. • Save context at invocation site of this method • Attach context information with object returned by this method

abstract class ActorRef extends java.lang.Comparable[ActorRef] with Serializable { .. ! /** * Sends the specified message to this ActorRef, i.e. fire-‐and-‐forget * semantics, including the sender reference if possible. * * Pass [[akka.actor.ActorRef]] `noSender` or `null` as sender if there is * nobody to reply to */ @asyncSaveContext(msg) final def tell(msg: Any, sender: ActorRef): Unit = this.!(msg)(sender) ! ..

abstract class ActorRef extends java.lang.Comparable[ActorRef] with Serializable { .. ! /** * Sends the specified message to this ActorRef, i.e. fire-‐and-‐forget * semantics, including the sender reference if possible. * * Pass [[akka.actor.ActorRef]] `noSender` or `null` as sender if there is * nobody to reply to */ @asyncSaveContext(msg) final def tell(msg: Any, sender: ActorRef): Unit = this.!(msg)(sender) ! .. Akka actor framework

abstract class ActorRef extends java.lang.Comparable[ActorRef] with Serializable { .. ! /** * Sends the specified message to this ActorRef, i.e. fire-‐and-‐forget * semantics, including the sender reference if possible. * * Pass [[akka.actor.ActorRef]] `noSender` or `null` as sender if there is * nobody to reply to */ @asyncSaveContext(msg) final def tell(msg: Any, sender: ActorRef): Unit = this.!(msg)(sender) ! .. • Save context at invocation site of this method • Attach context information with argument `msg` Akka actor framework

Haller High-Level Concurrency Libraries: Challenges for Tools Summary • Low
implementation effort! • Leverages knowledge of library designers! • High payoff! • Still limited 26

Haller High-Level Concurrency Libraries: Challenges for Tools More Challenges 27

Haller High-Level Concurrency Libraries: Challenges for Tools More Challenges •
“Step over”, “step into”, etc. 27

“Step over”, “step into”, etc. • Lost meaning when “steps” cross thread boundaries 27

“Step over”, “step into”, etc. • Lost meaning when “steps” cross thread boundaries • What’s the most useful interpretations of “step over/into/return” for a given library? 27

“Step over”, “step into”, etc. • Lost meaning when “steps” cross thread boundaries • What’s the most useful interpretations of “step over/into/return” for a given library? • Atomicity of library abstractions 27

“Step over”, “step into”, etc. • Lost meaning when “steps” cross thread boundaries • What’s the most useful interpretations of “step over/into/return” for a given library? • Atomicity of library abstractions • Atomic methods OK: can step over. Otherwise? 27

“Step over”, “step into”, etc. • Lost meaning when “steps” cross thread boundaries • What’s the most useful interpretations of “step over/into/return” for a given library? • Atomicity of library abstractions • Atomic methods OK: can step over. Otherwise? • Combining concurrency abstractions 27

Haller High-Level Concurrency Libraries: Challenges for Tools Combining Libraries •
Integration of multiple concurrency libraries natural (to some extent)! • Developers will do it anyway (see ECOOP ’13)! • Have to play nicely together! • We discovered many challenges! • Most of them have implications for tools 28

Haller High-Level Concurrency Libraries: Challenges for Tools Example: Actors/Futures 29
class ActorWithWorkers(workers: ...) extends Actor { ... ! def receive = { case Do(tasks) => val requests = (tasks zip workers).map { case (task, worker) => worker ? task } val allDone = Future.sequence(requests) allDone andThen { seq => sender ! seq.mkString(",") } } }

Haller High-Level Concurrency Libraries: Challenges for Tools Fixed Version 30
class ActorWithWorkers(workers: ...) extends Actor { ... ! def receive = { case Do(tasks) => val from = sender ! val requests = (tasks zip workers).map { case (task, worker) => worker ? task } val allDone = Future.sequence(requests) allDone andThen { seq => from ! seq.mkString(",") } } }

Haller High-Level Concurrency Libraries: Challenges for Tools Fixed Version 30
class ActorWithWorkers(workers: ...) extends Actor { ... ! def receive = { case Do(tasks) => val from = sender ! val requests = (tasks zip workers).map { case (task, worker) => worker ? task } val allDone = Future.sequence(requests) allDone andThen { seq => from ! seq.mkString(",") } } } Tool support?

Haller High-Level Concurrency Libraries: Challenges for Tools Spores • Reﬁnement
of closures that demands an explicit environment! • Supports type-based constraints on captured variables 31

of closures that demands an explicit environment! • Supports type-based constraints on captured variables 31 case Do(tasks) => .. ! val allDone = Future.sequence(requests) allDone andThen spore { val from = sender seq => from ! seq.mkString(",") } Miller et al. Spores: A Type-Based Foundation for Closures in the Age of Concurrency and Distribution. ECOOP ‘14

of closures that demands an explicit environment! • Supports type-based constraints on captured variables 31 case Do(tasks) => .. ! val allDone = Future.sequence(requests) allDone andThen spore { val from = sender seq => from ! seq.mkString(",") } Miller et al. Spores: A Type-Based Foundation for Closures in the Age of Concurrency and Distribution. ECOOP ‘14 Using spores instead of closures disallows erroneous version!

Haller High-Level Concurrency Libraries: Challenges for Tools Conclusion • Concurrency
libraries present numerous challenges for development tools! • Both low-hanging fruit and major challenges! • Can we improve the interface between library and tool designers? 32

High-Level Concurrency Libraries: Challenges fo...

High-Level Concurrency Libraries: Challenges for Tool Support

More Decks by Philipp Haller

Featured

Transcript