Concurrency Patterns

How to achieve concurrent processing and how to deal with it programmatically in elegant ways? Slides of a presentation given to the students of the software engineering course at U. Nijmegen.

Transcript

1. Concurrency patterns Georgios Gousios
2. Concurrency and Parallelism 4 Concurrency: doing many things at the same time 4 Requests from multiple clients on a server 4 Parallelism: doing one thing on multiple processors 4 Big matrix multiplication on CPUs/GPUs
3. Enemies of concurrency 4 State 4 Sharing
4. Common concurrency patterns 4 Processes 4 Threads 4 Event loops 4 Actors 4 Asynchronous processing 4 Streams and reactive programming
5. Processes 4 Same code, different memory space 4 Always have a parent, that forks(3) them # Shared memory (owned by the parent) children = [] pid = Process::fork if pid == 0 puts pid, a # Child specific memory do_work() else puts pid, a # Parent specific memory children += pid end children.each do |pid| Process.waitpid(pid, 0) end
6. Processes — when to use 4 Simplest concurrency pattern 4 fork(3) is expensive 4 Large work units, taking significant time to setup 4 (usually) not per request 4 Can be used in combination with finer- grained options (e.g. event loops)
7. Threads 4 A self-executing unit of computation within a bigger group of computations 4 OS or application based 4 N x M mapping called ‘green threading’ 4 Access same memory with initiator 4 Need concurrency primitives to avoid races
8. Threads example t = Thread.new do puts 'Tock' end puts 'Tick' t.join
9. Thread-based patterns 4 Producer/consumer queue 4 Thread pool 4 Fork/join
10. Producer/consumer queue 4 In memory blocking queue 4 Identical work items queue = Queue.new producer = Thread.new do 5.times do |i| queue << i end end consumer = Thread.new do while true do |i| value = queue.pop process value end end consumer.join
11. Thread pool A collection of initialised threads waiting for work to be assigned to them
12. Thread pool class Pool def initialize(size) @size = size @jobs = Queue.new @pool = Array.new(@size) do |i| Thread.new do Thread.current[:id] = i catch(:exit) do loop do job, args = @jobs.pop job.call(*args) end end end end def schedule(*args, &block) @jobs << [block, args] end end
13. Fork/Join solve(problem): if problem is small enough: solve problem directly else: for part in subdivide fork subtask to solve part join all subtasks spawned in previous loop combine results from subtasks
14. Threads — when to use 4 Reasoning about correctness with threads is hard 4 Especially when other isolation primitives (e.g. transactions) are involved 4 When one of the ready made patterns work 4 Better use higher level abstractions
15. Event loops 4 Basic pattern 4 Wait on a file descriptor to emit a result 4 Run list of registered callbacks 4 More advanced 4 Wait on multiple file descriptors 4 Schedule background work in threads 4 An event loop is always single threaded
16. Basic event loop fd = File.open("/var/log/messages.log") callbacks = [Proc.new{|x| print x}] while true do active = select([fd]) data = fd.read_lines callbacks.each{|x| x.invoke(data)} end
17. Pragmatic event loops
18. Event loops — where is the concurrency? 4 I/O is 10^4 - 10^6 slower than processing 4 On most systems, most time is spent waiting 4 select(3) or epoll(3) or kqueue(3) can handle thousands of open file descriptors
20. Event loops — when to use 4 When I/O wait >> CPU wait 4 User interfaces 4 Network/Protocol servers 4 Excessive use on the client side leads to “callback hell” 4 Use function composition to streamline callbacks
21. Actors 4 Unit of computation 4 3 axioms. An actor can: 4 Create new actors 4 Send messages to actors it knows about 4 Handle incoming messages
22. Actors in practice 4 Actors maintain internal state 4 Promise: Only one thread can process msgs for a single actor at any time 4 Actors have mailboxes 4 Messages should be immutable 4 Actors are supervised by other actors
23. Actors in practice 4 Work in a container 4 Actor registry 4 Manages assignment of actors to processors 4 Fault tolerance, supervision 4 Can be distributed (explicitly/implicitly) 4 Processing code be hot swapped
24. A simple actor case class Msg(contents: String) class ActorExample extends Actor { def receive = { case Msg(contents) => print(contents) case _ => "Don't know what to do" } }
25. Actor with state case object Add case object Added class ActorExample extends Actor { var state = 0 def receive = { case Add => state += 1 sender ! Added case _ => "Don't know what to do with it" } }
26. Types of communication 4 Fire and forget 4 Fire and get Future 4 Fire and wait
27. Orchestrating actors val f1 = ask(actor1, msg1) val f2 = ask(actor2, msg2) val f3 = for { a ← f1.mapTo[Int] b ← f2.mapTo[Int] c ← ask(actor3, (a + b)).mapTo[Int] } yield c f3 onSuccess { case _ => println(“Result: ” + result) }
28. Failure management 4 Let it crash 4 Supervisor knows an exception is thrown 4 Respawns crashed actor 4 caveat: actor mailbox?
29. Actors — when to use 4 Asynchronous co-ordination of shared state 4 High-nine SLAs
30. Dealing with concurrency programmatically
31. Effects Effects is what computations produce other than the actual result 4 State modification 4 Exceptions 4 Latency 4 Nulls Effects complicate reasoning. We must make them explicit.
32. Monads A type with 3 methods: 4 Type constructor: The monad name 4 A unit function: t -> M t 4 A flatMap function: M t -> (t -> M u) - > M u trait Monad[M[_]] { def unit[S](a: S) : M[S] def flatMap[S, T] (m: M[S])(f: S => M[T]) : Monad[T] }
33. Popular Monads to deal with effects 4 Nulls: Option[T] 4 Exceptions: Try[T] 4 Latency: Future[T]
34. Monads — exception handling Exception handling with the Try Monad object Converter extends App { def toInt(a: String): Try[Int] = Try{Integer.parseInt(a)} def toString(b: Int): Try[String] = Try{b.toString} val a = toInt("4").flatMap(x => toString(x)) println(a) val b = toInt("foo").flatMap(x => toString(x)) println(b) }
35. Monads — exception handling Equivalent code in Java public class Converter { public Integer toInt(String a) throws NumberFormatException {return Integer.parseInt(a)} public String toString(Integer a) throws NullPointerException {return b.toString();} public static void main(String[] args) { try{ Integer five = toInt('5'); try { return toString(five); } catch(NullPointerException e) { //baaah } } catch(NumberFormatException r) { //ooofff } } }
36. Async Programming The art of making latency work in our advantage
37. Async Programming The art of making latency work in our advantage CODE MUST NEVER WAIT
38. Futures A placeholder for a value that may become available //Blocks val amazon = new Amazon().login(username, password) // Does not block val search : Future[List[Product]] = future { amazon.search("lego") }
39. Futures class Amazon { def login(a: String, b: String): Amazon = ??? def search(s: String): Future[List[String]] = Future {???} } class Main extends App { var amazon = new Amazon().login(username, password) val result = amazon.search("lego") } Quiz: How can we make the code non- blocking?
40. Future compositions flatMap is our friend class Amazon { def login(a: String, b: String): Future[Amazon] = Future{???} def search(s: String): Future[List[String]] = Future {???} } class Main extends App { val amazon = new Amazon() val result = amazon.login("a", "b") .flatMap(a => a.search("foo")) }
41. More asynchrony We now want search to login automatically and still be non-blocking
42. Async/Await async/await convert sequential code to Continuation Passing Style class AmazonAsync(a: String, b: String) { private def login(a: String, b: String): Future[Amazon] = Future{???} def search(s: String): Future[List[String]] = async { val login = await(this.login(a, b)) searchAPICall(login, s) } }
43. Without Async/Await class AmazonAsync(a: String, b: String) { private def login(a: String, b: String): Future[Amazon] = Future{???} def search(s: String): Future[List[String]] = async { val login = await(this.login(a, b)) searchAPICall(login, s) } // Probably wrong code def searchNotAsync(q: String) : Future[List[String]] = { val login = this.login(a, b) var result : Future[List[String]] = null login.onComplete(l => result = searchAPICall(l.get, q)) Await.result(login, Duration.Inf) result } }
44. Dealing with errors Make effects explicit in the type system class Amazon { def login(a: String, b: String): Future[Try[Amazon]] = Future{Try{throw new Throwable()}} def search(s: String): Seq[Future[Try[String]]] = List(Future{Try{"item1"}}, Future{Try{"item2"}}) }
45. Dealing with errors class Amazon { def login(a: String, b: String): Future[Try[Amazon]] = Future{Try{throw new Throwable()}} def search(s: String): Seq[Future[Try[String]]] = List(Future{Try{"item1"}}, Future{Try{"item2"}}) } Quiz: should search return 4 Seq[Future[Try]] 4 Future[Seq[Try]] 4 Future[Try[Seq]]
46. Producing multiple values We have seen how we deal with asynchrony in functions that produce one value. How do we deal with functions producing multiple values? Is Future[List[T]] the way to go?
47. Iterables and Iterators trait Iterable[T] { def iterator : Iterator[T] } trait Iterator[T] { def next(): T def hasNext(): Boolean }
48. Iterables and Iterators trait Iterable[T] { def iterator : Iterator[T] } trait Iterator[T] { def next(): T def hasNext(): Boolean } next() represents a computation that blocks!
49. Iterables and Iterators trait Iterator[T] { def next(): T def hasNext(): Boolean } Idea: what if we could pass a callback to next()?
50. Observables and Observers trait Observable[T] { def subscribe(observer: Observer[T]) : Subscription } trait Observer[T] { def onNext(value: T): Unit def onError(error: Throwable) : Unit def onCompleted() : Unit }
51. Rx - Reactive Programming 4 Express computations as a series of transformations on Observables 4 Libraries in all programming languages def hello(names: String*) { Observable.from(names).subscribe { n => println(s"Hello $n!") } }
52. Rx example trait Result case class Open(s: Socket) extends Result case class Closed(port: Int) extends Result // Types Observable.from(1 to 65536).flatMap( // Observable[Int] port => // f: Int => Observable[Result] Observable.from( future { try { Open(new Socket(host, port)) } catch { case e: Exception => Closed(port) } } ) //Observable[Result] ).subscribe( res => res match { case y: Open => println("Port " + y.s.getPort + " is open") y.s.close case y: Closed => _ }, err => err.printStackTrace(), () => println("Done.")) } )
53. Rx Observable transformations 4 One stream: Map, Filter, Buffer, Distinct 4 Multiple streams: Merge, Zip, Group By
54. Rx map Observable.map[R](func: T 㱺 R): Observable[R]
55. Rx filter Observable.filter(predicate: T 㱺 Boolean): Observable[T]
56. Rx Buffer Observable.slidingBuffer(timespan: Duration, timeshift: Duration): Observable[Seq[T]] Observable.slidingBuffer(count: Int, skip: Int): Observable[Seq[T]]
57. Rx Distinct Observable.distinct[U](keySelector: (T) 㱺 U): Observable[T] Observable.distinctUntilChanged: Observable[T]
58. Rx merge Observable.merge[U >: T](that: Observable[U]): Observable[U]
59. Rx zip Observable.zip[U](that: Observable[U]): Observable[(T, U)]
60. Rx zip example val o = Observable.zip( imageService.getUserPhoto(id), imageService.getPhotoMetadata(id), {(photo, metadata) => createPhotoWithData(photo, metadata)}) o.subscribe({photoWithData => showPhoto(photoWithData)});
61. Rx groupBy Observable.groupBy[K](f: (T) 㱺 K): Observable[(K, Observable[T])]
62. Rx groupBy example Observable(keyboard). groupBy{x : String => x}. subscribe { group : Observable[(String, Observable[String])] => var keyPresses = 0 group.subscribe{key : String => println(s"You pressed $key for $keyPresses times") } }
63. Rx concurrency 4 Concurrency both when subscribing and observing 4 Concurrency mechanism abstracted as Scheduler imageService.getImage(url) // Returns Observable[Image] .subscribeOn(Schedulers.io) .observeOn(AndroidSchedulers.mainThread) .subscribe{bitmap => myImageView.setImageBitmap(bitmap)};
64. Further reading 1. P. Walder, Monads for functional programming 2. E. Meijer, Your mouse is a database 3. M. Odersky, E. Meijer and R. Kuhn, Reactive Programming MOOC 4. Akka Team, Akka Documentation 5. T. Hughes-Croucher and M. Wilson, Node: Up and Running, Ch3
65. Acks 4 Event loop figure from SO question 21596172 4 Latency numbers every programmer should know by Colin Scott 4 All Rx marble diagram figures are from reactivex.io
66. Questions? Comments? Fixes? @gousiosg / gousiosg@gmail.com