Concurrency Patterns - Speaker Deck

Slide 1

Slide 1 text

Concurrency patterns Georgios Gousios

Slide 2

Slide 2 text

Concurrency and Parallelism 4 Concurrency: doing many things at the same time 4 Requests from multiple clients on a server 4 Parallelism: doing one thing on multiple processors 4 Big matrix multiplication on CPUs/GPUs

Slide 3

Slide 3 text

Enemies of concurrency 4 State 4 Sharing

Slide 4

Slide 4 text

Common concurrency patterns 4 Processes 4 Threads 4 Event loops 4 Actors 4 Asynchronous processing 4 Streams and reactive programming

Slide 5

Slide 5 text

Processes 4 Same code, different memory space 4 Always have a parent, that forks(3) them # Shared memory (owned by the parent) children = [] pid = Process::fork if pid == 0 puts pid, a # Child specific memory do_work() else puts pid, a # Parent specific memory children += pid end children.each do |pid| Process.waitpid(pid, 0) end

Slide 6

Slide 6 text

Processes — when to use 4 Simplest concurrency pattern 4 fork(3) is expensive 4 Large work units, taking significant time to setup 4 (usually) not per request 4 Can be used in combination with finer- grained options (e.g. event loops)

Slide 7

Slide 7 text

Threads 4 A self-executing unit of computation within a bigger group of computations 4 OS or application based 4 N x M mapping called ‘green threading’ 4 Access same memory with initiator 4 Need concurrency primitives to avoid races

Slide 8

Slide 8 text

Threads example t = Thread.new do puts 'Tock' end puts 'Tick' t.join

Slide 9

Slide 9 text

Thread-based patterns 4 Producer/consumer queue 4 Thread pool 4 Fork/join

Slide 10

Slide 10 text

Producer/consumer queue 4 In memory blocking queue 4 Identical work items queue = Queue.new producer = Thread.new do 5.times do |i| queue << i end end consumer = Thread.new do while true do |i| value = queue.pop process value end end consumer.join

Slide 11

Slide 11 text

Thread pool A collection of initialised threads waiting for work to be assigned to them

Slide 12

Slide 12 text

Thread pool class Pool def initialize(size) @size = size @jobs = Queue.new @pool = Array.new(@size) do |i| Thread.new do Thread.current[:id] = i catch(:exit) do loop do job, args = @jobs.pop job.call(*args) end end end end def schedule(*args, &block) @jobs << [block, args] end end

Slide 13

Slide 13 text

Fork/Join solve(problem): if problem is small enough: solve problem directly else: for part in subdivide fork subtask to solve part join all subtasks spawned in previous loop combine results from subtasks

Slide 14

Slide 14 text

Threads — when to use 4 Reasoning about correctness with threads is hard 4 Especially when other isolation primitives (e.g. transactions) are involved 4 When one of the ready made patterns work 4 Better use higher level abstractions

Slide 15

Slide 15 text

Event loops 4 Basic pattern 4 Wait on a file descriptor to emit a result 4 Run list of registered callbacks 4 More advanced 4 Wait on multiple file descriptors 4 Schedule background work in threads 4 An event loop is always single threaded

Slide 16

Slide 16 text

Basic event loop fd = File.open("/var/log/messages.log") callbacks = [Proc.new{|x| print x}] while true do active = select([fd]) data = fd.read_lines callbacks.each{|x| x.invoke(data)} end

Slide 17

Slide 17 text

Pragmatic event loops

Slide 18

Slide 18 text

Event loops — where is the concurrency? 4 I/O is 10^4 - 10^6 slower than processing 4 On most systems, most time is spent waiting 4 select(3) or epoll(3) or kqueue(3) can handle thousands of open file descriptors

Slide 19

Slide 19 text

No content

Slide 20

Slide 20 text

Event loops — when to use 4 When I/O wait >> CPU wait 4 User interfaces 4 Network/Protocol servers 4 Excessive use on the client side leads to “callback hell” 4 Use function composition to streamline callbacks

Slide 21

Slide 21 text

Actors 4 Unit of computation 4 3 axioms. An actor can: 4 Create new actors 4 Send messages to actors it knows about 4 Handle incoming messages

Slide 22

Slide 22 text

Actors in practice 4 Actors maintain internal state 4 Promise: Only one thread can process msgs for a single actor at any time 4 Actors have mailboxes 4 Messages should be immutable 4 Actors are supervised by other actors

Slide 23

Slide 23 text

Actors in practice 4 Work in a container 4 Actor registry 4 Manages assignment of actors to processors 4 Fault tolerance, supervision 4 Can be distributed (explicitly/implicitly) 4 Processing code be hot swapped

Slide 24

Slide 24 text

A simple actor case class Msg(contents: String) class ActorExample extends Actor { def receive = { case Msg(contents) => print(contents) case _ => "Don't know what to do" } }

Slide 25

Slide 25 text

Actor with state case object Add case object Added class ActorExample extends Actor { var state = 0 def receive = { case Add => state += 1 sender ! Added case _ => "Don't know what to do with it" } }

Slide 26

Slide 26 text

Types of communication 4 Fire and forget 4 Fire and get Future 4 Fire and wait

Slide 27

Slide 27 text

Orchestrating actors val f1 = ask(actor1, msg1) val f2 = ask(actor2, msg2) val f3 = for { a ← f1.mapTo[Int] b ← f2.mapTo[Int] c ← ask(actor3, (a + b)).mapTo[Int] } yield c f3 onSuccess { case _ => println(“Result: ” + result) }

Slide 28

Slide 28 text

Failure management 4 Let it crash 4 Supervisor knows an exception is thrown 4 Respawns crashed actor 4 caveat: actor mailbox?

Slide 29

Slide 29 text

Actors — when to use 4 Asynchronous co-ordination of shared state 4 High-nine SLAs

Slide 30

Slide 30 text

Dealing with concurrency programmatically

Slide 31

Slide 31 text

Effects Effects is what computations produce other than the actual result 4 State modification 4 Exceptions 4 Latency 4 Nulls Effects complicate reasoning. We must make them explicit.

Slide 32

Slide 32 text

Monads A type with 3 methods: 4 Type constructor: The monad name 4 A unit function: t -> M t 4 A flatMap function: M t -> (t -> M u) - > M u trait Monad[M[_]] { def unit[S](a: S) : M[S] def flatMap[S, T] (m: M[S])(f: S => M[T]) : Monad[T] }

Slide 33

Slide 33 text

Popular Monads to deal with effects 4 Nulls: Option[T] 4 Exceptions: Try[T] 4 Latency: Future[T]

Slide 34

Slide 34 text

Monads — exception handling Exception handling with the Try Monad object Converter extends App { def toInt(a: String): Try[Int] = Try{Integer.parseInt(a)} def toString(b: Int): Try[String] = Try{b.toString} val a = toInt("4").flatMap(x => toString(x)) println(a) val b = toInt("foo").flatMap(x => toString(x)) println(b) }

Slide 35

Slide 35 text

Monads — exception handling Equivalent code in Java public class Converter { public Integer toInt(String a) throws NumberFormatException {return Integer.parseInt(a)} public String toString(Integer a) throws NullPointerException {return b.toString();} public static void main(String[] args) { try{ Integer five = toInt('5'); try { return toString(five); } catch(NullPointerException e) { //baaah } } catch(NumberFormatException r) { //ooofff } } }

Slide 36

Slide 36 text

Async Programming The art of making latency work in our advantage

Slide 37

Slide 37 text

Async Programming The art of making latency work in our advantage CODE MUST NEVER WAIT

Slide 38

Slide 38 text

Futures A placeholder for a value that may become available //Blocks val amazon = new Amazon().login(username, password) // Does not block val search : Future[List[Product]] = future { amazon.search("lego") }

Slide 39

Slide 39 text

Futures class Amazon { def login(a: String, b: String): Amazon = ??? def search(s: String): Future[List[String]] = Future {???} } class Main extends App { var amazon = new Amazon().login(username, password) val result = amazon.search("lego") } Quiz: How can we make the code non- blocking?

Slide 40

Slide 40 text

Future compositions flatMap is our friend class Amazon { def login(a: String, b: String): Future[Amazon] = Future{???} def search(s: String): Future[List[String]] = Future {???} } class Main extends App { val amazon = new Amazon() val result = amazon.login("a", "b") .flatMap(a => a.search("foo")) }

Slide 41

Slide 41 text

More asynchrony We now want search to login automatically and still be non-blocking

Slide 42

Slide 42 text

Async/Await async/await convert sequential code to Continuation Passing Style class AmazonAsync(a: String, b: String) { private def login(a: String, b: String): Future[Amazon] = Future{???} def search(s: String): Future[List[String]] = async { val login = await(this.login(a, b)) searchAPICall(login, s) } }

Slide 43

Slide 43 text

Without Async/Await class AmazonAsync(a: String, b: String) { private def login(a: String, b: String): Future[Amazon] = Future{???} def search(s: String): Future[List[String]] = async { val login = await(this.login(a, b)) searchAPICall(login, s) } // Probably wrong code def searchNotAsync(q: String) : Future[List[String]] = { val login = this.login(a, b) var result : Future[List[String]] = null login.onComplete(l => result = searchAPICall(l.get, q)) Await.result(login, Duration.Inf) result } }

Slide 44

Slide 44 text

Dealing with errors Make effects explicit in the type system class Amazon { def login(a: String, b: String): Future[Try[Amazon]] = Future{Try{throw new Throwable()}} def search(s: String): Seq[Future[Try[String]]] = List(Future{Try{"item1"}}, Future{Try{"item2"}}) }

Slide 45

Slide 45 text

Dealing with errors class Amazon { def login(a: String, b: String): Future[Try[Amazon]] = Future{Try{throw new Throwable()}} def search(s: String): Seq[Future[Try[String]]] = List(Future{Try{"item1"}}, Future{Try{"item2"}}) } Quiz: should search return 4 Seq[Future[Try]] 4 Future[Seq[Try]] 4 Future[Try[Seq]]

Slide 46

Slide 46 text

Producing multiple values We have seen how we deal with asynchrony in functions that produce one value. How do we deal with functions producing multiple values? Is Future[List[T]] the way to go?

Slide 47

Slide 47 text

Iterables and Iterators trait Iterable[T] { def iterator : Iterator[T] } trait Iterator[T] { def next(): T def hasNext(): Boolean }

Slide 48

Slide 48 text

Iterables and Iterators trait Iterable[T] { def iterator : Iterator[T] } trait Iterator[T] { def next(): T def hasNext(): Boolean } next() represents a computation that blocks!

Slide 49

Slide 49 text

Iterables and Iterators trait Iterator[T] { def next(): T def hasNext(): Boolean } Idea: what if we could pass a callback to next()?

Slide 50

Slide 50 text

Observables and Observers trait Observable[T] { def subscribe(observer: Observer[T]) : Subscription } trait Observer[T] { def onNext(value: T): Unit def onError(error: Throwable) : Unit def onCompleted() : Unit }

Slide 51

Slide 51 text

Rx - Reactive Programming 4 Express computations as a series of transformations on Observables 4 Libraries in all programming languages def hello(names: String*) { Observable.from(names).subscribe { n => println(s"Hello $n!") } }

Slide 52

Slide 52 text

Rx example trait Result case class Open(s: Socket) extends Result case class Closed(port: Int) extends Result // Types Observable.from(1 to 65536).flatMap( // Observable[Int] port => // f: Int => Observable[Result] Observable.from( future { try { Open(new Socket(host, port)) } catch { case e: Exception => Closed(port) } } ) //Observable[Result] ).subscribe( res => res match { case y: Open => println("Port " + y.s.getPort + " is open") y.s.close case y: Closed => _ }, err => err.printStackTrace(), () => println("Done.")) } )

Slide 53

Slide 53 text

Rx Observable transformations 4 One stream: Map, Filter, Buffer, Distinct 4 Multiple streams: Merge, Zip, Group By

Slide 54

Slide 54 text

Rx map Observable.map[R](func: T 㱺 R): Observable[R]

Slide 55

Slide 55 text

Rx filter Observable.filter(predicate: T 㱺 Boolean): Observable[T]

Slide 56

Slide 56 text

Rx Buffer Observable.slidingBuffer(timespan: Duration, timeshift: Duration): Observable[Seq[T]] Observable.slidingBuffer(count: Int, skip: Int): Observable[Seq[T]]

Slide 57

Slide 57 text

Rx Distinct Observable.distinct[U](keySelector: (T) 㱺 U): Observable[T] Observable.distinctUntilChanged: Observable[T]

Slide 58

Slide 58 text

Rx merge Observable.merge[U >: T](that: Observable[U]): Observable[U]

Slide 59

Slide 59 text

Rx zip Observable.zip[U](that: Observable[U]): Observable[(T, U)]

Slide 60

Slide 60 text

Rx zip example val o = Observable.zip( imageService.getUserPhoto(id), imageService.getPhotoMetadata(id), {(photo, metadata) => createPhotoWithData(photo, metadata)}) o.subscribe({photoWithData => showPhoto(photoWithData)});

Slide 61

Slide 61 text

Rx groupBy Observable.groupBy[K](f: (T) 㱺 K): Observable[(K, Observable[T])]

Slide 62

Slide 62 text

Rx groupBy example Observable(keyboard). groupBy{x : String => x}. subscribe { group : Observable[(String, Observable[String])] => var keyPresses = 0 group.subscribe{key : String => println(s"You pressed $key for $keyPresses times") } }

Slide 63

Slide 63 text

Rx concurrency 4 Concurrency both when subscribing and observing 4 Concurrency mechanism abstracted as Scheduler imageService.getImage(url) // Returns Observable[Image] .subscribeOn(Schedulers.io) .observeOn(AndroidSchedulers.mainThread) .subscribe{bitmap => myImageView.setImageBitmap(bitmap)};

Slide 64

Slide 64 text

Further reading 1. P. Walder, Monads for functional programming 2. E. Meijer, Your mouse is a database 3. M. Odersky, E. Meijer and R. Kuhn, Reactive Programming MOOC 4. Akka Team, Akka Documentation 5. T. Hughes-Croucher and M. Wilson, Node: Up and Running, Ch3

Slide 65

Slide 65 text

Acks 4 Event loop figure from SO question 21596172 4 Latency numbers every programmer should know by Colin Scott 4 All Rx marble diagram figures are from reactivex.io

Slide 66

Slide 66 text

Questions? Comments? Fixes? @gousiosg / [email protected]