Tweet
Tweet

Slide 1

Slide 1 text

Functional Reactive Programming Wednesday 2 October 13

Slide 2

Slide 2 text

@gideondk Functional, actor based programming and native application development Gideon de Kok Wednesday 2 October 13

Slide 3

Slide 3 text

Functional / Reactive Programming Wednesday 2 October 13

Slide 4

Slide 4 text

We need “new” tools, why? Wednesday 2 October 13

Slide 5

Slide 5 text

End of “traditional” web era Wednesday 2 October 13

Slide 6

Slide 6 text

Real-time Web / Big Data Wednesday 2 October 13

Slide 7

Slide 7 text

Increasing demands in respect to scalability, fault tolerence Wednesday 2 October 13

Slide 8

Slide 8 text

In terms of vertical scalability : Wednesday 2 October 13

Slide 9

Slide 9 text

Free lunch is over! Herb Sutter, 2005 Wednesday 2 October 13

Slide 10

Slide 10 text

1970 1975 1980 1985 1990 1995 2000 2005 2010 2015 Transistors Frequency Cores Wednesday 2 October 13

Slide 11

Slide 11 text

“Traditional” threads are too hard to manage Wednesday 2 October 13

Slide 12

Slide 12 text

Shared mutable state, locks, mutexes Wednesday 2 October 13

Slide 13

Slide 13 text

In terms of horizontal scalability : Wednesday 2 October 13

Slide 14

Slide 14 text

Difficult to build and manage distributed enviroments Wednesday 2 October 13

Slide 15

Slide 15 text

Supervision, fault tolerence, redundancy often not by default Wednesday 2 October 13

Slide 16

Slide 16 text

Distributed mutable state, objects, transactions == Distributed evil Wednesday 2 October 13

Slide 17

Slide 17 text

Let’s look at the two specific domains Wednesday 2 October 13

Slide 18

Slide 18 text

Big data applying filtering, transformations, and reductions to huge streams of data Wednesday 2 October 13

Slide 19

Slide 19 text

The real-time Web constant exchange of data, both in large and small packages Wednesday 2 October 13

Slide 20

Slide 20 text

Big data implications Wednesday 2 October 13

Slide 21

Slide 21 text

OO isn’t that well suited transformations are not inherent properties of objects Wednesday 2 October 13

Slide 22

Slide 22 text

A flat imperative model neither imperative functions are not easily composable Wednesday 2 October 13

Slide 23

Slide 23 text

Real-time web implications Wednesday 2 October 13

Slide 24

Slide 24 text

Traditional stacks aren’t suited Wednesday 2 October 13

Slide 25

Slide 25 text

Modern stacks neither ;-) Wednesday 2 October 13

Slide 26

Slide 26 text

Callback hell anyone? fs.readdir(source, function(err, files) { if (err) { console.log('Error finding files: ' + err) } else { files.forEach(function(filename, fileIndex) { console.log(filename) gm(source + filename).size(function(err, values) { if (err) { console.log('Error identifying file size: ' + err) } else { console.log(filename + ' : ' + values) aspect = (values.width / values.height) widths.forEach(function(width, widthIndex) { height = Math.round(width / aspect) console.log('resizing ' + filename + 'to ' + height + 'x' + height) this.resize(width, height).write(destination + 'w' + width + '_' + filename, function(err) { if (err) console.log('Error writing file: ' + err) }) }.bind(this)) } }) }) } }) Wednesday 2 October 13

Slide 27

Slide 27 text

We need a reactive system Wednesday 2 October 13

Slide 28

Slide 28 text

a system capable of reacting to events – load – failure – users Wednesday 2 October 13

Slide 29

Slide 29 text

a system which is event-driven, scalable, resilient and responsive Wednesday 2 October 13

Slide 30

Slide 30 text

(Re)defining the stack Wednesday 2 October 13

Slide 31

Slide 31 text

Starting with a deterministic core is essential : Wednesday 2 October 13

Slide 32

Slide 32 text

Non determinism is the root of all evil in both concurrent as distributed environments Wednesday 2 October 13

Slide 33

Slide 33 text

Starting with a expressive core is just as essential : Wednesday 2 October 13

Slide 34

Slide 34 text

Accidental complexity should be eliminated while working in complex domains Wednesday 2 October 13

Slide 35

Slide 35 text

Welcoming Functional Programming (and Scala) * *Doesn’t need batteries Wednesday 2 October 13

Slide 36

Slide 36 text

State Wednesday 2 October 13

Slide 37

Slide 37 text

State Wednesday 2 October 13

Slide 38

Slide 38 text

Immutable values Wednesday 2 October 13

Slide 39

Slide 39 text

scala> val a = 7 a: Int = 7 scala> a = 9 :8: error: reassignment to val Wednesday 2 October 13

Slide 40

Slide 40 text

val res = async { a + 2 } + async { val d = a; d - 2 } Wednesday 2 October 13

Slide 41

Slide 41 text

Purity Wednesday 2 October 13

Slide 42

Slide 42 text

function(5) == function(5) Wednesday 2 October 13

Slide 43

Slide 43 text

Functions first class citizens Wednesday 2 October 13

Slide 44

Slide 44 text

def a(): Function0[Int] def b(f: Function0[String]) functions taking functions returning functions Wednesday 2 October 13

Slide 45

Slide 45 text

Mapping Wednesday 2 October 13

Slide 46

Slide 46 text

Category Theory Wednesday 2 October 13

Slide 47

Slide 47 text

Objects & Arrows Wednesday 2 October 13

Slide 48

Slide 48 text

def length(s: String): Int = s.length Simple function Wednesday 2 October 13

Slide 49

Slide 49 text

String => Int String maps to Int Wednesday 2 October 13

Slide 50

Slide 50 text

Function as value val length = (s: String) => s.length Wednesday 2 October 13

Slide 51

Slide 51 text

Multiple arguments (Int, Int) => Int Wednesday 2 October 13

Slide 52

Slide 52 text

Multiple arguments val addAndDivide = (a: Int, b: Int) => (b + a) / a addAndDivide(10,4) res0: Int = 3 Wednesday 2 October 13

Slide 53

Slide 53 text

Curried Int => Int => Int Wednesday 2 October 13

Slide 54

Slide 54 text

Curried val addAndDivide = (a: Int) => (b: Int) => (b + a) / a addAndDivide(10)(4) res0: Int = 3 Wednesday 2 October 13

Slide 55

Slide 55 text

Partial application val addTwoAndDivideByTwo: Int => Int = addAndDivide(2) addTwoAndDivideByTwo(20) res1: Int = 11 Wednesday 2 October 13

Slide 56

Slide 56 text

Higher ordered functions Wednesday 2 October 13

Slide 57

Slide 57 text

Higher ordered functions def mapToStringList(l: List[Int], f: Int 㱺 String): List[String] Wednesday 2 October 13

Slide 58

Slide 58 text

Type signature (List[Int], Int 㱺 String) 㱺 List[String] Wednesday 2 October 13

Slide 59

Slide 59 text

More sane example val isInRangeF = (min: Int, max: Int) 㱺 (x: Int) 㱺 x > min && x < max Wednesday 2 October 13

Slide 60

Slide 60 text

Type signature (Int, Int) 㱺 Int 㱺 Boolean Wednesday 2 October 13

Slide 61

Slide 61 text

Usage val newFunction = isInRangeF(20,30) newFunction: Int 㱺 Boolean = newFunction(22) res0: Boolean = true isInRangeF(40,45)(50) res1: Boolean = false Wednesday 2 October 13

Slide 62

Slide 62 text

In the wild (Ruby) a = [ "a", "b", "c", "d" ] b = a.map {|x| x + "!" } b #=> [ "a!", "b!", "c!", "d!" ] Wednesday 2 October 13

Slide 63

Slide 63 text

In the wild (Scala) List(List(1,2), List(2), List(4,5)).map(_.sum) res0: List(3, 2, 9) List(2,3,4,5,6,7).foldLeft(0)(_+_) res1: Int = 27 List("Hello", "I", "want", "to", "be", "a", "sentence").reduceLeft(_ + " " + _) res2: String = Hello I want to be a sentence Wednesday 2 October 13

Slide 64

Slide 64 text

Map val l = List(1,2,3,4,5,6) l.map(x => x + 2) res0: List[Int] = List(3,4,5,6,7,8) Wednesday 2 October 13

Slide 65

Slide 65 text

Map trait List[Int] { def map[Int](f: Int 㱺 String): List[String] } Wednesday 2 October 13

Slide 66

Slide 66 text

Map trait List[A] { def map[B](f: A 㱺 B): List[B] } Wednesday 2 October 13

Slide 67

Slide 67 text

Deep mapping? Wednesday 2 October 13

Slide 68

Slide 68 text

What is a List? Wednesday 2 October 13

Slide 69

Slide 69 text

List[Int] 1 2 3 4 5 Wednesday 2 October 13

Slide 70

Slide 70 text

Context zero to a infinite number of values Wednesday 2 October 13

Slide 71

Slide 71 text

Map applies function to context Wednesday 2 October 13

Slide 72

Slide 72 text

Map: List applies function to zero or a infinite number of values Wednesday 2 October 13

Slide 73

Slide 73 text

Map also available in different shaped types Wednesday 2 October 13

Slide 74

Slide 74 text

val a: Option[Int] = Some(2) a.map(x => x + 4) res0: Some(6) Option Wednesday 2 October 13

Slide 75

Slide 75 text

val a: Future[String] = Future(“a”) a.map(x => x + “b”) Future Wednesday 2 October 13

Slide 76

Slide 76 text

IO val a: IO[File] a.map(x => x.path) res0: IO[String] Wednesday 2 October 13

Slide 77

Slide 77 text

Context mapping behavior attribute of specific “type class” Wednesday 2 October 13

Slide 78

Slide 78 text

Functor Wednesday 2 October 13

Slide 79

Slide 79 text

trait Functor[A] { def map[B](f: A 㱺 B): Functor[B] } Functor Wednesday 2 October 13

Slide 80

Slide 80 text

Null? def fetchUserFromDatabase(email: String): User val user = fetchUserFromDatabase("[email protected]") var firstName = "" if (user != null) { firstName = user.firstName } else { throw Exception("User doesn't exist") } Wednesday 2 October 13

Slide 81

Slide 81 text

Option! def fetchUserFromDatabase(email: String): Option[User] val firstName: Option[String] = fetchUserFromDatabase("[email protected]").map(_.firstName) Wednesday 2 October 13

Slide 82

Slide 82 text

Option: context a potentially existing value Wednesday 2 October 13

Slide 83

Slide 83 text

Map: Option applies function to a potentially existing value Wednesday 2 October 13

Slide 84

Slide 84 text

Future def fetchFromSite(url: String): Future[HttpResponse] val json: Future[JsObject] = fetchFromSite("http://google.com/ search.json?q=cats").map(x => toJson(x.toString)) val value: JsObject = Await.result(json, 2 seconds) Wednesday 2 October 13

Slide 85

Slide 85 text

Future: context context with a eventually resolved value Wednesday 2 October 13

Slide 86

Slide 86 text

Map: Future applies function to a eventually resolved value Wednesday 2 October 13

Slide 87

Slide 87 text

What happens in context, stays in context Wednesday 2 October 13

Slide 88

Slide 88 text

But what if a function returns a context within a context.. Wednesday 2 October 13

Slide 89

Slide 89 text

within a context within a context within a context within a context within a context? Wednesday 2 October 13

Slide 90

Slide 90 text

Wednesday 2 October 13

Slide 91

Slide 91 text

val userFut: Future[User] = db.user.fetch("[email protected]") val orgFut: Future[Future[Organisation]] = userFut.map(x 㱺 db.organisation.fetch(x.organisationId)) Wednesday 2 October 13

Slide 92

Slide 92 text

val fut: Future[Organisation] = Await.result(orgFut, 1 second) val org: Organisation = Await.result(org, 1 second) Wednesday 2 October 13

Slide 93

Slide 93 text

Monads Wednesday 2 October 13

Slide 94

Slide 94 text

Monads: flatten multiple contexts as a “stack” Wednesday 2 October 13

Slide 95

Slide 95 text

trait Monad[A] { def flatMap[B](f: A 㱺 Monad[B]): Monad[B] } Monad Wednesday 2 October 13

Slide 96

Slide 96 text

trait Future[A] { def flatMap[B](f: A 㱺 Future[B]): Future[B] } Future Monad Wednesday 2 October 13

Slide 97

Slide 97 text

Future: Monad val userFut: Future[User] = db.user.fetch("[email protected]") val orgFut: Future[Organisation] = userFut.flatMap(x 㱺 db.organisation.fetch(x.organisationId)) val org: Organisation = Await.result(orgFut, 1 second) Wednesday 2 October 13

Slide 98

Slide 98 text

Option: Monad val email: Option[String] = request.headers.get(“email”) val domain: Option[String] = email.flatMap(x => x.split("@").tail.headOption) val domainLength: Option[Int] = domain.map(_.length) Wednesday 2 October 13

Slide 99

Slide 99 text

Multiple Monads db.user.fetch("[email protected]").flatMap { u => db.organisation.fetch(u.organizationId).flatMap { o 㱺 db.city.fetch(o.cityId) flatMap { c => db.country.fetch(c.countryId) } } } Wednesday 2 October 13

Slide 100

Slide 100 text

Multiple Monads for { user <- db.user.fetch("[email protected]") org <- db.organisation.fetch(user.organizationId) city <- db.city.fetch(org.cityId) country <- db.coutry.fetch(city.countryId) } yield country Wednesday 2 October 13

Slide 101

Slide 101 text

Break! Wednesday 2 October 13

Slide 102

Slide 102 text

Wednesday 2 October 13

Slide 103

Slide 103 text

Dataflows Wednesday 2 October 13

Slide 104

Slide 104 text

Dataflows Wednesday 2 October 13

Slide 105

Slide 105 text

Contexts are able to “box” behavior Wednesday 2 October 13

Slide 106

Slide 106 text

Contexts help to seperate the determistic and non-deterministic Wednesday 2 October 13

Slide 107

Slide 107 text

Contexts purify the unpure* *Book of Monad – 2:22 Wednesday 2 October 13

Slide 108

Slide 108 text

While immutability helps determinism and contexts box non-determinism Wednesday 2 October 13

Slide 109

Slide 109 text

A lot of data flows are based on state Wednesday 2 October 13

Slide 110

Slide 110 text

Concurrent state Wednesday 2 October 13

Slide 111

Slide 111 text

State monads work great in singular data flows Wednesday 2 October 13

Slide 112

Slide 112 text

But what should we use to enable stateful programming in concurrent and distributed environments? Wednesday 2 October 13

Slide 113

Slide 113 text

Concurrency, Parallelism? Wednesday 2 October 13

Slide 114

Slide 114 text

customers cashier Wednesday 2 October 13

Slide 115

Slide 115 text

customers cashiers Wednesday 2 October 13

Slide 116

Slide 116 text

Concurrency & Parallelism no or, but and! Wednesday 2 October 13

Slide 117

Slide 117 text

Concurrency is hard… Wednesday 2 October 13

Slide 118

Slide 118 text

Techological Error handling is tricky Race conditions, mutexes, locks Wednesday 2 October 13

Slide 119

Slide 119 text

Cognitive Concurrency proof data structures are difficult to design Understanding in-code bottlenecks is painful Bugfixing and error hunting sometimes mere impossible Wednesday 2 October 13

Slide 120

Slide 120 text

Multiple concurrency methologies Wednesday 2 October 13

Slide 121

Slide 121 text

Shared-state concurrency Wednesday 2 October 13

Slide 122

Slide 122 text

class User(var firstName: String, var lastName: String) { ! def setFirstName(name: String) = firstName = name ! def setLastName(name: String) = lastName = name ! def getFullName = firstName + " " + lastName } val user = User("John", "Doe") val a = Async { ! user.setFirstName("Johnny") ! user.setLastName("Walker") } val b = Async { ! user.getFullName } b == "Johnny Doe" Wednesday 2 October 13

Slide 123

Slide 123 text

class User(var firstName: String, var lastName: String) { ! val lock = new ReentrantLock ! def setFirstName(name: String) = { ! ! lock.lock ! ! firstName = name ! ! lock.unlock ! } ! def setLastName(name: String) = { ! ! lock.lock ! ! lastName = name ! ! lock.unlock ! } ! def getFullName = { ! ! lock.lock ! ! firstName + " " + lastName ! ! lock.unlock ! } } // Thread 1 user.setFirstName("Johnny") // Thread 2 user.getFullName // Thread 1 user.setLastName("Walker") Wednesday 2 October 13

Slide 124

Slide 124 text

Wednesday 2 October 13

Slide 125

Slide 125 text

Message passing for concurrency Wednesday 2 October 13

Slide 126

Slide 126 text

What would it look like if we didn’t create the problem in the first place? Wednesday 2 October 13

Slide 127

Slide 127 text

Focussing on isolated state through non-mutable messages Wednesday 2 October 13

Slide 128

Slide 128 text

Welcoming actors! Wednesday 2 October 13

Slide 129

Slide 129 text

Let’s illustrate the concept! Wednesday 2 October 13

Slide 130

Slide 130 text

request response request parser response formatter data query data result ? ! Wednesday 2 October 13

Slide 131

Slide 131 text

request response request parser response formatter data query data result ? ! Wednesday 2 October 13

Slide 132

Slide 132 text

Wednesday 2 October 13

Slide 133

Slide 133 text

customer cashier barista Wednesday 2 October 13

Slide 134

Slide 134 text

m oney coffee prepare order retrieve order em pty cup filled cup Wednesday 2 October 13

Slide 135

Slide 135 text

m oney coffee prepare order retrieve order em pty cup filled cup Wednesday 2 October 13

Slide 136

Slide 136 text

scaling the café (naively) Wednesday 2 October 13

Slide 137

Slide 137 text

Wednesday 2 October 13

Slide 138

Slide 138 text

work duration 15s 120s 5s Wednesday 2 October 13

Slide 139

Slide 139 text

unnecessary scaling of cashiers Wednesday 2 October 13

Slide 140

Slide 140 text

businesses scale where it hurts Wednesday 2 October 13

Slide 141

Slide 141 text

incoming served waiting Wednesday 2 October 13

Slide 142

Slide 142 text

incoming served waiting Wednesday 2 October 13

Slide 143

Slide 143 text

Wednesday 2 October 13

Slide 144

Slide 144 text

state behavior mailbox Actor Wednesday 2 October 13

Slide 145

Slide 145 text

Actor state behavior mailbox event driven threat Wednesday 2 October 13

Slide 146

Slide 146 text

state behavior mailbox event driven threat Actor Wednesday 2 October 13

Slide 147

Slide 147 text

Tell & Ask / ! & ? Wednesday 2 October 13

Slide 148

Slide 148 text

val printer = actor(new Act { ! become { ! ! case x: String ⇒ println(x) ! } }) printer ! "Print this!" Wednesday 2 October 13

Slide 149

Slide 149 text

val a = actor(new Act { ! become { ! ! case "hello" ⇒ sender ! "hi" !} }) val result: Future[String] = (a ? "hello").mapTo[String] Wednesday 2 October 13

Slide 150

Slide 150 text

In action! Wednesday 2 October 13

Slide 151

Slide 151 text

object CoffeeShop { ! case class Money ! case class Coffee ! case class Cup(contents: Option[Coffee]) ! case class OrderCoffee(cashier: ActorRef) ! implicit val system = ActorSystem("coffee-shop") ! class Customer extends Actor { ! var money = Some(Money()) ! var cup = None ! def receive = { ! ! case OrderCoffee(cashier) => ! ! ! val m = money ! ! ! money = None ! ! ! cashier ! m ! ! case c: Cup => ! ! ! cup = c ! ! ! println("Whooohooo! Coffee :-)") ! } } } Wednesday 2 October 13

Slide 152

Slide 152 text

class Cassier(barista: ActorRef) extends Actor { ! var orders = Map[String, ActorRef] ! def receive = { ! ! case Money => ! ! ! val customer = sender ! ! ! val newOrder = Order(java.util.UUID.randomUUID.toString) ! ! ! orders = orders ++ (newOrder.id -> customer) ! ! ! val cup = Cup(contents = None) ! ! ! barista ! (order, cup) ! ! case x: (Order, Cup) => val customer = orders(x._1.id) ! ! customer ! x._2 ! } } Wednesday 2 October 13

Slide 153

Slide 153 text

class Barista extends Actor { ! def receive = { ! ! case x: (Order, Cup) => ! ! ! sender ! (x._1 -> prepareCoffee(x._2))) ! } ! def prepareCoffee(cup: Cup) = { ! ! thread.sleep(120) ! ! cup.copy(contents = Some(Coffee())) ! } } Wednesday 2 October 13

Slide 154

Slide 154 text

val customer = system.actorOf(Props[Customer]) val barista = system.actorOf(Props[Barista]) val cassier = system.actorOf(Props(new Cassier(barista))) customer ! OrderCoffee(cassier) Wednesday 2 October 13

Slide 155

Slide 155 text

One actor processes one message at a time Wednesday 2 October 13

Slide 156

Slide 156 text

“Actors do one thing at a time, they do it very well, and then they quickly move on.” Wednesday 2 October 13

Slide 157

Slide 157 text

Multiple actors proces multiple messages at a time Wednesday 2 October 13

Slide 158

Slide 158 text

Meet routers! Wednesday 2 October 13

Slide 159

Slide 159 text

? Wednesday 2 October 13

Slide 160

Slide 160 text

Transparent load balancing Wednesday 2 October 13

Slide 161

Slide 161 text

Without any additional code in actor implementation Wednesday 2 October 13

Slide 162

Slide 162 text

val baristas = system.actorOf(Props[Barista]).withRouter(RoundRobinRouter(nrOfInstances = 5))) val cassiers = system.actorOf(Props(new Cassier(barista))).withRouter(RoundRobinRouter(nrOfInstances = 3))) customer ! OrderCoffee(cassiers) customerTwo ! OrderCoffee(cassiers) customerThree ! OrderCoffee(cassiers) Wednesday 2 October 13

Slide 163

Slide 163 text

People die, actors too Wednesday 2 October 13

Slide 164

Slide 164 text

Exception handling in threads is tough Wednesday 2 October 13

Slide 165

Slide 165 text

Barista burns hand :-( Wednesday 2 October 13

Slide 166

Slide 166 text

Probable choice: stop working, tell supervisor Wednesday 2 October 13

Slide 167

Slide 167 text

Supervisor responsable for barista replacement Wednesday 2 October 13

Slide 168

Slide 168 text

Actor state behavior Wednesday 2 October 13

Slide 169

Slide 169 text

Actor state behavior Wednesday 2 October 13

Slide 170

Slide 170 text

Actor behavior Supervisor Escalate Wednesday 2 October 13

Slide 171

Slide 171 text

class CoffeeShopOwner extends Actor { ! var baristas: List[Barista] ! override val supervisorStrategy = ! OneForOneStrategy(maxNrOfRetries = 10, withinTimeRange = 1 minute) { ! case _: BurnedHandException ⇒ Resume ! case _: PassedOut ! ! ⇒ Restart ! case _: Died ! ! ! ! ⇒ Stop ! case _: Exception ⇒ Escalate ! } } Wednesday 2 October 13

Slide 172

Slide 172 text

Actors are restarted by default (UserGuardian) Wednesday 2 October 13

Slide 173

Slide 173 text

Just let it crash™ Wednesday 2 October 13

Slide 174

Slide 174 text

But what about multiple server instances? Wednesday 2 October 13

Slide 175

Slide 175 text

Actors run transparantly local & remote Wednesday 2 October 13

Slide 176

Slide 176 text

val actor = context.actorOf[Barista]("my-service") Wednesday 2 October 13

Slide 177

Slide 177 text

akka { ! actor { ! ! deployment { ! ! ! /path/to/service { ! ! ! ! router = "round-robin" ! ! ! ! nr-of-instances = 2 ! ! ! ! remote { ! ! ! ! ! nodes = ["node1:2552", "node2:2552"] ! ! ! ! } ! ! ! } ! ! } ! } } Wednesday 2 October 13

Slide 178

Slide 178 text

Write as local, deploy in the cloud (zero code change) Wednesday 2 October 13

Slide 179

Slide 179 text

? Wednesday 2 October 13

Slide 180

Slide 180 text

Pushing configurations can be cumbersome Wednesday 2 October 13

Slide 181

Slide 181 text

Akka Cluster Wednesday 2 October 13

Slide 182

Slide 182 text

Functional Groningen Wednesday 2 October 13