Introduction to Building Distributed Systems with Scala and Akka

Slide 1

Slide 1 text

Building(Distributed(Systems with%the%Typesafe%Reac/ve%Pla3orm

Slide 2

Slide 2 text

Who$am$I? • Kevin'Webber,'Enterprise'Advocate,'Typesafe • Graduated'from'Georgian'College,'1999 • Java'developer'since'2001'(Java'1.1.8'→'Java'8),'Scala' developer'since'2012 • Previous'roles'include'developer,'team'lead,'and'handsKon' architect • Interviewed'>'100'developers

Slide 3

Slide 3 text

About&Typesafe • Founded(in(2011(in(an.cipa.on(of(a(genera.onal(shi7(to(cloud9 based(compu.ng(by • Mar$n&Odersky,(creator(of(Scala,(Java(Generics,(and(the( current(genera.on(of(javac • Jonas&Bonér,(creator(of(Akka

Slide 4

Slide 4 text

Who$is$Typesafe? • Typesafe)is)a)technology)company)providing)stewardship)for) open7source)so9ware)targe:ng)the)JVM • Scala)7)a)hybrid)object7func:onal)programming)language • Play)7)an)MVC)framework • Akka)7)a)concurrency)toolkit) • Apache,Spark*)7)a)fast)and)general)engine)for)big)data) processing)(streaming,)SQL,)machine)learning,)and)graph) processing)

Slide 5

Slide 5 text

No content

Slide 6

Slide 6 text

Typesafe's)Customers

Slide 7

Slide 7 text

What%are%distributed%systems?

Slide 8

Slide 8 text

You$have$a$distributed$system$when$ the$crash$of$a$computer$you’ve$ never$heard$of$stops$you$from$ ge8ng$any$work$done. —"Leslie"Lamport:"A"Guide"to"Building"Dependable"Distributed"Systems

Slide 9

Slide 9 text

A"collec'on"of"independent" computers"that"appears"to"its"users" as"a"single"coherent"system. —"Tanenbaum"and"Steen:"Distributed"Systems,"Principles"and" Paradigms

Slide 10

Slide 10 text

Distributed*architecture

Slide 11

Slide 11 text

Why$build$distributed$systems?

Slide 12

Slide 12 text

Size%of%the%internet

Slide 13

Slide 13 text

Market'forces'leading'to'change Concurrent)connec*ons • "Internet(of(Things",(mobile(devices

Slide 14

Slide 14 text

Market'forces'leading'to'change Concurrent)connec*ons • "Internet(of(Things",(mobile(devices Big$data • Size&of&data&is&overwhelming&our&ability&to&manage&it

Slide 15

Slide 15 text

Market'forces'leading'to'change Concurrent)connec*ons • "Internet(of(Things",(mobile(devices Big$data • Size&of&data&is&overwhelming&our&ability&to&manage&it Response'(mes • Real&'me)results)(e.g,)analy'cs))with)sub&second)latencies

Slide 16

Slide 16 text

Physical)factors)leading)to)change • Expensive*hardware*→*cheap&hardware

Slide 17

Slide 17 text

Physical)factors)leading)to)change • Expensive*hardware*→*cheap&hardware • A*single*machine*→*clusters&of&machines

Slide 18

Slide 18 text

Physical)factors)leading)to)change • Expensive*hardware*→*cheap&hardware • A*single*machine*→*clusters&of&machines • A*single*core*→*mul3ple&cores

Slide 19

Slide 19 text

Physical)factors)leading)to)change • Expensive*hardware*→*cheap&hardware • A*single*machine*→*clusters&of&machines • A*single*core*→*mul3ple&cores • Slow*networks*→*fast&networks

Slide 20

Slide 20 text

Physical)factors)leading)to)change • Expensive*hardware*→*cheap&hardware • A*single*machine*→*clusters&of&machines • A*single*core*→*mul3ple&cores • Slow*networks*→*fast&networks • Small*data*snapshots*→*big&data,&streams&of&data

Slide 21

Slide 21 text

2005$architecture

Slide 22

Slide 22 text

2015%architecture

Slide 23

Slide 23 text

2020#architecture

Slide 24

Slide 24 text

New$skills$demanded$by$industry • Local'→'distributed

Slide 25

Slide 25 text

New$skills$demanded$by$industry • Local'→'distributed • Impera.ve'→'func,onal

Slide 26

Slide 26 text

New$skills$demanded$by$industry • Local'→'distributed • Impera.ve'→'func,onal • Security'agnos.c'→'security1aware

Slide 27

Slide 27 text

New$skills$demanded$by$industry • Local'→'distributed • Impera.ve'→'func,onal • Security'agnos.c'→'security1aware • Single'database'→'reads/writes1across1many1data1stores

Slide 28

Slide 28 text

New$skills$demanded$by$industry • Local'→'distributed • Impera.ve'→'func,onal • Security'agnos.c'→'security1aware • Single'database'→'reads/writes1across1many1data1stores • Batch'processing'→'streams1of1data,1real6,me1analy,cs

Slide 29

Slide 29 text

Slide 30

Slide 30 text

Changing'how'we'think'about' building'so2ware

Slide 31

Slide 31 text

Thread'based* concurrency • Shared(mutable(state • Impera1ve,(sequen1al,(side7eﬀec1ng • Threads(for(concurrency • Locks,(monitors,(etc • Implementa)ons( • Java(Enterprise(Edi1on,(Ruby(on(Rails

Slide 32

Slide 32 text

Message&driven, concurrency • Isolated*state • Asynchronous*message*passing* between*mailboxes • Process*isola:on*for*fault*tolerance • Actors*can*live*in*diﬀerent*thread*pools,* JVMs,*nodes,*or*clusters • Implementa)ons* • Erlang,*Play,*Akka

Slide 33

Slide 33 text

Event&driven*concurrency • Thread(based+event+loops • Typically+single+threaded • Popular+among+JavaScript+frameworks • Can+lead+to+the+pyramid(of(doom • Implementa)ons+ • Node.js,+Twisted+Python,+ EventMachine+(Ruby)

Slide 34

Slide 34 text

Entry&level*Java*developer*perspec2ve

Slide 35

Slide 35 text

Reality

Slide 36

Slide 36 text

Building(so+ware(in(2015 Exploring*the*fallacies*of*distributed*compu6ng,*originally*wri:en* by*L.*Peter*Deutsch*in*1994*at*Sun*Microsystems

Slide 37

Slide 37 text

Fallacies(of(distributed(compu2ng • The$network$is$reliable"→"build"systems"that"func3on"even" when"integrated"systems"go"dark

Slide 38

Slide 38 text

Fallacies(of(distributed(compu2ng • The$network$is$reliable"→"build"systems"that"func3on"even" when"integrated"systems"go"dark • Latency$is$zero"→"there's"no"way"to"change"the"speed"of"light

Slide 39

Slide 39 text

Fallacies(of(distributed(compu2ng • The$network$is$reliable"→"build"systems"that"func3on"even" when"integrated"systems"go"dark • Latency$is$zero"→"there's"no"way"to"change"the"speed"of"light • Bandwidth$is$inﬁnite"→"using"that"fancy"JS"framework"is" wonderful"un3l"it"blocks"page"loading"across"100,000"browsers

Slide 40

Slide 40 text

Slide 41

Slide 41 text

Fallacies(of(distributed(compu2ng • Topology'doesn’t'change"→"moving"applica.ons"to"the"cloud" gives"the"ability"to"change"topology"at"will;"add"new"app"servers," scale"on"demand,"etc

Slide 42

Slide 42 text

Fallacies(of(distributed(compu2ng • Topology'doesn’t'change"→"moving"applica.ons"to"the"cloud" gives"the"ability"to"change"topology"at"will;"add"new"app"servers," scale"on"demand,"etc • There'is'one'administrator"→"external"systems"have" performance,"availability,"or"security"issues"that"you"have"no" direct"inﬂuence"over

Slide 43

Slide 43 text

Slide 44

Slide 44 text

Why$func)onal$programming?

Slide 45

Slide 45 text

Side%eﬀect)free)func.ons)enable)parallelism val lastNames = List("Smith","Jones","Frankenstein","Bach","Jackson","Rodin").par // lastNames: scala.collection.parallel.immutable.ParSeq[String] = // ParVector(Smith, Jones, Frankenstein, Bach, Jackson, Rodin) val res0 = lastNames.map(_.toUpperCase) // toUpperCase, higher-order function // res0: scala.collection.parallel.immutable.ParSeq[String] = // ParVector(SMITH, JONES, FRANKENSTEIN, BACH, JACKSON, RODIN) • Crea&ng)a)parallel)collec&on)is)simple • Parallel)opera&ons)are)implemented)with)divide)and)conquer) style)algorithms)that)parallelize)well

Slide 46

Slide 46 text

Example(of(a(chained(transforma3on(and(ﬁlter coll.map(_.toUpperCase).filter(predicate)

Slide 47

Slide 47 text

Enable'concurrency'in'a'transparent'way import scala.util.{Success, Failure} // obtain a future list of recent posts // executes on a different thread pool val f: Future[List[String]] = future { session.getRecentPosts } // success and failure callbacks f onComplete { case Success(posts) => for (post <- posts) println(post) case Failure(t) => println("An error has occured: " + t.getMessage) }

Slide 48

Slide 48 text

Message&passing&to&a&remote&worker val remoteGameActor = system.actorOf( GameActor.props.withDeploy(Deploy(scope = RemoteScope(address))), name = “gameActor”) val remoteGameActor = context.actorSelection( “akka.tcp://[email protected]:2552/gameActor") remoteGameActor ! message

Slide 49

Slide 49 text

Loca%on'transparency

Slide 50

Slide 50 text

Easier'to'maintain'and'reason'about Scala%enables%DRY%code%with%the%following%features: • Higher(order+func0ons • Func0onal+composi0on+&+collec0ons+with+map,+ﬁlter,+reduce,+etc • Case+classes • Pa>ern+matching • Trait+composi0on+ • Etc...

Slide 51

Slide 51 text

As#of#April#7th,#2015,#29#jobs#are# posted#on#LinkedIn#for#Apple# requiring#Scala#or#other#funcAonal# programming#experience.

Slide 52

Slide 52 text

Web$Services$Engineer,$Apple Key$Qualiﬁca,ons • Experience*in*developing*web2services*in*a*commercial*se6ng* and*familiar*with*HTTP,*REST,*JSON • Know*how*to*write*reliable,*resilient,*unit2tested,*performant,* baEle2tested*code • Deep*experience*in*any*one*of*Java,*Scala,*C*,*C#,*F# • Familiar*with*funcKonal*and*reacKve*programming*concepts

Slide 53

Slide 53 text

Slide 54

Slide 54 text

Do#I#expect#College#graduates#to#be#qualified#for#fully#qualified#to# build#distributed#systems#right#out#of#school? • No!%Building%distributed%systems%is%difficult%and%takes%years%of% experience%to%design%and%implement • Building%blocks%can%be%taught%at%the%College%level,%posi?oning% students%for%future%on@the@job%learning,%e.g,%func?onal,%side@ effect@free%programming

Slide 55

Slide 55 text

Slide 56

Slide 56 text

Interested(in(learning(more? • What%Is%Reac+ve%Programming?"#"h%p:/ /bit.ly/1Ay3lHR • Reac+veTO"#"h%p:/ /www.meetup.com/Reac;ve#TO • Func+onal%programming%in%Scala%(Coursera)"#"h%ps:/ / www.coursera.org/course/progfun • Typesafe%Ac+vator"#"h%ps:/ /typesafe.com/community/core# tools/ac;vator#and#sbt

Slide 57

Slide 57 text

Ques%ons?