Reactive Streams at "Underscore" Scala User Group Tel Aviv, 2.7.2015

http://www.meetup.com/de/underscore/events/223584473/

Transcript

2. Reactive Streams and Akka Streams Lutz Hühnken (@lutzhuehnken)
3. Streams
4. Reactive Streams What is a Stream? • Ephemeral flow of data • Potentially unbounded in size • Processed by describing transformation of data 4
5. Reactive Streams Common Use of Streams • Bulk Data Transfer • Real Time Data Sources • Batch Processing of large data sets • Monitoring and Analytics 5
6. Reactive Streams
7. Reactive Streams Reactive 7
8. Reactive Streams Reactive Streams Overview • How do we: • Handle potentially infinite streams of data? • Handle data in a reactive manner? • Achieve asynchronous non-blocking data flow? • Avoid out of memory errors? 8
9. Reactive Streams Reactive Streams Projects / Companies • Typesafe • Akka Streams • Netflix • rxJava / rxScala • Pivotal • Spring Reactor • Redhat • Vert.x • Oracle 9
10. Reactive Streams Supply and Demand • Data Items Flow Downstream • Demand Flows Upstream • Data Items flow only when there is demand. 10
11. Reactive Streams Dynamic Push-Pull • “Push” behavior when consumer is faster • “Pull” behavior when producer is faster • Switches automatically between these • Batching demand allows batching data 11
12. Reactive Streams Reactive Streams Specification • Interface Specification • Java interfaces for implementations • TCK • Test Harness to validate implementations • Specification Website • http://www.reactive-streams.org/ 12
13. Reactive Streams Publisher package org.reactivestreams; public interface Publisher<T> { public void subscribe<T>( Subscriber<T> subscriber); } 13
14. Reactive Streams Subscriber public interface Subscriber<T> { public void onSubscribe( Subscription subscription); public void onNext(T element); public void onComplete(); public void onError(Throwable cause); } 14
15. Reactive Streams Subscription public interface Subscription { public void cancel(); public void request(long elements); } 15
16. Reactive Streams Backpressure • Downstream consumers pushing back on the producer to prevent flooding. • In reactive-streams: • Consumers stop requesting more elements until they are ready to process more. • Producers only fire elements if there is demand. 16
17. Reactive Streams Why Backpressure? • Explicitly design for system overload • Demand is propagated throughout the WHOLE flow • Can decide WHERE to handle overload • Limit the number of in-flight messages throughout the system • Bounded memory consumption • Bounded cpu contention • Recipient is in control of incoming data rate • Data in flight is bounded by signaled demand 17
18. Reactive Streams Buffering • We can prefetch stream elements by requesting more than we really need. • We can use this technique to ensure no "sputter" in the stream. • We can also use this technique to pull faster than downstream consumer. 18
19. Akka Streams & Actors
20. Reactive Streams Basic Actor 20 case class Greeting(who: String) class GreetingActor extends Actor with ActorLogging { def receive = { case Greeting(who) => log.info("Hello " + who) } } val system = ActorSystem("MySystem") val greeter: ActorRef = system.actorOf(Props[GreetingActor]) greeter ! Greeting("London Scala User Group")
21. Reactive Streams Properties of Actors • Message Based / Event Driven • Isolated State • Sane Concurrency Model • Isolated Failure Handling (Supervision) 21
22. Reactive Streams Akka Streams – A Bridge • Converts Publisher/Subscriber API into Actor messages • Simplify creating Publisher/Subscribers using Actors • Attach Reactive streams into existing Akka applications 22
23. Reactive Streams Creating an ActorSubscriber import akka.stream._ class PrintlnActor extends Actor with ActorSubscriber { val requestStrategy = OneByOneRequestStrategy def receive: Receive = { case ActorSubscriberMessage.OnNext(element) => println(element) } } val printlnActor:ActorRef = system.actorOf(Props[PrintlnActor], "println") val subscriber = ActorSubscriber(printlnActor) 23
24. Reactive Streams Creating an ActorPublisher import akka.stream._ class IntActor extends Actor with ActorPublisher[Int] { def receive: Receive = { case ActorPublisherMessage.Request(elements) => while (totalDemand > 0) { onNext(1) } } } val intActor: ActorRef = system.actorOf(Props[IntActor], "intActor") val publisher = ActorPublisher(intActor) 24
25. Reactive Streams Why use Actors as Publishers? • Actors are smart • They can keep internal state to track demand and supply • They can buffer data to meet anticipated demand • Actors are powerful • They can spin up child actors to meet demand • With Akka clustering, can spread load across multiple machines • Actors are resilient • On exception, actor can be killed and restarted by supervisor • Actor interaction is thread-safe, and actor state is private 25
26. Switch to code
27. Flows
28. Reactive Streams Linear Transformations 28 Demand Source Sink map …
29. Reactive Streams Linear Transformations 29 Source Demand Sink drop map
30. Reactive Streams Linear Stream Transformations • Deterministic (like for collections) • map, filter, collect, grouped, drop, take, groupBy, ... • Time-Based • takeWithin, dropWithin, groupedWithin, ... • Rate-Detached • expand, conflate, buffer, ... • asynchronous • mapAsync, mapAsyncUnordered, ... 30
31. Reactive Streams Non-linear Stream Transformations • Fan-In • merge, concat, zip • Fan-Out • broadcast, route, unzip 31
32. Reactive Streams Fan In 32
33. Reactive Streams Fan Out 33
34. Reactive Streams Materialization • Akka Streams separate the what from the how • declarative Source/Flow/Sink DSL to create blueprint • FlowMaterializer turns this into running Actors • this allows alternative materialization strategies • optimization • verification / validation • cluster deployment • only Akka Actors for now, but more to come! 34
35. Switch to code
36. Slick
37. Reactive Streams JDBC • Synchronous I/O • Will block the thread it’s running on. • This is true not only for JDBC, but any DB lib running on top of JDBC, Slick included. We like to avoid that. 37
38. Reactive Streams Imperfect Solution: Use Future / blocking combo • Put blocking database calls in Future(blocking( ... )) • Contention for Connections (but may be limited by the ExecutionContext) • A saturated thread pool blocks everything. 38
39. Reactive Streams Slick Every Database contains an AsyncExecutor that manages the thread pool for asynchronous execution of Database I/O Actions. 39 mydb = { dataSourceClass = "org.postgresql.ds.PGSimpleDataSource" properties = { databaseName = "mydb" user = "myuser" password = "secret" } numThreads = 10 }
40. Akka HTTP
41. Image by Mathias Doenitz
43. Interoperability
44. Image by Mathias Doenitz
45. Switch to code
46. Thanks @lutzhuehnken
47. ©Typesafe 2014 – All Rights Reserved