Slide 1

Slide 1 text

reactive programming @mnxfst on example of the basar platform

Slide 2

Slide 2 text

/Me ● Christian Kreutzfeldt, 35yrs, lives near Hamburg …. ● studied computer science at University of Lübeck ● started software development w/ JAVA in 1999 at university projects on distributed systems ● entered commercial sw dev in 2002 at subshell HH (popfile.de) ● moved to work with the Otto Group in 2006 as dev & architect ● tech lead of central ecommerce middleware & marketplace platform ● since may 2013: protoyping architect & dev at Otto Group BI team

Slide 3

Slide 3 text

Agenda ● parallel processing - classical problems ● reactive programming - principles ● reactive programming & eCommerce - friendship! ● basar platform - akka.io based reactive playground ● q/a

Slide 4

Slide 4 text

Parallel Processing = classical problems standard approach threads shared mutable state indeterministic root of all evil locks too much/less, wrong, order...

Slide 5

Slide 5 text

Parallel Processing = …amdahl http://upload.wikimedia.org/wikipedia/commons/6/6b/AmdahlsLaw.png 95% code parallelization leads to a speedup factor of only 20

Slide 6

Slide 6 text

Parallel Processing - a solution approach never block go async be domain driven think in workflows rather than seq processes use message passing as communication model share-nothing-architecture event-driven communication from top to bottom Follow Reactive Programming Principles

Slide 7

Slide 7 text

Reactive Programming - principles reactive programming is oriented around data flows and the propagation of change (wikipedia) react to load react to users scalable event-driven resilient responsive react to failure react to events

Slide 8

Slide 8 text

Reactive Programming & eCommerce = friendship! “only a few years ago a large application had tens of servers, seconds of response time, hours of offline maintenance and gigabites of data” “scaling was achieved through buying larger servers and concurrent processing via multi-threading.” reducing costs & satisfy customers makes you king ...BUT... (there is always a BUT) “today applications are deployed on everything from mobile ... to ... clusters” “users expect millisecond ... response times and 100% uptime” (reactivemanifesto.org)

Slide 9

Slide 9 text

Reactive Programming & eCommerce = challenges how to ... ... sync browsers with message passing workflows ... describe workflows / processes ... define & establish application monitoring ...

Slide 10

Slide 10 text

Reactive Programming & eCommerce = requirements change in paradigms ... ... requires change in system design & architecture ... requires change in software development ... requires change in application management ... must have no impact on the business

Slide 11

Slide 11 text

Basar Platform - reactive programming playground objective 1: investigate differences in architecture objective 2: explore differences in programming effort objective 3: prove applicability in ecommerce (master challenges) objective 4: get a feeling from proven ground

Slide 12

Slide 12 text

Basar Platform - reactive programming playground Reco Avail Search Fraud Track Basket Client HTTP !Built-In Multi-Tenancy Support!

Slide 13

Slide 13 text

Basar Platform - first artifact - proven ground get into architecture and code quickly - simple and straightforward use case - serves as sandbox for daily work common ground required event tracking component

Slide 14

Slide 14 text

Basar Platform - event tracking ● process arbitrary events (web tracking, log, purchases...) ● multi-tenancy support, contractor registers for metrics/kpis ● let customer select relevant metrics & KPIs ● everything is stateless & transparent, share code base ● persist all events as raw data stream ● provide inbound & outbound API ● compute & update selected KPIs in near-realtime

Slide 15

Slide 15 text

Basar Platform - (very) high level architecture HTTP JMS Remote Actor FS ... Convert Convert Convert Convert Convert Contractor Gateway KPI KPI KPI KPI KPI Metric DB root Cache root kpi db writer metric cache writer

Slide 16

Slide 16 text

Basar Platform - introducing the tools: akka.io Akka is a toolkit and runtime for building highly concurrent, distributed, and fault tolerant event-driven applications on the JVM Actors process messages asynchronously using an event-driven receive loop. They raise abstraction level. You focus on workflow instead of low level primitives like threads, locks and socket IO. Actors are location transparent and distributable by design.

Slide 17

Slide 17 text

Basar Platform - it’s all about hierarchies (akka.io) Root A 1 2 B 3 4 Ownership Supervise Server A Server B

Slide 18

Slide 18 text

Basar Platform - meet the challenges challenge 1: getting from a to b - “hello world” akka style challenge 2: shared code & component base challenge 3: providing multi-tenancy support challenge 4: handling sync’ed workflow responses challenge 5: testing / unit & integration challenge 6: error handling/fault tolerance → F. Esser @ Room 6

Slide 19

Slide 19 text

Basar Platform - “Hello World “akka style final ActorSystem actorSystem = ActorSystem.create(); final ActorRef worldActorRef = actorSystem.actorOf(Props.create(World.class)); final ActorRef helloActorRef = actorSystem.actorOf(Props.create(Hello.class, worldActorRef)); Patterns.ask(helloActorRef, “tell”, new Timeout(500, TimeUnit.MILLISECONDS)); public Hello extends UntypedActor { private final ActorRef worldActorRef; public Hello(final ActorRef worldActorRef) { this.worldActorRef = worldActorRef; } public onReceive(Object msg) throws Exception { if(msg instanceof String.class) { System.out.print(“Hello ”); this.worldActorRef.tell(“Hello”,getSelf()); } } } public World extends UntypedActor { public onReceive(Object msg) throws Exception { if(msg instanceof String.class) { System.out.println(“World!”); } else { unhandledMessag(msg); } } }

Slide 20

Slide 20 text

Basar Platform - share code & component base ● metrics and kpis are stateless ● technical components (db & cache) located in separate tree ● separation of concerns and dedicated supervision strategy ● metrics/kpis register their tech components with tree ● metrics/kpis forward write events into tree KPI/metric database & cache register

Slide 21

Slide 21 text

Basar Platform - multi-tenancy support ● cool: manual implementation having one actor serving as gateway node ● much better: use CustomRouterConfig ○ seemless integration with framework ○ keeps actors as stateless as possible

Slide 22

Slide 22 text

Basar Platform - sync workflow handling ● message passing apps are async by definition … but ● requirements exists where sync’ing is required ● use Patterns.ask and Await.result ○ Patterns.ask provide you with Future object ○ Await.result lets you wait for the process to finish ● but be aware that these are BLOCKING calls ● upcoming akka-io layer has promising features getting async closer to the frontend

Slide 23

Slide 23 text

Basar Platform - testing ● akka.io provides integrated testing framework ● be aware that you are testing a message passing app ● use TestActorRef to instantiate and access actors, for unit testing instance features -> direct access will fail ● use JavaTestKit for integration testing

Slide 24

Slide 24 text

Basar Platform - forecast ● basar-track: ○ more testing on existing code base ○ implement more kpis/metrics ○ connector api for writing to different destinations (mongo, memcached, hadoop …) ● platform: ○ implement interconnection framework (research done in iservices project to servce as foundation)

Slide 25

Slide 25 text

Q & A

Slide 26

Slide 26 text

inspired by projects @ Otto Group BI we are hiring www.ottogroup.com/careers

Slide 27

Slide 27 text

@mnxfst twitter.com/mnxfst github.com/mnxfst xing.com/profiles/Christian_Kreutzfeldt