reactive programming on example of the basar platform

Transcript

1. reactive programming @mnxfst on example of the basar platform
2. /Me • Christian Kreutzfeldt, 35yrs, lives near Hamburg …. • studied computer science at University of Lübeck • started software development w/ JAVA in 1999 at university projects on distributed systems • entered commercial sw dev in 2002 at subshell HH (popfile.de) • moved to work with the Otto Group in 2006 as dev & architect • tech lead of central ecommerce middleware & marketplace platform • since may 2013: protoyping architect & dev at Otto Group BI team
3. Agenda • parallel processing - classical problems • reactive programming - principles • reactive programming & eCommerce - friendship! • basar platform - akka.io based reactive playground • q/a
4. Parallel Processing = classical problems standard approach threads shared mutable state indeterministic root of all evil locks too much/less, wrong, order...
5. Parallel Processing = …amdahl http://upload.wikimedia.org/wikipedia/commons/6/6b/AmdahlsLaw.png 95% code parallelization leads to a speedup factor of only 20
6. Parallel Processing - a solution approach never block go async be domain driven think in workflows rather than seq processes use message passing as communication model share-nothing-architecture event-driven communication from top to bottom Follow Reactive Programming Principles
7. Reactive Programming - principles reactive programming is oriented around data flows and the propagation of change (wikipedia) react to load react to users scalable event-driven resilient responsive react to failure react to events
8. Reactive Programming & eCommerce = friendship! “only a few years ago a large application had tens of servers, seconds of response time, hours of offline maintenance and gigabites of data” “scaling was achieved through buying larger servers and concurrent processing via multi-threading.” reducing costs & satisfy customers makes you king ...BUT... (there is always a BUT) “today applications are deployed on everything from mobile ... to ... clusters” “users expect millisecond ... response times and 100% uptime” (reactivemanifesto.org)
9. Reactive Programming & eCommerce = challenges how to ... ... sync browsers with message passing workflows ... describe workflows / processes ... define & establish application monitoring ...
10. Reactive Programming & eCommerce = requirements change in paradigms ... ... requires change in system design & architecture ... requires change in software development ... requires change in application management ... must have no impact on the business
11. Basar Platform - reactive programming playground objective 1: investigate differences in architecture objective 2: explore differences in programming effort objective 3: prove applicability in ecommerce (master challenges) objective 4: get a feeling from proven ground
12. Basar Platform - reactive programming playground Reco Avail Search Fraud Track Basket Client HTTP !Built-In Multi-Tenancy Support!
13. Basar Platform - first artifact - proven ground get into architecture and code quickly - simple and straightforward use case - serves as sandbox for daily work common ground required event tracking component
14. Basar Platform - event tracking • process arbitrary events (web tracking, log, purchases...) • multi-tenancy support, contractor registers for metrics/kpis • let customer select relevant metrics & KPIs • everything is stateless & transparent, share code base • persist all events as raw data stream • provide inbound & outbound API • compute & update selected KPIs in near-realtime
15. Basar Platform - (very) high level architecture HTTP JMS Remote Actor FS ... Convert Convert Convert Convert Convert Contractor Gateway KPI KPI KPI KPI KPI Metric DB root Cache root kpi db writer metric cache writer
16. Basar Platform - introducing the tools: akka.io Akka is a toolkit and runtime for building highly concurrent, distributed, and fault tolerant event-driven applications on the JVM Actors process messages asynchronously using an event-driven receive loop. They raise abstraction level. You focus on workflow instead of low level primitives like threads, locks and socket IO. Actors are location transparent and distributable by design.
17. Basar Platform - it’s all about hierarchies (akka.io) Root A 1 2 B 3 4 Ownership Supervise Server A Server B
18. Basar Platform - meet the challenges challenge 1: getting from a to b - “hello world” akka style challenge 2: shared code & component base challenge 3: providing multi-tenancy support challenge 4: handling sync’ed workflow responses challenge 5: testing / unit & integration challenge 6: error handling/fault tolerance → F. Esser @ Room 6
19. Basar Platform - “Hello World “akka style final ActorSystem actorSystem = ActorSystem.create(); final ActorRef worldActorRef = actorSystem.actorOf(Props.create(World.class)); final ActorRef helloActorRef = actorSystem.actorOf(Props.create(Hello.class, worldActorRef)); Patterns.ask(helloActorRef, “tell”, new Timeout(500, TimeUnit.MILLISECONDS)); public Hello extends UntypedActor { private final ActorRef worldActorRef; public Hello(final ActorRef worldActorRef) { this.worldActorRef = worldActorRef; } public onReceive(Object msg) throws Exception { if(msg instanceof String.class) { System.out.print(“Hello ”); this.worldActorRef.tell(“Hello”,getSelf()); } } } public World extends UntypedActor { public onReceive(Object msg) throws Exception { if(msg instanceof String.class) { System.out.println(“World!”); } else { unhandledMessag(msg); } } }
20. Basar Platform - share code & component base • metrics and kpis are stateless • technical components (db & cache) located in separate tree • separation of concerns and dedicated supervision strategy • metrics/kpis register their tech components with tree • metrics/kpis forward write events into tree KPI/metric database & cache register
21. Basar Platform - multi-tenancy support • cool: manual implementation having one actor serving as gateway node • much better: use CustomRouterConfig ◦ seemless integration with framework ◦ keeps actors as stateless as possible
22. Basar Platform - sync workflow handling • message passing apps are async by definition … but • requirements exists where sync’ing is required • use Patterns.ask and Await.result ◦ Patterns.ask provide you with Future object ◦ Await.result lets you wait for the process to finish • but be aware that these are BLOCKING calls • upcoming akka-io layer has promising features getting async closer to the frontend
23. Basar Platform - testing • akka.io provides integrated testing framework • be aware that you are testing a message passing app • use TestActorRef to instantiate and access actors, for unit testing instance features -> direct access will fail • use JavaTestKit for integration testing
24. Basar Platform - forecast • basar-track: ◦ more testing on existing code base ◦ implement more kpis/metrics ◦ connector api for writing to different destinations (mongo, memcached, hadoop …) • platform: ◦ implement interconnection framework (research done in iservices project to servce as foundation)
25. Q & A
26. inspired by projects @ Otto Group BI we are hiring www.ottogroup.com/careers
27. @mnxfst twitter.com/mnxfst github.com/mnxfst xing.com/profiles/Christian_Kreutzfeldt