Revitalizing Aging Architectures (Gartner Catalyst)

We take a look into the Walmart Canada case study and how Scala, Play, and Akka was leveraged to modernize a heritage architecture. We then dive into the latest in reactive microservices development and walk through a modern architecture on how a heritage architecture can be re-architected as an event-driven real-time system.

Transcript

1. Revitalizing Aging Architectures Kevin Webber Enterprise Architect, Lightbend
3. The world by 2020 » 4 billion connected people » 25+ million apps » 25+ billion embedded systems » 40 zettabytes (40 trillion gigabytes) » 5,200 GB of data for every person on Earth
6. Goals » ~100% availability ("nine nines")
7. Goals » ~100% availability ("nine nines") » Consistent responsiveness under varying load conditions
8. Goals » ~100% availability ("nine nines") » Consistent responsiveness under varying load conditions » Predictable spikes of traffic, e.g, Black Friday » Less predictable spikes of traffic, e.g, marketing campaign driving traffic
9. Goals » ~100% availability ("nine nines") » Consistent responsiveness under varying load conditions » Predictable spikes of traffic, e.g, Black Friday » Less predictable spikes of traffic, e.g, marketing campaign driving traffic » Enable LOBs to rapidly adapt to market conditions
10. walmart.ca » Added Play in front of customer experience » API enabled heritage architecture for scale » Added Akka in front of back-office » Circuit breakers for resilience » Async message-passing for parallelism
11. Walmart's Business Uplift » Conversions up 20% » Mobile orders up 98% » No downtime Black Friday or Boxing Day
12. Walmart's Operational Savings » Moved off expensive hardware » On cheap virtual x86 servers » 20% - 50% cost savings » ~ 40% compute cycles
13. Heritage challenges
14. Challenges of scale Heritage solution: » Replicate stateless servers in front of middlware » Push out state to the database tier Reality: » The database becomes a single point of failure
15. Challenges of resilience Heritage solution: » DR, active/passive Reality: » No graceful way to handle cascading failure in a heritage system without downtime
16. Challenges in development Heritage solution: » Hire more developers! Reality: » ...
17. heritage applications » Single points of failure » Difficult to scale out (and back in) due to state » Different non-functional requirements between logical boundaries » Long, risky release cycles
18. “Architecture is hard enough without adding unnecessary complexity.” Ross Mason, CTO, MuleSoft
20. SOA Stateless: messaging, message exchange patterns (sync req/resp, async p2p messaging, pub/sub, streams), routing, service orchestration (service aggregation), security (encryption and signing), monitoring, protocol conversion, legacy adapters, payload transformation, schema validation, governance (the ability to apply business rules uniformly), service catalog
21. SOA Stateful: process choreography (BPM), complex event processing (CEP), reliable delivery, transaction management, split and merge (conflating, windowing, ordering)
22. heritage systems » Productivity depends on working within the system context » Creates difficult productivity barriers » E.g, systems-level exploratory development on a laptop?
23. "Jack of all trades, master of none, often times better than the master of one." Give up control to gain flexibility. Embrace new patterns to achieve balance.
24. Solution
25. We need to build these types of systems. asynchronous, non-blocking, real-time, highly- available, loosely coupled, scalable, fault-tolerant, concurrent, reactive, event-driven, push instead of pull, distributed, low latency, high throughput
28. Reactive A maturity model for modern distributed systems. » Responsive » Resilient » Elastic » Message-driven
29. Principles Reactive: » Responsive » Resilient » Elastic » Message-driven
30. Strategy Microservices: » Domain Driven Design (DDD) » Bounded contexts » Process isolation » Distributed systems » Event sourcing » CQRS » Eventual consistency
31. Tools Lightbend Reactive Platform: » Lagom » Play » Akka » Spark
32. The Lightbend Platform
33. Overview » Lagom - Microservices framework » Play - RESTful API framework » Akka - Distributed computing toolkit » Spark - General purpose compute engine
35. Lagom
36. Lagom » Core abstract are Persistent Entities » They move around the cluster automatically » Can replay all state changes » One command to launch all services in a developer environment
37. Play
38. Play » Familiar MVC paradigm » Embraces flows of data » WebSockets, SSE » Reactive Streams » File uploads » Integration with Akka » Distribute work via messaging
39. Akka
40. Actor model » Actors » Conceptually they are very lightweight threads (~300 bytes each) that communicate with each other by messaging » Inspired by Erlang
42. Spark
45. Fast Data Architecture » Akka Streams » < 1 second latency per- event processing » Spark Streaming » 1-10+ second latency for mini-batch event processing » Kafka captures the complete event stream
46. A reactive microservices architecture
47. Microservices » Single responsibility » Independently deployable, scalable, resilient » Communicate via async messaging » Own their data » Don't expose a public API
48. API gateway » Authentication, authorization » Quality of service » Protocol conversions » Well-documented (self- documented) API » Completely stateless
49. Services & integration » Stateful services » Async messaging within service boundaries » Publish/subscribe rather than request/response between services » A system made up entirely of commands, queries, and events
50. Kafka » Publish-subscribe messaging rethought as a distributed commit log » The event stream becomes the database » Distributed and replicated for resilience » Ordering and at-least once delivery
51. Clustered microservices » Responsive to events, load, failure, users » Clusters are resilient » Across nodes, regions, data centers » Actors can fail on one node and come back on another node, with state intact
52. booth 317 Reactive Microservices Architecture: » Go deeper into the theory behind microservices » Learn more about the new architectural patterns required for a successful microservices project » Come by the booth and pick up a copy!
54. Thank you! Visit https://www.lightbend.com to get started Get in touch: Kevin Webber, Enterprise Architect, Lightbend Email: kevin.webber@lightbend.com Medium: medium.com/@kvnwbbr Twitter: @kvnwbbr Booth: 317