Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Revitalizing Aging Architectures (Gartner Catalyst)

Revitalizing Aging Architectures (Gartner Catalyst)

We take a look into the Walmart Canada case study and how Scala, Play, and Akka was leveraged to modernize a heritage architecture. We then dive into the latest in reactive microservices development and walk through a modern architecture on how a heritage architecture can be re-architected as an event-driven real-time system.


Kevin Webber

August 16, 2016

More Decks by Kevin Webber

Other Decks in Programming


  1. Revitalizing Aging Architectures Kevin Webber Enterprise Architect, Lightbend

  2. None
  3. The world by 2020 » 4 billion connected people »

    25+ million apps » 25+ billion embedded systems » 40 zettabytes (40 trillion gigabytes) » 5,200 GB of data for every person on Earth
  4. None
  5. None
  6. Goals » ~100% availability ("nine nines")

  7. Goals » ~100% availability ("nine nines") » Consistent responsiveness under

    varying load conditions
  8. Goals » ~100% availability ("nine nines") » Consistent responsiveness under

    varying load conditions » Predictable spikes of traffic, e.g, Black Friday » Less predictable spikes of traffic, e.g, marketing campaign driving traffic
  9. Goals » ~100% availability ("nine nines") » Consistent responsiveness under

    varying load conditions » Predictable spikes of traffic, e.g, Black Friday » Less predictable spikes of traffic, e.g, marketing campaign driving traffic » Enable LOBs to rapidly adapt to market conditions
  10. walmart.ca » Added Play in front of customer experience »

    API enabled heritage architecture for scale » Added Akka in front of back-office » Circuit breakers for resilience » Async message-passing for parallelism
  11. Walmart's Business Uplift » Conversions up 20% » Mobile orders

    up 98% » No downtime Black Friday or Boxing Day
  12. Walmart's Operational Savings » Moved off expensive hardware » On

    cheap virtual x86 servers » 20% - 50% cost savings » ~ 40% compute cycles
  13. Heritage challenges

  14. Challenges of scale Heritage solution: » Replicate stateless servers in

    front of middlware » Push out state to the database tier Reality: » The database becomes a single point of failure
  15. Challenges of resilience Heritage solution: » DR, active/passive Reality: »

    No graceful way to handle cascading failure in a heritage system without downtime
  16. Challenges in development Heritage solution: » Hire more developers! Reality:

    » ...
  17. heritage applications » Single points of failure » Difficult to

    scale out (and back in) due to state » Different non-functional requirements between logical boundaries » Long, risky release cycles
  18. “Architecture is hard enough without adding unnecessary complexity.” Ross Mason,

    CTO, MuleSoft
  19. None
  20. SOA Stateless: messaging, message exchange patterns (sync req/resp, async p2p

    messaging, pub/sub, streams), routing, service orchestration (service aggregation), security (encryption and signing), monitoring, protocol conversion, legacy adapters, payload transformation, schema validation, governance (the ability to apply business rules uniformly), service catalog
  21. SOA Stateful: process choreography (BPM), complex event processing (CEP), reliable

    delivery, transaction management, split and merge (conflating, windowing, ordering)
  22. heritage systems » Productivity depends on working within the system

    context » Creates difficult productivity barriers » E.g, systems-level exploratory development on a laptop?
  23. "Jack of all trades, master of none, often times better

    than the master of one." Give up control to gain flexibility. Embrace new patterns to achieve balance.
  24. Solution

  25. We need to build these types of systems. asynchronous, non-blocking,

    real-time, highly- available, loosely coupled, scalable, fault-tolerant, concurrent, reactive, event-driven, push instead of pull, distributed, low latency, high throughput
  26. None
  27. None
  28. Reactive A maturity model for modern distributed systems. » Responsive

    » Resilient » Elastic » Message-driven
  29. Principles Reactive: » Responsive » Resilient » Elastic » Message-driven

  30. Strategy Microservices: » Domain Driven Design (DDD) » Bounded contexts

    » Process isolation » Distributed systems » Event sourcing » CQRS » Eventual consistency
  31. Tools Lightbend Reactive Platform: » Lagom » Play » Akka

    » Spark
  32. The Lightbend Platform

  33. Overview » Lagom - Microservices framework » Play - RESTful

    API framework » Akka - Distributed computing toolkit » Spark - General purpose compute engine
  34. None
  35. Lagom

  36. Lagom » Core abstract are Persistent Entities » They move

    around the cluster automatically » Can replay all state changes » One command to launch all services in a developer environment
  37. Play

  38. Play » Familiar MVC paradigm » Embraces flows of data

    » WebSockets, SSE » Reactive Streams » File uploads » Integration with Akka » Distribute work via messaging
  39. Akka

  40. Actor model » Actors » Conceptually they are very lightweight

    threads (~300 bytes each) that communicate with each other by messaging » Inspired by Erlang
  41. None
  42. Spark

  43. None
  44. None
  45. Fast Data Architecture » Akka Streams » < 1 second

    latency per- event processing » Spark Streaming » 1-10+ second latency for mini-batch event processing » Kafka captures the complete event stream
  46. A reactive microservices architecture

  47. Microservices » Single responsibility » Independently deployable, scalable, resilient »

    Communicate via async messaging » Own their data » Don't expose a public API
  48. API gateway » Authentication, authorization » Quality of service »

    Protocol conversions » Well-documented (self- documented) API » Completely stateless
  49. Services & integration » Stateful services » Async messaging within

    service boundaries » Publish/subscribe rather than request/response between services » A system made up entirely of commands, queries, and events
  50. Kafka » Publish-subscribe messaging rethought as a distributed commit log

    » The event stream becomes the database » Distributed and replicated for resilience » Ordering and at-least once delivery
  51. Clustered microservices » Responsive to events, load, failure, users »

    Clusters are resilient » Across nodes, regions, data centers » Actors can fail on one node and come back on another node, with state intact
  52. booth 317 Reactive Microservices Architecture: » Go deeper into the

    theory behind microservices » Learn more about the new architectural patterns required for a successful microservices project » Come by the booth and pick up a copy!
  53. None
  54. Thank you! Visit https://www.lightbend.com to get started Get in touch:

    Kevin Webber, Enterprise Architect, Lightbend Email: kevin.webber@lightbend.com Medium: medium.com/@kvnwbbr Twitter: @kvnwbbr Booth: 317