Events-First Microservices with Lagom

Events-First Microservices with Lagom

Even with microservices starting to become a commodity, a lot of implementations still fail as microliths. To ensure the autonomy of services and its development process, strong coupling between services should be limited as much as possible. The way to achieve this is to isolate the inner working of a service at cost and to switch from a commanding and synchronous to a promising and asynchronous way of communication between services.

A route to success is to stop architectural design from a perspective of services with well-defined and canonical relationships and to switch to a model which is based on the production of immutable domain events. Not only does this way of architecting ensure a better match with the things that happen within your business. The focus on the timeline of facts over the structures producing them ensures far less coupling within your system.

In this talk, I will share how Lagom embraces this way of thinking by pushing events-first development as a default. From the event-sourcing principles within aggregates to the message-broker functionality between services, I’ll give you the background of the elements which are in place to shift your system to an event-driven microservice architecture.

01abe62e641a626ed2dccc08ea2f8a14?s=128

Kenny Baas-Schwegler

April 17, 2018
Tweet

Transcript

  1. Events-First Microservices With Lagom Gideon de Kok @gideondk Kenny Baas

    @kenny_baas
  2. 2 IT’S YOUR BUSINESS. WE ACCELERATE IT. XEBIA GROUP CONSISTS

    OF SEVEN SPECIALIZED, INTERLINKED COMPANIES WITH OFFICES IN AMSTERDAM AND HILVERSUM (NETHERLANDS), PARIS, DELHI, BANGALORE AND BOSTON, WE EMPLOY OVER 700 PEOPLE WORLDWIDE
  3. Microservices

  4. We’re often forcing our current problem into microservices instead of

    complimenting it
  5. None
  6. ShipOrder Order Shipping

  7. ShipOrder Order Shipping

  8. “Any organization that designs a system will produce a design

    whose structure is a copy of the organization's communication structure.”
 
 — Melvin Conway, 1967
  9. None
  10. Order Team Shipping Team ShipOrder Order Shipping

  11. None
  12. None
  13. Boundaries aren’t only about form, they’re also about communication patterns

    and language
  14. None
  15. We not only need boundaries, We need an architectural pattern

    which enable autonomy within and over those boundaries
  16. None
  17. Structuring business processes into domains, and aligning those with technology

    Domain Driven Design
  18. An explicit boundary for language, complexity, logic, context Bounded Contexts

  19. Structure-First Domain-Driven Design

  20. Structure weights you down. Something that has been visualised is

    hard to be unvisualised.
  21. Design is not just what it looks like and feels

    like. Design is how it works.
  22. It is not the things that matter (in early stages

    of design). It is the things that happen — Russ Miles
  23. “Event”
 1. A thing that happens or takes place, especially

    one of importance.
  24. Events-First Domain-Driven Design™,

  25. "Software development is a learning process; working code is a

    side effect." — Alberto Brandolini
  26. None
  27. None
  28. None
  29. None
  30. None
  31. None
  32. None
  33. None
  34. Order Created

  35. Order Confirmed Order Created Order Paid Order Shipped Order Delivered

    Order Picked
  36. Order Shipping Order Confirmed Order Created Order Paid Order Shipped

    Order Delivered Order Picked
  37. package com.xebia.webshop.order.api sealed trait OrderEvent object OrderEvent { case class

    OrderCreated(orderId: String, orderLines: Seq[OrderLine], shippingInfo: ShippingInfo) extends OrderEvent case class OrderConfirmed(orderId: String) extends OrderEvent case class OrderPaid(orderId: String, orderLines: Seq[OrderLine], shippingInfo: ShippingInfo) extends OrderEvent object OrderCreated { implicit val format: Format[OrderCreated] = Json.format } object OrderConfirmed { implicit val format: Format[OrderConfirmed] = Json.format } object OrderPaid { implicit val format: Format[OrderPaid] = Json.format } implicit val format: Format[OrderEvent] = derived.flat.oformat((__ \ "type").format[String]) }
  38. package com.xebia.webshop.shipping.api sealed trait OrderEvent object OrderEvent { case class

    OrderPicked(orderId: String, pickedBy: Employee) extends OrderEvent case class OrderShipped(orderId: String, shippingReference: String) extends OrderEvent case class OrderDelivered(orderId: String, wasDeliveredAt: LocalDateTime) extends OrderEvent object OrderPicked { implicit val format: Format[OrderPicked] = Json.format } object OrderShipped { implicit val format: Format[OrderShipped] = Json.format } object OrderDelivered { implicit val format: Format[OrderDelivered] = Json.format } implicit val format: Format[OrderEvent] = derived.flat.oformat((__ \ "type").format[String]) }
  39. “Lagom” /lah-gome/ 1. (just) right, fitting, neither too much nor

    too little
  40. Lagom provides an opinionated way of building microservices that intentionally

    constrains what a developer can do and how they should do it.
  41. package com.xebia.webshop.order.api import akka.NotUsed import com.lightbend.lagom.scaladsl.api.broker.Topic import com.lightbend.lagom.scaladsl.api.transport._ import com.lightbend.lagom.scaladsl.api.{Descriptor,

    Service, ServiceCall} trait OrderService extends Service { import Method._ import Service._ def orderEvents: Topic[OrderEvent] final override def descriptor = { named("order").withTopics( topic("order-OrderEvent", orderEvents) ) } }
  42. Order Paid Order Picked Order Shipping

  43. Order Paid Order Picked Whenever an Order is Paid Order

    Shipping
  44. class OrderServicePolicy(orderService: OrderService) { orderService.orderEvents.subscribe.atLeastOnce(Flow[api.OrderEvent].mapAsync(1) { case x: OrderPaid =>

    ??? // Where we should trigger our side-effect case other => Future.successful(Done) }) }
  45. package com.xebia.webshop.order.api import akka.NotUsed import com.lightbend.lagom.scaladsl.api.broker.Topic import com.lightbend.lagom.scaladsl.api.transport._ import com.lightbend.lagom.scaladsl.api.{Descriptor,

    Service, ServiceCall} trait OrderService extends Service { import Method._ import Service._ def orderEvents: Topic[OrderEvent] final override def descriptor = { named("order").withTopics( topic("order-OrderEvent", orderEvents) ) } }
  46. Order Paid Order Picked Whenever an Order is Paid Pick

    Order Pay Order Order Shipping
  47. Order Order Picked Whenever an Order is Paid Pick Order

    Order Order Paid Pay Order Order Shipping
  48. Create Order Order Add Payment Information Order Confirmed Order Created

    Order Paid
  49. Order OrderLine OrderLine OrderLine CreateOrder Ack OrderCreated Akka Actor

  50. State OrderPaid OrderConfirmed OrderUpdated Akka Persistence

  51. Order 1 Order Domain Order 2 Order 3 Akka Singleton

    (Entity Framework)
  52. Hash(Id(Order 1)) Hash(Id(Order 2)) Hash(Id(Order 3)) A B C Akka

    Cluster (Auto-Sharding)
  53. Create Order Order Pay Order Order Confirmed Order Created Order

    Paid
  54. trait POrderCommand object POrderCommand { case class CreateOrder(orderLines: Seq[POrderLine], info:

    PShippingInfo) extends POrderCommand with ReplyType[Done] case class PayOrder(reference: String) extends POrderCommand with ReplyType[Done] case object GetOrder extends POrderCommand with ReplyType[POrder] }
  55. trait POrderEvent extends AggregateEvent[POrderEvent] { override def aggregateTag: AggregateEventTagger[POrderEvent] =

    POrderEvent.Tag } object POrderEvent { val Tag = AggregateEventTag[POrderEvent] case class OrderCreated( orderLines: Seq[POrderLine], shippingInformation: PShippingInfo) extends POrderEvent case object OrderConfirmed extends POrderEvent case class OrderPaid(reference: String) extends POrderEvent }
  56. case class POrder(id: String, orderLines: Seq[POrderLine], shippingInfo: Option[PShippingInfo], paymentReference: Option[String],

    confirmed: Boolean) object POrder { case class PShippingInfo(address: String, zipcode: String, city: String, country: String) case class PSKU(id: String) case class POrderLine(sku: PSKU, amount: Int) }
  57. class OrderEntity extends PersistentEntity { override type Command = POrderCommand

    override type Event = POrderEvent override type State = POrder override def initialState = POrder(this.entityId, Seq.empty, None, None, false) override def behavior: Behavior = Actions() .onCommand[POrderCommand.CreateOrder, Done] { case (POrderCommand.CreateOrder(orderLines, shippingInfo), ctx, state) => { ctx.thenPersist(POrderEvent.OrderCreated(orderLines, shippingInfo)) { evt => ctx.reply(Done) } } }.onCommand[POrderCommand.PayOrder, Done] { case (POrderCommand.PayOrder(reference), ctx, state) => { ctx.thenPersistAll(POrderEvent.OrderPaid(reference), POrderEvent.OrderConfirmed) { () => ctx.reply(Done) } } }.onReadOnlyCommand[GetOrder.type, POrder] { case (GetOrder, ctx, state) => ctx.reply(state) }.onEvent { case (POrderEvent.OrderCreated(lines, shippingInfo), state) => state.copy(orderLines = lines, shippingInfo = Some(shippingInfo)) case (POrderEvent.OrderPaid(reference), state) => state.copy(paymentReference = Some(reference)) case (POrderEvent.OrderConfirmed, state) => state.copy(confirmed = true) } }
  58. class OrderEntity extends PersistentEntity { override type Command = POrderCommand

    override type Event = POrderEvent override type State = POrder override def initialState = POrder(this.entityId, Seq.empty, None, None, false) override def behavior: Behavior = Actions() .onCommand[POrderCommand.CreateOrder, Done] { case (POrderCommand.CreateOrder(orderLines, shippingInfo), ctx, state) => { ctx.thenPersist(POrderEvent.OrderCreated(orderLines, shippingInfo)) { evt => ctx.reply(Done) } } }.onCommand[POrderCommand.PayOrder, Done] { case (POrderCommand.PayOrder(reference), ctx, state) => { ctx.thenPersistAll(POrderEvent.OrderPaid(reference), POrderEvent.OrderConfirmed) { () => ctx.reply(Done) } } }.onReadOnlyCommand[GetOrder.type, POrder] { case (GetOrder, ctx, state) => ctx.reply(state) }.onEvent { case (POrderEvent.OrderCreated(lines, shippingInfo), state) => state.copy(orderLines = lines, shippingInfo = Some(shippingInfo)) case (POrderEvent.OrderPaid(reference), state) => state.copy(paymentReference = Some(reference)) case (POrderEvent.OrderConfirmed, state) => state.copy(confirmed = true) } }
  59. class OrderEntity extends PersistentEntity { override type Command = POrderCommand

    override type Event = POrderEvent override type State = POrder override def initialState = POrder(this.entityId, Seq.empty, None, None, false) override def behavior: Behavior = Actions() .onCommand[POrderCommand.CreateOrder, Done] { case (POrderCommand.CreateOrder(orderLines, shippingInfo), ctx, state) => { ctx.thenPersist(POrderEvent.OrderCreated(orderLines, shippingInfo)) { evt => ctx.reply(Done) } } }.onCommand[POrderCommand.PayOrder, Done] { case (POrderCommand.PayOrder(reference), ctx, state) => { ctx.thenPersistAll(POrderEvent.OrderPaid(reference), POrderEvent.OrderConfirmed) { () => ctx.reply(Done) } } }.onReadOnlyCommand[GetOrder.type, POrder] { case (GetOrder, ctx, state) => ctx.reply(state) }.onEvent { case (POrderEvent.OrderCreated(lines, shippingInfo), state) => state.copy(orderLines = lines, shippingInfo = Some(shippingInfo)) case (POrderEvent.OrderPaid(reference), state) => state.copy(paymentReference = Some(reference)) case (POrderEvent.OrderConfirmed, state) => state.copy(confirmed = true) } }
  60. class OrderEntity extends PersistentEntity { override type Command = POrderCommand

    override type Event = POrderEvent override type State = POrder override def initialState = POrder(this.entityId, Seq.empty, None, None, false) override def behavior: Behavior = Actions() .onCommand[POrderCommand.CreateOrder, Done] { case (POrderCommand.CreateOrder(orderLines, shippingInfo), ctx, state) => { ctx.thenPersist(POrderEvent.OrderCreated(orderLines, shippingInfo)) { evt => ctx.reply(Done) } } }.onCommand[POrderCommand.PayOrder, Done] { case (POrderCommand.PayOrder(reference), ctx, state) => { ctx.thenPersistAll(POrderEvent.OrderPaid(reference), POrderEvent.OrderConfirmed) { () => ctx.reply(Done) } } }.onReadOnlyCommand[GetOrder.type, POrder] { case (GetOrder, ctx, state) => ctx.reply(state) }.onEvent { case (POrderEvent.OrderCreated(lines, shippingInfo), state) => state.copy(orderLines = lines, shippingInfo = Some(shippingInfo)) case (POrderEvent.OrderPaid(reference), state) => state.copy(paymentReference = Some(reference)) case (POrderEvent.OrderConfirmed, state) => state.copy(confirmed = true) } }
  61. Event Processor OrderConfirmed OrderPaid OrderChanged OrderCreated Repository (Payment => Order)

    ID: A ID: B ID: C OrderConfirmed OrderPaid OrderChanged OrderCreated OrderConfirmed OrderPaid OrderChanged OrderCreated
  62. Event Processor SEQ: 12 #1 #2 #3 #4 #5 #6

    #7 #8 #9 #10 #11 #12 SEQ: 0 ID: A ID: B ID: C OrderConfirmed OrderPaid OrderChanged OrderCreated OrderConfirmed OrderPaid OrderChanged OrderCreated OrderConfirmed OrderPaid OrderChanged OrderCreated Repository (Payment => Order)
  63. Event Processor SEQ: 12 #1 #2 #3 #4 #5 #6

    #7 #8 #9 #10 #11 #12 SEQ: 12 ID: A ID: B ID: C OrderConfirmed OrderPaid OrderChanged OrderCreated OrderConfirmed OrderPaid OrderChanged OrderCreated OrderConfirmed OrderPaid OrderChanged OrderCreated Repository (Payment => Order)
  64. class OrderServiceImpl(persistentEntityRegistry: PersistentEntityRegistry)(implicit ec: ExecutionContext) extends OrderService { override def

    payOrder(orderId: String): ServiceCall[ConfirmOrderRequest, NotUsed] = ServiceCall { req => persistentEntityRegistry.refFor[OrderEntity](orderId).ask(POrderCommand.PayOrder(req.paymentReference)).map(_ => NotUsed) } override def orderEvents = TopicProducer.singleStreamWithOffset { offset => persistentEntityRegistry.eventStream(POrderEvent.Tag, offset).filter(e => /* Filter some of the internal events to isolate some of the entities internal logic */ e.event.isInstanceOf[POrderEvent.OrderCreated] || e.event == POrderEvent.OrderPaid ).mapAsync(1) { event => event.event match { case POrderEvent.OrderCreated(orderLines, shippingInfo) => val message = api.OrderEvent.OrderCreated(event.entityId, orderLines.map(x => api.Order.OrderLine(api.Order.SKU(x.sku.id), x.amount)), ShippingInfo(shippingInfo.address, shippingInfo.zipcode, shippingInfo.city, shippingInfo.country)) Future.successful((message, event.offset)) case x: POrderEvent.OrderPaid => persistentEntityRegistry.refFor[OrderEntity](event.entityId).ask(POrderCommand.GetOrder).map { x => val shippingInfo = x.shippingInfo.getOrElse(throw new Exception("Shipping info is expected to be set when the order is confirmed")) val message = api.OrderEvent.OrderPaid( orderId = event.entityId, orderLines = x.orderLines.map(x => api.Order.OrderLine(api.Order.SKU(x.sku.id), x.amount)), shippingInfo = api.Order.ShippingInfo(shippingInfo.address, shippingInfo.zipcode, shippingInfo.city, shippingInfo.country) ) (message, event.offset) } } } } }
  65. Order Shipping Message Broker OrderPaid PickOrder OrderPaid

  66. Order Order Picked Whenever an Order is Paid Pick Order

    Order Order Paid Pay Order Order Shipping
  67. Order Shipping Message Broker OrderPaid PickOrder OrderPaid Policy

  68. class OrderServicePolicy(persistentEntityRegistry: PersistentEntityRegistry, orderService: OrderService) { orderService.orderEvents.subscribe.atLeastOnce(Flow[OrderEvent].mapAsync(1) { case x:

    OrderPaid => entityRef(x.orderId).ask( PickOrder(x.orderId, PAddress(x.shippingInfo.address, x.shippingInfo.city, x.shippingInfo.address, x.shippingInfo.zipcode) )) case other => Future.successful(Done) }) private def entityRef(orderId: UUID) = persistentEntityRegistry.refFor[POrder](orderId.toString) }
  69. Purposes are deduced from behaviour, not from rhetoric or stated

    goals — Donella H. Meadows
  70. Autonomy is reached by implementing policies for behaviour instead of

    commanding behaviour
  71. In this, we can treat REST as an anti-pattern for

    domain to domain communication
  72. As long as events are readable on a topic, the

    origin is of less importance
  73. Decouples behaviour, communication from structure

  74. Events-first delivers on a better way of modelling businesses in

    technology, while getting the rest for free
  75. Q&A