Building a Reactive e-commerce platform - Sting

Transcript

1. Events-first Microservices Moving away from the monolith with success Gideon de Kok @gideondk
2. 2 IT’S YOUR BUSINESS. WE ACCELERATE IT. XEBIA GROUP CONSISTS OF SEVEN SPECIALISED, INTERLINKED COMPANY WITH OFFICES IN AMSTERDAM AND HILVERSUM (NETHERLANDS), PARIS, DELHI, BANGALORE AND BOSTON, WE EMPLOY OVER 700 PEOPLE WORLDWIDE
3. Microservices
6. We’re often forcing our problem into microservices instead of complimenting it
8. Complexity Productivity Working in a highly coupled system increases collaboration and productivity in initial phases Increased complexity from larger code base start to hinder progress Overall velocity reaches a potential stable but low point
9. For systems with less complexity, the overhead required to get things running reduces initial productivity A true modular architecture with correct boundaries profits from localised complexity The complexity of a business will always impact productivity in the end but overhead is reduced significantly Complexity Productivity
10. Complexity Productivity Working in a highly coupled system increases collaboration and productivity in initial phases Increased complexity from larger code base start to hinder progress Overall velocity reaches a potential stable but low point
11. /
12. Two separate monoliths on Hybris, abysmal test coverage, quality issues, tight coupling everywhere
13. Basket Shipping Order Product Stock Hybris
14. Desire to build a platform which enables a multi-brand platform in a business flexibility enhancing environment
16. Issues don’t come from software stacks, but the way they’re used to compose and develop our software (it’s a learning process…)
17. We should not be focused on microservices, we should be focused on creating architectures which enable clarity, loose coupling and autonomy
18. Command Service A Service B
19. Command Service A Service B
21. We not only need boundaries, We need an architectural pattern which enable autonomy within and over those boundaries
22. Structuring business processes into domains, and aligning those with technology Domain Driven Design
23. An explicit boundary for language, complexity, logic Bounded Contexts
24. Structure-First Domain-Driven Design
25. Structure weights you down. Something that has been visualised is hard to be unvisualised.
26. It is not the things that matter (in early stages of design), it is the things that happen — Russ Miles
27. “Event”
 1. A thing that happens or takes place, especially one of importance.
28. Events-First Domain-Driven Design™,
33. Order Created
34. Order Confirmed Order Created Order Paid Order Shipped Order Delivered Order Picked
35. Order Shipping Order Confirmed Order Created Order Paid Order Shipped Order Delivered Order Picked
36. package com.xebia.webshop.order.api sealed trait OrderEvent object OrderEvent { case class OrderCreated(orderId: String, orderLines: Seq[OrderLine], shippingInfo: ShippingInfo) extends OrderEvent case class OrderConfirmed(orderId: String) extends OrderEvent case class OrderPaid(orderId: String, orderLines: Seq[OrderLine], shippingInfo: ShippingInfo) extends OrderEvent object OrderCreated { implicit val format: Format[OrderCreated] = Json.format } object OrderConfirmed { implicit val format: Format[OrderConfirmed] = Json.format } object OrderPaid { implicit val format: Format[OrderPaid] = Json.format } implicit val format: Format[OrderEvent] = derived.flat.oformat((__ \ "type").format[String]) }
37. package com.xebia.webshop.shipping.api sealed trait OrderEvent object OrderEvent { case class OrderPicked(orderId: String, pickedBy: Employee) extends OrderEvent case class OrderShipped(orderId: String, shippingReference: String) extends OrderEvent case class OrderDelivered(orderId: String, wasDeliveredAt: LocalDateTime) extends OrderEvent object OrderPicked { implicit val format: Format[OrderPicked] = Json.format } object OrderShipped { implicit val format: Format[OrderShipped] = Json.format } object OrderDelivered { implicit val format: Format[OrderDelivered] = Json.format } implicit val format: Format[OrderEvent] = derived.flat.oformat((__ \ "type").format[String]) }
38. “Lagom” /lah-gome/ 1. (just) right, fitting, neither too much nor too little
39. Lagom provides an opinionated way of building microservices that intentionally constrains what a developer can do and how they should do it.
40. • Build your services using the concepts of Domain Driven Design • Services communicate on basis of contracts • Your event-log is the single point of truth, Event Source all your state* • Separate your writes from your reads / queries (CQRS) • Make your services stateful as a default** • Make your services elastic and resilient as a default
41. • Persistent-ignorant design with support for multiple storage- engines out of the box • Default end-to-end support for message brokers • Able to run, test and continuously compile a complete microservice system on your local machine • Java / Scala support (and Kotlin if you bend it) • Spawns Cassandra & Kafka during local execution to get from idea to development after installation • Service locators & Circuit Breakers
42. package com.xebia.webshop.order.api import akka.NotUsed import com.lightbend.lagom.scaladsl.api.broker.Topic import com.lightbend.lagom.scaladsl.api.transport._ import com.lightbend.lagom.scaladsl.api.{Descriptor, Service, ServiceCall} trait OrderService extends Service { import Method._ import Service._ def orderEvents: Topic[OrderEvent] final override def descriptor = { named("order").withTopics( topic("order-OrderEvent", orderEvents) ) } }
43. Order Paid Order Picked Order Shipping
44. Order Paid Order Picked Whenever an Order is Paid Order Shipping
45. class OrderServiceSubscriber(orderService: OrderService) { orderService.orderEvents.subscribe.atLeastOnce(Flow[api.OrderEvent].mapAsync(1) { case x: OrderPaid => ??? // Where we should trigger our side-effect case other => Future.successful(Done) }) }
46. Order Paid Order Picked Whenever an Order is Paid Pick Order Add Payment Information Order Shipping
47. Create Order Order Add Payment Information Order Confirmed Order Created Order Paid
48. Order OrderLine OrderLine OrderLine CreateOrder Ack OrderCreated
49. State OrderPaid OrderConfirmed OrderUpdated OrderCreated
50. Order 1 Order Domain Order 2 Order 3
51. Hash(Id(Order 1)) Hash(Id(Order 2)) Hash(Id(Order 3)) Akka Cluster (Auto-Sharding)
52. Hash(Id(Order 1)) Hash(Id(Order 2)) Hash(Id(Order 3)) A B C Akka Cluster (Auto-Sharding)
53. Hash(Id(Order 1)) Hash(Id(Order 2)) Hash(Id(Order 3)) A B C Akka Cluster (Auto-Sharding)
54. trait POrderCommand object POrderCommand { case class CreateOrder(orderLines: Seq[POrderLine], info: PShippingInfo) extends POrderCommand with ReplyType[Done] case class AddPayment(reference: String) extends POrderCommand with ReplyType[Done] case object GetOrder extends POrderCommand with ReplyType[POrder] }
55. trait POrderEvent extends AggregateEvent[POrderEvent] { override def aggregateTag: AggregateEventTagger[POrderEvent] = POrderEvent.Tag } object POrderEvent { val Tag = AggregateEventTag[POrderEvent] case class OrderCreated( orderLines: Seq[POrderLine], shippingInformation: PShippingInfo) extends POrderEvent case object OrderConfirmed extends POrderEvent case class OrderPaid(reference: String) extends POrderEvent }
56. case class POrder(id: String, orderLines: Seq[POrderLine], shippingInfo: Option[PShippingInfo], paymentReference: Option[String], confirmed: Boolean) object POrder { case class PShippingInfo(address: String, zipcode: String, city: String, country: String) case class PSKU(id: String) case class POrderLine(sku: PSKU, amount: Int) }
57. class OrderEntity extends PersistentEntity { override type Command = POrderCommand override type Event = POrderEvent override type State = POrder override def initialState = POrder(this.entityId, Seq.empty, None, None, false) override def behavior: Behavior = Actions() .onCommand[POrderCommand.CreateOrder, Done] { case (POrderCommand.CreateOrder(orderLines, shippingInfo), ctx, state) => { ctx.thenPersist(POrderEvent.OrderCreated(orderLines, shippingInfo)) { evt => ctx.reply(Done) } } }.onCommand[POrderCommand.AddPayment, Done] { case (POrderCommand.AddPayment(reference), ctx, state) => { ctx.thenPersistAll(POrderEvent.OrderPaid(reference), POrderEvent.OrderConfirmed) { () => ctx.reply(Done) } } }.onReadOnlyCommand[GetOrder.type, POrder] { case (GetOrder, ctx, state) => ctx.reply(state) }.onEvent { case (POrderEvent.OrderCreated(lines, shippingInfo), state) => state.copy(orderLines = lines, shippingInfo = Some(shippingInfo)) case (POrderEvent.OrderPaid(reference), state) => state.copy(paymentReference = Some(reference)) case (POrderEvent.OrderConfirmed, state) => state.copy(confirmed = true) } }
58. Read Model ItemAdded ItemRemoved ItemAdded ItemChanged ItemAdded ItemRemoved ItemAdded ItemChanged ItemAdded ItemRemoved ItemAdded ItemChanged Repository ID: A ID: B ID: C
59. Read Model ItemAdded ItemRemoved ItemAdded ItemChanged ItemAdded ItemRemoved ItemAdded ItemChanged ItemAdded ItemRemoved ItemAdded ItemChanged Repository SEQ: 12 #1 #2 #3 #4 #5 #6 #7 #8 #9 #10 #11 #12 SEQ: 0 ID: A ID: B ID: C
60. Read Model ItemAdded ItemRemoved ItemAdded ItemChanged ItemAdded ItemRemoved ItemAdded ItemChanged ItemAdded ItemRemoved ItemAdded ItemChanged Repository SEQ: 12 #1 #2 #3 #4 #5 #6 #7 #8 #9 #10 #11 #12 SEQ: 12 ID: A ID: B ID: C
61. class OrderServiceImpl(persistentEntityRegistry: PersistentEntityRegistry)(implicit ec: ExecutionContext) extends OrderService { override def confirmOrder(orderId: String): ServiceCall[ConfirmOrderRequest, NotUsed] = ServiceCall { req => persistentEntityRegistry.refFor[OrderEntity](orderId).ask(POrderCommand.AddPayment(req.paymentReference)).map(_ => NotUsed) } override def orderEvents = TopicProducer.singleStreamWithOffset { offset => persistentEntityRegistry.eventStream(POrderEvent.Tag, offset).filter(e => /* Filter some of the internal events to isolate some of the entities internal logic */ e.event.isInstanceOf[POrderEvent.OrderCreated] || e.event == POrderEvent.OrderPaid ).mapAsync(1) { event => event.event match { case POrderEvent.OrderCreated(orderLines, shippingInfo) => val message = api.OrderEvent.OrderCreated(event.entityId, orderLines.map(x => api.Order.OrderLine(api.Order.SKU(x.sku.id), x.amount)), ShippingInfo(shippingInfo.address, shippingInfo.zipcode, shippingInfo.city, shippingInfo.country)) Future.successful((message, event.offset)) case x: POrderEvent.OrderPaid => persistentEntityRegistry.refFor[OrderEntity](event.entityId).ask(POrderCommand.GetOrder).map { x => val shippingInfo = x.shippingInfo.getOrElse(throw new Exception("Shipping info is expected to be set when the order is confirmed")) val message = api.OrderEvent.OrderPaid( orderId = event.entityId, orderLines = x.orderLines.map(x => api.Order.OrderLine(api.Order.SKU(x.sku.id), x.amount)), shippingInfo = api.Order.ShippingInfo(shippingInfo.address, shippingInfo.zipcode, shippingInfo.city, shippingInfo.country) ) (message, event.offset) } } } } }
62. Order Shipping Message Broker OrderPaid ShipOrder OrderPaid
63. class OrderServiceSubscriber(persistentEntityRegistry: PersistentEntityRegistry, orderService: OrderService) { orderService.orderEvents.subscribe.atLeastOnce(Flow[OrderEvent].mapAsync(1) { case x: OrderPaid => entityRef(x.orderId) .ask(ShipOrder(x.orderId, PAddress(x.shippingInfo.address, x.shippingInfo.city, x.shippingInfo.address, x.shippingInfo.zipcode) )) case other => Future.successful(Done) }) private def entityRef(orderId: UUID) = persistentEntityRegistry.refFor[POrder](orderId.toString) }
64. Autonomy is reached by implementing policies for behaviour instead of commanding behaviour
65. As long as the events are readable on a topic, the origin is of less importance
66. Message Broker Basket Shipping Order Product Stock Hybris Service Gateway Domain Events Product UpdateProduct ProductUpdated / StockChanged Anti Corruption Layer / Data Pumps
67. Message Broker Domain Events Anti Corruption Layer / Data Pumps API Gateway CORE SHOPPING Product Event Command Basket Command Event
68. Message Broker Domain Events Anti Corruption Layer / Data Pumps API Gateway CORE SHOPPING CHECK-OUT SELF-SERVICE Product Event Payment Command Event Shipping Command Event Order Command Event Command Basket Command Event
69. Events-first delivers on a better way of modelling businesses in technology, better suiting an environment of incremental learning
70. Q&A
71. 71 INTERESTED TO JOIN THE RIDE? CHARLENE BENJAMINS TALENT MANAGER cbenjamins@xebia.com GIDEON DE KOK MANAGER – XEBIA SOFTWARE DEVELOPMENT gdekok@xebia.com /