Slide 1

Slide 1 text

Events-first Microservices Moving away from the monolith with success Gideon de Kok @gideondk

Slide 2

Slide 2 text

2 IT’S YOUR BUSINESS. WE ACCELERATE IT. XEBIA GROUP CONSISTS OF SEVEN SPECIALISED, INTERLINKED COMPANY WITH OFFICES IN AMSTERDAM AND HILVERSUM (NETHERLANDS), PARIS, DELHI, BANGALORE AND BOSTON, WE EMPLOY OVER 700 PEOPLE WORLDWIDE

Slide 3

Slide 3 text

Microservices

Slide 4

Slide 4 text

No content

Slide 5

Slide 5 text

No content

Slide 6

Slide 6 text

We’re often forcing our problem into microservices instead of complimenting it

Slide 7

Slide 7 text

No content

Slide 8

Slide 8 text

Complexity Productivity Working in a highly coupled system increases collaboration and productivity in initial phases Increased complexity from larger code base start to hinder progress Overall velocity reaches a potential stable but low point

Slide 9

Slide 9 text

For systems with less complexity, the overhead required to get things running reduces initial productivity A true modular architecture with correct boundaries profits from localised complexity The complexity of a business will always impact productivity in the end but overhead is reduced significantly Complexity Productivity

Slide 10

Slide 10 text

Complexity Productivity Working in a highly coupled system increases collaboration and productivity in initial phases Increased complexity from larger code base start to hinder progress Overall velocity reaches a potential stable but low point

Slide 11

Slide 11 text

/

Slide 12

Slide 12 text

Two separate monoliths on Hybris, abysmal test coverage, quality issues, tight coupling everywhere

Slide 13

Slide 13 text

Basket Shipping Order Product Stock Hybris

Slide 14

Slide 14 text

Desire to build a platform which enables a multi-brand platform in a business flexibility enhancing environment

Slide 15

Slide 15 text

No content

Slide 16

Slide 16 text

Issues don’t come from software stacks, but the way they’re used to compose and develop our software (it’s a learning process…)

Slide 17

Slide 17 text

We should not be focused on microservices, we should be focused on creating architectures which enable clarity, loose coupling and autonomy

Slide 18

Slide 18 text

Command Service A Service B

Slide 19

Slide 19 text

Command Service A Service B

Slide 20

Slide 20 text

No content

Slide 21

Slide 21 text

We not only need boundaries, We need an architectural pattern which enable autonomy within and over those boundaries

Slide 22

Slide 22 text

Structuring business processes into domains, and aligning those with technology Domain Driven Design

Slide 23

Slide 23 text

An explicit boundary for language, complexity, logic Bounded Contexts

Slide 24

Slide 24 text

Structure-First Domain-Driven Design

Slide 25

Slide 25 text

Structure weights you down. Something that has been visualised is hard to be unvisualised.

Slide 26

Slide 26 text

It is not the things that matter (in early stages of design), it is the things that happen — Russ Miles

Slide 27

Slide 27 text

“Event”
 1. A thing that happens or takes place, especially one of importance.

Slide 28

Slide 28 text

Events-First Domain-Driven Design™,

Slide 29

Slide 29 text

No content

Slide 30

Slide 30 text

No content

Slide 31

Slide 31 text

No content

Slide 32

Slide 32 text

No content

Slide 33

Slide 33 text

Order Created

Slide 34

Slide 34 text

Order Confirmed Order Created Order Paid Order Shipped Order Delivered Order Picked

Slide 35

Slide 35 text

Order Shipping Order Confirmed Order Created Order Paid Order Shipped Order Delivered Order Picked

Slide 36

Slide 36 text

package com.xebia.webshop.order.api sealed trait OrderEvent object OrderEvent { case class OrderCreated(orderId: String, orderLines: Seq[OrderLine], shippingInfo: ShippingInfo) extends OrderEvent case class OrderConfirmed(orderId: String) extends OrderEvent case class OrderPaid(orderId: String, orderLines: Seq[OrderLine], shippingInfo: ShippingInfo) extends OrderEvent object OrderCreated { implicit val format: Format[OrderCreated] = Json.format } object OrderConfirmed { implicit val format: Format[OrderConfirmed] = Json.format } object OrderPaid { implicit val format: Format[OrderPaid] = Json.format } implicit val format: Format[OrderEvent] = derived.flat.oformat((__ \ "type").format[String]) }

Slide 37

Slide 37 text

package com.xebia.webshop.shipping.api sealed trait OrderEvent object OrderEvent { case class OrderPicked(orderId: String, pickedBy: Employee) extends OrderEvent case class OrderShipped(orderId: String, shippingReference: String) extends OrderEvent case class OrderDelivered(orderId: String, wasDeliveredAt: LocalDateTime) extends OrderEvent object OrderPicked { implicit val format: Format[OrderPicked] = Json.format } object OrderShipped { implicit val format: Format[OrderShipped] = Json.format } object OrderDelivered { implicit val format: Format[OrderDelivered] = Json.format } implicit val format: Format[OrderEvent] = derived.flat.oformat((__ \ "type").format[String]) }

Slide 38

Slide 38 text

“Lagom” /lah-gome/ 1. (just) right, fitting, neither too much nor too little

Slide 39

Slide 39 text

Lagom provides an opinionated way of building microservices that intentionally constrains what a developer can do and how they should do it.

Slide 40

Slide 40 text

• Build your services using the concepts of Domain Driven Design • Services communicate on basis of contracts • Your event-log is the single point of truth, Event Source all your state* • Separate your writes from your reads / queries (CQRS) • Make your services stateful as a default** • Make your services elastic and resilient as a default

Slide 41

Slide 41 text

• Persistent-ignorant design with support for multiple storage- engines out of the box • Default end-to-end support for message brokers • Able to run, test and continuously compile a complete microservice system on your local machine • Java / Scala support (and Kotlin if you bend it) • Spawns Cassandra & Kafka during local execution to get from idea to development after installation • Service locators & Circuit Breakers

Slide 42

Slide 42 text

package com.xebia.webshop.order.api import akka.NotUsed import com.lightbend.lagom.scaladsl.api.broker.Topic import com.lightbend.lagom.scaladsl.api.transport._ import com.lightbend.lagom.scaladsl.api.{Descriptor, Service, ServiceCall} trait OrderService extends Service { import Method._ import Service._ def orderEvents: Topic[OrderEvent] final override def descriptor = { named("order").withTopics( topic("order-OrderEvent", orderEvents) ) } }

Slide 43

Slide 43 text

Order Paid Order Picked Order Shipping

Slide 44

Slide 44 text

Order Paid Order Picked Whenever an Order is Paid Order Shipping

Slide 45

Slide 45 text

class OrderServiceSubscriber(orderService: OrderService) { orderService.orderEvents.subscribe.atLeastOnce(Flow[api.OrderEvent].mapAsync(1) { case x: OrderPaid => ??? // Where we should trigger our side-effect case other => Future.successful(Done) }) }

Slide 46

Slide 46 text

Order Paid Order Picked Whenever an Order is Paid Pick Order Add Payment Information Order Shipping

Slide 47

Slide 47 text

Create Order Order Add Payment Information Order Confirmed Order Created Order Paid

Slide 48

Slide 48 text

Order OrderLine OrderLine OrderLine CreateOrder Ack OrderCreated

Slide 49

Slide 49 text

State OrderPaid OrderConfirmed OrderUpdated OrderCreated

Slide 50

Slide 50 text

Order 1 Order Domain Order 2 Order 3

Slide 51

Slide 51 text

Hash(Id(Order 1)) Hash(Id(Order 2)) Hash(Id(Order 3)) Akka Cluster (Auto-Sharding)

Slide 52

Slide 52 text

Hash(Id(Order 1)) Hash(Id(Order 2)) Hash(Id(Order 3)) A B C Akka Cluster (Auto-Sharding)

Slide 53

Slide 53 text

Hash(Id(Order 1)) Hash(Id(Order 2)) Hash(Id(Order 3)) A B C Akka Cluster (Auto-Sharding)

Slide 54

Slide 54 text

trait POrderCommand object POrderCommand { case class CreateOrder(orderLines: Seq[POrderLine], info: PShippingInfo) extends POrderCommand with ReplyType[Done] case class AddPayment(reference: String) extends POrderCommand with ReplyType[Done] case object GetOrder extends POrderCommand with ReplyType[POrder] }

Slide 55

Slide 55 text

trait POrderEvent extends AggregateEvent[POrderEvent] { override def aggregateTag: AggregateEventTagger[POrderEvent] = POrderEvent.Tag } object POrderEvent { val Tag = AggregateEventTag[POrderEvent] case class OrderCreated( orderLines: Seq[POrderLine], shippingInformation: PShippingInfo) extends POrderEvent case object OrderConfirmed extends POrderEvent case class OrderPaid(reference: String) extends POrderEvent }

Slide 56

Slide 56 text

case class POrder(id: String, orderLines: Seq[POrderLine], shippingInfo: Option[PShippingInfo], paymentReference: Option[String], confirmed: Boolean) object POrder { case class PShippingInfo(address: String, zipcode: String, city: String, country: String) case class PSKU(id: String) case class POrderLine(sku: PSKU, amount: Int) }

Slide 57

Slide 57 text

class OrderEntity extends PersistentEntity { override type Command = POrderCommand override type Event = POrderEvent override type State = POrder override def initialState = POrder(this.entityId, Seq.empty, None, None, false) override def behavior: Behavior = Actions() .onCommand[POrderCommand.CreateOrder, Done] { case (POrderCommand.CreateOrder(orderLines, shippingInfo), ctx, state) => { ctx.thenPersist(POrderEvent.OrderCreated(orderLines, shippingInfo)) { evt => ctx.reply(Done) } } }.onCommand[POrderCommand.AddPayment, Done] { case (POrderCommand.AddPayment(reference), ctx, state) => { ctx.thenPersistAll(POrderEvent.OrderPaid(reference), POrderEvent.OrderConfirmed) { () => ctx.reply(Done) } } }.onReadOnlyCommand[GetOrder.type, POrder] { case (GetOrder, ctx, state) => ctx.reply(state) }.onEvent { case (POrderEvent.OrderCreated(lines, shippingInfo), state) => state.copy(orderLines = lines, shippingInfo = Some(shippingInfo)) case (POrderEvent.OrderPaid(reference), state) => state.copy(paymentReference = Some(reference)) case (POrderEvent.OrderConfirmed, state) => state.copy(confirmed = true) } }

Slide 58

Slide 58 text

Read Model ItemAdded ItemRemoved ItemAdded ItemChanged ItemAdded ItemRemoved ItemAdded ItemChanged ItemAdded ItemRemoved ItemAdded ItemChanged Repository ID: A ID: B ID: C

Slide 59

Slide 59 text

Read Model ItemAdded ItemRemoved ItemAdded ItemChanged ItemAdded ItemRemoved ItemAdded ItemChanged ItemAdded ItemRemoved ItemAdded ItemChanged Repository SEQ: 12 #1 #2 #3 #4 #5 #6 #7 #8 #9 #10 #11 #12 SEQ: 0 ID: A ID: B ID: C

Slide 60

Slide 60 text

Read Model ItemAdded ItemRemoved ItemAdded ItemChanged ItemAdded ItemRemoved ItemAdded ItemChanged ItemAdded ItemRemoved ItemAdded ItemChanged Repository SEQ: 12 #1 #2 #3 #4 #5 #6 #7 #8 #9 #10 #11 #12 SEQ: 12 ID: A ID: B ID: C

Slide 61

Slide 61 text

class OrderServiceImpl(persistentEntityRegistry: PersistentEntityRegistry)(implicit ec: ExecutionContext) extends OrderService { override def confirmOrder(orderId: String): ServiceCall[ConfirmOrderRequest, NotUsed] = ServiceCall { req => persistentEntityRegistry.refFor[OrderEntity](orderId).ask(POrderCommand.AddPayment(req.paymentReference)).map(_ => NotUsed) } override def orderEvents = TopicProducer.singleStreamWithOffset { offset => persistentEntityRegistry.eventStream(POrderEvent.Tag, offset).filter(e => /* Filter some of the internal events to isolate some of the entities internal logic */ e.event.isInstanceOf[POrderEvent.OrderCreated] || e.event == POrderEvent.OrderPaid ).mapAsync(1) { event => event.event match { case POrderEvent.OrderCreated(orderLines, shippingInfo) => val message = api.OrderEvent.OrderCreated(event.entityId, orderLines.map(x => api.Order.OrderLine(api.Order.SKU(x.sku.id), x.amount)), ShippingInfo(shippingInfo.address, shippingInfo.zipcode, shippingInfo.city, shippingInfo.country)) Future.successful((message, event.offset)) case x: POrderEvent.OrderPaid => persistentEntityRegistry.refFor[OrderEntity](event.entityId).ask(POrderCommand.GetOrder).map { x => val shippingInfo = x.shippingInfo.getOrElse(throw new Exception("Shipping info is expected to be set when the order is confirmed")) val message = api.OrderEvent.OrderPaid( orderId = event.entityId, orderLines = x.orderLines.map(x => api.Order.OrderLine(api.Order.SKU(x.sku.id), x.amount)), shippingInfo = api.Order.ShippingInfo(shippingInfo.address, shippingInfo.zipcode, shippingInfo.city, shippingInfo.country) ) (message, event.offset) } } } } }

Slide 62

Slide 62 text

Order Shipping Message Broker OrderPaid ShipOrder OrderPaid

Slide 63

Slide 63 text

class OrderServiceSubscriber(persistentEntityRegistry: PersistentEntityRegistry, orderService: OrderService) { orderService.orderEvents.subscribe.atLeastOnce(Flow[OrderEvent].mapAsync(1) { case x: OrderPaid => entityRef(x.orderId) .ask(ShipOrder(x.orderId, PAddress(x.shippingInfo.address, x.shippingInfo.city, x.shippingInfo.address, x.shippingInfo.zipcode) )) case other => Future.successful(Done) }) private def entityRef(orderId: UUID) = persistentEntityRegistry.refFor[POrder](orderId.toString) }

Slide 64

Slide 64 text

Autonomy is reached by implementing policies for behaviour instead of commanding behaviour

Slide 65

Slide 65 text

As long as the events are readable on a topic, the origin is of less importance

Slide 66

Slide 66 text

Message Broker Basket Shipping Order Product Stock Hybris Service Gateway Domain Events Product UpdateProduct ProductUpdated / StockChanged Anti Corruption Layer / Data Pumps

Slide 67

Slide 67 text

Message Broker Domain Events Anti Corruption Layer / Data Pumps API Gateway CORE SHOPPING Product Event Command Basket Command Event

Slide 68

Slide 68 text

Message Broker Domain Events Anti Corruption Layer / Data Pumps API Gateway CORE SHOPPING CHECK-OUT SELF-SERVICE Product Event Payment Command Event Shipping Command Event Order Command Event Command Basket Command Event

Slide 69

Slide 69 text

Events-first delivers on a better way of modelling businesses in technology, better suiting an environment of incremental learning

Slide 70

Slide 70 text

Q&A

Slide 71

Slide 71 text

71 INTERESTED TO JOIN THE RIDE? CHARLENE BENJAMINS TALENT MANAGER [email protected] GIDEON DE KOK MANAGER – XEBIA SOFTWARE DEVELOPMENT [email protected] /