Slide 1

Slide 1 text

Eberhard Wolff Fellow INNOQ @ewolff http://ewolff.com

Slide 2

Slide 2 text

http://continuous-delivery-buch.de/ http://continuous-delivery-book.com/

Slide 3

Slide 3 text

http://microservices-buch.de/ http://microservices-book.com/

Slide 4

Slide 4 text

http://microservices-buch.de/ ueberblick.html http://microservices-book.com/ primer.html FREE!!!!

Slide 5

Slide 5 text

http://microservices-praxisbuch.de http://practical-microservices.com/

Slide 6

Slide 6 text

http://microservices-praxisbuch.de/ rezepte.html http://practical-microservices.com/ recipes.html FREE!!!!

Slide 7

Slide 7 text

No content

Slide 8

Slide 8 text

Independent Systems Architecture: ISA

Slide 9

Slide 9 text

Modules Macro / Micro Architecture Independent Continuous Delivery Pipeline Resilience Integration & Communication Authentication & Metadata Standardized Operations Standards: Interface only Container Independent Systems Architecture

Slide 10

Slide 10 text

Microservices = Extremely decoupled modules

Slide 11

Slide 11 text

Most important factor for microservice: Getting the split right!!!

Slide 12

Slide 12 text

Bounded Context: Specialized domain model for some use cases.

Slide 13

Slide 13 text

Order Invoice Delivery Items Customer Credit cards Taxes Parcel service Returns

Slide 14

Slide 14 text

No content

Slide 15

Slide 15 text

Events • Communication between Bounded Contexts • Event in business domain • Past tense • Immutable

Slide 16

Slide 16 text

Events enable decoupling.

Slide 17

Slide 17 text

Events & microservices:

Slide 18

Slide 18 text

Events & microservices: A match made in heaven.

Slide 19

Slide 19 text

Event: Decoupling • Time: Might handle event later • Availability: Down? Handle event later • Change?

Slide 20

Slide 20 text

Order Accepted Invoice Delivery Asynchronous Communication ???

Slide 21

Slide 21 text

No content

Slide 22

Slide 22 text

Order Accepted Item prices Billing address Item sizes Item weights Delivery address ??? Invoice Delivery ??? • Lots of data • Which Bounded Context uses what? • Changes can impact all modules • DDD: Published Language • THE order model?

Slide 23

Slide 23 text

Delivery Order Accepted Invoice Order Accepted Item prices Billing address Item sizes Item weights Delivery address • Specialized “events” • Independent evolution • Easy to figure out which Bounded Context uses what • Extensibility = new event • DDD: Customer / Supplier ??? Order Accepted ???

Slide 24

Slide 24 text

Order Accepted Order ID • Just an ID • Provide additional specialized API for data for each consumer • Compromise between extensibility and specialized events

Slide 25

Slide 25 text

Events: Conclusion • Events per se provide limited decoupling. • What data should be included? • Prefer specialized events and just IDs

Slide 26

Slide 26 text

Events: Conclusion • Best option for communication between Bounded Contexts. • Great fit for a Ubiquitous Language. • But: Implicit replication

Slide 27

Slide 27 text

Inconsistencies!!!

Slide 28

Slide 28 text

No content

Slide 29

Slide 29 text

What Kind of Inconsistencies? • One system for invoices • One for deliveries • One for orders • One source of truth for invoices/deliveries/orders. • No “invoice might be there or not”

Slide 30

Slide 30 text

Inconsistencies • Either delivery without invoice • …or invoice without delivery. • Will eventually be solved. • Delays are not unheard of. • What is the alternative? • Centralized complex database?

Slide 31

Slide 31 text

No content

Slide 32

Slide 32 text

Inconsistencies • Consequences if data appears after 0.1s / 1s / 1min / 1h / 1d ? • “Invoice has to be there in <1min!!!!!”

Slide 33

Slide 33 text

No content

Slide 34

Slide 34 text

Event Sourcing • Store the events that lead to a specific state + state (optional) • System of record: Event or state • No clear definition https://martinfowler.com/ eaaDev/EventSourcing.html

Slide 35

Slide 35 text

Microservice Interface Calls, messages, … Event Store Order Cancelled 42 Order Accepted 23 Order Accepted 42 Order 42 Order 23 Order Delivered 23 Order 23 ✔ State

Slide 36

Slide 36 text

State Events Calculate State on Demand State+ Events Event Sourcing

Slide 37

Slide 37 text

Event Sourcing: Advantages • Might be a better model • History log • Audit log • Can rebuild state • Even after the domain model changed

Slide 38

Slide 38 text

Event Sourcing: Advantages • Temporal queries easy (yesterday’s state) • Replay • Reversal • Scalability – might replicate state using events

Slide 39

Slide 39 text

Event Sourcing: Challenges • External systems might have different data • …and behave differently during replay. • Code changes might change response to events • …even if it is just a bug fix.

Slide 40

Slide 40 text

Event Sourcing: Pattern for Persistence

Slide 41

Slide 41 text

Persistence • Responsibility of the microservices • Might be changed • Might be different for each microservice • Event Sourcing should be a local decision • Event sourcing makes it easier to fix inconsistencies.

Slide 42

Slide 42 text

event sourcing

Slide 43

Slide 43 text

No content

Slide 44

Slide 44 text

Kafka Records • Key • Value • Timestamp • Headers • Stored forever (!) Record Key Value Timestamp

Slide 45

Slide 45 text

Kafka Topics • Topics have a name • Topics have partitions • Order guaranteed per partition • Consumer commits offset per partition • Consumer group: One consumer per partition Topic Partition Record Record Record

Slide 46

Slide 46 text

Topic Partition Record Record Record Consumer Offset Offset committed per consumer Offset Partition Record Record Record Partition Record Record Record Producer Consumer Consumer group

Slide 47

Slide 47 text

Kafka Replication • N Replicas (configurable) • In-sync replicas (configurable): write successful if all written

Slide 48

Slide 48 text

Order Invoice Delivery

Slide 49

Slide 49 text

https://github.com/ ewolff/microservice-kafka

Slide 50

Slide 50 text

Docker-Host Apache httpd Load Balancer Postgres Order Shipping Invoicing Kafka Zookeeper HTTP-Routing 8080 8080 80 8080 8080 9092

Slide 51

Slide 51 text

Events • Events contain all information for invoice and delivery • Alternative: specific events • …more independence • Alternative: just ID in the event • …plus service to get detailed information

Slide 52

Slide 52 text

Data • Data is copied into invoice / delivery database • …and denormalized • Each invoice and delivery contains customer and item data from when the order was created. • Bounded Context • No complex historization

Slide 53

Slide 53 text

Kafka Conclusion • Provides access to old events • Guaranteed order per partition • Consumer groups send records to one consumer • Additional (complex) infrastructure

Slide 54

Slide 54 text

No content

Slide 55

Slide 55 text

Atom • Data format • …for feeds, blogs, podcasts etc • Access through HTTP • Idea: Provide a feeds of events / orders

Slide 56

Slide 56 text

Atom Feed • Some unneeded information author, summary, title… • Mostly links to details • ...and timestamps • Content negotiation can provide different data formats or details

Slide 57

Slide 57 text

Subscribing to Feed • Poll the feed (HTTP GET) • Client decides when to process new events • …but very inefficient • Lots of unchanged data

Slide 58

Slide 58 text

HTTP Caching • Client GETs feed • Server send data + Last-Modified header • Client sends GET + If-Modified-Since header • Server: 304 (Not modified) …or new data

Slide 59

Slide 59 text

https://github.com/ ewolff/microservice-atom

Slide 60

Slide 60 text

Docker-Host 80 8080 Apache httpd Postgres Order Shipping Invoicing HTTP-Routing HTTP + Atom 8080 8080 8080

Slide 61

Slide 61 text

Do you need Atom?

Slide 62

Slide 62 text

Do you need Atom? No!

Slide 63

Slide 63 text

Alternative • Roll your own format …that has no information only relevant for blogs …with links to details ...use content negotiation for details • Only issue: Atom reader cannot handle format

Slide 64

Slide 64 text

No content

Slide 65

Slide 65 text

Atom Conclusion • Can provides access to old events if they are stored anyway. • One consumer: idempotency • No additional infrastructure • But: Can’t send event to just one receiver

Slide 66

Slide 66 text

No content

Slide 67

Slide 67 text

Conclusion • Most important: Getting the split right! • Bounded Context • …communicate via business events • Inconsistencies can be dealt with • What is in the event? • Don’t share events for event sourcing! • Do not overuse event sourcing. • Kafka: Great solution for messaging • …but REST might be enough.

Slide 68

Slide 68 text

Send email to [email protected] Slides + Microservices Primer DE / EN + Microservices Recipes DE / EN + Sample Microservices Book DE / EN + Sample Practical Microservices DE/EN + Sample of Continuous Delivery Book DE Powered by Amazon Lambda & Microservices