Tweet
Tweet

Slide 1

Slide 1 text

Implementing the Strangler Fig Pattern for microservices migrations @KeithResar Kafka Developer Confluent

Slide 2

Slide 2 text

No content

Slide 3

Slide 3 text

1 2 3 Monoliths and Microservices Kafka for Microservices Messaging Strangler Fig in Depth

Slide 4

Slide 4 text

No content

Slide 5

Slide 5 text

Monolith Application // Defining A application Applications are typically born to serve a unified purpose. Simple is the design goal, right? ● Easier to architect ● Easier to debug ● Easier to operate @KeithResar

Slide 6

Slide 6 text

Monolith Application // Evolution A B C monolith container Organic growth leads to increased functionality. This usually makes sense – ● Module A / B / C commonality ● Shared resources (e.g. state store) ● Development overlap ● Business process / rules overlap independent modules @KeithResar

Slide 7

Slide 7 text

In a bizarre twist of fate, further growth turns every strength into a weakness.

Slide 8

Slide 8 text

Monolith Application // Weaknesses Runtime scalability. ● All or nothing scaling ● Decreased resource efficiency ● Deployment interlock ● Lowered performance

Slide 9

Slide 9 text

Monolith Application // Weaknesses Development scalability. ● Single team → Overloaded ● Multiple teams → Interlock cost ● Decreased availability @KeithResar

Slide 10

Slide 10 text

Monolith Application // Weaknesses Extensibility ● Single framework / vendor ● Sync / Async / Performance SLOs

Slide 11

Slide 11 text

Microservices // Solution A B C The microservices solution often looks like the initial clean-sheet design for the first component. Each element is developed, deploys, and scales independently. State and inter-process coordination facilitates loosely coupled integrations. independent services state / message bus @KeithResar

Slide 12

Slide 12 text

No content

Slide 13

Slide 13 text

2010 Apache Kafka created at LinkedIn 2022 Most fortune 100 companies trust and use Kafka Kafka and the Rise of Event Streaming

Slide 14

Slide 14 text

@KeithResar

Slide 15

Slide 15 text

@KeithResar

Slide 16

Slide 16 text

Anatomy of a Kafka Topic 1 2 3 4 5 6 8 9 7 Partition 1 Old New 1 2 3 4 5 6 8 7 Partition 0 10 9 11 12 Partition 2 1 2 3 4 5 6 8 7 10 9 11 12 Writes 1 2 3 4 5 6 8 7 10 9 11 12 Producers Writes Consumer A (offset=4) Consumer B (offset=7) Reads @KeithResar

Slide 17

Slide 17 text

No content

Slide 18

Slide 18 text

Strangler Fig – explaining the metaphor.

Slide 19

Slide 19 text

No content

Slide 20

Slide 20 text

No content

Slide 21

Slide 21 text

No content

Slide 22

Slide 22 text

No content

Slide 23

Slide 23 text

No content

Slide 24

Slide 24 text

Monolith → Microservices? Strangler Fig enables migrations from monolithic applications to microservices without the risks associated with a large cut-over or re-write. @KeithResar

Slide 25

Slide 25 text

Monolith → Microservices? De-risking benefits of Strangler Fig: ● Incremental migration - Baby Steps! ● Pause or stop migration ● Reversible @KeithResar

Slide 26

Slide 26 text

No content

Slide 27

Slide 27 text

Strangler Fig by Example A B C monolith container database Module A is a write endpoint that persists data to the database Module B is a read endpoint that returns data from the database @KeithResar

Slide 28

Slide 28 text

Strangler Fig by Example client Monolith application before applying any change. Clients directly access application via standard interfaces (could be network, message queues, etc.) Next we begin to replace services as atomic units. Step 0 A B C monolith container database @KeithResar

Slide 29

Slide 29 text

Strangler Fig by Example proxy Create a proxy which intercepts network / messaging calls between clients and the monolith application. (sometimes referred to as an anti-corruption layer) Initially this is more of a no-op – just supporting transparent pass-through. client Step 1 A B C database monolith container @KeithResar

Slide 30

Slide 30 text

Strangler Fig by Example proxy Write traffic to continues to monolith unchanged (1) We could read results of service directly from the database (2). Need to resolve timing issues. Also leads to future scalability concerns. A B C client database Step 2 2 A B 1 monolith container @KeithResar

Slide 31

Slide 31 text

Strangler Fig by Example proxy A B C client database Step 3 Write traffic to continues to monolith unchanged (1) Use CDC to get realtime updates for from database (2). Async results are pushed from Kafka back to proxy / client (3, 4) A B 4 B’ 3 1 2 monolith container @KeithResar

Slide 32

Slide 32 text

Strangler Fig by Example proxy A B C client database Step 4 Write traffic to hits the new microservice (1) and state is persisted in Kafka (2) Updates are pushed to from Kafka (3) and async are available to client via proxy (4) A 1 A’ 2 4 B’ 3 B x x monolith container @KeithResar

Slide 33

Slide 33 text

Strangler Fig by Example and were an easy use case. What about ? Many intricacies to account for… let’s review some examples. ● Non-atomic modules ● Internal state / messaging ● Async / sync external calls A B C database x x A B C monolith container @KeithResar

Slide 34

Slide 34 text

Atomic Modules A B monolith container state A B monolith container state Internal state representation hides intermediate work product within the monolith Externalize intermediate state to support decomposition. @KeithResar

Slide 35

Slide 35 text

Caches Caches hidden in monoliths are deadly. ● DNS ● Objects ● Intermediate results @KeithResar

Slide 36

Slide 36 text

Large Data Objects A microservice Large data in object storage s3://../payload/.. Leverage Claim Check pattern to mitigate impact of large intermediate objects. Microservice makes two writes, ● Commits serialized data to S3 ● Produces message with data address @KeithResar

Slide 37

Slide 37 text

Change Data Capture microservice @KeithResar A Taps into transaction log to capture INSERT/UPDATE/DELETE events Propagated to constream consumers of target topics. database Debezium CDC table 1 table 2 B microservice

Slide 38

Slide 38 text

Event Commands @KeithResar microservice A B microservice Events are direct commands from A to B. A: “Disable sensor, B!” Optional result stream

Slide 39

Slide 39 text

Proxy vs. Anti-corruption Layer proxy client A B C Anti-corruption layer client A B C @KeithResar

Slide 40

Slide 40 text

No content

Slide 41

Slide 41 text

Where to go from here? 4 Resources

Slide 42

Slide 42 text

Everything Kafka docs, animations, and video on demand developer.confluent.io

Slide 43

Slide 43 text

O’Reilly Books free e-book bundle cnfl.io/book-bundle

Slide 44

Slide 44 text

Confluent Cloud Free access with new accounts

Slide 45

Slide 45 text

Encore Happy Hour The Gnome Craft Pub

Slide 46

Slide 46 text

No content

Slide 47

Slide 47 text

Thank You @KeithResar Kafka Developer Confluent

Slide 48

Slide 48 text

No content

Slide 49

Slide 49 text

#bc8474 #788c0c #7584ae #28300c #14170a #788c0c #7584ae #28300c