Implementing Strangler pattern for microservices migrations

Decomposing existing monolithic applications into microservices or other migration / hybrid application transitions requires a balancing act between moving efficiently and keeping existing services online. Strangler Fig is the most common pattern used for migrations of these types. See how to successfully implement Strangler Fig in an example monolithic to microservices application rewrite.

Transcript

1. Implementing the Strangler Fig Pattern for microservices migrations @KeithResar Kafka Developer Confluent
3. 1 2 3 Monoliths and Microservices Kafka for Microservices Messaging Strangler Fig in Depth
5. Monolith Application // Defining A application Applications are typically born to serve a unified purpose. Simple is the design goal, right? • Easier to architect • Easier to debug • Easier to operate @KeithResar
6. Monolith Application // Evolution A B C monolith container Organic growth leads to increased functionality. This usually makes sense – • Module A / B / C commonality • Shared resources (e.g. state store) • Development overlap • Business process / rules overlap independent modules @KeithResar
7. In a bizarre twist of fate, further growth turns every strength into a weakness.
8. Monolith Application // Weaknesses Runtime scalability. • All or nothing scaling • Decreased resource efficiency • Deployment interlock • Lowered performance
9. Monolith Application // Weaknesses Development scalability. • Single team → Overloaded • Multiple teams → Interlock cost • Decreased availability @KeithResar
10. Monolith Application // Weaknesses Extensibility • Single framework / vendor • Sync / Async / Performance SLOs
11. Microservices // Solution A B C The microservices solution often looks like the initial clean-sheet design for the first component. Each element is developed, deploys, and scales independently. State and inter-process coordination facilitates loosely coupled integrations. independent services state / message bus @KeithResar
13. 2010 Apache Kafka created at LinkedIn 2022 Most fortune 100 companies trust and use Kafka Kafka and the Rise of Event Streaming
14. @KeithResar
15. @KeithResar
16. Anatomy of a Kafka Topic 1 2 3 4 5 6 8 9 7 Partition 1 Old New 1 2 3 4 5 6 8 7 Partition 0 10 9 11 12 Partition 2 1 2 3 4 5 6 8 7 10 9 11 12 Writes 1 2 3 4 5 6 8 7 10 9 11 12 Producers Writes Consumer A (offset=4) Consumer B (offset=7) Reads @KeithResar
18. Strangler Fig – explaining the metaphor.
24. Monolith → Microservices? Strangler Fig enables migrations from monolithic applications to microservices without the risks associated with a large cut-over or re-write. @KeithResar
25. Monolith → Microservices? De-risking benefits of Strangler Fig: • Incremental migration - Baby Steps! • Pause or stop migration • Reversible @KeithResar
27. Strangler Fig by Example A B C monolith container database Module A is a write endpoint that persists data to the database Module B is a read endpoint that returns data from the database @KeithResar
28. Strangler Fig by Example client Monolith application before applying any change. Clients directly access application via standard interfaces (could be network, message queues, etc.) Next we begin to replace services as atomic units. Step 0 A B C monolith container database @KeithResar
29. Strangler Fig by Example proxy Create a proxy which intercepts network / messaging calls between clients and the monolith application. (sometimes referred to as an anti-corruption layer) Initially this is more of a no-op – just supporting transparent pass-through. client Step 1 A B C database monolith container @KeithResar
30. Strangler Fig by Example proxy Write traffic to continues to monolith unchanged (1) We could read results of service directly from the database (2). Need to resolve timing issues. Also leads to future scalability concerns. A B C client database Step 2 2 A B 1 monolith container @KeithResar
31. Strangler Fig by Example proxy A B C client database Step 3 Write traffic to continues to monolith unchanged (1) Use CDC to get realtime updates for from database (2). Async results are pushed from Kafka back to proxy / client (3, 4) A B 4 B’ 3 1 2 monolith container @KeithResar
32. Strangler Fig by Example proxy A B C client database Step 4 Write traffic to hits the new microservice (1) and state is persisted in Kafka (2) Updates are pushed to from Kafka (3) and async are available to client via proxy (4) A 1 A’ 2 4 B’ 3 B x x monolith container @KeithResar
33. Strangler Fig by Example and were an easy use case. What about ? Many intricacies to account for… let’s review some examples. • Non-atomic modules • Internal state / messaging • Async / sync external calls A B C database x x A B C monolith container @KeithResar
34. Atomic Modules A B monolith container state A B monolith container state Internal state representation hides intermediate work product within the monolith Externalize intermediate state to support decomposition. @KeithResar
35. Caches Caches hidden in monoliths are deadly. • DNS • Objects • Intermediate results @KeithResar
36. Large Data Objects A microservice Large data in object storage s3://../payload/.. Leverage Claim Check pattern to mitigate impact of large intermediate objects. Microservice makes two writes, • Commits serialized data to S3 • Produces message with data address @KeithResar
37. Change Data Capture microservice @KeithResar A Taps into transaction log to capture INSERT/UPDATE/DELETE events Propagated to constream consumers of target topics. database Debezium CDC table 1 table 2 B microservice
38. Event Commands @KeithResar microservice A B microservice Events are direct commands from A to B. A: “Disable sensor, B!” Optional result stream
39. Proxy vs. Anti-corruption Layer proxy client A B C Anti-corruption layer client A B C @KeithResar
41. Where to go from here? 4 Resources
42. Everything Kafka docs, animations, and video on demand developer.confluent.io
43. O’Reilly Books free e-book bundle cnfl.io/book-bundle
44. Confluent Cloud Free access with new accounts
45. Encore Happy Hour The Gnome Craft Pub
47. Thank You @KeithResar Kafka Developer Confluent
49. #bc8474 #788c0c #7584ae #28300c #14170a #788c0c #7584ae #28300c