Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Dissecting our Legacy: The Strangler Fig Pattern with Apache Kafka, Debezium and MongoDB @ VoxxedDays Romania 2021

Dissecting our Legacy: The Strangler Fig Pattern with Apache Kafka, Debezium and MongoDB @ VoxxedDays Romania 2021

Abstract:

There is no denying the fact that many development efforts have to be spent on existing applications - legacy that is - which typically exhibit a monolithic design based on traditional tech stacks. Thus, affected companies strive to move towards distributed architectures and modern technologies. This talk introduces you to the strangler fig pattern, which aids a smooth and step-wise migration of monolithic applications into separate services. The practical part shows how to apply this pattern to extract parts of a fictional monolith into their own service by featuring:

* Apache Kafka, the de-facto standard for event streaming
* MongoDB and its official connector for Apache Kafka
* plus Debezium, a distributed open-source change data capture platform.

After this talk, you’ll have a better understanding of and a concrete blueprint for extracting functionality from your monoliths, thereby gradually evolving into a (micro)service architecture and an en vogue tech stack.

Demo Repository: https://github.com/hpgrahsl/voxxedromania21-sfp-demo

Video Recording:
https://www.youtube.com/watch?v=rl06nAIdIhQ

Hans-Peter Grahsl

October 21, 2021
Tweet

More Decks by Hans-Peter Grahsl

Other Decks in Programming

Transcript

  1. Dissecting our Legacy The Strangler Fig Pattern with Apache Kafka,

    Debezium and MongoDB Hans-Peter Grahsl Technical Trainer @hpgrahsl Gunnar Morling Software Engineer @gunnarmorling
  2. @gunnarmorling @hpgrahsl • Technical Trainer at NETCONOMY • Independent Consultant

    & Engineer • Confluent Community Catalyst • MongoDB Champion Hans-Peter Grahsl
  3. @gunnarmorling @hpgrahsl • Open source software engineer at Red Hat

    ◦ Debezium ◦ Quarkus • Spec Lead for Bean Validation 2.0 • Java Champion Gunnar Morling
  4. @gunnarmorling @hpgrahsl System Migration Goal • gradually evolve from old

    into new • support temporary coexistence • avoid big bang cut-over @hpgrahsl
  5. @gunnarmorling @hpgrahsl Open-Source Change-Data-Capture With Debezium • Captures INSERT/UPDATE/DELETE events

    by tapping into DB’s TX log • Changes propagated via Apache Kafka & Kafka Connect
  6. @gunnarmorling @hpgrahsl Debezium in a Nutshell • Feature-rich CDC Platform

    ◦ Based on transaction logs ◦ Snapshotting, filtering, etc. ◦ Outbox support ◦ Web-based UI • Very active OSS community • Large production deployments
  7. @gunnarmorling @hpgrahsl Debezium Connectors • Stable ◦ MySQL ◦ Postgres

    ◦ MongoDB ◦ SQL Server ◦ Db2 ◦ Oracle • Incubating ◦ Vitess ◦ Cassandra
  8. @gunnarmorling @hpgrahsl Debezium – Data Change Events • Message Key:

    table PK • Message Value: ◦ Old and new row state ◦ Metadata on table, TX id, etc. ◦ Operation type, timestamp • Serialized as JSON, Avro, etc.
  9. @gunnarmorling @hpgrahsl Debezium – Data Change Events • Message Key:

    table PK • Message Value: ◦ Old and new row state ◦ Metadata on table, TX id, etc. ◦ Operation type, timestamp • Serialized as JSON, Avro, etc.
  10. @gunnarmorling @hpgrahsl • Message Key: table PK • Message Value:

    ◦ Old and new row state ◦ Metadata on table, TX id, etc. ◦ Operation type, timestamp • Serialized as JSON, Avro, etc. Debezium – Data Change Events
  11. @gunnarmorling @hpgrahsl CDC Pipeline Considerations • data model leaking from

    monolith ? • “1:1 replication” → building aggregates ? • transactional boundaries in the monolith ? @hpgrahsl
  12. @gunnarmorling @hpgrahsl Wrapping Up Strangler Fig Pattern Benefits • incremental

    migration → “baby steps” • pause or stop without losing spent efforts • extraction steps ideally reversible • rationale: ⚠ minimize risk ⚠ @hpgrahsl
  13. @gunnarmorling @hpgrahsl Resources • Debezium Website: https://debezium.io/ • MongoDB Connector

    for Apache Kafka: https://www.mongodb.com/kafka-connector • Debezium Example Repositories: https://github.com/debezium/debezium-examples • Demo Repository: https://bit.ly/vdr21-sfp-demo
  14. @gunnarmorling @hpgrahsl Image Credits in order of appearance: @davidclode https://unsplash.com/photos/T49WTav4LgU

    @mparzuchowski https://unsplash.com/photos/geNNFqfvw48 @florianolv https://unsplash.com/photos/ca7tJ48YD64 @mischievous_penguins https://unsplash.com/photos/wqAuyugJIeU @jernejcic https://unsplash.com/photos/Oi31uKsnM1Q @ahmad_elsafty https://unsplash.com/photos/x97saLWeMNI @anniespratt https://unsplash.com/photos/t3IYuQZRDNE @janoschphotos https://unsplash.com/photos/w9rYOScXf6E @morvanic https://unsplash.com/photos/GiUJ02Yj_io