Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Debezium: What's new (KafkaSummit 2021)

Debezium: What's new (KafkaSummit 2021)

Overview of latest changes in Debezium, from a presentation at the Red Hat booth at Kafka Summit Americas 2021.

8e25c0ca4bf25113bd9c0ccc5d118164?s=128

Gunnar Morling

September 15, 2021
Tweet

Transcript

  1. 1 Debezium - What’s new, what’s coming? Gunnar Morling

  2. #Debezium • Taps into TX log to capture INSERT/UPDATE/DELETE events

    • Propagated to consumers via Apache Kafka and Kafka Connect Debezium — Log-based Change Data Capture
  3. #Debezium • CDC for multiple databases ◦ Based on transaction

    logs ◦ Snapshotting, Filtering etc. • Fully open-source, very active community • Latest version: 1.6 • Large production deployments at multiple companies Debezium Streaming Changes out of the Database
  4. #Debezium Debezium Release Model 2021 Q3 2021 Q4 Red Hat

    Integration Debezium 1.6 2021 Q1 Upstream Debezium 1.5 2021 Q2 Red Hat Integration Debezium 1.4 Upstream Debezium 1.6 Upstream Debezium 1.7 Upstream Debezium 1.8 Red Hat Integration Debezium 1.5
  5. #Debezium • Distributed Tracing • Incubating Vitess connector (community-led) Debezium

    1.4 What’s New?
  6. #Debezium

  7. #Debezium • Stable LogMiner-based connector for Oracle • Transaction metadata

    topic for MySQL • Support for Redis Streams in Debezium Server Debezium 1.5 What’s New?
  8. #Debezium • Incremental snapshots ◦ Resumeable ◦ Re-snapshot selected tables

    ◦ Modify filter config ◦ Begin streaming while snapshotting • Support for Pravega in Debezium Server Debezium 1.6 What’s New?
  9. #Debezium • Debezium UI • Stabilization and improvements to incremental

    snapshotting • Schema history compaction • Off-heap transaction buffer for the Oracle connector • Multi-database support for the SQL Server connector • Performance improvements Debezium Roadmap What’s Coming?
  10. #Debezium • Debezium Examples https://github.com/debezium/debezium-examples/ • Saga orchestration with Debezium

    and CDC https://www.infoq.com/articles/saga-orchestration-outbox/ • Getting started with Debezium in Red Hat Integration https://access.redhat.com/documentation/en-us/red_hat_integra tion/2021.q3/html/getting_started_with_debezium/index Resources