Debezium: What's new (KafkaSummit 2021)

Overview of latest changes in Debezium, from a presentation at the Red Hat booth at Kafka Summit Americas 2021.

Transcript

1. 1 Debezium - What’s new, what’s coming? Gunnar Morling
2. #Debezium • Taps into TX log to capture INSERT/UPDATE/DELETE events • Propagated to consumers via Apache Kafka and Kafka Connect Debezium — Log-based Change Data Capture
3. #Debezium • CDC for multiple databases ◦ Based on transaction logs ◦ Snapshotting, Filtering etc. • Fully open-source, very active community • Latest version: 1.6 • Large production deployments at multiple companies Debezium Streaming Changes out of the Database
4. #Debezium Debezium Release Model 2021 Q3 2021 Q4 Red Hat Integration Debezium 1.6 2021 Q1 Upstream Debezium 1.5 2021 Q2 Red Hat Integration Debezium 1.4 Upstream Debezium 1.6 Upstream Debezium 1.7 Upstream Debezium 1.8 Red Hat Integration Debezium 1.5
5. #Debezium • Distributed Tracing • Incubating Vitess connector (community-led) Debezium 1.4 What’s New?
6. #Debezium
7. #Debezium • Stable LogMiner-based connector for Oracle • Transaction metadata topic for MySQL • Support for Redis Streams in Debezium Server Debezium 1.5 What’s New?
8. #Debezium • Incremental snapshots ◦ Resumeable ◦ Re-snapshot selected tables ◦ Modify filter config ◦ Begin streaming while snapshotting • Support for Pravega in Debezium Server Debezium 1.6 What’s New?
9. #Debezium • Debezium UI • Stabilization and improvements to incremental snapshotting • Schema history compaction • Off-heap transaction buffer for the Oracle connector • Multi-database support for the SQL Server connector • Performance improvements Debezium Roadmap What’s Coming?
10. #Debezium • Debezium Examples https://github.com/debezium/debezium-examples/ • Saga orchestration with Debezium and CDC https://www.infoq.com/articles/saga-orchestration-outbox/ • Getting started with Debezium in Red Hat Integration https://access.redhat.com/documentation/en-us/red_hat_integra tion/2021.q3/html/getting_started_with_debezium/index Resources