Lock in $30 Savings on PRO—Offer Ends Soon! ⏳

Track Database Changes with Apache Kafka

FTisiot
September 11, 2021
Technology
0
240

Track Database Changes with Apache Kafka

Tired of the spaghetti architecture created by hundreds applications and ETL flows dependencies making every database table change a matter of 100 people meeting? Understand how you can decouple and integrate better your systems with Apache Kafka and Kafka Connect!

FTisiot

September 11, 2021
Tweet

More Decks by FTisiot

See All by FTisiot
Apache Kafka JDBC Source Connector: What could go wrong?
ftisiot
0
58
Solve the Knapsack problem with recursive queries and PostgreSQL
ftisiot
0
95
Build an Open Source Data Pipeline
ftisiot
0
260
Get to know Apache Kafka with Jupyter Notebooks
ftisiot
0
640
Fix Your Strings in PostgreSQL
ftisiot
0
230
Practical Tips & Tricks for Apache Kafka Messages Integration
ftisiot
0
170
Apache Kafka and Flink: Stateful Streaming Data Pipelines made easy with SQL
ftisiot
0
880
Kickstart your Kafka with Faker Data
ftisiot
0
340
Event-Driven Applications: Apache Kafka and Python
ftisiot
0
880

Other Decks in Technology

See All in Technology
メインテーマはKubernetes
nwiizo
2
300
Behind the scenes of 24-hour global online event “JAWS PANKRATION 2024”
syoshie
0
110
SLMをエッジAIとして検証してみて分かったこと
iotcomjpadmin
0
180
asumikamというカンファレンスオーガナイザの凄さを語る / The Brilliance of Asumikam
tomzoh
1
160
日本全国・都市3D化プロジェクト「PLATEAU」とデータ変換OSS「PLATEAU GIS Converter」の公開
nokonoko1203
2
260
マルチプロダクト、マルチデータ基盤での Looker活用事例 〜BQじゃなくてもLookerはいいぞ〜
gappy50
0
100
Mastering Quickfix
daisuzu
2
470
共創するアーキテクチャ ～チーム全体で築く持続可能な開発エコシステム～ / Co-Creating Architecture - A Sustainable Development Ecosystem Built by the Entire Team
bitkey
PRO
1
3.8k
もう一度、 事業を支えるシステムに。
leveragestech
6
3k
LLMを「速く」「安く」 動かすには / CloudNative Days Winter 2024
pfn
PRO
4
1.2k
【CNDW2024】SIerで200人クラウドネイティブのファンを増やした話
yuta1979
1
240
140年の歴史あるエンタープライズ企業の内製化×マイクロサービス化への航海
yussugi
0
3.5k

Featured

See All Featured
Six Lessons from altMBA
skipperchong
27
3.5k
Reflections from 52 weeks, 52 projects
jeffersonlam
346
20k
Teambox: Starting and Learning
jrom
133
8.8k
Mobile First: as difficult as doing things right
swwweet
222
8.9k
Why You Should Never Use an ORM
jnunemaker
PRO
54
9.1k
Designing Experiences People Love
moore
138
23k
Practical Orchestrator
shlominoach
186
10k
Docker and Python
trallard
40
3.1k
Rebuilding a faster, lazier Slack
samanthasiow
79
8.7k
The MySQL Ecosystem @ GitHub 2015
samlambert
250
12k
Responsive Adventures: Dirty Tricks From The Dark Corners of Front-End
smashingmag
250
21k
Unsuck your backbone
ammeep
669
57k

Transcript

  1. I don’t Want to Miss a Thing Track Database Changes

    with Apache Kafka Francesco Tisiot - Developer Advocate @ftisiot

  2. @ftisiot

  3. @ftisiot

  4. @ftisiot

  5. @ftisiot

  6. @ftisiot What is Apache Kafka?

  7. @ftisiot Topic A Topic B 0 1 2 3 4

    0 1 2 3 Producer Consumer Producer Consumer Consumer

  8. @ftisiot Events in a Database?

  9. @ftisiot Table Log vs The Fridge Dilemma

  10. @ftisiot Table Log 5 3 6 +7 +3 +6 -5

    -2 +5

  11. @ftisiot Integrating Apache Kafka

  12. @ftisiot Kafka Connect Source Kafka Connect Sink

  13. @ftisiot Changes in Kafka?

  14. @ftisiot Query Based Approach

  15. @ftisiot

  16. @ftisiot Log Based Approach

  17. @ftisiot

  18. @ftisiot Debezium

  19. @ftisiot

  20. @ftisiot Query Bulk Query Timestamp Log INSERT UPDATE DELETE NOTES

  21. @ftisiot Resources https://aiven.io https://debezium.io/ https://aiven.io/blog/using-kafka-connect-jdbc-source- a-postgresql-example https://kafka.apache.org/