Track Database Changes with Apache Kafka

Tired of the spaghetti architecture created by hundreds applications and ETL flows dependencies making every database table change a matter of 100 people meeting? Understand how you can decouple and integrate better your systems with Apache Kafka and Kafka Connect!

Transcript

1. I don’t Want to Miss a Thing Track Database Changes with Apache Kafka Francesco Tisiot - Developer Advocate @ftisiot
2. @ftisiot
3. @ftisiot
4. @ftisiot
5. @ftisiot
6. @ftisiot What is Apache Kafka?
7. @ftisiot Topic A Topic B 0 1 2 3 4 0 1 2 3 Producer Consumer Producer Consumer Consumer
8. @ftisiot Events in a Database?
9. @ftisiot Table Log vs The Fridge Dilemma
10. @ftisiot Table Log 5 3 6 +7 +3 +6 -5 -2 +5
11. @ftisiot Integrating Apache Kafka
12. @ftisiot Kafka Connect Source Kafka Connect Sink
13. @ftisiot Changes in Kafka?
14. @ftisiot Query Based Approach
15. @ftisiot
16. @ftisiot Log Based Approach
17. @ftisiot
18. @ftisiot Debezium
19. @ftisiot
20. @ftisiot Query Bulk Query Timestamp Log INSERT UPDATE DELETE NOTES
21. @ftisiot Resources https://aiven.io https://debezium.io/ https://aiven.io/blog/using-kafka-connect-jdbc-source- a-postgresql-example https://kafka.apache.org/