Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Event-Driven Applications: Apache Kafka and Python
Search
FTisiot
March 24, 2021
Technology
0
930
Event-Driven Applications: Apache Kafka and Python
How to create Event-Driven Applications with Apache Kafka and Python
FTisiot
March 24, 2021
Tweet
Share
More Decks by FTisiot
See All by FTisiot
Apache Kafka JDBC Source Connector: What could go wrong?
ftisiot
0
88
Solve the Knapsack problem with recursive queries and PostgreSQL
ftisiot
0
110
Build an Open Source Data Pipeline
ftisiot
0
280
Get to know Apache Kafka with Jupyter Notebooks
ftisiot
0
690
Fix Your Strings in PostgreSQL
ftisiot
0
250
Practical Tips & Tricks for Apache Kafka Messages Integration
ftisiot
0
200
Track Database Changes with Apache Kafka
ftisiot
0
260
Apache Kafka and Flink: Stateful Streaming Data Pipelines made easy with SQL
ftisiot
0
930
Kickstart your Kafka with Faker Data
ftisiot
0
390
Other Decks in Technology
See All in Technology
GitHub Coding Agent 概要
kkamegawa
1
1.3k
AIコードエディタは開発を変えるか?Cursorをチームに導入して1ヶ月経った本音
ota1022
1
640
面接を通過するためにやってて良かったこと3選
sansantech
PRO
0
110
declaration mergingの威力:ライブラリアップデート時の書き換え作業を90%短縮するテクニック
yutake27
2
270
プロジェクトマネジメント実践論|現役エンジニアが語る!~チームでモノづくりをする時のコツとは?~
mixi_engineers
PRO
3
160
やさしいClaude Code入門
minorun365
PRO
10
4.9k
Houtou.pm #1
papix
0
590
大事なのは、AIの精度だけじゃない!〜1円のズレも許されない経理領域とAI〜
jun_nemoto
10
5k
FastMCPでSQLをチェックしてくれるMCPサーバーを自作してCursorから動かしてみた
nayuts
1
170
Introduction to Sansan, inc / Sansan Global Development Center, Inc.
sansan33
PRO
0
2.6k
What's Next in OpenShift Q2 CY2025
redhatlivestreaming
1
450
Digitization部 紹介資料
sansan33
PRO
1
3.8k
Featured
See All Featured
Building a Modern Day E-commerce SEO Strategy
aleyda
40
7.3k
Building an army of robots
kneath
306
45k
Done Done
chrislema
184
16k
The Art of Delivering Value - GDevCon NA Keynote
reverentgeek
14
1.5k
I Don’t Have Time: Getting Over the Fear to Launch Your Podcast
jcasabona
32
2.3k
The Invisible Side of Design
smashingmag
299
50k
StorybookのUI Testing Handbookを読んだ
zakiyama
30
5.8k
Faster Mobile Websites
deanohume
307
31k
How GitHub (no longer) Works
holman
314
140k
Documentation Writing (for coders)
carmenintech
71
4.8k
Agile that works and the tools we love
rasmusluckow
329
21k
It's Worth the Effort
3n
184
28k
Transcript
Event-Driven Applications: Apache Kafka and Python Francesco Tisiot - Developer
Advocate @ftisiot
@ftisiot Why Kafka?
@ftisiot Why Kafka?
@ftisiot Why Kafka?
@ftisiot Event Driven Applications
@ftisiot What is an Event?
@ftisiot Why Event-driven?
@ftisiot What is Kafka? Topic A Topic B 0 1
2 3 4 0 1 2 3
@ftisiot Distributed Brokers Replication Factor 3 2
@ftisiot Distributed Replication Factor 2
@ftisiot What is a Kafka Event? <K, V> <"temp_max", 35.3>
<{ shop: "Mario's Pizza", phone_line: 3 } { orderId: 123456 name: "Andrea Pirlo", pizzas: [ { pizza: "Margherita", additionalToppings: ["Tuna","Onions","Ham"] }, ... ] } , >
@ftisiot Write to Kafka Producer Topic A Hostname + Port
Authentication Encoding
@ftisiot Hostname + Port Authentication Decoding Topic Name(s) Read from
Kafka Consumer Offset 0 1 2 3
@ftisiot Pizza Demo!
@ftisiot Log Size
@ftisiot Topic Partitions
@ftisiot Distributed
@ftisiot Topic Retention Policies Time Size
@ftisiot Selecting a Partition <K, V> Same Key Same Partition
@ftisiot Ordering 2 1 3 2 1 3 Partition 0
Partition 1 Partition 1 Guaranteed per Partition
@ftisiot Scaling Out P1 P2 P3 C1 C2
@ftisiot Partitions Demo!
@ftisiot Multiple Applications C1 C2 Consumer Group 1 CX Consumer
Group 2
@ftisiot Consumer Groups Demo!
@ftisiot Kafka Connect Kafka Connect Kafka Connect
@ftisiot Evolve Existing Applications Kafka Connect CDC
@ftisiot Distribute Events Config File Only!
@ftisiot Kafka Connect Demo!
@ftisiot Resources https://github.com/aiven/kafka-python-fake-data-producer https://aiven.io/ @ftisiot https://github.com/aiven/kafka-python-notebooks
@ftisiot Ask me Anything