Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Event-Driven Applications: Apache Kafka and Python
Search
FTisiot
March 24, 2021
Technology
0
930
Event-Driven Applications: Apache Kafka and Python
How to create Event-Driven Applications with Apache Kafka and Python
FTisiot
March 24, 2021
Tweet
Share
More Decks by FTisiot
See All by FTisiot
Apache Kafka JDBC Source Connector: What could go wrong?
ftisiot
0
90
Solve the Knapsack problem with recursive queries and PostgreSQL
ftisiot
0
120
Build an Open Source Data Pipeline
ftisiot
0
280
Get to know Apache Kafka with Jupyter Notebooks
ftisiot
0
690
Fix Your Strings in PostgreSQL
ftisiot
0
250
Practical Tips & Tricks for Apache Kafka Messages Integration
ftisiot
0
200
Track Database Changes with Apache Kafka
ftisiot
0
260
Apache Kafka and Flink: Stateful Streaming Data Pipelines made easy with SQL
ftisiot
0
930
Kickstart your Kafka with Faker Data
ftisiot
0
390
Other Decks in Technology
See All in Technology
Amazon ECS & AWS Fargate 運用アーキテクチャ2025 / Amazon ECS and AWS Fargate Ops Architecture 2025
iselegant
12
3.5k
実践! AIエージェント導入記
1mono2prod
0
120
成立するElixirの再束縛(再代入)可という選択
kubell_hr
0
510
生成AIをテストプロセスに活用し"よう"としている話 #jasstnano
makky_tyuyan
0
250
vLLM meetup Tokyo
jpishikawa
1
270
In Praise of "Normal" Engineers (LDX3)
charity
2
1.1k
VISITS_AIIoTビジネス共創ラボ登壇資料.pdf
iotcomjpadmin
0
140
DB 醬,嗨!哪泥嘎斯基?
line_developers_tw
PRO
0
1k
Observability infrastructure behind the trillion-messages scale Kafka platform
lycorptech_jp
PRO
0
120
初めてのAzure FunctionsをClaude Codeで作ってみた / My first Azure Functions using Claude Code
hideakiaoyagi
1
160
工具人的一生: 開發很多 AI 工具讓我 慵懶過一生
line_developers_tw
PRO
0
1k
25分で解説する「最小権限の原則」を実現するための AWS「ポリシー」大全 / 20250625-aws-summit-aws-policy
opelab
4
420
Featured
See All Featured
How To Stay Up To Date on Web Technology
chriscoyier
790
250k
Docker and Python
trallard
44
3.4k
Stop Working from a Prison Cell
hatefulcrawdad
269
20k
Building a Modern Day E-commerce SEO Strategy
aleyda
41
7.3k
Designing Dashboards & Data Visualisations in Web Apps
destraynor
231
53k
Reflections from 52 weeks, 52 projects
jeffersonlam
351
20k
Building Better People: How to give real-time feedback that sticks.
wjessup
367
19k
Bootstrapping a Software Product
garrettdimon
PRO
307
110k
Put a Button on it: Removing Barriers to Going Fast.
kastner
60
3.9k
Creating an realtime collaboration tool: Agile Flush - .NET Oxford
marcduiker
30
2.1k
Making the Leap to Tech Lead
cromwellryan
134
9.3k
The Pragmatic Product Professional
lauravandoore
35
6.7k
Transcript
Event-Driven Applications: Apache Kafka and Python Francesco Tisiot - Developer
Advocate @ftisiot
@ftisiot Why Kafka?
@ftisiot Why Kafka?
@ftisiot Why Kafka?
@ftisiot Event Driven Applications
@ftisiot What is an Event?
@ftisiot Why Event-driven?
@ftisiot What is Kafka? Topic A Topic B 0 1
2 3 4 0 1 2 3
@ftisiot Distributed Brokers Replication Factor 3 2
@ftisiot Distributed Replication Factor 2
@ftisiot What is a Kafka Event? <K, V> <"temp_max", 35.3>
<{ shop: "Mario's Pizza", phone_line: 3 } { orderId: 123456 name: "Andrea Pirlo", pizzas: [ { pizza: "Margherita", additionalToppings: ["Tuna","Onions","Ham"] }, ... ] } , >
@ftisiot Write to Kafka Producer Topic A Hostname + Port
Authentication Encoding
@ftisiot Hostname + Port Authentication Decoding Topic Name(s) Read from
Kafka Consumer Offset 0 1 2 3
@ftisiot Pizza Demo!
@ftisiot Log Size
@ftisiot Topic Partitions
@ftisiot Distributed
@ftisiot Topic Retention Policies Time Size
@ftisiot Selecting a Partition <K, V> Same Key Same Partition
@ftisiot Ordering 2 1 3 2 1 3 Partition 0
Partition 1 Partition 1 Guaranteed per Partition
@ftisiot Scaling Out P1 P2 P3 C1 C2
@ftisiot Partitions Demo!
@ftisiot Multiple Applications C1 C2 Consumer Group 1 CX Consumer
Group 2
@ftisiot Consumer Groups Demo!
@ftisiot Kafka Connect Kafka Connect Kafka Connect
@ftisiot Evolve Existing Applications Kafka Connect CDC
@ftisiot Distribute Events Config File Only!
@ftisiot Kafka Connect Demo!
@ftisiot Resources https://github.com/aiven/kafka-python-fake-data-producer https://aiven.io/ @ftisiot https://github.com/aiven/kafka-python-notebooks
@ftisiot Ask me Anything