Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Event-Driven Applications: Apache Kafka and Python
Search
FTisiot
March 24, 2021
Technology
0
910
Event-Driven Applications: Apache Kafka and Python
How to create Event-Driven Applications with Apache Kafka and Python
FTisiot
March 24, 2021
Tweet
Share
More Decks by FTisiot
See All by FTisiot
Apache Kafka JDBC Source Connector: What could go wrong?
ftisiot
0
81
Solve the Knapsack problem with recursive queries and PostgreSQL
ftisiot
0
110
Build an Open Source Data Pipeline
ftisiot
0
270
Get to know Apache Kafka with Jupyter Notebooks
ftisiot
0
680
Fix Your Strings in PostgreSQL
ftisiot
0
250
Practical Tips & Tricks for Apache Kafka Messages Integration
ftisiot
0
190
Track Database Changes with Apache Kafka
ftisiot
0
250
Apache Kafka and Flink: Stateful Streaming Data Pipelines made easy with SQL
ftisiot
0
920
Kickstart your Kafka with Faker Data
ftisiot
0
370
Other Decks in Technology
See All in Technology
「家族アルバム みてね」を支えるS3ライフサイクル戦略
fanglang
4
640
AIと開発者の共創: エージェント時代におけるAIフレンドリーなDevOpsの実践
bicstone
1
200
.mdc駆動ナレッジマネジメント/.mdc-driven knowledge management
yodakeisuke
24
10k
Classmethod AI Talks(CATs) #20 司会進行スライド(2025.04.10) / classmethod-ai-talks-aka-cats_moderator-slides_vol20_2025-04-10
shinyaa31
0
110
LangfuseでAIエージェントの 可観測性を高めよう!/Enhancing AI Agent Observability with Langfuse!
jnymyk
0
150
ソフトウェアプロジェクトの成功率が上がらない原因-「社会価値を考える」ということ-
ytanaka5569
0
150
Lightdashの利活用状況 ー導入から2年経った現在地_20250409
hirokiigeta
2
270
LangChainとLangGiraphによるRAG・AIエージェント実践入門「10章 要件定義書生成Alエージェントの開発」輪読会スライド
takaakiinada
0
120
Spice up your notifications/try!Swift25
noppefoxwolf
2
160
30 代子育て SRE が考える SRE ナレッジマネジメントの現在と将来
kworkdev
PRO
0
200
DuckDB MCPサーバーを使ってAWSコストを分析させてみた / AWS cost analysis with DuckDB MCP server
masahirokawahara
0
210
システムとの会話から生まれる先手のDevOps
kakehashi
PRO
0
180
Featured
See All Featured
Reflections from 52 weeks, 52 projects
jeffersonlam
349
20k
Producing Creativity
orderedlist
PRO
344
40k
Into the Great Unknown - MozCon
thekraken
37
1.7k
How to Think Like a Performance Engineer
csswizardry
23
1.5k
The Cult of Friendly URLs
andyhume
78
6.3k
A Tale of Four Properties
chriscoyier
158
23k
The Straight Up "How To Draw Better" Workshop
denniskardys
232
140k
Bootstrapping a Software Product
garrettdimon
PRO
307
110k
Navigating Team Friction
lara
184
15k
Imperfection Machines: The Place of Print at Facebook
scottboms
267
13k
Code Review Best Practice
trishagee
67
18k
Documentation Writing (for coders)
carmenintech
69
4.7k
Transcript
Event-Driven Applications: Apache Kafka and Python Francesco Tisiot - Developer
Advocate @ftisiot
@ftisiot Why Kafka?
@ftisiot Why Kafka?
@ftisiot Why Kafka?
@ftisiot Event Driven Applications
@ftisiot What is an Event?
@ftisiot Why Event-driven?
@ftisiot What is Kafka? Topic A Topic B 0 1
2 3 4 0 1 2 3
@ftisiot Distributed Brokers Replication Factor 3 2
@ftisiot Distributed Replication Factor 2
@ftisiot What is a Kafka Event? <K, V> <"temp_max", 35.3>
<{ shop: "Mario's Pizza", phone_line: 3 } { orderId: 123456 name: "Andrea Pirlo", pizzas: [ { pizza: "Margherita", additionalToppings: ["Tuna","Onions","Ham"] }, ... ] } , >
@ftisiot Write to Kafka Producer Topic A Hostname + Port
Authentication Encoding
@ftisiot Hostname + Port Authentication Decoding Topic Name(s) Read from
Kafka Consumer Offset 0 1 2 3
@ftisiot Pizza Demo!
@ftisiot Log Size
@ftisiot Topic Partitions
@ftisiot Distributed
@ftisiot Topic Retention Policies Time Size
@ftisiot Selecting a Partition <K, V> Same Key Same Partition
@ftisiot Ordering 2 1 3 2 1 3 Partition 0
Partition 1 Partition 1 Guaranteed per Partition
@ftisiot Scaling Out P1 P2 P3 C1 C2
@ftisiot Partitions Demo!
@ftisiot Multiple Applications C1 C2 Consumer Group 1 CX Consumer
Group 2
@ftisiot Consumer Groups Demo!
@ftisiot Kafka Connect Kafka Connect Kafka Connect
@ftisiot Evolve Existing Applications Kafka Connect CDC
@ftisiot Distribute Events Config File Only!
@ftisiot Kafka Connect Demo!
@ftisiot Resources https://github.com/aiven/kafka-python-fake-data-producer https://aiven.io/ @ftisiot https://github.com/aiven/kafka-python-notebooks
@ftisiot Ask me Anything