Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Event-Driven Applications: Apache Kafka and Python
Search
FTisiot
March 24, 2021
Technology
0
930
Event-Driven Applications: Apache Kafka and Python
How to create Event-Driven Applications with Apache Kafka and Python
FTisiot
March 24, 2021
Tweet
Share
More Decks by FTisiot
See All by FTisiot
Apache Kafka JDBC Source Connector: What could go wrong?
ftisiot
0
92
Solve the Knapsack problem with recursive queries and PostgreSQL
ftisiot
0
120
Build an Open Source Data Pipeline
ftisiot
0
280
Get to know Apache Kafka with Jupyter Notebooks
ftisiot
0
690
Fix Your Strings in PostgreSQL
ftisiot
0
250
Practical Tips & Tricks for Apache Kafka Messages Integration
ftisiot
0
200
Track Database Changes with Apache Kafka
ftisiot
0
260
Apache Kafka and Flink: Stateful Streaming Data Pipelines made easy with SQL
ftisiot
0
930
Kickstart your Kafka with Faker Data
ftisiot
0
390
Other Decks in Technology
See All in Technology
BigQuery Remote FunctionでLooker Studioをインタラクティブ化
cuebic9bic
3
310
Fabric + Databricks 2025.6 の最新情報ピックアップ
ryomaru0825
1
140
なぜ私はいま、ここにいるのか? #もがく中堅デザイナー #プロダクトデザイナー
bengo4com
0
470
5min GuardDuty Extended Threat Detection EKS
takakuni
0
150
Абьюзим random_bytes(). Фёдор Кулаков, разработчик Lamoda Tech
lamodatech
0
340
SalesforceArchitectGroupOsaka#20_CNX'25_Report
atomica7sei
0
170
Claude Code Actionを使ったコード品質改善の取り組み
potix2
PRO
6
2.3k
急成長を支える基盤作り〜地道な改善からコツコツと〜 #cre_meetup
stefafafan
0
120
2年でここまで成長!AWSで育てたAI Slack botの軌跡
iwamot
PRO
4
720
Model Mondays S2E02: Model Context Protocol
nitya
0
220
Oracle Cloud Infrastructure:2025年6月度サービス・アップデート
oracle4engineer
PRO
2
250
地図も、未来も、オープンに。 〜OSGeo.JPとFOSS4Gのご紹介〜
wata909
0
110
Featured
See All Featured
How STYLIGHT went responsive
nonsquared
100
5.6k
Large-scale JavaScript Application Architecture
addyosmani
512
110k
Principles of Awesome APIs and How to Build Them.
keavy
126
17k
Templates, Plugins, & Blocks: Oh My! Creating the theme that thinks of everything
marktimemedia
31
2.4k
Design and Strategy: How to Deal with People Who Don’t "Get" Design
morganepeng
130
19k
Optimising Largest Contentful Paint
csswizardry
37
3.3k
Dealing with People You Can't Stand - Big Design 2015
cassininazir
367
26k
Designing for Performance
lara
609
69k
Let's Do A Bunch of Simple Stuff to Make Websites Faster
chriscoyier
507
140k
RailsConf & Balkan Ruby 2019: The Past, Present, and Future of Rails at GitHub
eileencodes
138
34k
Git: the NoSQL Database
bkeepers
PRO
430
65k
[RailsConf 2023] Rails as a piece of cake
palkan
55
5.6k
Transcript
Event-Driven Applications: Apache Kafka and Python Francesco Tisiot - Developer
Advocate @ftisiot
@ftisiot Why Kafka?
@ftisiot Why Kafka?
@ftisiot Why Kafka?
@ftisiot Event Driven Applications
@ftisiot What is an Event?
@ftisiot Why Event-driven?
@ftisiot What is Kafka? Topic A Topic B 0 1
2 3 4 0 1 2 3
@ftisiot Distributed Brokers Replication Factor 3 2
@ftisiot Distributed Replication Factor 2
@ftisiot What is a Kafka Event? <K, V> <"temp_max", 35.3>
<{ shop: "Mario's Pizza", phone_line: 3 } { orderId: 123456 name: "Andrea Pirlo", pizzas: [ { pizza: "Margherita", additionalToppings: ["Tuna","Onions","Ham"] }, ... ] } , >
@ftisiot Write to Kafka Producer Topic A Hostname + Port
Authentication Encoding
@ftisiot Hostname + Port Authentication Decoding Topic Name(s) Read from
Kafka Consumer Offset 0 1 2 3
@ftisiot Pizza Demo!
@ftisiot Log Size
@ftisiot Topic Partitions
@ftisiot Distributed
@ftisiot Topic Retention Policies Time Size
@ftisiot Selecting a Partition <K, V> Same Key Same Partition
@ftisiot Ordering 2 1 3 2 1 3 Partition 0
Partition 1 Partition 1 Guaranteed per Partition
@ftisiot Scaling Out P1 P2 P3 C1 C2
@ftisiot Partitions Demo!
@ftisiot Multiple Applications C1 C2 Consumer Group 1 CX Consumer
Group 2
@ftisiot Consumer Groups Demo!
@ftisiot Kafka Connect Kafka Connect Kafka Connect
@ftisiot Evolve Existing Applications Kafka Connect CDC
@ftisiot Distribute Events Config File Only!
@ftisiot Kafka Connect Demo!
@ftisiot Resources https://github.com/aiven/kafka-python-fake-data-producer https://aiven.io/ @ftisiot https://github.com/aiven/kafka-python-notebooks
@ftisiot Ask me Anything