Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Event-Driven Applications: Apache Kafka and Python
Search
FTisiot
March 24, 2021
Technology
0
950
Event-Driven Applications: Apache Kafka and Python
How to create Event-Driven Applications with Apache Kafka and Python
FTisiot
March 24, 2021
Tweet
Share
More Decks by FTisiot
See All by FTisiot
Apache Kafka JDBC Source Connector: What could go wrong?
ftisiot
0
100
Solve the Knapsack problem with recursive queries and PostgreSQL
ftisiot
0
120
Build an Open Source Data Pipeline
ftisiot
0
290
Get to know Apache Kafka with Jupyter Notebooks
ftisiot
0
710
Fix Your Strings in PostgreSQL
ftisiot
0
260
Practical Tips & Tricks for Apache Kafka Messages Integration
ftisiot
0
210
Track Database Changes with Apache Kafka
ftisiot
0
270
Apache Kafka and Flink: Stateful Streaming Data Pipelines made easy with SQL
ftisiot
0
940
Kickstart your Kafka with Faker Data
ftisiot
0
410
Other Decks in Technology
See All in Technology
フルカイテン株式会社 エンジニア向け採用資料
fullkaiten
0
9k
about #74462 go/token#FileSet
tomtwinkle
1
290
それでも私はContextに値を詰めたい | Go Conference 2025 / go conference 2025 fill context
budougumi0617
4
1.2k
Green Tea Garbage Collector の今
zchee
PRO
2
390
Azure SynapseからAzure Databricksへ 移行してわかった新時代のコスト問題!?
databricksjapan
0
140
Exadata Database Service on Dedicated Infrastructure(ExaDB-D) UI スクリーン・キャプチャ集
oracle4engineer
PRO
2
5.4k
PLaMo2シリーズのvLLM実装 / PFN LLM セミナー
pfn
PRO
2
980
AI駆動開発を推進するためにサービス開発チームで 取り組んでいること
noayaoshiro
0
160
リーダーになったら未来を語れるようになろう/Speak the Future
sanogemaru
0
280
いまさら聞けない ABテスト入門
skmr2348
1
200
社内お問い合わせBotの仕組みと学び
nish01
0
240
10年の共創が示す、これからの開発者と企業の関係 ~ Crossroad
soracom
PRO
1
170
Featured
See All Featured
Learning to Love Humans: Emotional Interface Design
aarron
274
40k
Faster Mobile Websites
deanohume
310
31k
The World Runs on Bad Software
bkeepers
PRO
71
11k
How GitHub (no longer) Works
holman
315
140k
Done Done
chrislema
185
16k
Keith and Marios Guide to Fast Websites
keithpitt
411
22k
Product Roadmaps are Hard
iamctodd
PRO
54
11k
Testing 201, or: Great Expectations
jmmastey
45
7.7k
Intergalactic Javascript Robots from Outer Space
tanoku
273
27k
Building Adaptive Systems
keathley
43
2.8k
Exploring the Power of Turbo Streams & Action Cable | RailsConf2023
kevinliebholz
34
6.1k
Site-Speed That Sticks
csswizardry
11
880
Transcript
Event-Driven Applications: Apache Kafka and Python Francesco Tisiot - Developer
Advocate @ftisiot
@ftisiot Why Kafka?
@ftisiot Why Kafka?
@ftisiot Why Kafka?
@ftisiot Event Driven Applications
@ftisiot What is an Event?
@ftisiot Why Event-driven?
@ftisiot What is Kafka? Topic A Topic B 0 1
2 3 4 0 1 2 3
@ftisiot Distributed Brokers Replication Factor 3 2
@ftisiot Distributed Replication Factor 2
@ftisiot What is a Kafka Event? <K, V> <"temp_max", 35.3>
<{ shop: "Mario's Pizza", phone_line: 3 } { orderId: 123456 name: "Andrea Pirlo", pizzas: [ { pizza: "Margherita", additionalToppings: ["Tuna","Onions","Ham"] }, ... ] } , >
@ftisiot Write to Kafka Producer Topic A Hostname + Port
Authentication Encoding
@ftisiot Hostname + Port Authentication Decoding Topic Name(s) Read from
Kafka Consumer Offset 0 1 2 3
@ftisiot Pizza Demo!
@ftisiot Log Size
@ftisiot Topic Partitions
@ftisiot Distributed
@ftisiot Topic Retention Policies Time Size
@ftisiot Selecting a Partition <K, V> Same Key Same Partition
@ftisiot Ordering 2 1 3 2 1 3 Partition 0
Partition 1 Partition 1 Guaranteed per Partition
@ftisiot Scaling Out P1 P2 P3 C1 C2
@ftisiot Partitions Demo!
@ftisiot Multiple Applications C1 C2 Consumer Group 1 CX Consumer
Group 2
@ftisiot Consumer Groups Demo!
@ftisiot Kafka Connect Kafka Connect Kafka Connect
@ftisiot Evolve Existing Applications Kafka Connect CDC
@ftisiot Distribute Events Config File Only!
@ftisiot Kafka Connect Demo!
@ftisiot Resources https://github.com/aiven/kafka-python-fake-data-producer https://aiven.io/ @ftisiot https://github.com/aiven/kafka-python-notebooks
@ftisiot Ask me Anything