Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Speaker Deck
PRO
Sign in
Sign up
for free
Event-Driven Applications: Apache Kafka and Python
FTisiot
March 24, 2021
Technology
0
41
Event-Driven Applications: Apache Kafka and Python
How to create Event-Driven Applications with Apache Kafka and Python
FTisiot
March 24, 2021
Tweet
Share
More Decks by FTisiot
See All by FTisiot
ftisiot
0
11
ftisiot
0
80
ftisiot
0
410
ftisiot
0
79
ftisiot
0
130
ftisiot
0
120
ftisiot
0
160
ftisiot
0
28
ftisiot
0
190
Other Decks in Technology
See All in Technology
clustervr
PRO
0
190
sei88888
5
380
ishiitetsuji
0
110
bolonio
4
560
oracle4engineer
0
130
okuken
0
320
pauloxnet
1
300
yuzoiwasaki
0
160
kateinoigakukun
0
120
opelab
1
430
techclouderp
0
260
bcrikko
1
160
Featured
See All Featured
roundedbygravity
242
21k
revolveconf
200
9.7k
brianwarren
82
4.8k
jponch
103
5.1k
tammielis
237
23k
jasonvnalue
81
8.1k
malarkey
193
8.7k
eitanlees
115
10k
morganepeng
94
14k
marktimemedia
7
440
thoeni
3
650
bkeepers
PRO
322
53k
Transcript
Event-Driven Applications: Apache Kafka and Python Francesco Tisiot - Developer
Advocate @ftisiot
@ftisiot Why Kafka?
@ftisiot Why Kafka?
@ftisiot Why Kafka?
@ftisiot Event Driven Applications
@ftisiot What is an Event?
@ftisiot Why Event-driven?
@ftisiot What is Kafka? Topic A Topic B 0 1
2 3 4 0 1 2 3
@ftisiot Distributed Brokers Replication Factor 3 2
@ftisiot Distributed Replication Factor 2
@ftisiot What is a Kafka Event? <K, V> <"temp_max", 35.3>
<{ shop: "Mario's Pizza", phone_line: 3 } { orderId: 123456 name: "Andrea Pirlo", pizzas: [ { pizza: "Margherita", additionalToppings: ["Tuna","Onions","Ham"] }, ... ] } , >
@ftisiot Write to Kafka Producer Topic A Hostname + Port
Authentication Encoding
@ftisiot Hostname + Port Authentication Decoding Topic Name(s) Read from
Kafka Consumer Offset 0 1 2 3
@ftisiot Pizza Demo!
@ftisiot Log Size
@ftisiot Topic Partitions
@ftisiot Distributed
@ftisiot Topic Retention Policies Time Size
@ftisiot Selecting a Partition <K, V> Same Key Same Partition
@ftisiot Ordering 2 1 3 2 1 3 Partition 0
Partition 1 Partition 1 Guaranteed per Partition
@ftisiot Scaling Out P1 P2 P3 C1 C2
@ftisiot Partitions Demo!
@ftisiot Multiple Applications C1 C2 Consumer Group 1 CX Consumer
Group 2
@ftisiot Consumer Groups Demo!
@ftisiot Kafka Connect Kafka Connect Kafka Connect
@ftisiot Evolve Existing Applications Kafka Connect CDC
@ftisiot Distribute Events Config File Only!
@ftisiot Kafka Connect Demo!
@ftisiot Resources https://github.com/aiven/kafka-python-fake-data-producer https://aiven.io/ @ftisiot https://github.com/aiven/kafka-python-notebooks
@ftisiot Ask me Anything