Event-Driven Applications: Apache Kafka and Python

How to create Event-Driven Applications with Apache Kafka and Python

Transcript

1. Event-Driven Applications: Apache Kafka and Python Francesco Tisiot - Developer Advocate @ftisiot
2. @ftisiot Why Kafka?
3. @ftisiot Why Kafka?
4. @ftisiot Why Kafka?
5. @ftisiot Event Driven Applications
6. @ftisiot What is an Event?
7. @ftisiot Why Event-driven?
8. @ftisiot What is Kafka? Topic A Topic B 0 1 2 3 4 0 1 2 3
9. @ftisiot Distributed Brokers Replication Factor 3 2
10. @ftisiot Distributed Replication Factor 2
11. @ftisiot What is a Kafka Event? <K, V> <"temp_max", 35.3> <{ shop: "Mario's Pizza", phone_line: 3 } { orderId: 123456 name: "Andrea Pirlo", pizzas: [ { pizza: "Margherita", additionalToppings: ["Tuna","Onions","Ham"] }, ... ] } , >
12. @ftisiot Write to Kafka Producer Topic A Hostname + Port Authentication Encoding
13. @ftisiot Hostname + Port Authentication Decoding Topic Name(s) Read from Kafka Consumer Offset 0 1 2 3
14. @ftisiot Pizza Demo!
15. @ftisiot Log Size
16. @ftisiot Topic Partitions
17. @ftisiot Distributed
18. @ftisiot Topic Retention Policies Time Size
19. @ftisiot Selecting a Partition <K, V> Same Key Same Partition
20. @ftisiot Ordering 2 1 3 2 1 3 Partition 0 Partition 1 Partition 1 Guaranteed per Partition
21. @ftisiot Scaling Out P1 P2 P3 C1 C2
22. @ftisiot Partitions Demo!
23. @ftisiot Multiple Applications C1 C2 Consumer Group 1 CX Consumer Group 2
24. @ftisiot Consumer Groups Demo!
25. @ftisiot Kafka Connect Kafka Connect Kafka Connect
26. @ftisiot Evolve Existing Applications Kafka Connect CDC
27. @ftisiot Distribute Events Config File Only!
28. @ftisiot Kafka Connect Demo!
29. @ftisiot Resources https://github.com/aiven/kafka-python-fake-data-producer https://aiven.io/ @ftisiot https://github.com/aiven/kafka-python-notebooks
30. @ftisiot Ask me Anything