Save 37% off PRO during our Black Friday Sale! »

Event-Driven Applications: Apache Kafka and Python

A23789f299ed06fe7d9f1c6940440bfa?s=47 FTisiot
March 24, 2021

Event-Driven Applications: Apache Kafka and Python

How to create Event-Driven Applications with Apache Kafka and Python

A23789f299ed06fe7d9f1c6940440bfa?s=128

FTisiot

March 24, 2021
Tweet

Transcript

  1. Event-Driven Applications: Apache Kafka and Python Francesco Tisiot - Developer

    Advocate @ftisiot
  2. @ftisiot Why Kafka?

  3. @ftisiot Why Kafka?

  4. @ftisiot Why Kafka?

  5. @ftisiot Event Driven Applications

  6. @ftisiot What is an Event?

  7. @ftisiot Why Event-driven?

  8. @ftisiot What is Kafka? Topic A Topic B 0 1

    2 3 4 0 1 2 3
  9. @ftisiot Distributed Brokers Replication Factor 3 2

  10. @ftisiot Distributed Replication Factor 2

  11. @ftisiot What is a Kafka Event? <K, V> <"temp_max", 35.3>

    <{ shop: "Mario's Pizza", phone_line: 3 } { orderId: 123456 name: "Andrea Pirlo", pizzas: [ { pizza: "Margherita", additionalToppings: ["Tuna","Onions","Ham"] }, ... ] } , >
  12. @ftisiot Write to Kafka Producer Topic A Hostname + Port

    Authentication Encoding
  13. @ftisiot Hostname + Port Authentication Decoding Topic Name(s) Read from

    Kafka Consumer Offset 0 1 2 3
  14. @ftisiot Pizza Demo!

  15. @ftisiot Log Size

  16. @ftisiot Topic Partitions

  17. @ftisiot Distributed

  18. @ftisiot Topic Retention Policies Time Size

  19. @ftisiot Selecting a Partition <K, V> Same Key Same Partition

  20. @ftisiot Ordering 2 1 3 2 1 3 Partition 0

    Partition 1 Partition 1 Guaranteed per Partition
  21. @ftisiot Scaling Out P1 P2 P3 C1 C2

  22. @ftisiot Partitions Demo!

  23. @ftisiot Multiple Applications C1 C2 Consumer Group 1 CX Consumer

    Group 2
  24. @ftisiot Consumer Groups Demo!

  25. @ftisiot Kafka Connect Kafka Connect Kafka Connect

  26. @ftisiot Evolve Existing Applications Kafka Connect CDC

  27. @ftisiot Distribute Events Config File Only!

  28. @ftisiot Kafka Connect Demo!

  29. @ftisiot Resources https://github.com/aiven/kafka-python-fake-data-producer https://aiven.io/ @ftisiot https://github.com/aiven/kafka-python-notebooks

  30. @ftisiot Ask me Anything