Get to know Apache Kafka with Jupyter Notebooks

Apache Kafka is recognised as the best data streaming platform around, but it can be difficult to observe what is happening when you are just getting started with this excellent technology platform. In this session, you will get a tour of key Kafka features using the delightful web UI of Jupyter notebooks.

Use the notebooks to see Kafka in action, producing and consuming records. We’ll also cover how to get the best from your application by making good use of topic partitioning and consumer groups. From high-level concepts to nitty-gritty details, this session will equip you with the skills you need to build applications with Kafka. Recommended for anyone curious about Apache Kafka or using Jupyter to work with or learn new data platforms.

Transcript

1. Francesco Tisiot - Developer Advocate @ftisiot Get to Know Apache Kafka with Jupyter Notebooks
2. @ftisiot | @aiven_io Why Jupyter Notebooks
3. @ftisiot | @aiven_io Title Text Markdown Text Images Executable Code
4. @ftisiot | @aiven_io Title Text https:/ /aiven.io/blog/teach-yourself-apache-kafka-and-python-with-a-jupyter-notebook
5. @ftisiot | @aiven_io Why Apache Kafka
6. @ftisiot | @aiven_io
7. @ftisiot | @aiven_io
8. @ftisiot | @aiven_io Event-Driven Applications
9. @ftisiot | @aiven_io What is an Event
10. @ftisiot | @aiven_io Why Event Driven?
11. @ftisiot | @aiven_io What is Kafka Topic A Topic B 0 1 2 3 4 0 1 2 3
12. @ftisiot | @aiven_io Distributed Brokers Replication Factor 3 2
13. @ftisiot | @aiven_io What’s a Kafka Event? <K, V> <"temp_max", 35.3> <{ shop: "Mario's Pizza", phone_line: 3 } { orderId: 123456 name: "Andrea Pirlo", pizzas: [ { pizza: "Margherita", additionalToppings: ["Tuna","Onions","Ham"] }, ... ] } , >
14. @ftisiot | @aiven_io Write to Kafka Producer Topic A Hostname + Port Authentication Encoding
15. @ftisiot | @aiven_io Read from Kafka Hostname + Port Authentication Decoding Topic Name(s) Consumer Offset 0 1 2 3
16. @ftisiot | @aiven_io Pizza Demo!
17. @ftisiot | @aiven_io Log Size
18. @ftisiot | @aiven_io Topic Partitions
19. @ftisiot | @aiven_io
20. @ftisiot | @aiven_io Selecting the Partition <K, V> Same Key Same Partition
21. @ftisiot | @aiven_io Topic Partitions 2 1 3 2 1 3 Partition 0 Partition 1 Guaranteed per Partition
22. @ftisiot | @aiven_io Scaling Out P1 P2 P3 C1 C2
23. @ftisiot | @aiven_io Partitions Demo!
24. @ftisiot | @aiven_io Multiple Applications C1 C2 Consumer Group 1 CX Consumer Group 2
25. @ftisiot | @aiven_io Kafka Connect Kafka Connect Kafka Connect
26. @ftisiot | @aiven_io References https:/ /github.com/aiven/kafka-python-fake-data-producer https:/ /aiven.io @ftisiot https:/ /github.com/aiven/kafka-python-notebooks
27. @ftisiot | @aiven_io Title Text