Slide 1

Slide 1 text

The talk will start shortly - before then, have a chat with your fellow attendees! Questions at the end This is unless the speaker says otherwise. But generally, because of internet delays and lags it can be a bit disruptive during an Online talk. Online Meetup Etiquette Mute during talk But remember to unmute when you want to ask a question at the end (or chat with fellow attendees before/After) Use chat to engage Use Zoom chat and/or create a thread in the #events channel on our Community Slack space so that your discussion can continue afterwards with fellow Kafkateers. Be on Camera & React! Speakers during these events are talking to blank screens. And it can be exhausting! Use Zoom reactions at the bottom of the page to give them something to work off of! 1. 2. Continue learning and collaborating WELCOME! Thank you for joining us in these unique circumstances. We hope you’re safe and well. 3. If you haven’t already joined, get on the our slack workspace and continue the conversation with other Kafkateers. If you’re already a member, start a thread for this event in #events. A single online source of everything you’ll need to learn Kafka. Plus it’s totally free and ungated… Confluent Developer developer.confluent.io https://cnfl.io/slack Confluent Community Slack You are being recorded and this footage may be added to public channels

Slide 2

Slide 2 text

About me @riferrei | #kafkameetup | @CONFLUENTINC • RICARDO FERREIRA • Works for confluent • Developer advocate • Ricardo@confluent.iO • HTTPS://RIFERREI.NET

Slide 3

Slide 3 text

Building event streaming apps with pac-man @riferrei | #kafkameetup | @CONFLUENTINC

Slide 4

Slide 4 text

@riferrei | #kafkameetup | @CONFLUENTINC Current state SQL SQL SQL application application application database LOG

Slide 5

Slide 5 text

@riferrei | #kafkameetup | @CONFLUENTINC Why?

Slide 6

Slide 6 text

@riferrei | #kafkameetup | @CONFLUENTINC I don’t know “my app only understands sql” “the source only understands sql” “I can only process atomic data” “that is what current books says” “it just feels right doing like this”

Slide 7

Slide 7 text

@riferrei | #kafkameetup | @CONFLUENTINC But what if… database 1000x more volume Non-transactional events Transactional events LOG

Slide 8

Slide 8 text

what? Are you kidding me?

Slide 9

Slide 9 text

@riferrei | #kafkameetup | @CONFLUENTINC This is fine

Slide 10

Slide 10 text

@riferrei | #kafkameetup | @CONFLUENTINC Solving the problem

Slide 11

Slide 11 text

@riferrei | #kafkameetup | @CONFLUENTINC Using the log database LOG 0 1 2 3 4 5 6 7 8 LOG reads New records application a (time = 1) application b (time = 3)

Slide 12

Slide 12 text

@riferrei | #kafkameetup | @CONFLUENTINC But log programming hurts public void readTransactions() { while (true) { ConsumerRecords records = consumer.poll(Duration.ofMillis(100)); for (Record record : records) { if (record.offset() != lastSavedOffset) {...} } } }

Slide 13

Slide 13 text

@riferrei | #kafkameetup | @CONFLUENTINC What then?

Slide 14

Slide 14 text

@riferrei | #kafkameetup | @CONFLUENTINC Event streaming appS

Slide 15

Slide 15 text

@riferrei | @confluentinc | @itau

Slide 16

Slide 16 text

@riferrei | @confluentinc | @itau

Slide 17

Slide 17 text

@riferrei | #kafkameetup | @CONFLUENTINC PAC-MAN

Slide 18

Slide 18 text

@riferrei | #kafkameetup | @CONFLUENTINC https://github.com/confluentinc/demo-scene <> Getting started

Slide 19

Slide 19 text

@riferrei | #kafkameetup | @CONFLUENTINC IMPLEMENTATION Api gateway Lambda function Kafka topic Ksqldb apps Kafka topic scoreboard

Slide 20

Slide 20 text

@riferrei | #kafkameetup | @CONFLUENTINC IMPLEMENTATION Kafka topic Ksqldb apps Kafka topic

Slide 21

Slide 21 text

@riferrei | #kafkameetup | @CONFLUENTINC Let’s create the streams

Slide 22

Slide 22 text

@riferrei | #kafkameetup | @CONFLUENTINC instructions 1. Get the game 2. Name yourself

Slide 23

Slide 23 text

@riferrei | #kafkameetup | @CONFLUENTINC BUILDING the scoreboard

Slide 24

Slide 24 text

@riferrei | #kafkameetup | @CONFLUENTINC Introduction to ksqldb https://ksqldb.io

Slide 25

Slide 25 text

@riferrei | #kafkameetup | @CONFLUENTINC Source: USER_GAME TOPIC

Slide 26

Slide 26 text

@riferrei | #kafkameetup | @CONFLUENTINC Creating User_game stream

Slide 27

Slide 27 text

@riferrei | #kafkameetup | @CONFLUENTINC Querying USER_GAME STREAM

Slide 28

Slide 28 text

@riferrei | #kafkameetup | @CONFLUENTINC Creating Stats_per_user table

Slide 29

Slide 29 text

@riferrei | #kafkameetup | @CONFLUENTINC Querying STATS_PER_USER TABLE

Slide 30

Slide 30 text

@riferrei | #kafkameetup | @CONFLUENTINC Low latency Pull queries

Slide 31

Slide 31 text

@riferrei | #kafkameetup | @CONFLUENTINC Source: User_losses topic

Slide 32

Slide 32 text

@riferrei | #kafkameetup | @CONFLUENTINC Creating USER_LOSSES STREAM

Slide 33

Slide 33 text

@riferrei | #kafkameetup | @CONFLUENTINC querying USER_LOSSES STREAM

Slide 34

Slide 34 text

@riferrei | #kafkameetup | @CONFLUENTINC Creating LOSSES_PER_USER TABLE

Slide 35

Slide 35 text

@riferrei | #kafkameetup | @CONFLUENTINC Querying LOSSES_PER_USER TABLE

Slide 36

Slide 36 text

@riferrei | #kafkameetup | @CONFLUENTINC Creating SCOREBOARD TABLE

Slide 37

Slide 37 text

@riferrei | #kafkameetup | @CONFLUENTINC Querying SCOREBOARD TABLE

Slide 38

Slide 38 text

@riferrei | #kafkameetup | @CONFLUENTINC Playing with The data

Slide 39

Slide 39 text

@riferrei | #kafkameetup | @CONFLUENTINC Read from kafka Api gateway Lambda function scoreboard Redis cache push

Slide 40

Slide 40 text

@riferrei | #kafkameetup | @CONFLUENTINC Read from kafka Amazon alexa Lambda function scoreboard Redis cache push

Slide 41

Slide 41 text

@riferrei | #kafkameetup | @CONFLUENTINC Read from ksqldb Your code Ksqldb table pull Kafka topic

Slide 42

Slide 42 text

@riferrei | #kafkameetup | @CONFLUENTINC Creating highest_score TABLE

Slide 43

Slide 43 text

@riferrei | #kafkameetup | @CONFLUENTINC Low latency Pull queries 69920

Slide 44

Slide 44 text

@riferrei | #kafkameetup | @CONFLUENTINC how can I learn more?

Slide 45

Slide 45 text

@riferrei | #kafkameetup | @CONFLUENTINC Get kafka: confluent cloud Try free: https://cnfl.io/confluent-cloud

Slide 46

Slide 46 text

@riferrei | #kafkameetup | @CONFLUENTINC https://cnfl.io/tutorials Get examples: kafka tutorials

Slide 47

Slide 47 text

@riferrei | #kafkameetup | @CONFLUENTINC https://cnfl.io/books Get books: o’reilly bundle

Slide 48

Slide 48 text

@riferrei | #kafkameetup | @CONFLUENTINC https://kafka-summit.org/events/kafka-summit-austin-2020 join kafka summit https://myeventi.events/kafka20/aus Use 25% discount code: KSa20Meetup

Slide 49

Slide 49 text

@riferrei | #kafkameetup | @CONFLUENTINC Thank you