Kafka Meetup from March 2020

Meetup: https://www.meetup.com/Raleigh-Apache-Kafka-Meetup-by-Confluent/events/269215507

This meetup was originally posted on Raleigh's meetup user groups, but it is also available for: Charlotte, Richmond, Montreal, Boston, Atlanta, Toronto, Ottawa, Quebec, Waterloo, Philadelphia, Pittsburgh, Columbus, Cleveland, Cincinnati, Detroit, Western New York, Columbia, Charleston, Jacksonville, Orlando, Miami, Tampa, Louisville, Indianapolis, Nashville, Richmond, Ft. Wayne, Stamford, Hartford, Providence, Norfolk, and Washington, DC.

Transcript

1. The talk will start shortly - before then, have a chat with your fellow attendees! Questions at the end This is unless the speaker says otherwise. But generally, because of internet delays and lags it can be a bit disruptive during an Online talk. Online Meetup Etiquette Mute during talk But remember to unmute when you want to ask a question at the end (or chat with fellow attendees before/After) Use chat to engage Use Zoom chat and/or create a thread in the #events channel on our Community Slack space so that your discussion can continue afterwards with fellow Kafkateers. Be on Camera & React! Speakers during these events are talking to blank screens. And it can be exhausting! Use Zoom reactions at the bottom of the page to give them something to work off of! 1. 2. Continue learning and collaborating WELCOME! Thank you for joining us in these unique circumstances. We hope you’re safe and well. 3. If you haven’t already joined, get on the our slack workspace and continue the conversation with other Kafkateers. If you’re already a member, start a thread for this event in #events. A single online source of everything you’ll need to learn Kafka. Plus it’s totally free and ungated… Confluent Developer developer.confluent.io https://cnfl.io/slack Confluent Community Slack You are being recorded and this footage may be added to public channels
2. About me @riferrei | #kafkameetup | @CONFLUENTINC • RICARDO FERREIRA • Works for confluent • Developer advocate • Ricardo@confluent.iO • HTTPS://RIFERREI.NET
3. Building event streaming apps with pac-man @riferrei | #kafkameetup | @CONFLUENTINC
4. @riferrei | #kafkameetup | @CONFLUENTINC Current state SQL SQL SQL application application application database LOG
5. @riferrei | #kafkameetup | @CONFLUENTINC Why?
6. @riferrei | #kafkameetup | @CONFLUENTINC I don’t know “my app only understands sql” “the source only understands sql” “I can only process atomic data” “that is what current books says” “it just feels right doing like this”
7. @riferrei | #kafkameetup | @CONFLUENTINC But what if… database 1000x more volume Non-transactional events Transactional events LOG
8. what? Are you kidding me?
9. @riferrei | #kafkameetup | @CONFLUENTINC This is fine
10. @riferrei | #kafkameetup | @CONFLUENTINC Solving the problem
11. @riferrei | #kafkameetup | @CONFLUENTINC Using the log database LOG 0 1 2 3 4 5 6 7 8 LOG reads New records application a (time = 1) application b (time = 3)
12. @riferrei | #kafkameetup | @CONFLUENTINC But log programming hurts public void readTransactions() { while (true) { ConsumerRecords<String, Type> records = consumer.poll(Duration.ofMillis(100)); for (Record<String, Type> record : records) { if (record.offset() != lastSavedOffset) {...} } } }
13. @riferrei | #kafkameetup | @CONFLUENTINC What then?
14. @riferrei | #kafkameetup | @CONFLUENTINC Event streaming appS
15. @riferrei | @confluentinc | @itau
16. @riferrei | @confluentinc | @itau
17. @riferrei | #kafkameetup | @CONFLUENTINC PAC-MAN
18. @riferrei | #kafkameetup | @CONFLUENTINC https://github.com/confluentinc/demo-scene <<Pacman-ccloud>> Getting started
19. @riferrei | #kafkameetup | @CONFLUENTINC IMPLEMENTATION Api gateway Lambda function Kafka topic Ksqldb apps Kafka topic scoreboard
20. @riferrei | #kafkameetup | @CONFLUENTINC IMPLEMENTATION Kafka topic Ksqldb apps Kafka topic
21. @riferrei | #kafkameetup | @CONFLUENTINC Let’s create the streams
22. @riferrei | #kafkameetup | @CONFLUENTINC instructions 1. Get the game 2. Name yourself
23. @riferrei | #kafkameetup | @CONFLUENTINC BUILDING the scoreboard
24. @riferrei | #kafkameetup | @CONFLUENTINC Introduction to ksqldb https://ksqldb.io
25. @riferrei | #kafkameetup | @CONFLUENTINC Source: USER_GAME TOPIC
26. @riferrei | #kafkameetup | @CONFLUENTINC Creating User_game stream
27. @riferrei | #kafkameetup | @CONFLUENTINC Querying USER_GAME STREAM
28. @riferrei | #kafkameetup | @CONFLUENTINC Creating Stats_per_user table
29. @riferrei | #kafkameetup | @CONFLUENTINC Querying STATS_PER_USER TABLE
30. @riferrei | #kafkameetup | @CONFLUENTINC Low latency Pull queries
31. @riferrei | #kafkameetup | @CONFLUENTINC Source: User_losses topic
32. @riferrei | #kafkameetup | @CONFLUENTINC Creating USER_LOSSES STREAM
33. @riferrei | #kafkameetup | @CONFLUENTINC querying USER_LOSSES STREAM
34. @riferrei | #kafkameetup | @CONFLUENTINC Creating LOSSES_PER_USER TABLE
35. @riferrei | #kafkameetup | @CONFLUENTINC Querying LOSSES_PER_USER TABLE
36. @riferrei | #kafkameetup | @CONFLUENTINC Creating SCOREBOARD TABLE
37. @riferrei | #kafkameetup | @CONFLUENTINC Querying SCOREBOARD TABLE
38. @riferrei | #kafkameetup | @CONFLUENTINC Playing with The data
39. @riferrei | #kafkameetup | @CONFLUENTINC Read from kafka Api gateway Lambda function scoreboard Redis cache push
40. @riferrei | #kafkameetup | @CONFLUENTINC Read from kafka Amazon alexa Lambda function scoreboard Redis cache push
41. @riferrei | #kafkameetup | @CONFLUENTINC Read from ksqldb Your code Ksqldb table pull Kafka topic
42. @riferrei | #kafkameetup | @CONFLUENTINC Creating highest_score TABLE
43. @riferrei | #kafkameetup | @CONFLUENTINC Low latency Pull queries 69920
44. @riferrei | #kafkameetup | @CONFLUENTINC how can I learn more?
45. @riferrei | #kafkameetup | @CONFLUENTINC Get kafka: confluent cloud Try free: https://cnfl.io/confluent-cloud
46. @riferrei | #kafkameetup | @CONFLUENTINC https://cnfl.io/tutorials Get examples: kafka tutorials
47. @riferrei | #kafkameetup | @CONFLUENTINC https://cnfl.io/books Get books: o’reilly bundle
48. @riferrei | #kafkameetup | @CONFLUENTINC https://kafka-summit.org/events/kafka-summit-austin-2020 join kafka summit https://myeventi.events/kafka20/aus Use 25% discount code: KSa20Meetup
49. @riferrei | #kafkameetup | @CONFLUENTINC Thank you