Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Kafka Meetup from March 2020

Kafka Meetup from March 2020

Meetup: https://www.meetup.com/Raleigh-Apache-Kafka-Meetup-by-Confluent/events/269215507

This meetup was originally posted on Raleigh's meetup user groups, but it is also available for: Charlotte, Richmond, Montreal, Boston, Atlanta, Toronto, Ottawa, Quebec, Waterloo, Philadelphia, Pittsburgh, Columbus, Cleveland, Cincinnati, Detroit, Western New York, Columbia, Charleston, Jacksonville, Orlando, Miami, Tampa, Louisville, Indianapolis, Nashville, Richmond, Ft. Wayne, Stamford, Hartford, Providence, Norfolk, and Washington, DC.

Ricardo Ferreira

March 31, 2020

More Decks by Ricardo Ferreira

Other Decks in Programming


  1. The talk will start shortly - before then, have a

    chat with your fellow attendees! Questions at the end This is unless the speaker says otherwise. But generally, because of internet delays and lags it can be a bit disruptive during an Online talk. Online Meetup Etiquette Mute during talk But remember to unmute when you want to ask a question at the end (or chat with fellow attendees before/After) Use chat to engage Use Zoom chat and/or create a thread in the #events channel on our Community Slack space so that your discussion can continue afterwards with fellow Kafkateers. Be on Camera & React! Speakers during these events are talking to blank screens. And it can be exhausting! Use Zoom reactions at the bottom of the page to give them something to work off of! 1. 2. Continue learning and collaborating WELCOME! Thank you for joining us in these unique circumstances. We hope you’re safe and well. 3. If you haven’t already joined, get on the our slack workspace and continue the conversation with other Kafkateers. If you’re already a member, start a thread for this event in #events. A single online source of everything you’ll need to learn Kafka. Plus it’s totally free and ungated… Confluent Developer developer.confluent.io https://cnfl.io/slack Confluent Community Slack You are being recorded and this footage may be added to public channels
  2. About me @riferrei | #kafkameetup | @CONFLUENTINC • RICARDO FERREIRA

    • Works for confluent • Developer advocate • [email protected] • HTTPS://RIFERREI.NET
  3. @riferrei | #kafkameetup | @CONFLUENTINC Current state SQL SQL SQL

    application application application database LOG
  4. @riferrei | #kafkameetup | @CONFLUENTINC I don’t know “my app

    only understands sql” “the source only understands sql” “I can only process atomic data” “that is what current books says” “it just feels right doing like this”
  5. @riferrei | #kafkameetup | @CONFLUENTINC But what if… database 1000x

    more volume Non-transactional events Transactional events LOG
  6. @riferrei | #kafkameetup | @CONFLUENTINC Using the log database LOG

    0 1 2 3 4 5 6 7 8 LOG reads New records application a (time = 1) application b (time = 3)
  7. @riferrei | #kafkameetup | @CONFLUENTINC But log programming hurts public

    void readTransactions() { while (true) { ConsumerRecords<String, Type> records = consumer.poll(Duration.ofMillis(100)); for (Record<String, Type> record : records) { if (record.offset() != lastSavedOffset) {...} } } }
  8. @riferrei | #kafkameetup | @CONFLUENTINC Read from kafka Api gateway

    Lambda function scoreboard Redis cache push
  9. @riferrei | #kafkameetup | @CONFLUENTINC Read from kafka Amazon alexa

    Lambda function scoreboard Redis cache push