Apache Kafka is one the most popular open-source projects for building messaging and streaming applications. Kafka takes data, makes it available to different applications, and therefore helps to eliminate daily batch jobs.
Kafka plays an important role for Change Data Capture (CDC) and in the world of microservices. This presentation gives an overview of the new Amazon Managed Streaming for Kafka (Amazon MSK).
Based on knowledge gained from several on-prem Kafka implementation projects I will cover the technical underpinnings first. You will learn about brokers, topics, and Zookeeper. Then I will explain what makes Kafka special, analyse major pain points in on-prem Kafka projects, critically analyse how Kafka differs from Kinesis, and why the cloud is the best way to use Kafka