CLI Tools you need to know for your everyday Kafka | Current 2024

In Hugo Guerrero's presentation at Current, he delved into the essential **CLI tools** for managing **Apache Kafka** environments, perfect for users who prefer the terminal over graphical interfaces. He covered everything from setting up clusters to managing **Kafka Connect connectors** and users. The session introduced key scripts from the **Apache Kafka** distribution and useful tools like **kcat** for Kafka, **kcctl** for Kafka Connect, and even resources for **Kubernetes** users like **kubectl** and **strimzi-kafka-cli**. This presentation equips developers with practical commands and tools to boost productivity in Kafka environments.

Transcript

1. You Need To Know For Your Everyday Kafka Command Line Tools Hugo Guerrero | @hguerreroo Sr. Principal Developer Advocate, Red Hat 1
2. Hugo Guerrero Sr. Principal Developer Advocate @Red Hat Likes: Travel + Food + History Mexican @ Massachusetts, USA AsyncAPI Ambassador @hguerreroo Articles: https://developers.redhat.com/author/hugo-guerrero {“Hello”:”World!”}
3. 3 bit.ly/journey4kafka A great day out with Apache Kafka
4. 4 COMMAND LINE TOOLS YOU NEED TO KNOW @hguerreroo Command Line Tools
5. COMMAND LINE TOOLS YOU NEED TO KNOW 5 @hguerrero Why Use Docker CLI? • Simplifies the management of containerized applications and services. • Ideal for developers building, testing, and deploying applications in containers. • Supports complex workflows with volumes, networks, and multi-container setups. • Perfect for CI/CD, DevOps pipelines, and local development environments. Key Features: • Build and run containers: Build images from Dockerfiles and run containers from images. • Manage images: Pull, push, and list Docker images from local and remote repositories. • Container operations: Start, stop, remove, and inspect running containers. • Networking: Create and manage container networks. • Volumes: Persist data by managing Docker volumes across containers. docker Container Management Tool
6. COMMAND LINE TOOLS YOU NEED TO KNOW 6 @hguerrero Why use kcat? • Lightweight and powerful Kafka tool. • Great for developers, testers, and operators. • Simple interface for testing Kafka APIs. • Enables rapid debugging and local development. Key Features: • Produce messages: Send messages to Kafka topics. • Consume messages: Read messages from Kafka topics. • Query metadata: Inspect Kafka topics, brokers, and partitions. • Low-level debug: Helpful for troubleshooting Kafka clusters. kcat (formerly known as kafkacat) The Versatile Kafka CLI Tool
7. COMMAND LINE TOOLS YOU NEED TO KNOW 7 @hguerrero Why use kaf? • Comprehensive CLI for Kafka cluster management. • Combines Kafka topic, broker, and consumer group management in a single tool. • Fast and efficient – no need for multiple tools or interfaces. • Perfect for development, testing, and production environments. kaf Kafka CLI inspired by kubectl & docker Key Features: • Manage topics: Create, list, describe, and delete topics. • Produce and consume messages: Send and read messages to/from Kafka topics. • Monitor consumer groups: Check consumer group lags, offsets, and more. • View broker metadata: Inspect brokers, partitions, and replication details.
8. COMMAND LINE TOOLS YOU NEED TO KNOW 8 @hguerrero Why use kcli? • A simple, easy-to-use Kafka management tool. • Fast and efficient for basic Kafka operations. • Ideal for developers and operators who need quick access to Kafka functionality. • Minimal setup, making it perfect for local development and testing. kcli Kafka Command-Line Interface Key Features: • Manage topics: Create, list, delete, and describe topics. • Inspect partitions: View partition details, including leader and replicas. • Monitor consumer offsets: Track consumer offsets and lags. • Query broker metadata: Get broker information and cluster status.
9. COMMAND LINE TOOLS YOU NEED TO KNOW 9 @hguerrero Why use kafkactl? • A versatile tool for managing Kafka clusters via the REST Proxy. • Simplifies production and consumption of messages. • Ideal for users looking to perform CRUD operations and monitor Kafka resources efficiently. • Perfect for automation, CI/CD, and scripting use cases. kafkactl Kafka Command-Line Management Tool Key Features: • Manage topics: Create, delete, describe, and list Kafka topics. • Produce and consume messages: Send and read messages from Kafka topics. • Monitor consumer groups: List, describe, and check consumer group offsets. • Query Kafka resources: Inspect brokers, partitions, and configurations.
10. COMMAND LINE TOOLS YOU NEED TO KNOW 10 @hguerrero Why use kcctl? • Makes Kafka Connect cluster management simple and efficient. • No need for REST API calls – everything is done via CLI. • Streamlines development and operational workflows. • Ideal for managing multiple connectors and environments. 🧸 kcctl Command-Line Tool for Kafka Connect Key Features: • Register connectors: Easily register new Kafka Connect connectors. • Manage connectors: Pause, resume, or delete connectors with ease. • Monitor status: Check the status of connectors and tasks. • Update configurations: Update connector configurations from the terminal.
11. COMMAND LINE TOOLS YOU NEED TO KNOW 11 @hguerrero Why use kubectl? • The most comprehensive CLI for interacting with Kubernetes clusters. • Allows for full control over Kubernetes resources. • Supports automation and integration with CI/CD pipelines. • Ideal for developers, system administrators, and DevOps engineers managing production or development clusters. kubectl Kubernetes Command-Line Tool Key Features: • Manage resources: Create, update, and delete Kubernetes resources (e.g., pods, deployments, services). • Inspect cluster state: Get detailed information on pods, nodes, and workloads. • Port-forwarding: Forward local ports to services running inside the cluster. • Logs and events: View logs and monitor events for troubleshooting. • Apply configurations: Use declarative configurations with kubectl apply for Kubernetes resources.
12. COMMAND LINE TOOLS YOU NEED TO KNOW 12 @hguerrero Why Use Strimzi CLI? • Designed for managing Kafka on Kubernetes with Strimzi’s custom resource definitions (CRDs). • Simplifies the complexity of operating Kafka clusters in a cloud-native environment. • Ideal for automating Kafka operations in Kubernetes. • Perfect for DevOps teams managing Kafka in Kubernetes for scaling, resilience, and ease of deployment. Strimzi CLI Kafka Management on Kubernetes Key Features: • Manage Kafka clusters: Create, update, and delete Kafka clusters. • Topic operations: Create, list, and delete Kafka topics. • User management: Manage Kafka users and their permissions. • Cluster status: Check the health and status of Kafka clusters. • Configurable context: Easily switch between multiple Kubernetes clusters or namespaces.
13. 13 bit.ly/journey4kafka A great day out with Apache Kafka
14. Thank you! github.com/hguerrero x.com/hguerreroo linkedin.com/in/hugoguerrero youtube.com/hguerreroo