1 Tips & Tricks about Apache Kafka in the Cloud for Java Developers Ricardo Ferreira, Developer Advocate @riferrei

2

3 About Us: ● Ricardo Ferreira ○ Developer Advocate @ Confluent ○ Ex-Oracle, Red Hat, IONA Tech ○ Currently ~10% Smart, ~90 Dumb ○ https://riferrei.net ● Echo Dot (Alexa) ○ The voice behind Amazon ○ Ex-Raspberry Pi, Arduino, T3? ○ Currently ~90% Smart, ~10 Dumb @riferrei @alexa99

4 #1 Managed Kafka Services ● Kafka is a fine proven technology ● People don’t know/care internals ● Human labor is often too costly ● Install-patch-monitor-scale repeat ● Build applications, not clusters! https://www.confluent.io/confluent-cloud @riferrei @jeeconf

5 Let's Create a Cluster 1. Log in to the Confluent Cloud. 2. Create a new cluster on-the-fly. 3. Connect your Java applications. https://confluent.cloud @riferrei @jeeconf

6 http://kafka.apache.org/documentation.ht ml#connect https://docs.confluent.io/current/connect /quickstart.html https://debezium.io #2 Apache Kafka Connect ● Moving data in-and-out is hard ● +75 connectors available now ● Integrated with Schema Registry ● CDC via Red Hat's Debezium ● Connectivity code is you, let the Serialization, Partitioning, Fault- Tolerance, Schema Mgmt on us. @riferrei @jeeconf

7 Pushing Data to Redis Cache using Apache Kafka Connect @riferrei @jeeconf

8 Connecting the Layers 1. Please scan the barcode below. 2. Listen the song and try to guess. 3. Enter with your name and send. @riferrei @jeeconf

9 #3 Kafka Streams and KSQL ● Create stream processing apps ● Data computation outside Kafka ● Same partitioning/scaling model ● Complex pipelines using filters, aggregations, windows & more. ● Flavors: Java, Scala and KSQL https://kafka.apache.org/documentation/ streams https://www.confluent.io/product/ksql @riferrei @jeeconf

10 @riferrei @jeeconf Processing the Data In-Motion and Creating a New Data Set

11 #4 Distributed Tracing ● Monitor distributed transactions ● Glue your Java code with Kafka ● Trace producers and consumers ● Decorators and Interceptors API ● Trace Kafka services such as REST Proxy, Connect and KSQL https://github.com/opentracing-contrib/java-kafka- client https://github.com/riferrei/jaeger-tracing-support @riferrei @jeeconf

12 https://github.com/opentracing-contrib/java-kafka- client https://github.com/riferrei/jaeger-tracing-support @riferrei @jeeconf Tomorrow: 11:05 – 11:50 Implementing Distributed Tracing 'Like a Boss' in your Apache Kafka Deployments

13 #5 Infrastructure-as-Code ● Create immutable infrastructure ● Handle dependencies of the code ● Build your Java code on-the-fly ● Connect with Cloud automatically ● Blend Kafka services with infra https://github.com/confluentinc/cp-docker-images https://github.com/confluentinc/cp-helm-charts https://github.com/confluentinc/ccloud-tools https://github.com/confluentinc/cp-ansible @riferrei @jeeconf

14 14 Free Books by Confluent: https://www.confluent.io/apache-kafka-stream-processing-book-bundle

15 15 Let’s Keep in Touch: @riferrei riferrei riferrei [email protected] https://riferrei.net https://cnfl.io/slack Confluent is Hiring: https://www.confluent.io/careers

16