Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Tips & Tricks about Apache Kafka in the Cloud for Java Developers

Tips & Tricks about Apache Kafka in the Cloud for Java Developers

02ff2dde723b6e26f4ef03ee6b3f6eb9?s=128

Ricardo Ferreira

March 20, 2019
Tweet

Transcript

  1. 1 Tips & Tricks for Apache Kafka in the Cloud

    for Java Developers Ricardo Ferreira, Developer Advocate @riferrei
  2. 2 About me: • Hi, my name is Ricardo Ferreira

    • Developer Advocate @ Confluent • Ex-Oracle, Red Hat, IONA Tech • Currently ~70% Dev, ~30% Ops • https://riferrei.net @riferrei
  3. 3 #1 Managed Kafka Services • Kafka is a fine

    proven technology • People don’t know/care internals • Human labor is often too costly • Install-patch-monitor-scale repeat • Build applications, not clusters! @riferrei https://www.confluent.io/confluent-cloud
  4. 4 Let's create a cluster 1. Log in to the

    Confluent Cloud UI 2. Create a new cluster on-the-fly 3. Swap the credit card. That is it! @riferrei https://confluent.cloud
  5. 5 #2 Kafka Streams and KSQL • Create stream processing

    apps • Data computation outside Kafka • Same partitioning/scaling model • Complex pipelines using filters, aggregations, windows & more. • Flavors: Java, Scala and KSQL @riferrei https://kafka.apache.org/documentation/ streams https://www.confluent.io/product/ksql
  6. 6 Getting Number Three 1. Open the URL with your

    phone 2. Enter your name or nickname 3. Try to perfectly position the dice on number three: -180 | 0 | 180 https://bit.ly/2zCjHG8 @riferrei
  7. 7 #3 Kafka Connect API • Moving data in-and-out is

    hard • +75 connectors currently available • Integrated with Schema Registry • CDC via Red Hat's Debezium • Domain code is you, let the rest on us: Serialization, Partitioning, Fault-Tolerance, Schema Mgmt. @riferrei http://kafka.apache.org/documentation.ht ml#connect https://docs.confluent.io/current/connect /quickstart.html https://debezium.io
  8. 8 The Song Is... 1. Access your Twitter account 2.

    Tweet something as shown in the example below. Make sure you use @kcjug or #KansasCityJUG https://twitter.com @riferrei
  9. 9 #4 Distributed Tracing • Monitor distributed transactions • Glue

    your Java code with Kafka • Trace producers and consumers • Decorators and Interceptors API • Trace Kafka services such as REST Proxy, Connect and KSQL @riferrei https://github.com/opentracing-contrib/java- kafka-client https://github.com/riferrei/jaeger-tracing- support
  10. 10 #5 Infrastructure-as-Code • Create immutable infrastructure • Detect and

    handle all changes • Build your Java code on-the-fly • Connect with Cloud automatically • Blend Kafka services with infra @riferrei https://github.com/confluentinc/ccloud- tools https://docs.confluent.io/current/cloud/to ols.html https://www.terraform.io
  11. 11 11 Free Books by Confluent: https://www.confluent.io/apache-kafka-stream-processing-book-bundle

  12. 12 Please Stay in Touch: @riferrei riferrei riferrei ricardo@confluent.io https://riferrei.net

    https://cnfl.io/slack
  13. 13