Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Tips & Tricks about Apache Kafka in the Cloud for Java Developers

Tips & Tricks about Apache Kafka in the Cloud for Java Developers


Ricardo Ferreira

May 27, 2019


  1. Ricardo Ferreira Developer Advocate

  2. None
  3. Can We Though? "Dude, you are embarrassing me in front

    of the wizards…" SSH? Tunneling? VPC? Encryption? HPC? HA? RPO? RTO? Sizing? Existing Java Code? @riferrei #jbcn19
  4. About Us: • Ricardo Ferreira ◦ Developer Advocate @ Confluent

    ◦ Ex-Oracle, Red Hat, IONA Tech ◦ Currently ~70% Dev, ~30 Ops ◦ https://riferrei.net • Echo Dot (Alexa) ◦ The voice behind Amazon ◦ Ex-Raspberry Pi, Terminator? @riferrei @alexa99
  5. None
  6. None
  7. Apache Kafka Clusters are Not Easy to Manage!

  8. Use Serverless Managed Services to Offload your Projects! @riferrei #jbcn19

  9. Demo: Creating a cluster for a Java Application @riferrei #jbcn19

  10. None
  11. None
  12. @riferrei #jbcn19 Applications VPC consumer_01 conusmer_02 consumer_03 topic_name_01 topic_name_02 topic_name_03

    Schema Registry Apache Kafka Cluster VPC consumer_04
  13. Watch Out for These Guys… @riferrei #jbcn19

  14. @riferrei #jbcn19 On Premises topic_name_01 topic_name_02 topic_name_03 Confluent Replicator Schema

    Registry topic_name_01 topic_name_02 topic_name_03 Schema Registry Public Cloud
  15. Demo: Showing a Peered VPC in GCP @riferrei #jbcn19

  16. None
  17. None
  18. Observability = Metrics + Logs + Distributed Tracing @riferrei #jbcn19

  19. Why Distributed Tracing is Important? System IS NOT Working! System

    IS Working! Time Spent with Troubleshooting: Days, Weeks or even Months! @riferrei #jbcn19
  20. https://github.com/opentracing-contrib/java-kafka-client Apache Kafka Client + Spring Kafka + Kafka Streams

    https://github.com/riferrei/jaeger-tracing-support Kafka Connect, REST Proxy, Schema Registry, KSQL @riferrei #jbcn19
  21. None
  22. @riferrei #jbcn19

  23. Scan the barcode to create events @riferrei #jbcn19

  24. https://github.com/riferrei/the-song-is Demo Source-Code @riferrei #jbcn19

  25. None
  26. None
  27. https://github.com/confluentinc/ccloud-tools Terraform Scripts for Kafka Connect, REST Proxy, KSQL https://github.com/confluentinc/cp-ansible

    Ansible Playbooks are also Available as Well! @riferrei #jbcn19
  28. Free Books by Confluent

  29. ricardo@confluent.io @riferrei #jbcn19 http://cnfl.io/slack

  30. None