Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Tips & Tricks about Apache Kafka in the Cloud for Java Developers

Tips & Tricks about Apache Kafka in the Cloud for Java Developers

Ricardo Ferreira

May 27, 2019
Tweet

More Decks by Ricardo Ferreira

Other Decks in Programming

Transcript

  1. Ricardo Ferreira
    Developer Advocate

    View Slide

  2. View Slide

  3. Can We Though?
    "Dude, you are embarrassing
    me in front of the wizards…"
    SSH? Tunneling? VPC?
    Encryption? HPC? HA?
    RPO? RTO? Sizing?
    Existing Java Code?
    @riferrei #jbcn19

    View Slide

  4. About Us:
    ● Ricardo Ferreira
    ○ Developer Advocate @ Confluent
    ○ Ex-Oracle, Red Hat, IONA Tech
    ○ Currently ~70% Dev, ~30 Ops
    ○ https://riferrei.net
    ● Echo Dot (Alexa)
    ○ The voice behind Amazon
    ○ Ex-Raspberry Pi, Terminator?
    @riferrei
    @alexa99

    View Slide

  5. View Slide

  6. View Slide

  7. Apache
    Kafka
    Clusters are
    Not Easy to
    Manage!

    View Slide

  8. Use Serverless Managed
    Services to Offload your Projects!
    @riferrei #jbcn19

    View Slide

  9. Demo: Creating a
    cluster for a Java
    Application
    @riferrei #jbcn19

    View Slide

  10. View Slide

  11. View Slide

  12. @riferrei #jbcn19
    Applications
    VPC
    consumer_01
    conusmer_02
    consumer_03
    topic_name_01
    topic_name_02
    topic_name_03
    Schema Registry
    Apache Kafka
    Cluster VPC
    consumer_04

    View Slide

  13. Watch Out for These Guys…
    @riferrei #jbcn19

    View Slide

  14. @riferrei #jbcn19
    On
    Premises
    topic_name_01
    topic_name_02
    topic_name_03
    Confluent
    Replicator
    Schema Registry
    topic_name_01
    topic_name_02
    topic_name_03
    Schema Registry
    Public
    Cloud

    View Slide

  15. Demo: Showing a
    Peered VPC in GCP
    @riferrei #jbcn19

    View Slide

  16. View Slide

  17. View Slide

  18. Observability =
    Metrics + Logs +
    Distributed Tracing
    @riferrei #jbcn19

    View Slide

  19. Why Distributed Tracing is Important?
    System IS
    NOT Working!
    System IS
    Working!
    Time Spent with Troubleshooting: Days, Weeks or even Months!
    @riferrei #jbcn19

    View Slide

  20. https://github.com/opentracing-contrib/java-kafka-client
    Apache Kafka Client + Spring
    Kafka + Kafka Streams
    https://github.com/riferrei/jaeger-tracing-support
    Kafka Connect, REST Proxy,
    Schema Registry, KSQL
    @riferrei #jbcn19

    View Slide

  21. View Slide

  22. @riferrei #jbcn19

    View Slide

  23. Scan the
    barcode
    to create
    events
    @riferrei #jbcn19

    View Slide

  24. https://github.com/riferrei/the-song-is
    Demo Source-Code
    @riferrei #jbcn19

    View Slide

  25. View Slide

  26. View Slide

  27. https://github.com/confluentinc/ccloud-tools
    Terraform Scripts for Kafka
    Connect, REST Proxy, KSQL
    https://github.com/confluentinc/cp-ansible
    Ansible Playbooks are also
    Available as Well!
    @riferrei #jbcn19

    View Slide

  28. Free Books
    by Confluent

    View Slide

  29. [email protected] @riferrei #jbcn19
    http://cnfl.io/slack

    View Slide

  30. View Slide