Tips & Tricks about Apache Kafka in the Cloud for Java Developers

Transcript

1. Ricardo Ferreira
   Developer Advocate
   

   View Slide

2. View Slide

3. Can We Though?
   "Dude, you are embarrassing
   me in front of the wizards…"
   SSH? Tunneling? VPC?
   Encryption? HPC? HA?
   RPO? RTO? Sizing?
   Existing Java Code?
   @riferrei #jbcn19
   

   View Slide

4. About Us:
   ● Ricardo Ferreira
   ○ Developer Advocate @ Confluent
   ○ Ex-Oracle, Red Hat, IONA Tech
   ○ Currently ~70% Dev, ~30 Ops
   ○ https://riferrei.net
   ● Echo Dot (Alexa)
   ○ The voice behind Amazon
   ○ Ex-Raspberry Pi, Terminator?
   @riferrei
   @alexa99
   

   View Slide

5. View Slide

6. View Slide

7. Apache
   Kafka
   Clusters are
   Not Easy to
   Manage!
   

   View Slide

8. Use Serverless Managed
   Services to Offload your Projects!
   @riferrei #jbcn19
   

   View Slide

9. Demo: Creating a
   cluster for a Java
   Application
   @riferrei #jbcn19
   

   View Slide

10. View Slide

11. View Slide

12. @riferrei #jbcn19
    Applications
    VPC
    consumer_01
    conusmer_02
    consumer_03
    topic_name_01
    topic_name_02
    topic_name_03
    Schema Registry
    Apache Kafka
    Cluster VPC
    consumer_04
    

    View Slide

13. Watch Out for These Guys…
    @riferrei #jbcn19
    

    View Slide

14. @riferrei #jbcn19
    On
    Premises
    topic_name_01
    topic_name_02
    topic_name_03
    Confluent
    Replicator
    Schema Registry
    topic_name_01
    topic_name_02
    topic_name_03
    Schema Registry
    Public
    Cloud
    

    View Slide

15. Demo: Showing a
    Peered VPC in GCP
    @riferrei #jbcn19
    

    View Slide

16. View Slide

17. View Slide

18. Observability =
    Metrics + Logs +
    Distributed Tracing
    @riferrei #jbcn19
    

    View Slide

19. Why Distributed Tracing is Important?
    System IS
    NOT Working!
    System IS
    Working!
    Time Spent with Troubleshooting: Days, Weeks or even Months!
    @riferrei #jbcn19
    

    View Slide

20. https://github.com/opentracing-contrib/java-kafka-client
    Apache Kafka Client + Spring
    Kafka + Kafka Streams
    https://github.com/riferrei/jaeger-tracing-support
    Kafka Connect, REST Proxy,
    Schema Registry, KSQL
    @riferrei #jbcn19
    

    View Slide

21. View Slide

22. @riferrei #jbcn19
    

    View Slide

23. Scan the
    barcode
    to create
    events
    @riferrei #jbcn19
    

    View Slide

24. https://github.com/riferrei/the-song-is
    Demo Source-Code
    @riferrei #jbcn19
    

    View Slide

25. View Slide

26. View Slide

27. https://github.com/confluentinc/ccloud-tools
    Terraform Scripts for Kafka
    Connect, REST Proxy, KSQL
    https://github.com/confluentinc/cp-ansible
    Ansible Playbooks are also
    Available as Well!
    @riferrei #jbcn19
    

    View Slide

28. Free Books
    by Confluent
    

    View Slide

29. ricardo@confluent.io @riferrei #jbcn19
    http://cnfl.io/slack
    

    View Slide

30. View Slide