$30 off During Our Annual Pro Sale. View Details »

Implementing Distributed Tracing 'Like a Boss' in your Apache Kafka Deployments

Implementing Distributed Tracing 'Like a Boss' in your Apache Kafka Deployments

Ricardo Ferreira

April 27, 2019

More Decks by Ricardo Ferreira

Other Decks in Programming


  1. Implementing Distributed Tracing 'Like a Boss' in your Apache Kafka

    Ricardo Ferreira, Developer Advocate @riferrei
  2. About Us: • Ricardo Ferreira ◦ Developer Advocate @ Confluent

    ◦ Ex-Oracle, Red Hat, IONA Tech ◦ Currently ~10% Smart, ~90 Dumb ◦ https://riferrei.net • Echo Dot (Alexa) ◦ The voice behind Amazon ◦ Ex-Raspberry Pi, Arduino, T3? ◦ Currently ~90% Smart, ~10 Dumb @riferrei @alexa99
  3. A long time ago in a galaxy far, far away…

  4. How We Did IT Monitoring • Each system would have

    their very own monitoring system. • Developers should not worry about monitoring. This was supposed to be "IT stuff". • Architecture was deemed OK if all systems were showing green. • Monitoring systems were not standard. Each system could potentially use a different vendor. @riferrei @jeeconf
  5. OK… What was the Problem?

  6. System IS NOT Working! System IS working! Time spent with

    troubleshooting. Days, weeks, months! @riferrei @jeeconf
  7. System IS NOT Working! System IS working! Time spent with

    troubleshooting. Days, weeks, months! Lesson about Patience: That was only One Incident! @riferrei @jeeconf
  8. Solution: Observability!

  9. None
  10. Three Pillars of Observability • Logs: stream of events related

    to transactions of one application. • Metrics: Statistics and numeric measures aggregated by time. • Distributed Tracing: complete view of what happened within a single transaction. It answers: ◦ Which services were involved? ◦ If it was slow, who caused that? ◦ If failed, who actually failed? @riferrei @jeeconf
  11. Why Distributed Tracing Became So Relevant?

  12. Evolution of Concurrency • No Concurrency at all ◦ Ex:

    Apache HTTP Server • Basic Concurrency ◦ Ex: Multi-threaded Applications • Async Concurrency ◦ Ex: Actor-based Programming • Distributed Concurrency ◦ Ex: μServices Architecture Style @riferrei @jeeconf
  13. Distributed Tracing Today • There are many distributed tracing technologies

    available. • Standards are getting created to ensure a single programming model for each μService. • Deployment mechanisms such as Kubernetes are also taking care of that automatically. • Network proxies such as Service Meshes are also handling this. @riferrei @jeeconf
  14. Why Apache Kafka? • Apache Kafka is becoming the de-

    facto standard to handle data. • Prediction? It will be the central nervous system of any company. • μServices in general already use Kafka to exchange messages and keep their data stores in-sync. • With event streaming becoming even more popular, the Kafka adoption tend to grow even more. @riferrei @jeeconf
  15. Implementing Distributed Tracing in Apache Kafka

  16. OpenTracing for the Rescue • Monitor distributed transactions • Glue

    your Java code with Kafka • Trace producers and consumers • Decorators and Interceptors API • Pluggable tracers via classpath via the tracer resolver library. • Dense documentation available. https://github.com/opentracing-contrib/java-kafka- client @riferrei @jeeconf
  17. Extension for Bundled JVMs • Custom interceptors to handle distributed

    tracing in JVM-based applications that you don't have the source-code. • Examples: REST Proxy, Kafka Connect and KSQL Servers. • Allows you to group μServices based on the topics they use. https://github.com/riferrei/jaeger-tracing-support @riferrei @jeeconf
  18. DEMO

  19. 19 Free Books by Confluent: https://www.confluent.io/apache-kafka-stream-processing-book-bundle

  20. 20 Let’s Keep in Touch: @riferrei riferrei riferrei ricardo@confluent.io https://riferrei.net

    https://cnfl.io/slack Confluent is Hiring: https://www.confluent.io/careers
  21. None