Implementing Distributed Tracing 'Like a Boss' in your Apache Kafka Deployments

Transcript

1. Implementing Distributed Tracing 'Like a Boss' in your Apache Kafka Ricardo Ferreira, Developer Advocate @riferrei
2. About Us: • Ricardo Ferreira ◦ Developer Advocate @ Confluent ◦ Ex-Oracle, Red Hat, IONA Tech ◦ Currently ~10% Smart, ~90 Dumb ◦ https://riferrei.net • Echo Dot (Alexa) ◦ The voice behind Amazon ◦ Ex-Raspberry Pi, Arduino, T3? ◦ Currently ~90% Smart, ~10 Dumb @riferrei @alexa99
3. A long time ago in a galaxy far, far away…
4. How We Did IT Monitoring • Each system would have their very own monitoring system. • Developers should not worry about monitoring. This was supposed to be "IT stuff". • Architecture was deemed OK if all systems were showing green. • Monitoring systems were not standard. Each system could potentially use a different vendor. @riferrei @jeeconf
5. OK… What was the Problem?
6. System IS NOT Working! System IS working! Time spent with troubleshooting. Days, weeks, months! @riferrei @jeeconf
7. System IS NOT Working! System IS working! Time spent with troubleshooting. Days, weeks, months! Lesson about Patience: That was only One Incident! @riferrei @jeeconf
8. Solution: Observability!
10. Three Pillars of Observability • Logs: stream of events related to transactions of one application. • Metrics: Statistics and numeric measures aggregated by time. • Distributed Tracing: complete view of what happened within a single transaction. It answers: ◦ Which services were involved? ◦ If it was slow, who caused that? ◦ If failed, who actually failed? @riferrei @jeeconf
11. Why Distributed Tracing Became So Relevant?
12. Evolution of Concurrency • No Concurrency at all ◦ Ex: Apache HTTP Server • Basic Concurrency ◦ Ex: Multi-threaded Applications • Async Concurrency ◦ Ex: Actor-based Programming • Distributed Concurrency ◦ Ex: μServices Architecture Style @riferrei @jeeconf
13. Distributed Tracing Today • There are many distributed tracing technologies available. • Standards are getting created to ensure a single programming model for each μService. • Deployment mechanisms such as Kubernetes are also taking care of that automatically. • Network proxies such as Service Meshes are also handling this. @riferrei @jeeconf
14. Why Apache Kafka? • Apache Kafka is becoming the de- facto standard to handle data. • Prediction? It will be the central nervous system of any company. • μServices in general already use Kafka to exchange messages and keep their data stores in-sync. • With event streaming becoming even more popular, the Kafka adoption tend to grow even more. @riferrei @jeeconf
15. Implementing Distributed Tracing in Apache Kafka
16. OpenTracing for the Rescue • Monitor distributed transactions • Glue your Java code with Kafka • Trace producers and consumers • Decorators and Interceptors API • Pluggable tracers via classpath via the tracer resolver library. • Dense documentation available. https://github.com/opentracing-contrib/java-kafka- client @riferrei @jeeconf
17. Extension for Bundled JVMs • Custom interceptors to handle distributed tracing in JVM-based applications that you don't have the source-code. • Examples: REST Proxy, Kafka Connect and KSQL Servers. • Allows you to group μServices based on the topics they use. https://github.com/riferrei/jaeger-tracing-support @riferrei @jeeconf
18. DEMO
19. 19 Free Books by Confluent: https://www.confluent.io/apache-kafka-stream-processing-book-bundle
20. 20 Let’s Keep in Touch: @riferrei riferrei riferrei ricardo@confluent.io https://riferrei.net https://cnfl.io/slack Confluent is Hiring: https://www.confluent.io/careers