Distributed tracing for big systems

As our systems grow, so does our software architecture complexity. As we scale and add more and more components, the interactions occurring between them become very complex and we start to lose visibility into the system. Traditional monitoring tools such as metrics and distributed logging still have their place, but they often fail to provide visibility across services. This is where distributed tracing thrives. During the talk we’ll explore what distributed tracing is, what open tools we can use to facilitate it and some lessons learned while implementing distributed tracing at Uber and how it helps us build big and impactful systems.

Transcript

2. Distributed tracing for big systems Nikolay Stoitsev
5. Reliability
6. Observability
7. Monitoring
8. Distributed logging
9. Business metrics
11. Distributed tracing?
12. opentracing.io
13. opentracing.io
14. opentracing.io
15. Distributed context propagation
16. https://research.google.com/pubs/pub36356.html
17. Low overhead
18. Application-level transparency
19. Scalability
20. Store context in thread-local storage
21. Async callbacks
22. RPC instrumentation
24. http://zipkin.io/pages/architecture.html
25. opentracing.io
26. opentracing.io
27. opentracing.io Trace
28. opentracing.io Span
29. ChildOf opentracing.io
30. Defines the data model
31. Defines the OpenTracing API
32. Cool but how to start?
33. Step 0 TChannel & Hyperbahn https://github.com/uber/tchannel
34. https://youtu.be/CwPwP3UaqrY
36. Finagle https://twitter.github.io/finagle
37. Step 1
39. https://github.com/jaegertracing
41. Sampling Strategy
42. Sample everything
43. Probabilistic approach
44. Rate limiting approach
45. Client libraries
47. Missing key-value logging API
48. Missing “child of” and “follows from”
51. Go Java Node.js Python
52. Jersey Dropwizard Apache http clients
56. Demo https://github.com/jaegertracing/jaeger/tree/master/examples/hotrod
57. https://eng.uber.com/distributed-tracing/
58. git clone git@github.com:jaegertracing/jaeger.git
59. Thanks!