What's new in Zipkin

Zipkin has been around long enough for some to not have looked since our team got together as OpenZipkin 4 years ago. We were invited to a Naver Pinpoint open house, and since the last time we saw each other was 4 years ago also, seemed a good time to recap.

Transcript

1. What’s new in Zipkin? @adrianfcole works at Pivotal works on Zipkin wandering around our last 4 years
2. @adrianfcole • spring cloud at pivotal • focused on distributed tracing • helped open zipkin
3. Wait what is Zipkin? @adrianfcole #zipkin
4. Zipkin is a distributed tracing system Zipkin instrumentation track requests using identifiers propagated through and across processes. Metadata such as latency, error and network context are sent later to a Zipkin compatible backend. The Zipkin UI allows users to lookup traces by ID or query parameters like error or duration. It also displays service dependency diagrams summarising traced traffic.
5. requisite screen shots http://zipkin.io/
6. requisite chart going up and right https://www.timqian.com/star-history/#openzipkin/zipkin
7. What changed in the UI? @adrianfcole #zipkin
8. Zipkin Lens is our new UI Universal search conditions pass between trace and dependency search (like haystack) Mini-map with stable zoom combines trace minimap (like Jaeger) with stable zoom (like zipkin classic) Site-specific tags auto-populate based on attributes like product code (like haystack) New inside & out react vizceral material-ui
9. Shinier Grafana This is used for the health of the collector side of zipkin.
10. What changed in the data? @adrianfcole #zipkin zipkin v1 -> v2
11. Much smaller $ wc -c 1e1a733a8e3d0f10-v1.json 1e1a733a8e3d0f10-v2.json 5433 1e1a733a8e3d0f10-v1.json 3377 1e1a733a8e3d0f10-v2.json We also support proto3 format now
12. No more need to translate annotations [ { "traceId": "1e1a733a8e3d0f10", "id": "1e1a733a8e3d0f10", "name": "get /", "timestamp": 1544062973621073, "duration": 6926, "annotations": [ { "timestamp": 1544062973621073, "value": "sr", "endpoint": { "serviceName": "frontend", "ipv4": "10.129.171.98" } }, { "timestamp": 1544062973627999, "value": "ss", "endpoint": { [ { "traceId": "1e1a733a8e3d0f10", "id": "1e1a733a8e3d0f10", "kind": "SERVER", "name": "get /", "timestamp": 1544062973621073, "duration": 6926, "localEndpoint": { "serviceName": "frontend", "ipv4": "10.129.171.98" }, "remoteEndpoint": { "ipv6": "::1", "port": 57317 }, "tags": { "http.method": "GET", "http.path": "/", "mvc.controller.class": "Frontend",
13. What changed in the headers? @adrianfcole #zipkin b3 -> b3 single
14. Much smaller $ wc -c /tmp/b3-multi /tmp/b3-single 129 /tmp/b3-multi 73 /tmp/b3-single
15. Easier to integrate X-B3-TraceId: 80f198ee56343ba864fe8b2a57d3eff7 X-B3-ParentSpanId: 05e3ac9a4f6e3b90 X-B3-SpanId: e457b5a2e4d86bd1 X-B3-Sampled: 1 b3: 80f198ee56343ba864fe8b2a57d3eff7-e457b5a2e4d86bd1-1-05e3ac9a4f6e3b90 The single format is optional, but elegantly allows integration in naming-constrained transports like JMS.
16. What’s new in the server You can send data over ActiveMQ, gRPC and RabbitMQ You can proxy data or send it directly to cloud services X-Ray and StackDriver with community options for others Storage throttling helps when your storage is out of breath Armeria Netty based backend used for http, gRPC and scribe
17. What’s new in instrumentation Http route for tags or span names. Ex /messages/:id Messaging relationships like producer and consumer are traceable Extra fields can often be propagated, like your correlation-id V2 style libraries Brave, zipkin-go and zipkin-php look the same
18. What changed in javascript tracing? Tracer now supports tracing local operations Instrumentation adds connect, grpc, postgres, request- promise, superagent v2 api will happen soon, compatible with existing libraries zipkin-js
19. What changed in java tracing? Tracer has simpler api for local tracing and “span finish” hooks Instrumentation adds dubbo, jersey, jms, kafka, kafka- streams, mysql8, netty, rabbit, vert.x spring-boot tracing (sleuth) uses Brave natively now brave
20. What changed in ruby tracing? Http route (ex /api/v1/messages/:id) is used for span names Instrumentation adds Sidekiq (job processing) zipkin-ruby
21. C# tracer Tracer is very similar to Finagle (scala library) Instrumentation includes Owin and ASP.NET Core Contributed by Criteo zipkin4net
22. golang tracer Tracer is “v2 native” and very similar to Brave (java library) Instrumentation includes http and grpc, with others in different repositories: redis, sql, sqlx Go-kit includes this library zipkin-go
23. PHP tracer Tracer is “v2 native” and very similar to Brave (java library) Instrumentation are in different repositories and include symfony, doctrine, guzzle OpenTracing bridge is optional zipkin-php
24. What’s next? @adrianfcole #zipkin
25. Zipkin Lens replaces classic UI
26. Messaging abstraction It is error prone to write instrumentation for similar libraries without common models or tests. While producer consumer tools are not as similar as http, it is possible to abstract some similar details, for example between Kafka and AMQP. Difficulties lie in instrumentation itself and also separation of consumption and processing of messages. https://github.com/openzipkin/openzipkin.github.io/wiki/Messaging-instrumentation-abstraction https://github.com/openzipkin/brave/pull/914
27. More widespread feature work Some features like advanced propagation controls and local root are not available yet in all libraries. These allow features like decaying sampling overlays and firehose handling that can let sites do more analysis cheaper. Ex. Currently only py_zipkin and Brave have firehose mode
28. Late processing We’ve work in progress for span and trace-scoped functions to work on notions of complete spans. This can be used for conditional sampling or reshaping of traces. • Kafka storage alternative • Forwarding storage adapter • In-memory triage with VoltDB https://github.com/adriancole/zipkin-voltdb https://github.com/jeqo/zipkin-storage-kafka https://github.com/jeqo/zipkin-forwarder
29. Truly cloud storage While we support forwards data to other tracing services, these do not provide read back. While we support cloud provisioned databases, these are still databases to manage. Our first truly cloud storage option is in the works for Amazon DynamoDB https://github.com/openzipkin/zipkin-aws/pull/125
30. UX UX UX When Lens becomes feature parity, we can eliminate the debt of maintaining two UI codebases and move to more exciting feature work. Ideas include • Grafana plug-in to embed Zipkin api data • Filtering within a single trace • Trace scoped aggregation and comparisons
31. Wrapping up Lots of stuff going on, but don’t get overwhelmed. Chat any time on Gitter @adrianfcole #zipkin gitter.im/openzipkin/zipkin github.com/openzipkin/zipkin