What's new in Zipkin

Zipkin has been around long enough for some to not have looked since our team got together as OpenZipkin 4 years ago. We were invited to a Naver Pinpoint open house, and since the last time we saw each other was 4 years ago also, seemed a good time to recap.

Transcript

1. What’s new in Zipkin?
   @adrianfcole
   works at Pivotal
   works on Zipkin
   wandering around our last 4 years
   

   View Slide

2. @adrianfcole
   • spring cloud at pivotal
   • focused on distributed tracing
   • helped open zipkin
   

   View Slide

3. Wait what is Zipkin?
   @adrianfcole
   #zipkin
   

   View Slide

4. Zipkin is a distributed tracing system
   Zipkin instrumentation track requests using identifiers propagated
   through and across processes. Metadata such as latency, error and
   network context are sent later to a Zipkin compatible backend.
   The Zipkin UI allows users to lookup traces by ID or query
   parameters like error or duration. It also displays service
   dependency diagrams summarising traced traffic.
   

   View Slide

5. requisite screen shots
   http://zipkin.io/
   

   View Slide

6. requisite chart going up and right
   https://www.timqian.com/star-history/#openzipkin/zipkin
   

   View Slide

7. What changed in the UI?
   @adrianfcole
   #zipkin
   

   View Slide

8. Zipkin Lens is our new UI
   Universal search conditions pass between
   trace and dependency search (like haystack)
   Mini-map with stable zoom combines trace
   minimap (like Jaeger) with stable zoom (like
   zipkin classic)
   Site-specific tags auto-populate based on
   attributes like product code (like haystack)
   New inside & out react vizceral material-ui
   

   View Slide

9. Shinier Grafana
   This is used for the
   health of the collector
   side of zipkin.
   

   View Slide

10. What changed in the data?
    @adrianfcole
    #zipkin
    zipkin v1 -> v2
    

    View Slide

11. Much smaller
    $ wc -c 1e1a733a8e3d0f10-v1.json 1e1a733a8e3d0f10-v2.json
    5433 1e1a733a8e3d0f10-v1.json
    3377 1e1a733a8e3d0f10-v2.json
    We also support proto3 format now
    

    View Slide

12. No more need to translate annotations
    [
    {
    "traceId": "1e1a733a8e3d0f10",
    "id": "1e1a733a8e3d0f10",
    "name": "get /",
    "timestamp": 1544062973621073,
    "duration": 6926,
    "annotations": [
    {
    "timestamp": 1544062973621073,
    "value": "sr",
    "endpoint": {
    "serviceName": "frontend",
    "ipv4": "10.129.171.98"
    }
    },
    {
    "timestamp": 1544062973627999,
    "value": "ss",
    "endpoint": {
    [
    {
    "traceId": "1e1a733a8e3d0f10",
    "id": "1e1a733a8e3d0f10",
    "kind": "SERVER",
    "name": "get /",
    "timestamp": 1544062973621073,
    "duration": 6926,
    "localEndpoint": {
    "serviceName": "frontend",
    "ipv4": "10.129.171.98"
    },
    "remoteEndpoint": {
    "ipv6": "::1",
    "port": 57317
    },
    "tags": {
    "http.method": "GET",
    "http.path": "/",
    "mvc.controller.class": "Frontend",
    

    View Slide

13. What changed in the headers?
    @adrianfcole
    #zipkin
    b3 -> b3 single
    

    View Slide

14. Much smaller
    $ wc -c /tmp/b3-multi /tmp/b3-single
    129 /tmp/b3-multi
    73 /tmp/b3-single
    

    View Slide

15. Easier to integrate
    X-B3-TraceId: 80f198ee56343ba864fe8b2a57d3eff7
    X-B3-ParentSpanId: 05e3ac9a4f6e3b90
    X-B3-SpanId: e457b5a2e4d86bd1
    X-B3-Sampled: 1
    b3: 80f198ee56343ba864fe8b2a57d3eff7-e457b5a2e4d86bd1-1-05e3ac9a4f6e3b90
    The single format is optional, but elegantly allows
    integration in naming-constrained transports like JMS.
    

    View Slide

16. What’s new in the server
    You can send data over ActiveMQ, gRPC and RabbitMQ
    You can proxy data or send it directly to cloud services
    X-Ray and StackDriver with community options for others
    Storage throttling helps when your storage is out of breath
    Armeria Netty based backend used for http, gRPC and scribe
    

    View Slide

17. What’s new in instrumentation
    Http route for tags or span names. Ex /messages/:id
    Messaging relationships like producer and consumer are traceable
    Extra fields can often be propagated, like your correlation-id
    V2 style libraries Brave, zipkin-go and zipkin-php look the same
    

    View Slide

18. What changed in javascript tracing?
    Tracer now supports tracing local operations
    Instrumentation adds connect, grpc, postgres, request-
    promise, superagent
    v2 api will happen soon, compatible with existing libraries
    zipkin-js
    

    View Slide

19. What changed in java tracing?
    Tracer has simpler api for local tracing and “span finish” hooks
    Instrumentation adds dubbo, jersey, jms, kafka, kafka-
    streams, mysql8, netty, rabbit, vert.x
    spring-boot tracing (sleuth) uses Brave natively now
    brave
    

    View Slide

20. What changed in ruby tracing?
    Http route (ex /api/v1/messages/:id) is used for span names
    Instrumentation adds Sidekiq (job processing)
    zipkin-ruby
    

    View Slide

21. C# tracer
    Tracer is very similar to Finagle (scala library)
    Instrumentation includes Owin and ASP.NET Core
    Contributed by Criteo
    zipkin4net
    

    View Slide

22. golang tracer
    Tracer is “v2 native” and very similar to Brave (java library)
    Instrumentation includes http and grpc, with others in different
    repositories: redis, sql, sqlx
    Go-kit includes this library
    zipkin-go
    

    View Slide

23. PHP tracer
    Tracer is “v2 native” and very similar to Brave (java library)
    Instrumentation are in different repositories and include
    symfony, doctrine, guzzle
    OpenTracing bridge is optional
    zipkin-php
    

    View Slide

24. What’s next?
    @adrianfcole
    #zipkin
    

    View Slide

25. Zipkin Lens replaces classic UI
    

    View Slide

26. Messaging abstraction
    It is error prone to write instrumentation for similar libraries without common
    models or tests.
    While producer consumer tools are not as similar as http, it is possible to
    abstract some similar details, for example between Kafka and AMQP.
    Difficulties lie in instrumentation itself and also separation of consumption and
    processing of messages.
    https://github.com/openzipkin/openzipkin.github.io/wiki/Messaging-instrumentation-abstraction
    
    https://github.com/openzipkin/brave/pull/914
    

    View Slide

27. More widespread feature work
    Some features like advanced propagation controls and local root
    are not available yet in all libraries.
    These allow features like decaying sampling overlays and firehose
    handling that can let sites do more analysis cheaper.
    Ex. Currently only py_zipkin and Brave have firehose mode
    

    View Slide

28. Late processing
    We’ve work in progress for span and trace-scoped functions to
    work on notions of complete spans. This can be used for
    conditional sampling or reshaping of traces.
    • Kafka storage alternative
    • Forwarding storage adapter
    • In-memory triage with VoltDB
    https://github.com/adriancole/zipkin-voltdb
    
    https://github.com/jeqo/zipkin-storage-kafka
    
    https://github.com/jeqo/zipkin-forwarder
    

    View Slide

29. Truly cloud storage
    While we support forwards data to other tracing services, these
    do not provide read back. While we support cloud provisioned
    databases, these are still databases to manage.
    Our first truly cloud storage option is in the works for Amazon
    DynamoDB
    https://github.com/openzipkin/zipkin-aws/pull/125
    

    View Slide

30. UX UX UX
    When Lens becomes feature parity, we can eliminate the debt of
    maintaining two UI codebases and move to more exciting feature
    work. Ideas include
    • Grafana plug-in to embed Zipkin api data
    • Filtering within a single trace
    • Trace scoped aggregation and comparisons
    

    View Slide

31. Wrapping up
    Lots of stuff going on, but don’t get overwhelmed.
    Chat any time on Gitter
    @adrianfcole
    #zipkin
    gitter.im/openzipkin/zipkin
    github.com/openzipkin/zipkin
    

    View Slide