Upgrade to Pro — share decks privately, control downloads, hide ads and more …

What's new in Zipkin

What's new in Zipkin

Zipkin has been around long enough for some to not have looked since our team got together as OpenZipkin 4 years ago. We were invited to a Naver Pinpoint open house, and since the last time we saw each other was 4 years ago also, seemed a good time to recap.

Adrian Cole

July 11, 2019
Tweet

More Decks by Adrian Cole

Other Decks in Technology

Transcript

  1. What’s new in Zipkin?
    @adrianfcole
    works at Pivotal
    works on Zipkin
    wandering around our last 4 years

    View Slide

  2. @adrianfcole
    • spring cloud at pivotal
    • focused on distributed tracing
    • helped open zipkin

    View Slide

  3. Wait what is Zipkin?
    @adrianfcole
    #zipkin

    View Slide

  4. Zipkin is a distributed tracing system
    Zipkin instrumentation track requests using identifiers propagated
    through and across processes. Metadata such as latency, error and
    network context are sent later to a Zipkin compatible backend.
    The Zipkin UI allows users to lookup traces by ID or query
    parameters like error or duration. It also displays service
    dependency diagrams summarising traced traffic.

    View Slide

  5. requisite screen shots
    http://zipkin.io/

    View Slide

  6. requisite chart going up and right
    https://www.timqian.com/star-history/#openzipkin/zipkin

    View Slide

  7. What changed in the UI?
    @adrianfcole
    #zipkin

    View Slide

  8. Zipkin Lens is our new UI
    Universal search conditions pass between
    trace and dependency search (like haystack)
    Mini-map with stable zoom combines trace
    minimap (like Jaeger) with stable zoom (like
    zipkin classic)
    Site-specific tags auto-populate based on
    attributes like product code (like haystack)
    New inside & out react vizceral material-ui

    View Slide

  9. Shinier Grafana
    This is used for the
    health of the collector
    side of zipkin.

    View Slide

  10. What changed in the data?
    @adrianfcole
    #zipkin
    zipkin v1 -> v2

    View Slide

  11. Much smaller
    $ wc -c 1e1a733a8e3d0f10-v1.json 1e1a733a8e3d0f10-v2.json
    5433 1e1a733a8e3d0f10-v1.json
    3377 1e1a733a8e3d0f10-v2.json
    We also support proto3 format now

    View Slide

  12. No more need to translate annotations
    [
    {
    "traceId": "1e1a733a8e3d0f10",
    "id": "1e1a733a8e3d0f10",
    "name": "get /",
    "timestamp": 1544062973621073,
    "duration": 6926,
    "annotations": [
    {
    "timestamp": 1544062973621073,
    "value": "sr",
    "endpoint": {
    "serviceName": "frontend",
    "ipv4": "10.129.171.98"
    }
    },
    {
    "timestamp": 1544062973627999,
    "value": "ss",
    "endpoint": {
    [
    {
    "traceId": "1e1a733a8e3d0f10",
    "id": "1e1a733a8e3d0f10",
    "kind": "SERVER",
    "name": "get /",
    "timestamp": 1544062973621073,
    "duration": 6926,
    "localEndpoint": {
    "serviceName": "frontend",
    "ipv4": "10.129.171.98"
    },
    "remoteEndpoint": {
    "ipv6": "::1",
    "port": 57317
    },
    "tags": {
    "http.method": "GET",
    "http.path": "/",
    "mvc.controller.class": "Frontend",

    View Slide

  13. What changed in the headers?
    @adrianfcole
    #zipkin
    b3 -> b3 single

    View Slide

  14. Much smaller
    $ wc -c /tmp/b3-multi /tmp/b3-single
    129 /tmp/b3-multi
    73 /tmp/b3-single

    View Slide

  15. Easier to integrate
    X-B3-TraceId: 80f198ee56343ba864fe8b2a57d3eff7
    X-B3-ParentSpanId: 05e3ac9a4f6e3b90
    X-B3-SpanId: e457b5a2e4d86bd1
    X-B3-Sampled: 1
    b3: 80f198ee56343ba864fe8b2a57d3eff7-e457b5a2e4d86bd1-1-05e3ac9a4f6e3b90
    The single format is optional, but elegantly allows
    integration in naming-constrained transports like JMS.

    View Slide

  16. What’s new in the server
    You can send data over ActiveMQ, gRPC and RabbitMQ
    You can proxy data or send it directly to cloud services
    X-Ray and StackDriver with community options for others
    Storage throttling helps when your storage is out of breath
    Armeria Netty based backend used for http, gRPC and scribe

    View Slide

  17. What’s new in instrumentation
    Http route for tags or span names. Ex /messages/:id
    Messaging relationships like producer and consumer are traceable
    Extra fields can often be propagated, like your correlation-id
    V2 style libraries Brave, zipkin-go and zipkin-php look the same

    View Slide

  18. What changed in javascript tracing?
    Tracer now supports tracing local operations
    Instrumentation adds connect, grpc, postgres, request-
    promise, superagent
    v2 api will happen soon, compatible with existing libraries
    zipkin-js

    View Slide

  19. What changed in java tracing?
    Tracer has simpler api for local tracing and “span finish” hooks
    Instrumentation adds dubbo, jersey, jms, kafka, kafka-
    streams, mysql8, netty, rabbit, vert.x
    spring-boot tracing (sleuth) uses Brave natively now
    brave

    View Slide

  20. What changed in ruby tracing?
    Http route (ex /api/v1/messages/:id) is used for span names
    Instrumentation adds Sidekiq (job processing)
    zipkin-ruby

    View Slide

  21. C# tracer
    Tracer is very similar to Finagle (scala library)
    Instrumentation includes Owin and ASP.NET Core
    Contributed by Criteo
    zipkin4net

    View Slide

  22. golang tracer
    Tracer is “v2 native” and very similar to Brave (java library)
    Instrumentation includes http and grpc, with others in different
    repositories: redis, sql, sqlx
    Go-kit includes this library
    zipkin-go

    View Slide

  23. PHP tracer
    Tracer is “v2 native” and very similar to Brave (java library)
    Instrumentation are in different repositories and include
    symfony, doctrine, guzzle
    OpenTracing bridge is optional
    zipkin-php

    View Slide

  24. What’s next?
    @adrianfcole
    #zipkin

    View Slide

  25. Zipkin Lens replaces classic UI

    View Slide

  26. Messaging abstraction
    It is error prone to write instrumentation for similar libraries without common
    models or tests.
    While producer consumer tools are not as similar as http, it is possible to
    abstract some similar details, for example between Kafka and AMQP.
    Difficulties lie in instrumentation itself and also separation of consumption and
    processing of messages.
    https://github.com/openzipkin/openzipkin.github.io/wiki/Messaging-instrumentation-abstraction

    https://github.com/openzipkin/brave/pull/914

    View Slide

  27. More widespread feature work
    Some features like advanced propagation controls and local root
    are not available yet in all libraries.
    These allow features like decaying sampling overlays and firehose
    handling that can let sites do more analysis cheaper.
    Ex. Currently only py_zipkin and Brave have firehose mode

    View Slide

  28. Late processing
    We’ve work in progress for span and trace-scoped functions to
    work on notions of complete spans. This can be used for
    conditional sampling or reshaping of traces.
    • Kafka storage alternative
    • Forwarding storage adapter
    • In-memory triage with VoltDB
    https://github.com/adriancole/zipkin-voltdb

    https://github.com/jeqo/zipkin-storage-kafka

    https://github.com/jeqo/zipkin-forwarder

    View Slide

  29. Truly cloud storage
    While we support forwards data to other tracing services, these
    do not provide read back. While we support cloud provisioned
    databases, these are still databases to manage.
    Our first truly cloud storage option is in the works for Amazon
    DynamoDB
    https://github.com/openzipkin/zipkin-aws/pull/125

    View Slide

  30. UX UX UX
    When Lens becomes feature parity, we can eliminate the debt of
    maintaining two UI codebases and move to more exciting feature
    work. Ideas include
    • Grafana plug-in to embed Zipkin api data
    • Filtering within a single trace
    • Trace scoped aggregation and comparisons

    View Slide

  31. Wrapping up
    Lots of stuff going on, but don’t get overwhelmed.
    Chat any time on Gitter
    @adrianfcole
    #zipkin
    gitter.im/openzipkin/zipkin
    github.com/openzipkin/zipkin

    View Slide