Zipkin (#SFScala meetup)

The slide deck used to present Zipkin to the #SFScala meet up hosted at @TwitterHQ on 07-29-2014

Transcript

1. Twitter Inc. Zipkin Jeff Smick @sprsquish jeff@twitter.com
2. Twitter Inc. Zipkin A distributed tracing system based on Google’s Dapper paper. ! Detailed structured logging of individual requests Full view of a request’s path through SOA Built into Finagle Add your own annotations
3. Twitter Inc. Metrics Counters / Histograms / Gauges Aggregated together Show overall service behavior Can show a long latency tail, but not why it’s so long
4. Twitter Inc. Zipkin: Per-request Visibility Latency of a single request Structured log of request-specific information begin/end (used to calculate latency) service specific info (arguments, SQL, cache miss, etc) Can show the request’s path through the architecture Are downstream reqs serial when they should be parallel?
5. Twitter Inc. How it works Four key events: Server Receiver/Send; Client Send/Receive An event has seven attributes: name: event name (sr/ss; cs/cr; or anything else) traceId: identifies the request spanId: identifies a piece of the request parentId: identifies the piece of the request that initiated this piece flags: a bit field, currently: debug, trace request timestamp: instant the event occurred according to the service creating it endpoint: a name and host/port pair identifying the service that created the event
6. Twitter Inc. How it works: Visualized sr, 1, 1a, n/a cs, 1, 1b, 1a ! ! ! cr, 1, 1b, 1a ss, 1, 1a, n/a ! sr, 1, 1b, 1a ! ! ! ss, 1, 1b, 1a
7. Twitter Inc. How it works: Visualized sr, 1, 1a, n/a cs, 1, 1b, 1a ! ! ! cr, 1, 1b, 1a ss, 1, 1a, n/a ! sr, 1, 1b, 1a cs, 1, 1c, 1b cs, 1, 1d, 1b cr, 1, 1c, 1b cr, 1, 1d, 1b ss, 1, 1b, 1a ! ! sr, 1, 1c, 1b ss, 1, 1c, 1b sr, 1, 1d, 1b ss, 1, 1d, 1b
8. Twitter Inc. Demo
9. Twitter Inc. Zipkin All-in-One sub-project: zipkin-example git clone https://github.com/twitter/zipkin.git cd zipkin bin/sbt zipkin-example/run open http://localhost:8080 ! to see all options: bin/sbt “zipkin-example/run -help”
10. Twitter Inc. Zipkin All-in-One - scribe collector - SQLite storage - Web UI - can be started with randomized example data ! - Start here. Split and expand as needed.
11. Twitter Inc. Custom Zipkin Receivers: Scribe, Kafka Storage: SQL, Cassandra, HBase, Redis ! Cake pattern via TwitterServer Config flags Admin interface
12. Twitter Inc. Example object Main extends TwitterServer with Closer with ZooKeeperClientFactory with ScribeSpanReceiverFactory with ZipkinWebFactory with AnormDBSpanStoreFactory with ZipkinSpanGenerator { val genSampleTraces = flag("genSampleTraces", false, "Generate sample traces") ! def main() { val store = newAnormSpanStore() if (genSampleTraces()) Await.result(generateTraces(store)) ! val convert: Seq[thrift.Span] => Seq[Span] = { _.map(_.toSpan) } val receiver = newScribeSpanReceiver(convert andThen store, statsReceiver.scope("scribeSpanReceiver")) val query = new ThriftQueryService(store, adjusters = DefaultAdjusters) val web = Http.serve(webServerPort(), newWebServer(query, statsReceiver.scope("web"))) ! closeOnExit(Closable.sequence(web, receiver, store)) Await.all(web, receiver, store) } }
13. Twitter Inc. Zipkin’s Architecture Service Service Service Service Zipkin Collectors Scribe ! Storage Zipkin Query Zipkin Web UI
14. Twitter Inc. Collector Only object ZipkinCollectorServer extends TwitterServer with Closer with ZipkinQueuedCollectorFactory with CassandraSpanStoreFactory with ZooKeeperClientFactory with ScribeSpanReceiverFactory { def newReceiver(receive: Seq[ThriftSpan] => Future[Unit], stats: StatsReceiver): SpanReceiver = newScribeSpanReceiver(receive, stats.scope("scribeSpanReceiver")) ! def newSpanStore(stats: StatsReceiver): WriteSpanStore = newCassandraStore(stats.scope("cassandra")) ! def main() { val collector = newCollector(statsReceiver) closeOnExit(collector) Await.ready(collector) } }
15. Twitter Inc. Query Server object ZipkinQueryServer extends TwitterServer with Closer with CassandraSpanStoreFactory with ZipkinQueryServerFactory { def main() { val spanStore = newCassandraStore(statsReceiver.scope("cassandra")) val query = newQueryServer(spanStore) closeOnExit(query) Await.ready(query) } }
16. Twitter Inc. Web UI com.twitter.zipkin.web.Main ! Set the dest of the query server via flags Set cacheResources to use static files from the jar
17. Twitter Inc. Getting Data Into Zipkin Serialized thrift structs via Scribe or Kafka (or write your own) ! trait WriteSpanStore extends (Seq[Span] => Future[Unit]) Receive Data via the collector Convert to Seq[Span] Send to the SpanStore
18. Twitter Inc. Zipkin and Finagle Simple! Depend on finagle-zipkin. Core events are handled automatically Use: Trace.record for user defined annotations Configuration flags are provided when using TwitterServer
19. Twitter Inc. Zipkin and Others A few contributions: https://github.com/search?q=zipkin Akka, Clojure, Rails, Node, Twisted, PHP ! Coming Soon(ish): Docs to describe how to provide trace info
20. Twitter Inc. Future Work Publish as a library Documentation Realtime UI
21. Twitter Inc. Get In Touch - Help Out ! https://github.com/twitter/zipkin @zipkinproject zipkin-user@googlegroups.com zipkin-dev@googlegroups.com ! Help! jeff@twitter.com