[EN] Zipkin + Elasticsearch

Elasticsearch meetup - Milan
Distributed tracing system for a microservice architecture

Stefano Massera and Mattia Piccinetti

Transcript

1. Mattia Piccinetti e Stefano Massera Senior Software Engineers 25-10-2016
2. Zipkin + Elasticsearch Distributed tracing system for a microservice architecture
3. 01 BZ – Before Zipkin
4. What we had: •  Graphite and Graphite only as metrics collector •  Increasing number of microservices •  Increasing dependencies between services •  Per-service type metrics stored in graphite •  Not easily scalable metrics storage •  Unable to highlight microservice dependencies and timings Main Title 01BZ – Before Zipkin
5. What we would like to have: •  Single and standard collection system •  An easy way to trace microservices interactions •  A UI to render all microservices involved in a single search request •  An easy way to analyze interaction in order to •  Lifecycle of a search request •  Find bottlenecks •  Full execution time overview per request Main Title 01BZ – Before Zipkin
6. 02 ZIPKIN
7. What is zipkin A distributed tracing system. It helps gather timing data needed to troubleshoot latency problems in microservice architectures. It manages both the collection and lookup of this data. Zipkin’s design is based on the Google Dapper. (http://zipkin.io/) Main Title 02 AZ – ZIPKIN Why zipkin •  Native support for elasticsearch •  Seamless integration in our Spring Boot microservices with Spring Cloud Sleuth •  Inspired by a lot of experiences shared by other adopters during craft conf 2016
8. Zipkin Dashboard Main Title 02 AZ – ZIPKIN Span Trace
9. 03 ELASTICSEARCH
10. Why Elasticsearch Main Title 01 Elasticsearch •  Knowledge & skills •  Painless infrastructure configuration (Thanks DevOps team J ) •  Tech support •  All good “Elastic” features ( REST api, distributed, fault tolerant, scalable ) •  Custom dashboards with Kibana
11. 04 AZE - AFTER ZIPKIN ELASTIC
12. Main Title 04 AZ – ZIPKIN What we have now: •  Zipkin as a service in a mesos marathon cluster •  Dedicated elasticsearch cluster for all tracing related document •  A lot of tools to find and profile our services •  Graphite as complementary tool
13. Main Title 04 AZ – ZIPKIN What we have now: elas,csearch cluster
14. Main Title 04 AZ – ZIPKIN Future<?> •  More Kibana dashboards: •  Custom Aggregations •  Correlation between services error log and zipkin tracing •  Add a “Watcher” to monitor performance trends
15. Beer? Or Q&A