Upgrade to Pro — share decks privately, control downloads, hide ads and more …

What Can I Do With the ELK Stack?

Tyler L
November 03, 2015

What Can I Do With the ELK Stack?

Tyler L

November 03, 2015

More Decks by Tyler L

Other Decks in Technology


  1. What Can I Do With the ELK Stack? DevOpsDays Charlotte

    2015 Tyler Langlois Infrastructure Engineer, Elastic
  2. This relates to DevOps… how? • Collect disparate data from

    across the organization • Expose it through an accessible interface • Let users create their own value from existing data • Get answers immediately instead of synchronously from dev/ops/admins Self-service unstructured data
  3. Vital Stats • All three are open source • JRuby

    / Java / Javascript / Go (beats) = contribute! • Most recent release (last week) brings LS+ES to 2.0, Kibana to 4.2 • Designed to scale • Very active open source community
  4. Implementation tl;dr • Data source logstash input { } {packet,file}beat

    POST :9200 • Document Store Elasticsearch on- premise, Found, Docker, etc. SaaS Options • Visualization Kibana runs in-browser Access controls/basic auth supported Really, anything that can throw JSON at a REST endpoint Scaled appropriately (1 - ??? nodes) Most settings are stored in an Elasticsearch index; simple deployment
  5. Metrics/Numerical Data • Emerging use case • Strong support in

    2.x series • Pipeline aggs ◦ moving averages, percentiles, derivatives
  6. All of this translates to... • Centralized logs & metrics

    • Self-service for: ◦ “Are we serving more 5xx errors than normal?” ◦ “What are response times like?” ◦ “Where is the influx of traffic coming from?” ◦ “How many $project build failures in the last 3 months?” • API for building alerts, dashboards, and tools across data sources … + easy scaling
  7. Files Filebeat • Tails files • Ships them elsewhere •

    Lightweight and suited for low- resource environments Metrics Topbeat • Think `top` metrics- ized • Cross-platform metric collection ...more? libbeat • Simple golang library • Create your own • Leverage library to ship along channels to Elasticsearch
  8. Use Your Imagination That’s the method I implemented to bring

    a sentiment analyzer into Logstash Create whatever your use case requires!
  9. Thank you! github.com/tylerjl irc/twitter: leothrix tjll.net Additional Information: • elastic.co

    • Discourse forums • IRC: #elasticsearch, #logstash, #kibana on freenode • github.com/elastic • Corner me anytime this conference with questions