Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Introduction to the ELK stack

John Hunt
January 17, 2017

Introduction to the ELK stack

I show you around the ELK stack and discuss why using centralised logging is not only essential in multi-server environments but why it opens up loads of cool possibilities with the ability to analyse data with Kibana.

First talk @ phpcambridge 17th Jan 2017

John Hunt

January 17, 2017
Tweet

Other Decks in Programming

Transcript

  1. About me: John Hunt • PHP developer since 2001 •

    Devops guy on and off since 1990-something (I’m getting old..) • Senior developer at BoilerJuice in St. Ives • Co-organiser of PHP Cambridge • Into road bikes and cycling too!
  2. What’s Elasticsearch? • It’s a search engine • It indexes

    data • It’s incredibly fast • It’s a bit (a lot?!) like Apache’s Solr • It uses a JSON REST API for input/output • Does all that horizontal scaling malarky • Runs on Java
  3. What’s Logstash? • Data parsing/processing tool • Common logfiles ->

    elastic • Vast library of input / output filters • Another product from Elastic
  4. What’s Kibana? • Search dashboard • Runs on node.js •

    Connects to elasticsearch • Is amazing
  5. Beats? • Sends data off to logstash and elasticsearch •

    Runs as client software on your servers • Sends log files • Sends system stas (cpu/mem/io…) • Sends network stats • Sends windows event log stuff
  6. Tips • Have a go with docker compose first! •

    Have an agreed structure for fields • Log everything (for cross-referencing) • 4GB minimum if all on one server! • Be careful logging input traffic (passwords!) • Set up securely!! No security out the box.