ELK how we use it

Transcript

1. Logging with ELK How we use it a secured environment or mabye a strange environment
2. Some basic infos • PHP developer • working at: Little Bird GmbH • Needed to handle our log (exception) much better
3. What is ELK Elasticsearch • search engine (Lucene based) • indices data • provides REST-API • saves data as NoSQL (JSON)
4. What is ELK Logstash • Log data processor / ETL • 165+ plugins for inputs (files, TCP/UDP-Ports, STDIN, etc.) • filter, normalise and transform data (Date-Format, GeoIP, RegExp, Anonymize, etc.) • send data to STDOUT, databases, files, chats, nagios etc.
5. What is ELK Kibana • nodeJS based GUI for elasticsearch • data visualisation (charts) • data analyses
6. Current situation and why to change • Several hosting partner with different systems • Writing separate log files for each customer (Symptoms and Apache) • Exception are catched and send via email to a shared mail account • Not every developer has access to mail account • Bugs are not always reported with stackstrace/infos • Hard to track if exception occurred on other system or many times
7. Our environment - our problems • Many hosting providers • Different OS: ◦ Debian ◦ Ubuntu ◦ SuSe • Different access rights ◦ Full access (internal system) ◦ Only one port: 443 ◦ Non-HTTP(S)/SMTP access only on request • No central server • One separate log file per customer (environment)
8. How we solved it • Central logging server on our in-house system • Tunneling the desired connection through port 443 • Application server only needs ◦ Stunnel - a tunneling software ◦ Java - we need it for the application, too ◦ Logstash as standalone downloaded application • Customer (environment) name is extracted from file name
9. The infrastructure with logging server
10. That’s how it looks like
11. That’s how it looks like
12. What to do next • Central log server fail over (more than one Redis, Elasticsearch server) • Automatical analysation of log and notify developers • Only process neccessary log messages
13. End