Open Source Logging with Logstash and Kibana

You can have your own logging collection & search system now with Logstash, and nice user interface with Kibana.

I gave this talk at JavaOne 2012 for the ScriptBowl session - a panel/competition between JRuby, Clojure, Scala and Groovy - as a demonstration of the awesomeness of Ruby/JRuby.

Transcript

1. Dr Nic Williams VP Developer Evangelism @drnic drnicwilliams.com
2. Open Source Logging?
3. Logstash
4. Kibana
5. From Servers to Search
6. Logstash Kibana
7. Logs & Events sys
8. Store & Search
9. Locate & Display
10. Why use Ruby? • Author’s preferred language at the time • Originally in PHP; rewritten in Ruby • Sinatra framework • Closer integration with Logstash
11. Why use JRuby? • Actual threads, better CPU usage • Debugging - jvisualvm, jstack • Single executable - including ElasticSearch • Some better libraries - Joda Time
12. Performance “Many users report 20k events/sec on a single m1.xlarge”
13. Data per day • 20k events per second (say 500b each) • 10Kb per second • 864Mb per day • Splunk is $2000/year for 500Mb per day, plus hosting costs
14. 10-min walk thru • Go to logstash.net (DOT NET, OH NO!) • Click “docs” • Click “10-minute walkthrough”
15. Live better • Use great tools • Live better with your apps • Find many more great tools with JRuby/Ruby