Tool Time with John

Presentation at Atlanta Puppet User's Group

Transcript

1. Tool Time with John! SE Puppet Users Group 02/16/2012
2. Everyone needs a tool belt
3. Why? • To fix things • To make things • To keep your pants up
4. Caveats
5. There are tools
6. There are the right tools for a given job
7. Then there's “butterknifing a phillips head”
8. Fixing Things
9. Logging is Broken(ish) • Commercial options are expensive • Syslog isn't flexible enough • Log lines are just data points with a timestamp (event)
10. Enter Logstash
11. Logstash • Unix pipe on steroids • If it has a timestamp, send it to Logstash • Slice it, Dice it, Munge it, Transform it • Now spit it out • This isn't a Logstash talk.
12. Simple Sample Configuration
13. More complex (inputs)
14. More complex (filters)
15. More complex (outputs)
16. 12 inputs, 11 filters, 21 outputs
17. Building stuff
18. Build something • Sockets the way you think sockets should work • Usable from almost every language • Flow independent • Captures common patterns • Freaking fast
19. Logstash Filters with ZeroMQ • You need to do something unique? • There's not a plugin yet? • Specific to your company? • Don't know Ruby? Write your own! (in whatever language you like)
20. Example – Logstash config
21. Example – External Filter
22. End Result
23. (don't get caught with your) Pants on the Ground
24. Statsd • Open sourced by Etsy last year • Simple UDP proxy to Graphite • Different “data types” • Client and Server implementations in almost every language
25. Graphite • Open sourced by Orbitz • Makes pretty pictures of data • Does dashboards • Simple API • Does realtime updates • Alternate dashboards available and easy to create
26. Logstash Configuration
27. End Result
28. Recap • Always be adding tools to your toolbelt • See a cool project? Bookmark it • Evernote/Read it Later are your friends • Always be hacking. Keep your skills fresh.
29. Links! • Logstash – http://logstash.net • ZeroMQ – http://zeromq.org • Statsd – http://github.com/etsy/statsd • Graphite – http://graphite.wikidot.com • Evernote – http://evernote.com
30. Thanks! • Twitter - @lusis • Github – lusis • Email – lusis.org@gmail.com