Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Tool Time with John

John Vincent
February 16, 2012

Tool Time with John

Presentation at Atlanta Puppet User's Group

John Vincent

February 16, 2012
Tweet

More Decks by John Vincent

Other Decks in Technology

Transcript

  1. Tool Time with John! SE Puppet Users Group 02/16/2012

  2. Everyone needs a tool belt

  3. Why? • To fix things • To make things •

    To keep your pants up
  4. Caveats

  5. There are tools

  6. There are the right tools for a given job

  7. Then there's “butterknifing a phillips head”

  8. Fixing Things

  9. Logging is Broken(ish) • Commercial options are expensive • Syslog

    isn't flexible enough • Log lines are just data points with a timestamp (event)
  10. Enter Logstash

  11. Logstash • Unix pipe on steroids • If it has

    a timestamp, send it to Logstash • Slice it, Dice it, Munge it, Transform it • Now spit it out • This isn't a Logstash talk.
  12. Simple Sample Configuration

  13. More complex (inputs)

  14. More complex (filters)

  15. More complex (outputs)

  16. 12 inputs, 11 filters, 21 outputs

  17. Building stuff

  18. Build something • Sockets the way you think sockets should

    work • Usable from almost every language • Flow independent • Captures common patterns • Freaking fast
  19. Logstash Filters with ZeroMQ • You need to do something

    unique? • There's not a plugin yet? • Specific to your company? • Don't know Ruby? Write your own! (in whatever language you like)
  20. Example – Logstash config

  21. Example – External Filter

  22. End Result

  23. (don't get caught with your) Pants on the Ground

  24. Statsd • Open sourced by Etsy last year • Simple

    UDP proxy to Graphite • Different “data types” • Client and Server implementations in almost every language
  25. Graphite • Open sourced by Orbitz • Makes pretty pictures

    of data • Does dashboards • Simple API • Does realtime updates • Alternate dashboards available and easy to create
  26. Logstash Configuration

  27. End Result

  28. Recap • Always be adding tools to your toolbelt •

    See a cool project? Bookmark it • Evernote/Read it Later are your friends • Always be hacking. Keep your skills fresh.
  29. Links! • Logstash – http://logstash.net • ZeroMQ – http://zeromq.org •

    Statsd – http://github.com/etsy/statsd • Graphite – http://graphite.wikidot.com • Evernote – http://evernote.com
  30. Thanks! • Twitter - @lusis • Github – lusis •

    Email – [email protected]