Tool Time with John!
SE Puppet Users Group
02/16/2012
Slide 2
Slide 2 text
Everyone needs a tool belt
Slide 3
Slide 3 text
Why?
●
To fix things
●
To make things
●
To keep your pants up
Slide 4
Slide 4 text
Caveats
Slide 5
Slide 5 text
There are tools
Slide 6
Slide 6 text
There are the right tools for a given job
Slide 7
Slide 7 text
Then there's “butterknifing a phillips head”
Slide 8
Slide 8 text
Fixing Things
Slide 9
Slide 9 text
Logging is Broken(ish)
●
Commercial options are expensive
●
Syslog isn't flexible enough
●
Log lines are just data points with a timestamp
(event)
Slide 10
Slide 10 text
Enter Logstash
Slide 11
Slide 11 text
Logstash
●
Unix pipe on steroids
●
If it has a timestamp, send it to Logstash
●
Slice it, Dice it, Munge it, Transform it
●
Now spit it out
●
This isn't a Logstash talk.
Slide 12
Slide 12 text
Simple Sample Configuration
Slide 13
Slide 13 text
More complex (inputs)
Slide 14
Slide 14 text
More complex (filters)
Slide 15
Slide 15 text
More complex (outputs)
Slide 16
Slide 16 text
12 inputs, 11 filters, 21 outputs
Slide 17
Slide 17 text
Building stuff
Slide 18
Slide 18 text
Build something
●
Sockets the way you
think sockets should
work
●
Usable from almost
every language
●
Flow independent
●
Captures common
patterns
●
Freaking fast
Slide 19
Slide 19 text
Logstash Filters with ZeroMQ
●
You need to do something unique?
●
There's not a plugin yet?
●
Specific to your company?
●
Don't know Ruby?
Write your own!
(in whatever language you like)
Slide 20
Slide 20 text
Example – Logstash config
Slide 21
Slide 21 text
Example – External Filter
Slide 22
Slide 22 text
End Result
Slide 23
Slide 23 text
(don't get caught with your)
Pants on the Ground
Slide 24
Slide 24 text
Statsd
●
Open sourced by Etsy
last year
●
Simple UDP proxy to
Graphite
●
Different “data types”
●
Client and Server
implementations in
almost every language
Slide 25
Slide 25 text
Graphite
●
Open sourced by Orbitz
●
Makes pretty pictures of
data
●
Does dashboards
●
Simple API
●
Does realtime updates
●
Alternate dashboards
available and easy to
create
Slide 26
Slide 26 text
Logstash Configuration
Slide 27
Slide 27 text
End Result
Slide 28
Slide 28 text
Recap
●
Always be adding tools to your toolbelt
●
See a cool project? Bookmark it
●
Evernote/Read it Later are your friends
●
Always be hacking. Keep your skills fresh.