Tool Time with John

Presentation at Atlanta Puppet User's Group

Transcript

1. Tool Time with John!
   SE Puppet Users Group
   02/16/2012
   

   View Slide

2. Everyone needs a tool belt
   

   View Slide

3. Why?
   ●
   To fix things
   ●
   To make things
   ●
   To keep your pants up
   

   View Slide

4. Caveats
   

   View Slide

5. There are tools
   

   View Slide

6. There are the right tools for a given job
   

   View Slide

7. Then there's “butterknifing a phillips head”
   

   View Slide

8. Fixing Things
   

   View Slide

9. Logging is Broken(ish)
   ●
   Commercial options are expensive
   ●
   Syslog isn't flexible enough
   ●
   Log lines are just data points with a timestamp
   (event)
   

   View Slide

10. Enter Logstash
    

    View Slide

11. Logstash
    ●
    Unix pipe on steroids
    ●
    If it has a timestamp, send it to Logstash
    ●
    Slice it, Dice it, Munge it, Transform it
    ●
    Now spit it out
    ●
    This isn't a Logstash talk.
    

    View Slide

12. Simple Sample Configuration
    

    View Slide

13. More complex (inputs)
    

    View Slide

14. More complex (filters)
    

    View Slide

15. More complex (outputs)
    

    View Slide

16. 12 inputs, 11 filters, 21 outputs
    

    View Slide

17. Building stuff
    

    View Slide

18. Build something
    ●
    Sockets the way you
    think sockets should
    work
    ●
    Usable from almost
    every language
    ●
    Flow independent
    ●
    Captures common
    patterns
    ●
    Freaking fast
    

    View Slide

19. Logstash Filters with ZeroMQ
    ●
    You need to do something unique?
    ●
    There's not a plugin yet?
    ●
    Specific to your company?
    ●
    Don't know Ruby?
    Write your own!
    (in whatever language you like)
    

    View Slide

20. Example – Logstash config
    

    View Slide

21. Example – External Filter
    

    View Slide

22. End Result
    

    View Slide

23. (don't get caught with your)
    Pants on the Ground
    

    View Slide

24. Statsd
    ●
    Open sourced by Etsy
    last year
    ●
    Simple UDP proxy to
    Graphite
    ●
    Different “data types”
    ●
    Client and Server
    implementations in
    almost every language
    

    View Slide

25. Graphite
    ●
    Open sourced by Orbitz
    ●
    Makes pretty pictures of
    data
    ●
    Does dashboards
    ●
    Simple API
    ●
    Does realtime updates
    ●
    Alternate dashboards
    available and easy to
    create
    

    View Slide

26. Logstash Configuration
    

    View Slide

27. End Result
    

    View Slide

28. Recap
    ●
    Always be adding tools to your toolbelt
    ●
    See a cool project? Bookmark it
    ●
    Evernote/Read it Later are your friends
    ●
    Always be hacking. Keep your skills fresh.
    

    View Slide

29. Links!
    ●
    Logstash – http://logstash.net
    ●
    ZeroMQ – http://zeromq.org
    ●
    Statsd – http://github.com/etsy/statsd
    ●
    Graphite – http://graphite.wikidot.com
    ●
    Evernote – http://evernote.com
    

    View Slide

30. Thanks!
    ●
    Twitter - @lusis
    ●
    Github – lusis
    ●
    Email –
    lusis.org@gmail.com
    

    View Slide