$30 off During Our Annual Pro Sale. View Details »

Tool Time with John

John Vincent
February 16, 2012

Tool Time with John

Presentation at Atlanta Puppet User's Group

John Vincent

February 16, 2012
Tweet

More Decks by John Vincent

Other Decks in Technology

Transcript

  1. Tool Time with John!
    SE Puppet Users Group
    02/16/2012

    View Slide

  2. Everyone needs a tool belt

    View Slide

  3. Why?

    To fix things

    To make things

    To keep your pants up

    View Slide

  4. Caveats

    View Slide

  5. There are tools

    View Slide

  6. There are the right tools for a given job

    View Slide

  7. Then there's “butterknifing a phillips head”

    View Slide

  8. Fixing Things

    View Slide

  9. Logging is Broken(ish)

    Commercial options are expensive

    Syslog isn't flexible enough

    Log lines are just data points with a timestamp
    (event)

    View Slide

  10. Enter Logstash

    View Slide

  11. Logstash

    Unix pipe on steroids

    If it has a timestamp, send it to Logstash

    Slice it, Dice it, Munge it, Transform it

    Now spit it out

    This isn't a Logstash talk.

    View Slide

  12. Simple Sample Configuration

    View Slide

  13. More complex (inputs)

    View Slide

  14. More complex (filters)

    View Slide

  15. More complex (outputs)

    View Slide

  16. 12 inputs, 11 filters, 21 outputs

    View Slide

  17. Building stuff

    View Slide

  18. Build something

    Sockets the way you
    think sockets should
    work

    Usable from almost
    every language

    Flow independent

    Captures common
    patterns

    Freaking fast

    View Slide

  19. Logstash Filters with ZeroMQ

    You need to do something unique?

    There's not a plugin yet?

    Specific to your company?

    Don't know Ruby?
    Write your own!
    (in whatever language you like)

    View Slide

  20. Example – Logstash config

    View Slide

  21. Example – External Filter

    View Slide

  22. End Result

    View Slide

  23. (don't get caught with your)
    Pants on the Ground

    View Slide

  24. Statsd

    Open sourced by Etsy
    last year

    Simple UDP proxy to
    Graphite

    Different “data types”

    Client and Server
    implementations in
    almost every language

    View Slide

  25. Graphite

    Open sourced by Orbitz

    Makes pretty pictures of
    data

    Does dashboards

    Simple API

    Does realtime updates

    Alternate dashboards
    available and easy to
    create

    View Slide

  26. Logstash Configuration

    View Slide

  27. End Result

    View Slide

  28. Recap

    Always be adding tools to your toolbelt

    See a cool project? Bookmark it

    Evernote/Read it Later are your friends

    Always be hacking. Keep your skills fresh.

    View Slide

  29. Links!

    Logstash – http://logstash.net

    ZeroMQ – http://zeromq.org

    Statsd – http://github.com/etsy/statsd

    Graphite – http://graphite.wikidot.com

    Evernote – http://evernote.com

    View Slide

  30. Thanks!

    Twitter - @lusis

    Github – lusis

    Email –
    [email protected]

    View Slide