Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Tool Time with John
Search
John Vincent
February 16, 2012
Technology
1
400
Tool Time with John
Presentation at Atlanta Puppet User's Group
John Vincent
February 16, 2012
Tweet
Share
More Decks by John Vincent
See All by John Vincent
Configuration management is a solved problem?
lusis
1
820
Everything about devops from metal
lusis
0
340
The Magic Omnibus
lusis
7
1.7k
Why Riak Matters
lusis
1
270
Monitorama 2013
lusis
7
1.8k
A Boy and His Logs
lusis
5
2.1k
Cross node orchestration with Chef and Noah
lusis
3
1.9k
Logging Patterns with Logstash and Chef
lusis
14
5.2k
The UnNamed Talk
lusis
7
1.4k
Other Decks in Technology
See All in Technology
NFV基盤のOpenStack更新 ~9世代バージョンアップへの挑戦~
vtj
0
360
急成長する企業で作った、エンジニアが輝ける制度/ 20250227 Rinto Ikenoue
shift_evolve
0
140
システム・ML活用を広げるdbtのデータモデリング / Expanding System & ML Use with dbt Modeling
i125
1
330
JavaにおけるNull非許容性
skrb
2
2.6k
AIエージェント開発のノウハウと課題
pharma_x_tech
4
2.5k
EDRの検知の仕組みと検知回避について
chayakonanaika
12
5k
日経のデータベース事業とElasticsearch
hinatades
PRO
0
250
Cracking the Coding Interview 6th Edition
gdplabs
14
28k
IAMポリシーのAllow/Denyについて、改めて理解する
smt7174
2
210
【5分でわかる】セーフィー エンジニア向け会社紹介
safie_recruit
0
19k
Visualize, Visualize, Visualize and rclone
tomoaki0705
9
83k
実は強い 非ViTな画像認識モデル
tattaka
3
1.3k
Featured
See All Featured
実際に使うSQLの書き方 徹底解説 / pgcon21j-tutorial
soudai
175
52k
How STYLIGHT went responsive
nonsquared
98
5.4k
Fashionably flexible responsive web design (full day workshop)
malarkey
406
66k
Why Our Code Smells
bkeepers
PRO
336
57k
Reflections from 52 weeks, 52 projects
jeffersonlam
348
20k
The Psychology of Web Performance [Beyond Tellerrand 2023]
tammyeverts
46
2.3k
[Rails World 2023 - Day 1 Closing Keynote] - The Magic of Rails
eileencodes
33
2.1k
Large-scale JavaScript Application Architecture
addyosmani
511
110k
Distributed Sagas: A Protocol for Coordinating Microservices
caitiem20
330
21k
Sharpening the Axe: The Primacy of Toolmaking
bcantrill
40
2k
Docker and Python
trallard
44
3.3k
Creating an realtime collaboration tool: Agile Flush - .NET Oxford
marcduiker
27
1.9k
Transcript
Tool Time with John! SE Puppet Users Group 02/16/2012
Everyone needs a tool belt
Why? • To fix things • To make things •
To keep your pants up
Caveats
There are tools
There are the right tools for a given job
Then there's “butterknifing a phillips head”
Fixing Things
Logging is Broken(ish) • Commercial options are expensive • Syslog
isn't flexible enough • Log lines are just data points with a timestamp (event)
Enter Logstash
Logstash • Unix pipe on steroids • If it has
a timestamp, send it to Logstash • Slice it, Dice it, Munge it, Transform it • Now spit it out • This isn't a Logstash talk.
Simple Sample Configuration
More complex (inputs)
More complex (filters)
More complex (outputs)
12 inputs, 11 filters, 21 outputs
Building stuff
Build something • Sockets the way you think sockets should
work • Usable from almost every language • Flow independent • Captures common patterns • Freaking fast
Logstash Filters with ZeroMQ • You need to do something
unique? • There's not a plugin yet? • Specific to your company? • Don't know Ruby? Write your own! (in whatever language you like)
Example – Logstash config
Example – External Filter
End Result
(don't get caught with your) Pants on the Ground
Statsd • Open sourced by Etsy last year • Simple
UDP proxy to Graphite • Different “data types” • Client and Server implementations in almost every language
Graphite • Open sourced by Orbitz • Makes pretty pictures
of data • Does dashboards • Simple API • Does realtime updates • Alternate dashboards available and easy to create
Logstash Configuration
End Result
Recap • Always be adding tools to your toolbelt •
See a cool project? Bookmark it • Evernote/Read it Later are your friends • Always be hacking. Keep your skills fresh.
Links! • Logstash – http://logstash.net • ZeroMQ – http://zeromq.org •
Statsd – http://github.com/etsy/statsd • Graphite – http://graphite.wikidot.com • Evernote – http://evernote.com
Thanks! • Twitter - @lusis • Github – lusis •
Email – lusis.org@gmail.com