Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Tool Time with John
Search
John Vincent
February 16, 2012
Technology
1
400
Tool Time with John
Presentation at Atlanta Puppet User's Group
John Vincent
February 16, 2012
Tweet
Share
More Decks by John Vincent
See All by John Vincent
Configuration management is a solved problem?
lusis
1
820
Everything about devops from metal
lusis
0
340
The Magic Omnibus
lusis
7
1.7k
Why Riak Matters
lusis
1
270
Monitorama 2013
lusis
7
1.8k
A Boy and His Logs
lusis
5
2.1k
Cross node orchestration with Chef and Noah
lusis
3
1.9k
Logging Patterns with Logstash and Chef
lusis
14
5.2k
The UnNamed Talk
lusis
7
1.4k
Other Decks in Technology
See All in Technology
クラウドサービス事業者におけるOSS
tagomoris
4
1k
EMConf JP 2025 懇親会LT / EMConf JP 2025 social gathering
sugamasao
2
190
コンピュータビジョンの社会実装について考えていたらゲームを作っていた話
takmin
1
590
遷移の高速化 ヤフートップの試行錯誤
narirou
6
1.1k
AWSではじめる Web APIテスト実践ガイド / A practical guide to testing Web APIs on AWS
yokawasa
7
630
短縮URLをお手軽に導入しよう
nakasho
0
140
【内製開発Summit 2025】イオンスマートテクノロジーの内製化組織の作り方/In-house-development-summit-AST
aeonpeople
2
590
大規模アジャイルフレームワークから学ぶエンジニアマネジメントの本質
staka121
PRO
3
1k
Aurora PostgreSQLがCloudWatch Logsに 出力するログの課金を削減してみる #jawsdays2025
non97
1
180
いまからでも遅くない!コンテナでWebアプリを動かしてみよう!コンテナハンズオン編
nomu
0
150
(機械学習システムでも) SLO から始める信頼性構築 - ゆる SRE#9 2025/02/21
daigo0927
0
260
システム・ML活用を広げるdbtのデータモデリング / Expanding System & ML Use with dbt Modeling
i125
1
320
Featured
See All Featured
Building Adaptive Systems
keathley
40
2.4k
ピンチをチャンスに:未来をつくるプロダクトロードマップ #pmconf2020
aki_iinuma
114
50k
Improving Core Web Vitals using Speculation Rules API
sergeychernyshev
10
510
Rebuilding a faster, lazier Slack
samanthasiow
80
8.9k
Understanding Cognitive Biases in Performance Measurement
bluesmoon
27
1.6k
The Straight Up "How To Draw Better" Workshop
denniskardys
232
140k
Refactoring Trust on Your Teams (GOTO; Chicago 2020)
rmw
33
2.8k
Code Review Best Practice
trishagee
67
18k
Git: the NoSQL Database
bkeepers
PRO
427
65k
Making Projects Easy
brettharned
116
6k
[Rails World 2023 - Day 1 Closing Keynote] - The Magic of Rails
eileencodes
33
2.1k
Docker and Python
trallard
44
3.3k
Transcript
Tool Time with John! SE Puppet Users Group 02/16/2012
Everyone needs a tool belt
Why? • To fix things • To make things •
To keep your pants up
Caveats
There are tools
There are the right tools for a given job
Then there's “butterknifing a phillips head”
Fixing Things
Logging is Broken(ish) • Commercial options are expensive • Syslog
isn't flexible enough • Log lines are just data points with a timestamp (event)
Enter Logstash
Logstash • Unix pipe on steroids • If it has
a timestamp, send it to Logstash • Slice it, Dice it, Munge it, Transform it • Now spit it out • This isn't a Logstash talk.
Simple Sample Configuration
More complex (inputs)
More complex (filters)
More complex (outputs)
12 inputs, 11 filters, 21 outputs
Building stuff
Build something • Sockets the way you think sockets should
work • Usable from almost every language • Flow independent • Captures common patterns • Freaking fast
Logstash Filters with ZeroMQ • You need to do something
unique? • There's not a plugin yet? • Specific to your company? • Don't know Ruby? Write your own! (in whatever language you like)
Example – Logstash config
Example – External Filter
End Result
(don't get caught with your) Pants on the Ground
Statsd • Open sourced by Etsy last year • Simple
UDP proxy to Graphite • Different “data types” • Client and Server implementations in almost every language
Graphite • Open sourced by Orbitz • Makes pretty pictures
of data • Does dashboards • Simple API • Does realtime updates • Alternate dashboards available and easy to create
Logstash Configuration
End Result
Recap • Always be adding tools to your toolbelt •
See a cool project? Bookmark it • Evernote/Read it Later are your friends • Always be hacking. Keep your skills fresh.
Links! • Logstash – http://logstash.net • ZeroMQ – http://zeromq.org •
Statsd – http://github.com/etsy/statsd • Graphite – http://graphite.wikidot.com • Evernote – http://evernote.com
Thanks! • Twitter - @lusis • Github – lusis •
Email –
[email protected]