Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Tool Time with John
Search
John Vincent
February 16, 2012
Technology
1
400
Tool Time with John
Presentation at Atlanta Puppet User's Group
John Vincent
February 16, 2012
Tweet
Share
More Decks by John Vincent
See All by John Vincent
Configuration management is a solved problem?
lusis
1
850
Everything about devops from metal
lusis
0
360
The Magic Omnibus
lusis
7
1.8k
Why Riak Matters
lusis
1
280
Monitorama 2013
lusis
7
1.9k
A Boy and His Logs
lusis
5
2.1k
Cross node orchestration with Chef and Noah
lusis
3
1.9k
Logging Patterns with Logstash and Chef
lusis
14
5.2k
The UnNamed Talk
lusis
7
1.4k
Other Decks in Technology
See All in Technology
20250913_JAWS_sysad_kobe
takuyay0ne
2
250
AI時代を生き抜くエンジニアキャリアの築き方 (AI-Native 時代、エンジニアという道は 「最大の挑戦の場」となる) / Building an Engineering Career to Thrive in the Age of AI (In the AI-Native Era, the Path of Engineering Becomes the Ultimate Arena of Challenge)
jeongjaesoon
0
240
💡Ruby 川辺で灯すPicoRubyからの光
bash0c7
0
120
スクラムガイドに載っていないスクラムのはじめかた - チームでスクラムをはじめるときに知っておきたい勘所を集めてみました! - / How to start Scrum that is not written in the Scrum Guide 2nd
takaking22
1
160
Create Ruby native extension gem with Go
sue445
0
130
スマートファクトリーの第一歩 〜AWSマネージドサービスで 実現する予知保全と生成AI活用まで
ganota
2
310
Firestore → Spanner 移行 を成功させた段階的移行プロセス
athug
1
500
20250912_RPALT_データを集める→とっ散らかる問題_Obsidian紹介
ratsbane666
0
100
今日から始めるAWSセキュリティ対策 3ステップでわかる実践ガイド
yoshidatakeshi1994
0
120
5分でカオスエンジニアリングを分かった気になろう
pandayumi
0
260
はじめてのOSS開発からみえたGo言語の強み
shibukazu
3
980
Autonomous Database - Dedicated 技術詳細 / adb-d_technical_detail_jp
oracle4engineer
PRO
4
10k
Featured
See All Featured
Rebuilding a faster, lazier Slack
samanthasiow
83
9.2k
Practical Tips for Bootstrapping Information Extraction Pipelines
honnibal
PRO
23
1.4k
The Web Performance Landscape in 2024 [PerfNow 2024]
tammyeverts
9
810
Fantastic passwords and where to find them - at NoRuKo
philnash
52
3.4k
The Art of Delivering Value - GDevCon NA Keynote
reverentgeek
15
1.7k
Dealing with People You Can't Stand - Big Design 2015
cassininazir
367
27k
CSS Pre-Processors: Stylus, Less & Sass
bermonpainter
358
30k
Unsuck your backbone
ammeep
671
58k
Building Applications with DynamoDB
mza
96
6.6k
Building an army of robots
kneath
306
46k
Principles of Awesome APIs and How to Build Them.
keavy
126
17k
Typedesign – Prime Four
hannesfritz
42
2.8k
Transcript
Tool Time with John! SE Puppet Users Group 02/16/2012
Everyone needs a tool belt
Why? • To fix things • To make things •
To keep your pants up
Caveats
There are tools
There are the right tools for a given job
Then there's “butterknifing a phillips head”
Fixing Things
Logging is Broken(ish) • Commercial options are expensive • Syslog
isn't flexible enough • Log lines are just data points with a timestamp (event)
Enter Logstash
Logstash • Unix pipe on steroids • If it has
a timestamp, send it to Logstash • Slice it, Dice it, Munge it, Transform it • Now spit it out • This isn't a Logstash talk.
Simple Sample Configuration
More complex (inputs)
More complex (filters)
More complex (outputs)
12 inputs, 11 filters, 21 outputs
Building stuff
Build something • Sockets the way you think sockets should
work • Usable from almost every language • Flow independent • Captures common patterns • Freaking fast
Logstash Filters with ZeroMQ • You need to do something
unique? • There's not a plugin yet? • Specific to your company? • Don't know Ruby? Write your own! (in whatever language you like)
Example – Logstash config
Example – External Filter
End Result
(don't get caught with your) Pants on the Ground
Statsd • Open sourced by Etsy last year • Simple
UDP proxy to Graphite • Different “data types” • Client and Server implementations in almost every language
Graphite • Open sourced by Orbitz • Makes pretty pictures
of data • Does dashboards • Simple API • Does realtime updates • Alternate dashboards available and easy to create
Logstash Configuration
End Result
Recap • Always be adding tools to your toolbelt •
See a cool project? Bookmark it • Evernote/Read it Later are your friends • Always be hacking. Keep your skills fresh.
Links! • Logstash – http://logstash.net • ZeroMQ – http://zeromq.org •
Statsd – http://github.com/etsy/statsd • Graphite – http://graphite.wikidot.com • Evernote – http://evernote.com
Thanks! • Twitter - @lusis • Github – lusis •
Email –
[email protected]