Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Tool Time with John
Search
John Vincent
February 16, 2012
Technology
1
380
Tool Time with John
Presentation at Atlanta Puppet User's Group
John Vincent
February 16, 2012
Tweet
Share
More Decks by John Vincent
See All by John Vincent
Configuration management is a solved problem?
lusis
1
750
Everything about devops from metal
lusis
0
320
The Magic Omnibus
lusis
7
1.7k
Why Riak Matters
lusis
1
260
Monitorama 2013
lusis
7
1.8k
A Boy and His Logs
lusis
5
2.1k
Cross node orchestration with Chef and Noah
lusis
3
1.8k
Logging Patterns with Logstash and Chef
lusis
14
5.1k
The UnNamed Talk
lusis
7
1.3k
Other Decks in Technology
See All in Technology
#51 “Empowering Azure Storage with RDMA”
cafenero_777
3
220
Kubeflow Pipelines v2 で変わる機械学習パイプライン開発
asei
4
340
中央集権体制からDataOpsへの転換 / centralized-to-dataops-transformation
pei0804
7
1.6k
Building a RAG app to chat with your data (on Azure)
pamelafox
0
130
Elementaryを用いたデータ品質の可視化とデータ基盤の運用改善
10xinc
6
1.7k
検証からプロダクトへ: シームレスなLLM開発の ためのしくみ作り
nunukim
1
210
プッシュ型子育てサービスを、先行プロジェクト実施自治体において開始します
govtechtokyo
0
290
依存ライブラリはどこに?
takesection
0
120
エンジニア候補者向け資料2024.03.28.pdf
macloud
0
2.9k
これまでのキャリアとこれからMLエンジニアとしてどう動くか
masatakashiwagi
1
640
コードレビューを支援するAI技術の応用
akkie76
2
160
LLM + RAG を使った SORACOM Support Bot の裏側の歴史
soracom
PRO
1
640
Featured
See All Featured
The Pragmatic Product Professional
lauravandoore
24
5.7k
Raft: Consensus for Rubyists
vanstee
130
6.2k
ReactJS: Keep Simple. Everything can be a component!
pedronauck
657
120k
KATA
mclloyd
14
11k
Producing Creativity
orderedlist
PRO
335
39k
Refactoring Trust on Your Teams (GOTO; Chicago 2020)
rmw
24
2.2k
Facilitating Awesome Meetings
lara
39
5.5k
Music & Morning Musume
bryan
39
5.5k
A designer walks into a library…
pauljervisheath
199
23k
ピンチをチャンスに:未来をつくるプロダクトロードマップ #pmconf2020
aki_iinuma
67
38k
Building Adaptive Systems
keathley
29
1.8k
Web development in the modern age
philhawksworth
201
10k
Transcript
Tool Time with John! SE Puppet Users Group 02/16/2012
Everyone needs a tool belt
Why? • To fix things • To make things •
To keep your pants up
Caveats
There are tools
There are the right tools for a given job
Then there's “butterknifing a phillips head”
Fixing Things
Logging is Broken(ish) • Commercial options are expensive • Syslog
isn't flexible enough • Log lines are just data points with a timestamp (event)
Enter Logstash
Logstash • Unix pipe on steroids • If it has
a timestamp, send it to Logstash • Slice it, Dice it, Munge it, Transform it • Now spit it out • This isn't a Logstash talk.
Simple Sample Configuration
More complex (inputs)
More complex (filters)
More complex (outputs)
12 inputs, 11 filters, 21 outputs
Building stuff
Build something • Sockets the way you think sockets should
work • Usable from almost every language • Flow independent • Captures common patterns • Freaking fast
Logstash Filters with ZeroMQ • You need to do something
unique? • There's not a plugin yet? • Specific to your company? • Don't know Ruby? Write your own! (in whatever language you like)
Example – Logstash config
Example – External Filter
End Result
(don't get caught with your) Pants on the Ground
Statsd • Open sourced by Etsy last year • Simple
UDP proxy to Graphite • Different “data types” • Client and Server implementations in almost every language
Graphite • Open sourced by Orbitz • Makes pretty pictures
of data • Does dashboards • Simple API • Does realtime updates • Alternate dashboards available and easy to create
Logstash Configuration
End Result
Recap • Always be adding tools to your toolbelt •
See a cool project? Bookmark it • Evernote/Read it Later are your friends • Always be hacking. Keep your skills fresh.
Links! • Logstash – http://logstash.net • ZeroMQ – http://zeromq.org •
Statsd – http://github.com/etsy/statsd • Graphite – http://graphite.wikidot.com • Evernote – http://evernote.com
Thanks! • Twitter - @lusis • Github – lusis •
Email –
[email protected]