Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
HOW DO YOU KNOW iT'S WORKiNG
Search
Mike Fiedler
May 20, 2016
Technology
0
87
HOW DO YOU KNOW iT'S WORKiNG
Monitoring NGiNX.Presented during
https://ngenious.website/
Mike Fiedler
May 20, 2016
Tweet
Share
More Decks by Mike Fiedler
See All by Mike Fiedler
Fixing AWS with AWS
miketheman
1
120
Instrumenting Application Stack in Dynamically Scaling Environment
miketheman
0
47
Giving (and Taking) Back
miketheman
0
52
Where Do We Go From Here?
miketheman
0
99
What's In This Cookbook?
miketheman
0
47
Treating Your Infrastructure Like Garbage
miketheman
2
1.7k
MongoDB - Operations Best Practices
miketheman
1
110
MongoDB in the Cloud - Amazon Web Services
miketheman
3
97
Other Decks in Technology
See All in Technology
プロトタイピングによる不確実性の低減 / Reducing Uncertainty through Prototyping
ohbarye
3
210
[2024年3月版] Databricksのシステムアーキテクチャ
databricksjapan
8
1.9k
A (short) History of AI
harishpillay
0
110
長期運用プロジェクトでのMySQLからTiDB移行の検証
colopl
1
610
Next'24 事例セッションの紹介とクラウド資格を活用したキャリア形成について語りMuscle
yasumuusan
1
310
元インフラエンジニアに成る / Human Resources to Human Relations
bobtani
3
750
Oracle Cloud Infrastructure:2024年4月度サービス・アップデート
oracle4engineer
PRO
1
100
Garoon 開発チーム / Garoon development team
cybozuinsideout
PRO
2
2.9k
【SORACOM UG】SIM Deep Dive セキュアエレメント編
soracom
PRO
0
250
コンテナセキュリティの基本と脅威への対策
kyohmizu
3
680
AIQ株式会社 エンジニア向け会社紹介資料
aiqlab
0
360
Databricks におけるデータエンジニアリング
databricksjapan
0
370
Featured
See All Featured
VelocityConf: Rendering Performance Case Studies
addyosmani
320
23k
Fantastic passwords and where to find them - at NoRuKo
philnash
36
2.5k
YesSQL, Process and Tooling at Scale
rocio
163
13k
Why Our Code Smells
bkeepers
PRO
331
56k
Large-scale JavaScript Application Architecture
addyosmani
503
110k
The Invisible Customer
myddelton
114
12k
How to train your dragon (web standard)
notwaldorf
72
5.1k
Learning to Love Humans: Emotional Interface Design
aarron
266
39k
GitHub's CSS Performance
jonrohan
1023
450k
The Power of CSS Pseudo Elements
geoffreycrofte
59
5k
Code Reviewing Like a Champion
maltzj
513
39k
The Psychology of Web Performance [Beyond Tellerrand 2023]
tammyeverts
5
1.5k
Transcript
HOW DO YOU KNOW iT’S WORKiNG MONiTORiNG
MiKE FiEDLER Wrangler of the Unusual Twitter: @mikefiedler Github: @miketheman
OpsSchool.org Chef Community Roller Derby Referee © Alex Erde
WHO CARES?
DOES iT EVEN?
DOES iT EVEN? YES iT DOES!
ACTiViTiES - ngx_http_stub_status_module $ nginx -V 2>&1 | grep -o
with-http_stub_status_module with-http_stub_status_module location /basic_status { stub_status; } $ curl http://localhost/basic_status Active connections: 1 server accepts handled requests 8 8 18 Reading: 0 Writing: 1 Waiting: 0
location /basic_status { allow 127.0.0.1; deny all; stub_status on; access_log
off; } Active connections: 45 server accepts handled requests 1023122 1023122 4270474 Reading: 0 Writing: 1 Waiting: 44 ACTiViTiES - SECURE-ER
MOAR??! - ngx_http_status_module
None
FREE AS A BiRD: ❖ awstats ❖ goaccess ❖ syslog/fluentd/logstash
❖ elasticsearch (ELK/EFK) LOGS - SO MUCH DATA DOLLA DOLLA BiLLS: ❖ Logentries ❖ Loggly ❖ Logz.io ❖ Splunk
• Emit log line • Log line is written to
file descriptor • Line is then: ◦ Read ◦ Shipped ◦ Tokenized ◦ Parsed ◦ Stored ◦ Eventually trashed • Oh, wait, the log line format changed?? FFS!! LOG PARSING iS HARD, MiKE!
STATSD (spec) • Push-based • UDP, connectionless • Lightweight datagram
<metric name>:<value>|c[|@<sample rate>] nginx.requests:1|c EMiTTERS TO THE RESCUE!
GET ME AN EMiTTER! Lua-based: • https://github.com/musicglue/ngx_lua_statsd • https://github.com/simplifi/ngx_lua_datadog •
https://github.com/lonelyplanet/openresty-statsd • https://github.com/knyar/nginx-lua-prometheus Native C: https://github.com/zebrafishlabs/nginx-statsd
DiD SOMEONE SAY C? HMMM…. • Get nginx source code
• Get emitter module • Patch a couple of files • Build a package • Config & run server { ... statsd_server localhost; statsd_count "nginx.requests" 1; statsd_count "nginx.responses.$status" 1 "$status"; statsd_count "nginx.request_length" "$request_length"; statsd_count "nginx.bytes_sent" "$bytes_sent"; location / { ...
PROFiT!
INSTRUMENT all resources • CPU • Network I/O • Disk
I/O & space (!!!) SUMMARY - MONiTOR FiRST ALERT on activities • Page loads • Dropped connections • Requests per second • Request timings • Error rates ESTABLiSH BASELiNES PRiOR TO CHANGE!
THANKS! https://github.com/miketheman/ngenious-statsd QUESTiONS?