Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
HOW DO YOU KNOW iT'S WORKiNG
Search
Mike Fiedler
May 20, 2016
Technology
0
100
HOW DO YOU KNOW iT'S WORKiNG
Monitoring NGiNX.Presented during
https://ngenious.website/
Mike Fiedler
May 20, 2016
Tweet
Share
More Decks by Mike Fiedler
See All by Mike Fiedler
Fixing AWS with AWS
miketheman
1
130
Instrumenting Application Stack in Dynamically Scaling Environment
miketheman
0
65
Giving (and Taking) Back
miketheman
0
57
Where Do We Go From Here?
miketheman
0
140
What's In This Cookbook?
miketheman
0
56
Treating Your Infrastructure Like Garbage
miketheman
2
1.7k
MongoDB - Operations Best Practices
miketheman
1
120
MongoDB in the Cloud - Amazon Web Services
miketheman
3
110
Other Decks in Technology
See All in Technology
FastAPIの魔法をgRPC/Connect RPCへ
monotaro
PRO
1
740
AI時代だからこそ考える、僕らが本当につくりたいスクラムチーム / A Scrum Team we really want to create in this AI era
takaking22
6
3.4k
Optuna DashboardにおけるPLaMo2連携機能の紹介 / PFN LLM セミナー
pfn
PRO
1
880
神回のメカニズムと再現方法/Mechanisms and Playbook for Kamikai scrumat2025
moriyuya
4
520
SoccerNet GSRの紹介と技術応用:選手視点映像を提供するサッカー作戦盤ツール
mixi_engineers
PRO
1
170
DataOpsNight#8_Terragruntを用いたスケーラブルなSnowflakeインフラ管理
roki18d
1
340
VCC 2025 Write-up
bata_24
0
180
許しとアジャイル
jnuank
1
120
「Verify with Wallet API」を アプリに導入するために
hinakko
1
230
自作LLM Native GORM Pluginで実現する AI Agentバックテスト基盤構築
po3rin
2
250
Goに育てられ開発者向けセキュリティ事業を立ち上げた僕が今向き合う、AI × セキュリティの最前線 / Go Conference 2025
flatt_security
0
350
From Prompt to Product @ How to Web 2025, Bucharest, Romania
janwerner
0
120
Featured
See All Featured
The Invisible Side of Design
smashingmag
301
51k
Refactoring Trust on Your Teams (GOTO; Chicago 2020)
rmw
35
3.2k
Scaling GitHub
holman
463
140k
Reflections from 52 weeks, 52 projects
jeffersonlam
352
21k
Designing for humans not robots
tammielis
254
25k
How to Create Impact in a Changing Tech Landscape [PerfNow 2023]
tammyeverts
54
3k
Thoughts on Productivity
jonyablonski
70
4.9k
How to Think Like a Performance Engineer
csswizardry
27
2k
StorybookのUI Testing Handbookを読んだ
zakiyama
31
6.2k
RailsConf & Balkan Ruby 2019: The Past, Present, and Future of Rails at GitHub
eileencodes
140
34k
CSS Pre-Processors: Stylus, Less & Sass
bermonpainter
358
30k
How GitHub (no longer) Works
holman
315
140k
Transcript
HOW DO YOU KNOW iT’S WORKiNG MONiTORiNG
MiKE FiEDLER Wrangler of the Unusual Twitter: @mikefiedler Github: @miketheman
OpsSchool.org Chef Community Roller Derby Referee © Alex Erde
WHO CARES?
DOES iT EVEN?
DOES iT EVEN? YES iT DOES!
ACTiViTiES - ngx_http_stub_status_module $ nginx -V 2>&1 | grep -o
with-http_stub_status_module with-http_stub_status_module location /basic_status { stub_status; } $ curl http://localhost/basic_status Active connections: 1 server accepts handled requests 8 8 18 Reading: 0 Writing: 1 Waiting: 0
location /basic_status { allow 127.0.0.1; deny all; stub_status on; access_log
off; } Active connections: 45 server accepts handled requests 1023122 1023122 4270474 Reading: 0 Writing: 1 Waiting: 44 ACTiViTiES - SECURE-ER
MOAR??! - ngx_http_status_module
None
FREE AS A BiRD: ❖ awstats ❖ goaccess ❖ syslog/fluentd/logstash
❖ elasticsearch (ELK/EFK) LOGS - SO MUCH DATA DOLLA DOLLA BiLLS: ❖ Logentries ❖ Loggly ❖ Logz.io ❖ Splunk
• Emit log line • Log line is written to
file descriptor • Line is then: ◦ Read ◦ Shipped ◦ Tokenized ◦ Parsed ◦ Stored ◦ Eventually trashed • Oh, wait, the log line format changed?? FFS!! LOG PARSING iS HARD, MiKE!
STATSD (spec) • Push-based • UDP, connectionless • Lightweight datagram
<metric name>:<value>|c[|@<sample rate>] nginx.requests:1|c EMiTTERS TO THE RESCUE!
GET ME AN EMiTTER! Lua-based: • https://github.com/musicglue/ngx_lua_statsd • https://github.com/simplifi/ngx_lua_datadog •
https://github.com/lonelyplanet/openresty-statsd • https://github.com/knyar/nginx-lua-prometheus Native C: https://github.com/zebrafishlabs/nginx-statsd
DiD SOMEONE SAY C? HMMM…. • Get nginx source code
• Get emitter module • Patch a couple of files • Build a package • Config & run server { ... statsd_server localhost; statsd_count "nginx.requests" 1; statsd_count "nginx.responses.$status" 1 "$status"; statsd_count "nginx.request_length" "$request_length"; statsd_count "nginx.bytes_sent" "$bytes_sent"; location / { ...
PROFiT!
INSTRUMENT all resources • CPU • Network I/O • Disk
I/O & space (!!!) SUMMARY - MONiTOR FiRST ALERT on activities • Page loads • Dropped connections • Requests per second • Request timings • Error rates ESTABLiSH BASELiNES PRiOR TO CHANGE!
THANKS! https://github.com/miketheman/ngenious-statsd QUESTiONS?