Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Beyond top: Command-Line Monitoring on the JVM ...
Search
Colin Jones
October 29, 2015
Programming
1
670
Beyond top: Command-Line Monitoring on the JVM (JavaOne 2015)
A session from JavaOne 2015
Colin Jones
October 29, 2015
Tweet
Share
More Decks by Colin Jones
See All by Colin Jones
A Bug's Life: What if `select` is Broken After All?
trptcolin
0
150
Underestimated costs of microservice architectures
trptcolin
3
1.5k
FP vs. OOP: Beyond the Bikeshed
trptcolin
0
410
Diving into the Details with DTrace! (RubyConf 2016 edition)
trptcolin
2
460
Diving into the Details with DTrace
trptcolin
3
510
Adopting FP: the good, the familiar, and the unknown
trptcolin
0
230
Finding out what's *really* going on, with DTrace!
trptcolin
1
370
Beyond top: Command-Line Monitoring on the JVM (ClojureRemote)
trptcolin
0
130
ZooKeeper: Wait-free coordination for Internet-scale systems
trptcolin
2
180
Other Decks in Programming
See All in Programming
PostgreSQLで手軽にDuckDBを使う!DuckDB&pg_duckdb入門/osk2025-duckdb
takahashiikki
1
240
フロントエンド開発に役立つクライアントプログラム共通のノウハウ / Universal client-side programming best practices for frontend development
nrslib
7
3.9k
Advance Your Career with Open Source
ivargrimstad
0
300
あなたの知らない「動画広告」の世界 - iOSDC Japan 2025
ukitaka
0
360
Railsだからできる 例外業務に禍根を残さない 設定設計パターン
ei_ei_eiichi
0
110
Swiftビルド弾丸ツアー - Swift Buildが作る新しいエコシステム
giginet
PRO
0
1.6k
アメ車でサンノゼを走ってきたよ!
s_shimotori
0
130
NetworkXとGNNで学ぶグラフデータ分析入門〜複雑な関係性を解き明かすPythonの力〜
mhrtech
3
980
CSC305 Lecture 03
javiergs
PRO
0
230
そのpreloadは必要?見過ごされたpreloadが技術的負債として爆発した日
mugitti9
2
2.9k
なぜGoのジェネリクスはこの形なのか? Featherweight Goが明かす設計の核心
ryotaros
7
1k
実践AIチャットボットUI実装入門
syumai
7
2.4k
Featured
See All Featured
The Success of Rails: Ensuring Growth for the Next 100 Years
eileencodes
46
7.6k
Build your cross-platform service in a week with App Engine
jlugia
232
18k
Rails Girls Zürich Keynote
gr2m
95
14k
A designer walks into a library…
pauljervisheath
209
24k
Become a Pro
speakerdeck
PRO
29
5.5k
The Cult of Friendly URLs
andyhume
79
6.6k
GitHub's CSS Performance
jonrohan
1032
460k
Git: the NoSQL Database
bkeepers
PRO
431
66k
Learning to Love Humans: Emotional Interface Design
aarron
274
40k
Designing for Performance
lara
610
69k
Responsive Adventures: Dirty Tricks From The Dark Corners of Front-End
smashingmag
252
21k
How to Create Impact in a Changing Tech Landscape [PerfNow 2023]
tammyeverts
54
3k
Transcript
Beyond top(1) Command-Line Monitoring on the JVM Colin Jones @trptcolin
8th Light
None
What to expect
command-line tooling
on the JVM
introspection & serviceability
--all-flags=false
war stories
real-life usage (well, re-enacted anyway)
A long time ago in a software shop far, far
away…
Things are going pretty well
What does this thing look like? app-architecture Postgres Web /
API Application Server Load Balancer Periodic Job Application Server 3rd-party Service A 3rd-party Service B Monitored email account End users: native mobile app Admin users: desktop browsers
But strange things are afoot
the server sometimes gets really slow
the team has to manually restart the application server
incident response time is ~5 minutes
Yes, strange things are afoot
Pain, frustration, anger
Just the facts
sometimes, things get slow
all requests seem to be affected
the JVM stays up
restart the JVM and everything is fine
What could it be?
Demo
More facts, please!
constant full GCs
what’s in the heap
what application code was running
The right tools for the job
vmstat system-level: CPU, memory, disk, context switching
top per-process: CPU & memory
jps what’s our PID?
jstack status of all threads (right now-ish!)
jcmd what can’t it do?! jcmd [PID] help (sorry, JVM
6 users: see jinfo/jmap/jstack)
jstat GC classloader compiler
Mystery solved!
Now “just” fix it
idea 1: eliminate the leak
idea 2: eliminate the cache altogether?
idea 3: delete the feature
idea 4: full-text search engine
So we’re good now… until the next incident
Lessons
“it’s slow” could mean lots of things
“high CPU” could mean lots of things
collecting data is crucial in a crisis
reproducing the issue helps me sleep at night
Other “right tools for the job”
Heap analyzers
Profilers
Constant monitoring & alerting
Dynamic tracing
Learning more
Books
operators are standing by! man jstat man jstack jcmd [PID]
help [COMMAND] etc.
Thank you! Colin Jones @trptcolin 8th Light