Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Beyond top: Command-Line Monitoring on the JVM ...

Avatar for Colin Jones Colin Jones
October 29, 2015
Programming
700
1

Beyond top: Command-Line Monitoring on the JVM (JavaOne 2015)

A session from JavaOne 2015

Avatar for Colin Jones

Colin Jones

October 29, 2015

More Decks by Colin Jones

See All by Colin Jones
A Bug's Life: What if `select` is Broken After All?
Avatar for Colin Jones trptcolin
0
180
Underestimated costs of microservice architectures
Avatar for Colin Jones trptcolin
3
1.6k
FP vs. OOP: Beyond the Bikeshed
Avatar for Colin Jones trptcolin
0
460
Diving into the Details with DTrace! (RubyConf 2016 edition)
Avatar for Colin Jones trptcolin
2
530
Diving into the Details with DTrace
Avatar for Colin Jones trptcolin
3
540
Adopting FP: the good, the familiar, and the unknown
Avatar for Colin Jones trptcolin
0
280
Finding out what's *really* going on, with DTrace!
Avatar for Colin Jones trptcolin
1
400
Beyond top: Command-Line Monitoring on the JVM (ClojureRemote)
Avatar for Colin Jones trptcolin
0
160
ZooKeeper: Wait-free coordination for Internet-scale systems
Avatar for Colin Jones trptcolin
2
220

Other Decks in Programming

See All in Programming
Symfonyの特性(設計思想)を手軽に活かす特性(trait)
Avatar for ICKX ickx
0
130
2026-03-27 #terminalnight 変数展開とコマンド展開でターミナル作業をスマートにする方法
Avatar for SUZUKI Masashi masasuzu
0
320
ふりがな Deep Dive try! Swift Tokyo 2026
Avatar for watura watura
0
180
「速くなった気がする」をデータで疑う
Avatar for Leaf senleaf24
0
150
年間50登壇、単著出版、雑誌寄稿、Podcast出演、YouTube、CM、カンファレンス主催……全部やってみたので面白さ等を比較してみよう / I’ve tried them all, so let’s compare how interesting they are.
Avatar for nrs nrslib
4
770
Don't Prompt Harder, Structure Better
Avatar for Yusuke Kita kitasuke
0
670
Offline should be the norm: building local-first apps with CRDTs & Kotlin Multiplatform
Avatar for Renaud MATHIEU renaudmathieu
0
190
Codex CLIのSubagentsによる並列API実装 / Parallel API Implementation with Codex CLI Subagents
Avatar for 冨澤宝斗 takatty
2
890
メッセージングを利用して時間的結合を分離しよう #phperkaigi
Avatar for Takuma Kajikawa kajitack
3
580
「接続」—パフォーマンスチューニングの最後の一手 〜点と点を結ぶ、その一瞬のために〜
Avatar for 武田 憲太郎 kentaroutakeda
5
2.5k
存在論的プログラミング： 時間と存在を記述する
Avatar for Akihito Koriyama koriym
5
860
ドメインイベントでビジネスロジックを解きほぐす #phpcon_odawara
Avatar for Takuma Kajikawa kajitack
2
130

Featured

See All Featured
The Success of Rails: Ensuring Growth for the Next 100 Years
Avatar for Eileen M. Uchitelle eileencodes
47
8k
A Guide to Academic Writing Using Generative AI - A Workshop
Avatar for Kenji Saito ks91
PRO
1
260
Reality Check: Gamification 10 Years Later
Avatar for Sebastian Deterding codingconduct
0
2.1k
Navigating the Design Leadership Dip - Product Design Week Design Leaders+ Conference 2024
Avatar for Andy Polaine apolaine
0
270
ピンチをチャンスに：未来をつくるプロダクトロードマップ #pmconf2020
Avatar for Aki / @LoveIdahoBurger aki_iinuma
128
55k
The B2B funnel & how to create a winning content strategy
Avatar for Katarina Dahlin katarinadahlin
PRO
1
330
The Director’s Chair: Orchestrating AI for Truly Effective Learning
Avatar for Mike Taylor tmiket
1
150
SEO in 2025: How to Prepare for the Future of Search
Avatar for Michael King ipullrank
3
3.4k
Future Trends and Review - Lecture 12 - Web Technologies (1019888BNR)
Avatar for Beat Signer signer
PRO
0
3.5k
The Straight Up "How To Draw Better" Workshop
Avatar for Dennis Kardys denniskardys
239
140k
B2B Lead Gen: Tactics, Traps & Triumph
Avatar for Sophie Logan marketingsoph
0
100
"I'm Feeling Lucky" - Building Great Search Experiences for Today's Users (#IAC19)
Avatar for Dan Newman danielanewman
231
23k

Transcript

  1. Beyond top(1) Command-Line Monitoring on the JVM Colin Jones @trptcolin

    8th Light
  2. None

  3. What to expect

  4. command-line tooling

  5. on the JVM

  6. introspection & serviceability

  7. --all-flags=false

  8. war stories

  9. real-life usage (well, re-enacted anyway)

  10. A long time ago in a software shop far, far

    away…

  11. Things are going pretty well

  12. What does this thing look like? app-architecture Postgres Web /

    API Application Server Load Balancer Periodic Job Application Server 3rd-party Service A 3rd-party Service B Monitored email account End users: native mobile app Admin users: desktop browsers

  13. But strange things are afoot

  14. the server sometimes gets really slow

  15. the team has to manually restart the application server

  16. incident response time is ~5 minutes

  17. Yes, strange things are afoot

  18. Pain, frustration, anger

  19. Just the facts

  20. sometimes, things get slow

  21. all requests seem to be affected

  22. the JVM stays up

  23. restart the JVM and everything is fine

  24. What could it be?

  25. Demo

  26. More facts, please!

  27. constant full GCs

  28. what’s in the heap

  29. what application code was running

  30. The right tools for the job

  31. vmstat system-level: CPU, memory, disk, context switching

  32. top per-process: CPU & memory

  33. jps what’s our PID?

  34. jstack status of all threads (right now-ish!)

  35. jcmd what can’t it do?! jcmd [PID] help (sorry, JVM

    6 users: see jinfo/jmap/jstack)

  36. jstat GC classloader compiler

  37. Mystery solved!

  38. Now “just” fix it

  39. idea 1: eliminate the leak

  40. idea 2: eliminate the cache altogether?

  41. idea 3: delete the feature

  42. idea 4: full-text search engine

  43. So we’re good now… until the next incident

  44. Lessons

  45. “it’s slow” could mean lots of things

  46. “high CPU” could mean lots of things

  47. collecting data is crucial in a crisis

  48. reproducing the issue helps me sleep at night

  49. Other “right tools for the job”

  50. Heap analyzers

  51. Profilers

  52. Constant monitoring & alerting

  53. Dynamic tracing

  54. Learning more

  55. Books

  56. operators are standing by! man jstat man jstack jcmd [PID]

    help [COMMAND] etc.

  57. Thank you! Colin Jones @trptcolin 8th Light