Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
なにもわからないところから始めるJVMモニタリング
Search
Yuuki Tsubouchi (yuuk1)
April 07, 2014
Technology
5
9.6k
なにもわからないところから始めるJVMモニタリング
JVM Operation Casual Talks
Yuuki Tsubouchi (yuuk1)
April 07, 2014
Tweet
Share
More Decks by Yuuki Tsubouchi (yuuk1)
See All by Yuuki Tsubouchi (yuuk1)
MetricSifter:クラウドアプリケーションにおける故障箇所特定の効率化のための多変量時系列データの特徴量削減 / FIT 2024
yuukit
2
91
工学としてのSRE再訪 / Revisiting SRE as Engineering
yuukit
18
10k
Cloudless Computingの論文紹介
yuukit
2
380
#SRE論文紹介 Detection is Better Than Cure: A Cloud Incidents Perspective V. Ganatra et. al., ESEC/FSE’23
yuukit
3
1.4k
エンジニアのためのSRE論文への招待 / Introduction to SRE Papers for Engineers
yuukit
2
10k
博士課程での研究まとめ 2023年1月版 / Summary of my research in the PhD course
yuukit
1
230
AI時代に向けたクラウドにおける信頼性エンジニアリングの未来構想 / DICOMO2022 6A-1
yuukit
7
2.8k
AIOps研究録―SREのための システム障害の自動原因診断 / SRE NEXT 2022
yuukit
10
12k
Interactive AIOps
yuukit
0
2k
Other Decks in Technology
See All in Technology
Azure App Service on Linux の Sidecar に Phi-3 を配置してインテリジェントなアプリケーションを作ってみよう/jazug-anniv14
thara0402
0
400
ガバメントクラウド開発と変化と成長する組織 / Organizational change and growth in developing a government cloud
kazeburo
4
750
Slackbot × RAG で実現する社内情報検索の最適化
howdy39
2
330
【shownet.conf_】ShowNet 2024 ~ Inter * Network ~
shownet
PRO
0
510
KubeVirt Networking ONIC 2024
orimanabu
3
340
[JAWS-UG GameTech] 第6回 各種事例紹介_18TRIPにおけるAWSサービスを活用した負荷テスト・障害テスト
naoto_yasuda
0
160
ドキュメントとの付き合い方を考える
leveragestech
1
130
「ばん・さく・つき・たー!」にならないためにSHIROBAKOから 学んだこと
ysknsid25
3
670
【shownet.conf_】ネットワークテストの最適化と利便性の追求
shownet
PRO
0
340
tenntennはなんでnewmoにnew社したの? - YAPC::Hakodate 2024
tenntenn
PRO
0
220
ラブグラフ紹介資料 〜プロダクト解体新書〜 / Lovegraph Product Deck
lovegraph
0
14k
Oracle Database 23ai 新機能#4 Rolling Maintenance
oracle4engineer
PRO
0
130
Featured
See All Featured
Visualization
eitanlees
143
15k
It's Worth the Effort
3n
183
27k
Navigating Team Friction
lara
183
14k
Raft: Consensus for Rubyists
vanstee
136
6.6k
KATA
mclloyd
27
13k
Art, The Web, and Tiny UX
lynnandtonic
296
20k
The Power of CSS Pseudo Elements
geoffreycrofte
71
5.3k
Intergalactic Javascript Robots from Outer Space
tanoku
268
27k
Designing with Data
zakiwarfel
98
5.1k
Visualizing Your Data: Incorporating Mongo into Loggly Infrastructure
mongodb
41
9.2k
Fireside Chat
paigeccino
32
3k
CSS Pre-Processors: Stylus, Less & Sass
bermonpainter
355
29k
Transcript
ͳʹΘ͔Βͳ͍ͱ͜Ζ͔Β ࢝ΊΔ+7.ϞχλϦϯά
id:y_uuki / @y_uuk1
None
None
✘
✘
PSGI/Plack Devel::NYTProf Devel::KYTProf Starlet Plack::Middleware::ServerStatus::Lite
PSGI/Plack Devel::NYTProf Devel::KYTProf Starlet Plack::Middleware::ServerStatus::Lite
ΠϯλʔωοτΈΔ • http://gihyo.jp/dev/serial/01/jvm-arc ! • http://www.slideshare.net/ChihiroIto1/java-16802525 JavaͲͷΑ͏ʹಈ͘ͷ͔ʙਤղͰΘ͔ΔJVMͷΈ JavaͲͷΑ͏ʹಈ͘ͷ͔ʙεϥΠυͰΘ͔ΔJVMͷΈ
ϝϞϦߏ / GCͷछྨ • Eden, Survivor, Tenured, Permanent… • γϦΞϧGCɺύϥϨϧGCɺίϯΧϨϯτGC
ϩάΛు͘ • http://n-agetsuma.hatenablog.com/entry/2014/03/30/175329 τϥϒϧʹඋ͑ΔJVMΦϓγϣϯ java -Xms?m -Xmx?m -XX:MaxPermSize=?m -Xloggc:/var/log/app/gc.log -XX:+PrintGCDetails
-XX:+PrintGCDateStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=10M -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/var/log/app -XX:ErrorFile=/var/log/app/jvm_err_pid%p.log
౷ܭใΈΔπʔϧ • jstat • jvmtop • Jolokia (JMX) • much
more…
jstat [y_uuki@hoge]$ sudo jstat -gc 30545 1000 S0C S1C S0U
S1U EC EU OC OU PC PU YGC YGCT FGC FGCT GCT 3072.0 3072.0 0.0 2400.0 343552.0 181900.4 699392.0 125193.0 55808.0 55592.9 719 8.034 0 0.000 8.034 3072.0 3072.0 0.0 2400.0 343552.0 189558.2 699392.0 125193.0 55808.0 55592.9 719 8.034 0 0.000 8.034 3072.0 3072.0 0.0 2400.0 343552.0 291417.5 699392.0 125193.0 55808.0 55592.9 719 8.034 0 0.000 8.034 3072.0 3072.0 0.0 2400.0 343552.0 291423.7 699392.0 125193.0 55808.0 55592.9 719 8.034 0 0.000 8.034 3072.0 3072.0 0.0 2400.0 343552.0 300260.1 699392.0 125193.0 55808.0 55592.9 719 8.034 0 0.000 8.034 3072.0 3072.0 0.0 2400.0 343552.0 300266.3 699392.0 125193.0 55808.0 55592.9 719 8.034 0 0.000 8.034 3072.0 3072.0 0.0 2400.0 343552.0 338223.4 699392.0 125193.0 55808.0 55592.9 719 8.034 0 0.000 8.034 3072.0 3072.0 2513.7 0.0 343552.0 5373.4 699392.0 125337.0 55808.0 55592.9 720 8.041 0 0.000 8.041 3072.0 3072.0 2513.7 0.0 343552.0 14378.4 699392.0 125337.0 55808.0 55592.9 720 8.041 0 0.000 8.041 3072.0 3072.0 2513.7 0.0 343552.0 14381.3 699392.0 125337.0 55808.0 55592.9 720 8.041 0 0.000 8.041 …
jvmtop $ jvmtop.sh <pid> JvmTop 0.8.0 alpha - 21:48:03, amd64,
4 cpus, Linux 3.10.23, load avg 0.30 http://code.google.com/p/jvmtop ! PID 30545: play.core.server.NettyServer ARGS: VMARGS: -Xms1024m -Xmx1024m -XX:MaxPermSize=256m -XX:ReservedCodeCacheSiz[...] VM: Oracle Corporation Java HotSpot(TM) 64-Bit Server VM 1.7.0_51 UP: 7:49m #THR: 41 #THRPEAK: 44 #THRCREATED: 5867 USER: apache GC-Time: 0: 0m #GC-Runs: 892 #TotalLoadedClasses: 10033 CPU: 2.21% GC: 0.00% HEAP: 454m /1019m NONHEAP: 64m / 384m ! TID NAME STATE CPU TOTALCPU BLOCKEDBY 5862 RMI TCP Connection(3)-127.0.0. RUNNABLE 9.20% 0.29% 31 play-scheduler-1 TIMED_WAITING 0.26% 4.43% 5867 JMX server connection timeout TIMED_WAITING 0.24% 0.01% 27 New I/O worker #8 RUNNABLE 0.01% 0.71% 21 New I/O worker #2 RUNNABLE 0.01% 0.75% 22 New I/O worker #3 RUNNABLE 0.01% 0.72% 20 New I/O worker #1 RUNNABLE 0.01% 0.72% 24 New I/O worker #5 RUNNABLE 0.00% 0.72% 26 New I/O worker #7 RUNNABLE 0.00% 0.72% 23 New I/O worker #4 RUNNABLE 0.00% 0.72% Note: Only top 10 threads (according cpu load) are shown!
Jolokia $ curl http://localhost:8778/jolokia/read/java.lang:type=Memory/ NonHeapMemoryUsage {"timestamp":1396794086,"status":200,"request": {"mbean":"java.lang:type=Memory","attribute":"NonHeapMemoryUsage","type":"re ad"},"value":{"max":402653184,"committed":59179008,"init":24576000,"used": 58814424}} -javaagent:/path/to/jolokia-jvm-1.2.0-agent.jar
ՄࢹԽ • Graphite • Sensu • Fluentd • Mackerel •
NewRelic
metrics ࣌ܥྻDB αʔόཧπʔϧ
• sensu / sensu-community-plugins • java/jstat-metrics.py
• fluent-plugin-jstat • fluent-plugin-jolokia
• sensu tcp handler • fluent-plugin-graphite
None
NewRelic -javaagent:/path/to/newrelic.jar
None
Stop The World ʹඋ͑ͯ • ϞχλϦϯάͷΈΛ͔͑ͯΒͷJVM ύϥ ϝʔλνϡʔχϯά
͓·͚
ॻ੶ 2010/03 2011/10 2014/05
Πϯλʔωοτ σϑΥϧτͷΧςΰϦશ෦ফͯ͠ Linux, Kernel, Docker, JVM, Perl, Scala, Graphite, Fluentd,
Sensu, … ͦΕͬΆ͍λάΛొ͠·͘Δ
None
IUUQTNBDLFSFMJP