Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
JCConf 2020 Observing in Microservices
Search
Shihyu Ho
November 18, 2020
Programming
0
760
JCConf 2020 Observing in Microservices
Lessons learned observing in Microservices
Shihyu Ho
November 18, 2020
Tweet
Share
More Decks by Shihyu Ho
See All by Shihyu Ho
JCConf 2024 Jib
shihyuho
0
59
JCConf 2022 - Using ArchUnit to test your architecture
shihyuho
0
140
hybrid-cloud-seminar.pdf
shihyuho
0
99
JCConf 2021 Access Kubernetes API in Java
shihyuho
0
120
iThome Kubernetes Summit 2018
shihyuho
0
68
JCConf 2016 zookeeper
shihyuho
0
61
Other Decks in Programming
See All in Programming
SQLアンチパターン第2版 データベースプログラミングで陥りがちな失敗とその対策 / Intro to SQL Antipatterns 2nd
twada
PRO
27
8k
Rails Frontend Evolution: It Was a Setup All Along
skryukov
0
320
「App Intent」よくわからんけどすごい!
rinngo0302
1
120
テスターからテストエンジニアへ ~新米テストエンジニアが歩んだ9ヶ月振り返り~
non0113
2
240
AWS Summit Japan 2024と2025の比較/はじめてのKiro、今あなたは岐路に立つ
satoshi256kbyte
1
250
テスト駆動Kaggle
isax1015
1
880
GPUを計算資源として使おう!
primenumber
1
290
The Niche of CDK Grant オブジェクトって何者?/the-niche-of-cdk-what-isgrant-object
hassaku63
1
690
チームのテスト力を総合的に鍛えて品質、スピード、レジリエンスを共立させる/Testing approach that improves quality, speed, and resilience
goyoki
5
1.2k
Streamlitで実現できるようになったこと、実現してくれたこと
ayumu_yamaguchi
2
220
状態遷移図を書こう / Sequence Chart vs State Diagram
orgachem
PRO
3
250
CIを整備してメンテナンスを生成AIに任せる
hazumirr
0
190
Featured
See All Featured
Fashionably flexible responsive web design (full day workshop)
malarkey
407
66k
The Power of CSS Pseudo Elements
geoffreycrofte
77
5.9k
Templates, Plugins, & Blocks: Oh My! Creating the theme that thinks of everything
marktimemedia
31
2.4k
Making the Leap to Tech Lead
cromwellryan
134
9.4k
Dealing with People You Can't Stand - Big Design 2015
cassininazir
367
26k
10 Git Anti Patterns You Should be Aware of
lemiorhan
PRO
656
60k
GraphQLとの向き合い方2022年版
quramy
49
14k
Facilitating Awesome Meetings
lara
54
6.5k
Let's Do A Bunch of Simple Stuff to Make Websites Faster
chriscoyier
507
140k
Documentation Writing (for coders)
carmenintech
72
4.9k
Distributed Sagas: A Protocol for Coordinating Microservices
caitiem20
331
22k
Art, The Web, and Tiny UX
lynnandtonic
301
21k
Transcript
Lessons learned observing in Microservices Matt Ho
SoftLeader RD Matt Ho https://github.com/shihyuho
Microservices!
Move from Spring Cloud to Kubernetes native!
好多 App, 平常沒事, But...
Infrastructure 0
Tracing App App App container platform
Monitoring App App App container platform cpu/memory/VM heap … cpu/memory/VM
heap …
Loggin g App App App container platform
Kubernetes on-premises
原⽣的, ⼀裝好什麼都沒有!
不能客製呦!
不是在做比較
Tracing 1
Using OpenTracing #
github.com/opentracing-contrib/ java-spring-cloud
Put tracing in response #
App filter response request api add trace info
Add tracing in logger MDC #
App A log MDC
None
None
None
None
None
Create your Spans #
time Span A Span B Span C Span D 外部呼叫
trace
None
Propagating messages #
None
App A App B App C App D Queue Baggage
App A App B App C App D REST gRPC
Queue Enqueue Dequeue Baggage
None
Monitoring 2
Prometheus & exporters #
- node-exporter - Hardware/OS metric s - cAdvisor - Container
metric s - kube-state-metrics - Kubernetes Object metrics 環境類 exporters
github.com/prometheus-operator/ kube-prometheus
None
None
None
- blackbox - Probing over HTTP, HTTPS, DNS … -
micrometer.io - Java app metrics 服務類 exporters
None
None
https://grafana.com/grafana/ dashboards
Logging 3
Define logging format #
JSON logging format
github.com/logstash/ logstash-logback-encoder
Append app information #
app_name, app_version…
None
None
ES index strategy #
By namespace and date
None
Forwarding logs from { … } #
Fluentd Forward Log format OCP DockerSwarm Forward Other Cluster
App App App App App App
好多 App, 平常沒事, But...
Thanks