Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
JCConf 2020 Observing in Microservices
Search
Shihyu Ho
November 18, 2020
Programming
0
710
JCConf 2020 Observing in Microservices
Lessons learned observing in Microservices
Shihyu Ho
November 18, 2020
Tweet
Share
More Decks by Shihyu Ho
See All by Shihyu Ho
JCConf 2024 Jib
shihyuho
0
35
JCConf 2022 - Using ArchUnit to test your architecture
shihyuho
0
130
hybrid-cloud-seminar.pdf
shihyuho
0
87
JCConf 2021 Access Kubernetes API in Java
shihyuho
0
110
iThome Kubernetes Summit 2018
shihyuho
0
57
JCConf 2016 zookeeper
shihyuho
0
54
Other Decks in Programming
See All in Programming
CI改善もDatadogとともに
taumu
0
120
第3回関東Kaggler会_AtCoderはKaggleの役に立つ
chettub
3
1k
Boost Performance and Developer Productivity with Jakarta EE 11
ivargrimstad
0
270
一休.com のログイン体験を支える技術 〜Web Components x Vue.js 活用事例と最適化について〜
atsumim
0
480
pylint custom ruleで始めるレビュー自動化
shogoujiie
0
120
もう僕は OpenAPI を書きたくない
sgash708
5
1.6k
ファインディLT_ポケモン対戦の定量的分析
fufufukakaka
0
710
Immutable ActiveRecord
megane42
0
140
Kubernetes History Inspector(KHI)を触ってみた
bells17
0
230
個人アプリを2年ぶりにアプデしたから褒めて / I just updated my personal app, praise me!
lovee
0
340
PHPのバージョンアップ時にも役立ったAST
matsuo_atsushi
0
110
Honoとフロントエンドの 型安全性について
yodaka
7
1.2k
Featured
See All Featured
Building Better People: How to give real-time feedback that sticks.
wjessup
367
19k
Refactoring Trust on Your Teams (GOTO; Chicago 2020)
rmw
33
2.8k
Templates, Plugins, & Blocks: Oh My! Creating the theme that thinks of everything
marktimemedia
30
2.2k
Building a Scalable Design System with Sketch
lauravandoore
461
33k
Why Our Code Smells
bkeepers
PRO
336
57k
What’s in a name? Adding method to the madness
productmarketing
PRO
22
3.3k
Bash Introduction
62gerente
611
210k
Cheating the UX When There Is Nothing More to Optimize - PixelPioneers
stephaniewalter
280
13k
How to Think Like a Performance Engineer
csswizardry
22
1.3k
Exploring the Power of Turbo Streams & Action Cable | RailsConf2023
kevinliebholz
30
4.6k
Adopting Sorbet at Scale
ufuk
74
9.2k
Visualizing Your Data: Incorporating Mongo into Loggly Infrastructure
mongodb
45
9.4k
Transcript
Lessons learned observing in Microservices Matt Ho
SoftLeader RD Matt Ho https://github.com/shihyuho
Microservices!
Move from Spring Cloud to Kubernetes native!
好多 App, 平常沒事, But...
Infrastructure 0
Tracing App App App container platform
Monitoring App App App container platform cpu/memory/VM heap … cpu/memory/VM
heap …
Loggin g App App App container platform
Kubernetes on-premises
原⽣的, ⼀裝好什麼都沒有!
不能客製呦!
不是在做比較
Tracing 1
Using OpenTracing #
github.com/opentracing-contrib/ java-spring-cloud
Put tracing in response #
App filter response request api add trace info
Add tracing in logger MDC #
App A log MDC
None
None
None
None
None
Create your Spans #
time Span A Span B Span C Span D 外部呼叫
trace
None
Propagating messages #
None
App A App B App C App D Queue Baggage
App A App B App C App D REST gRPC
Queue Enqueue Dequeue Baggage
None
Monitoring 2
Prometheus & exporters #
- node-exporter - Hardware/OS metric s - cAdvisor - Container
metric s - kube-state-metrics - Kubernetes Object metrics 環境類 exporters
github.com/prometheus-operator/ kube-prometheus
None
None
None
- blackbox - Probing over HTTP, HTTPS, DNS … -
micrometer.io - Java app metrics 服務類 exporters
None
None
https://grafana.com/grafana/ dashboards
Logging 3
Define logging format #
JSON logging format
github.com/logstash/ logstash-logback-encoder
Append app information #
app_name, app_version…
None
None
ES index strategy #
By namespace and date
None
Forwarding logs from { … } #
Fluentd Forward Log format OCP DockerSwarm Forward Other Cluster
App App App App App App
好多 App, 平常沒事, But...
Thanks