Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Monitoring in Motion: Challenges of Monitoring ...
Search
Ilan Rabinovitch
February 26, 2016
Technology
0
98
Monitoring in Motion: Challenges of Monitoring Containers and Kuberntes
Ilan Rabinovitch
February 26, 2016
Tweet
Share
More Decks by Ilan Rabinovitch
See All by Ilan Rabinovitch
Monitoring in Motion - ContainerCon 2016
irabinovitch
0
99
Data Driven Post Mortems at Datadog - LinuxCon 2016
irabinovitch
1
210
Introduction to Docker Monitoring
irabinovitch
0
150
OSCON 2016 - Monitoring in Motion
irabinovitch
2
170
Monitoring OpenStack at Lithium (OpenStack Summit Austin 2016)
irabinovitch
0
64
LinuxFest Northwest 2016 - Monitoring 101
irabinovitch
0
43
Monitoring ECS and Dynamic Infrastructure
irabinovitch
0
110
Doing DevOps Right with Datadog + Pagerduty
irabinovitch
0
120
Docker Usage Patterns - Docker Meetup Palo Alto - Nov 2015
irabinovitch
0
63
Other Decks in Technology
See All in Technology
Green Tea Garbage Collector の今
zchee
PRO
2
360
業務自動化プラットフォーム Google Agentspace に入門してみる #devio2025
maroon1st
0
170
KAGのLT会 #8 - 東京リージョンでGAしたAmazon Q in QuickSightを使って、報告用の資料を作ってみた
0air
0
190
Flaky Testへの現実解をGoのプロポーザルから考える | Go Conference 2025
upamune
1
340
PyCon JP 2025 DAY1 「Hello, satellite data! ~Pythonではじめる衛星データ解析~」
ra0kley
0
860
研究開発部メンバーの働き⽅ / Sansan R&D Profile
sansan33
PRO
3
20k
コンテキストエンジニアリングとは? 考え方と応用方法
findy_eventslides
4
840
AI×Data×SaaS×Operation
sansantech
PRO
0
110
Geospatialの世界最前線を探る [2025年版]
dayjournal
2
450
pprof vs runtime/trace (FlightRecorder)
task4233
0
140
C# 14 / .NET 10 の新機能 (RC 1 時点)
nenonaninu
1
1.3k
動画データのポテンシャルを引き出す! Databricks と AI活用への奮闘記(現在進行形)
databricksjapan
0
120
Featured
See All Featured
Done Done
chrislema
185
16k
The Art of Delivering Value - GDevCon NA Keynote
reverentgeek
15
1.7k
How To Stay Up To Date on Web Technology
chriscoyier
791
250k
[RailsConf 2023 Opening Keynote] The Magic of Rails
eileencodes
30
9.7k
Rails Girls Zürich Keynote
gr2m
95
14k
Keith and Marios Guide to Fast Websites
keithpitt
411
22k
The Myth of the Modular Monolith - Day 2 Keynote - Rails World 2024
eileencodes
26
3.1k
The Cost Of JavaScript in 2023
addyosmani
53
9k
The Web Performance Landscape in 2024 [PerfNow 2024]
tammyeverts
9
840
Creating an realtime collaboration tool: Agile Flush - .NET Oxford
marcduiker
32
2.2k
Build your cross-platform service in a week with App Engine
jlugia
231
18k
Making the Leap to Tech Lead
cromwellryan
135
9.5k
Transcript
Monitoring In Motion Challenges in Monitoring Kubernetes & Containers Cloud
Native SF Meetup Feb 25, 2016 Ilan Rabinovitch Director, Community Datadog
About Me • Long time Datadog user. • Prior to
Datadog built automation and monitoring tooling at Ooyala and Edmunds.com • SCALE and TXLF Co-Founder Ilan Rabinovitch Datadog
[email protected]
@irabinovitch
Agenda • Monitoring 101 - Crash Course • Challenges in
Monitoring Dynamic Infrastructure • Demo Time • Questions?
Monitoring Everything
None
@honest_update on Twitter
Quick Overview of Datadog • Monitoring for modern applications. •
Time series storage of metrics and events. • Trending, alerting and anomaly detection. • Hundreds of integrations out of the box.
Monitoring 101: Categorization More at: http://goo.gl/t1Rgcg
None
Monitoring 101: Focus on symptoms More at: http://goo.gl/t1Rgcg
Recurse until you find root cause. More at: http://goo.gl/t1Rgcg
Container Monitoring Challenges
https://www.datadoghq.com/docker-adoption/
None
None
Operational Complexity •Average containers per host: N (N=4, 10/2015) •N-times
as many “hosts” to manage •Affects everything
Operational Complexity: Scale 100 instances 400 containers
Operational Complexity: Scale 160 metrics per host 640 metrics per
host
Operational Complexity: Scale 100 instances 64,000 metrics
None
Host Centric vs Service Centric
Host Centric vs Service Centric
Query Based Monitoring … … …
•Use tags, labels, etc on your hosts and metrics. •Pull
in existing labels from your infrastructure (Region, Docker Images, K8S Tags..) Query Based Monitoring By using tags, auto-adapt!
Where is my application running ? What’s the total throughput
of App X ? What’s its response time per tag ? (pod, version, DC) What’s the distribution of 5xx from Nginx per pod ?
Auto Discovery
Docker API Kubelet API Monitoring Agent Container A O A
O A O Application Container Off-The-Shelf Application (Redis, PostgreSQL, …) Containers List Metadata Additional Metadata (Pod names, RC, …) Config Backend Integration Configurations Host Level Metrics
Some Pictures Dashboards and Metrics Alerts Sharing
Demo time