Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
マルチクラウドな大規模サービスKARTEを支えるDatadog/multi-cloud mon...
Search
Daiki Matsui
November 13, 2019
Technology
0
2.3k
マルチクラウドな大規模サービスKARTEを支えるDatadog/multi-cloud monitoring with Datadog
Datadog Summit Tokyoでの登壇資料です。
動画は
https://www.youtube.com/watch?v=_Z4HBiwODbc
Daiki Matsui
November 13, 2019
Tweet
Share
More Decks by Daiki Matsui
See All by Daiki Matsui
The Fourth Age of SRE
ikemonn
0
210
Multicloud deploy with Spinnaker / builderscon tokyo 2018
ikemonn
2
4.4k
KARTE を支えるマルチプラットフォームインフラ監視 /karte-multi-platform-monitoring
ikemonn
2
15k
Node Apps Performance Tuning
ikemonn
0
130
Other Decks in Technology
See All in Technology
[CV勉強会@関東 ECCV2024 読み会] オンラインマッピング x トラッキング MapTracker: Tracking with Strided Memory Fusion for Consistent Vector HD Mapping (Chen+, ECCV24)
abemii
0
220
【Pycon mini 東海 2024】Google Colaboratoryで試すVLM
kazuhitotakahashi
2
500
Evangelismo técnico: ¿qué, cómo y por qué?
trishagee
0
360
B2B SaaSから見た最近のC#/.NETの進化
sansantech
PRO
0
750
安心してください、日本語使えますよ―Ubuntu日本語Remix提供休止に寄せて― 2024-11-17
nobutomurata
1
990
EventHub Startup CTO of the year 2024 ピッチ資料
eventhub
0
110
Shopifyアプリ開発における Shopifyの機能活用
sonatard
4
250
スクラム成熟度セルフチェックツールを作って得た学びとその活用法
coincheck_recruit
1
140
リンクアンドモチベーション ソフトウェアエンジニア向け紹介資料 / Introduction to Link and Motivation for Software Engineers
lmi
4
300k
Incident Response Practices: Waroom's Features and Future Challenges
rrreeeyyy
0
160
TanStack Routerに移行するのかい しないのかい、どっちなんだい! / Are you going to migrate to TanStack Router or not? Which one is it?
kaminashi
0
580
AWS Media Services 最新サービスアップデート 2024
eijikominami
0
200
Featured
See All Featured
ReactJS: Keep Simple. Everything can be a component!
pedronauck
665
120k
Optimizing for Happiness
mojombo
376
70k
Save Time (by Creating Custom Rails Generators)
garrettdimon
PRO
27
840
Building a Scalable Design System with Sketch
lauravandoore
459
33k
Happy Clients
brianwarren
98
6.7k
Build The Right Thing And Hit Your Dates
maggiecrowley
33
2.4k
CoffeeScript is Beautiful & I Never Want to Write Plain JavaScript Again
sstephenson
159
15k
Become a Pro
speakerdeck
PRO
25
5k
How to train your dragon (web standard)
notwaldorf
88
5.7k
Typedesign – Prime Four
hannesfritz
40
2.4k
The Success of Rails: Ensuring Growth for the Next 100 Years
eileencodes
44
6.8k
Building Flexible Design Systems
yeseniaperezcruz
327
38k
Transcript
1 ϚϧνΫϥυͳେنαʔϏε KARTEΛࢧ͑ΔDatadog ɹɹʛɹɹ© 2019 PLAID Inc. 2019.11.13ɹɹʛɹɹDatadog Summit Tokyo
2019ɹɹʛɹ PLAID, Inc. Daiki Matsui
2 2019.11.13ɹɹʛɹɹDatadog Summit Tokyo 2019ɹɹʛɹ ɹɹʛɹɹ© 2019 PLAID Inc. ࣗݾհ
দҪ େٍ גࣜձࣾϓϨΠυ SRE Senior Architect 2012-2015·ͰגࣜձࣾαΠόʔυʹͯήʔϜ։ൃʹैࣄ 2015ΑΓϓϨΠυʹࢀը͠ɺSREͱͯ͠ϚϧνΫϥυΠϯϑϥͷߏஙɾӡ༻Λ୲ 2015ೖࣾ
ɹɹʛɹɹ© 2019 PLAID Inc. 3 2019.11.13ɹɹʛɹɹDatadog Summit Tokyo 2019ɹɹʛɹ גࣜձࣾϓϨΠυʹ͍ͭͯ
גࣜձࣾϓϨΠυ ౦ژதԝ۠ۜ࠲6-10-1 GINZA SIX 10F ઃཱɿ201110݄ ैۀһɿ130໊ ࢿຊۚɿ149,987,878ԁ ※ࢿຊ४උؚۚΉ
4 2019.11.13ɹɹʛɹɹDatadog Summit Tokyo 2019ɹɹʛɹ ɹɹʛɹɹ© 2019 PLAID Inc. KARTEʹ͍ͭͯ
֬ೝ͢Δ جຊαʔϏε͝ར༻ํ๏Λ͝հ͠·͢ɻ ͓ಘͳಛయཱͭใ͕ຬࡌͰ͢ɻ ॳΊͯͷํ ְתּׅ然钠ׅ 6*رؠ؎ش٦ֽוזַזַ♳麦׃זְծ➙ ״إٔؗ،حف׃ְהְֲ倯䗳铣דׅկ UIرؠ؎ش٦ָ濼גֶֻץֹ 7אךرؠ؎ٕٝ٦ٕ DESIGN RULES » CHECK ׆ה⢪ִ،؎ذيֽ䲧ִת׃կ ֿך堣⠓ֶ鋅鷕׃זֻկ 窫㼎ծ 妜׃ְ ౙͷओɺ Ξλʔ COLLECTION OUTER 嗚稊勴⟝㼰׃㢌刿ֽׅדծ֮זך椚 䟝ך暟⟝ח⳿⠓ִ〳腉䚍ָ넝ֻזתׅկ 勴⟝㢌ִג嗚稊׃גגְֻׁկ ׀䋞劄ך暟⟝כ 鋅אַתׇד׃ַ ꟗׄ ҰਓҰਓʹ߹Θͤͨ ސ٬ମݧΛఏڙ WebαΠτͷ๚ऀͷߦಈΛ ސ٬͝ͱʹϦΞϧλΠϜʹղੳ $9 ސ٬ମݧ ϓϥοτϑΥʔϜ
5 2019.11.13ɹɹʛɹɹDatadog Summit Tokyo 2019ɹɹʛɹ ɹɹʛɹɹ© 2019 PLAID Inc. ͢͜ͱ
KARTEͷಛ KARTEΛࢧ͑ΔࢹͷϙΠϯτ ࠓޙΓ͍ͨ͜ͱ 1 2 3
6 2019.11.13ɹɹʛɹɹDatadog Summit Tokyo 2019ɹɹʛɹ ɹɹʛɹɹ© 2019 PLAID Inc. ͢͜ͱ
KARTEͷಛ KARTEΛࢧ͑ΔࢹͷϙΠϯτ ࠓޙΓ͍ͨ͜ͱ
7 ɹɹʛɹɹ© 2019 PLAID Inc. 2019.11.13ɹɹʛɹɹDatadog Summit Tokyo 2019ɹɹʛɹ KARTEͷಛ
1.Ϛϧνςφϯτ 2.ϨΠςϯγʔ/ߴෛՙ 3.େنγεςϜ 4.ϚϧνΫϥυߏ ͢͜ͱ →
8 ɹɹʛɹɹ© 2019 PLAID Inc. 2019.11.13ɹɹʛɹɹDatadog Summit Tokyo 2019ɹɹʛɹ KARTEͷಛ
1.Ϛϧνςφϯτ 2.ϨΠςϯγʔ/ߴෛՙ 3.େنγεςϜ 4.ϚϧνΫϥυߏ ͢͜ͱ →
9 2019.11.13ɹɹʛɹɹDatadog Summit Tokyo 2019ɹɹʛɹ ɹɹʛɹɹ© 2019 PLAID Inc. Ϛϧνςφϯτ
ɹɹʛɹɹ© 2019 PLAID Inc. 10 2019.11.13ɹɹʛɹɹDatadog Summit Tokyo 2019ɹɹʛɹ ɾ༷ʑͳෛՙͷ͓٬͞Μ͕͍Δ
ɾΞΫηεύλʔϯ৭ʑ Ϛϧνςφϯτ
ɹɹʛɹɹ© 2019 PLAID Inc. 11 2019.11.13ɹɹʛɹɹDatadog Summit Tokyo 2019ɹɹʛɹ ɾෛՙ͕1Ͱ10ഒมԽ͢Δ
ɾηʔϧͰෛՙ͕ٸ૿͢Δ Ϛϧνςφϯτ 1ͷෛՙਪҠ
12 ɹɹʛɹɹ© 2019 PLAID Inc. 2019.11.13ɹɹʛɹɹDatadog Summit Tokyo 2019ɹɹʛɹ KARTEͷಛ
1.Ϛϧνςφϯτ 2.ϨΠςϯγʔ/ߴෛՙ 3.େنγεςϜ 4.ϚϧνΫϥυߏ ͢͜ͱ →
13 2019.11.13ɹɹʛɹɹDatadog Summit Tokyo 2019ɹɹʛɹ ɹɹʛɹɹ© 2019 PLAID Inc. ϨΠςϯγʔɺߴෛՙ
sec to real-time actions 0.x events per day 3 billion stored in BigQuery 2+ PB events (rows) / sec 65,000 streaming insert per month 1 PB analysis per month 60+ PB SRE 3 ϨΠςϯγʔ/ߴෛՙ
14 2019.11.13ɹɹʛɹɹDatadog Summit Tokyo 2019ɹɹʛɹ ɹɹʛɹɹ© 2019 PLAID Inc. ϨΠςϯγʔɺߴෛՙ
sec to real-time actions 0.x events per day 3 billion stored in BigQuery 2+ PB events (rows) / sec 65,000 streaming insert per month 1 PB analysis per month 60+ PB SRE 3 ϨΠςϯγʔ/ߴෛՙ
15 2019.11.13ɹɹʛɹɹDatadog Summit Tokyo 2019ɹɹʛɹ ɹɹʛɹɹ© 2019 PLAID Inc. ϨΠςϯγʔɺߴෛՙ
16 2019.11.13ɹɹʛɹɹDatadog Summit Tokyo 2019ɹɹʛɹ ɹɹʛɹɹ© 2019 PLAID Inc. ϨΠςϯγʔɺߴෛՙ
sec to real-time actions 0.x events per day 3 billion stored in BigQuery 2+ PB events (rows) / sec 65,000 streaming insert per month 1 PB analysis per month 60+ PB SRE 3 ϨΠςϯγʔ/ߴෛՙ
17 ɹɹʛɹɹ© 2019 PLAID Inc. 2019.11.13ɹɹʛɹɹDatadog Summit Tokyo 2019ɹɹʛɹ KARTEͷಛ
1.Ϛϧνςφϯτ 2.ϨΠςϯγʔ/ߴෛՙ 3.େنγεςϜ 4.ϚϧνΫϥυߏ ͢͜ͱ →
18 2019.11.13ɹɹʛɹɹDatadog Summit Tokyo 2019ɹɹʛɹ ɹɹʛɹɹ© 2019 PLAID Inc. େنγεςϜ
19 ɹɹʛɹɹ© 2019 PLAID Inc. 2019.11.13ɹɹʛɹɹDatadog Summit Tokyo 2019ɹɹʛɹ KARTEͷಛ
1.Ϛϧνςφϯτ 2.ϨΠςϯγʔ/ߴෛՙ 3.େنγεςϜ 4.ϚϧνΫϥυߏ ͢͜ͱ →
20 2019.11.13ɹɹʛɹɹDatadog Summit Tokyo 2019ɹɹʛɹ ɹɹʛɹɹ© 2019 PLAID Inc. ϚϧνΫϥυߏ
ɹɹʛɹɹ© 2019 PLAID Inc. 21 2019.11.13ɹɹʛɹɹDatadog Summit Tokyo 2019ɹɹʛɹ Pros
1. ֤ϓϩόΠμʔͷྑ͍ͱ͜औΓ͕Ͱ͖Δ 2. ϓϩόΠμʔͷোΛผϓϩόΠμʔͰΧ όʔͰ͖Δ Pros/Cons ϚϧνΫϥυߏ Cons 1. ӡ༻ͷख͕ؒ૿͑Δ
ɹɹʛɹɹ© 2019 PLAID Inc. 22 2019.11.13ɹɹʛɹɹDatadog Summit Tokyo 2019ɹɹʛɹ ৄࡉΛΓ͍ͨํ
ʮσϒαϛ 2018ʯͷࢿྉͰʂ ϚϧνΫϥυߏ https://speakerdeck.com/tikson/zhan-lue-topointonituite
23 2019.11.13ɹɹʛɹɹDatadog Summit Tokyo 2019ɹɹʛɹ ɹɹʛɹɹ© 2019 PLAID Inc. ͢͜ͱ
KARTEͷಛ KARTEΛࢧ͑ΔࢹͷϙΠϯτ ࠓޙΓ͍ͨ͜ͱ ✓
KARTEΛࢧ͑ΔࢹͷϙΠϯτ 24 ɹɹʛɹɹ© 2019 PLAID Inc. 2019.11.13ɹɹʛɹɹDatadog Summit Tokyo 2019ɹɹʛɹ
ɾϦΞϧλΠϜੑ ɾॊೈੑ ɾεέʔϥϏϦςΟ ɾ҆ఆੑ
25 ɹɹʛɹɹ© 2019 PLAID Inc. 2019.11.13ɹɹʛɹɹDatadog Summit Tokyo 2019ɹɹʛɹ KARTEͷಛ
→ 1.Ϛϧνςφϯτ 2.ϨΠςϯγʔ/ߴෛՙ 3.େنγεςϜ 4.ϚϧνΫϥυߏ
26 ɹɹʛɹɹ© 2019 PLAID Inc. 2019.11.13ɹɹʛɹɹDatadog Summit Tokyo 2019ɹɹʛɹ KARTEΛࢧ͑Δ
ࢹͷϙΠϯτ 1.Datadogʹશͯू 2.Custom metricsͷੵۃ׆༻ 3.LayerΛ͚ͨDashboard ͢͜ͱ →
27 ɹɹʛɹɹ© 2019 PLAID Inc. 2019.11.13ɹɹʛɹɹDatadog Summit Tokyo 2019ɹɹʛɹ →
1.Datadogʹશͯू 2.Custom metricsͷੵۃ׆༻ 3.LayerΛ͚ͨDashboard 1.Ϛϧνςφϯτ 2.ϨΠςϯγʔ/ߴෛՙ 3.େنγεςϜ 4.ϚϧνΫϥυߏ
ɹɹʛɹɹ© 2019 PLAID Inc. 28 2019.11.13ɹɹʛɹɹDatadog Summit Tokyo 2019ɹɹʛɹ োରԠ࣌ʹDatadogΛݟΕ
͙͢ʹঢ়گΛѲͰ͖ΔΑ͏ʹ Datadogʹશͯू
29 ɹɹʛɹɹ© 2019 PLAID Inc. 2019.11.13ɹɹʛɹɹDatadog Summit Tokyo 2019ɹɹʛɹ KARTEΛࢧ͑Δ
ࢹͷϙΠϯτ ͢͜ͱ → 1.Datadogʹશͯू 2.Custom metricsͷੵۃ׆༻ 3.LayerΛ͚ͨDashboard
30 ɹɹʛɹɹ© 2019 PLAID Inc. 2019.11.13ɹɹʛɹɹDatadog Summit Tokyo 2019ɹɹʛɹ →
1.Datadogʹશͯू 2.Custom metricsͷੵۃ׆༻ 3.LayerΛ͚ͨDashboard 1.Ϛϧνςφϯτ 2.ϨΠςϯγʔ/ߴෛՙ 3.େنγεςϜ 4.ϚϧνΫϥυߏ
ɹɹʛɹɹ© 2019 PLAID Inc. 31 2019.11.13ɹɹʛɹɹDatadog Summit Tokyo 2019ɹɹʛɹ 32M/mon
΄Ͳར༻ Custom metricsͷੵۃ׆༻
Custom metricsͷੵۃ׆༻ 32 ɹɹʛɹɹ© 2019 PLAID Inc. 2019.11.13ɹɹʛɹɹDatadog Summit Tokyo
2019ɹɹʛɹ https://github.com/makinoy/libs-dogstatsd ɾlibs-dogstatsdͰɺ؆୯ʹॲཧͷ࣮ߦճ ࣮ߦ࣌ؒΛଌఆͰ͖ΔΑ͏ʹ ɾؔΑΓࡉ͔͍୯ҐͰlatencyΛଌఆ ɾϢʔβ୯ҐͰͷύϑΥʔϚϯεΛଌఆ 1 const dd = require('libs-dogstatsd'); 2 const stats = dd.start(); 3 // measure hoge 4 hoge(); 5 stats.tick('hoge', 1, 1, [user_id:1234]); Node.jsͷwrapper library
ɹɹʛɹɹ© 2019 PLAID Inc. 33 2019.11.13ɹɹʛɹɹDatadog Summit Tokyo 2019ɹɹʛɹ tagͰϢʔβࣝผ༻IDΛૹΔ
ࠓى͍ͬͯ͜Δ͕ ʮಛఆͷϢʔβىҼͰى͍ͬͯ͜Δʁʯ ʮKARTEશମͰى͍ͬͯ͜Δʁʯ ͔ΛΓ͚͍ͯ͘͢͠͠Δ Custom metricsͷੵۃ׆༻
34 ɹɹʛɹɹ© 2019 PLAID Inc. 2019.11.13ɹɹʛɹɹDatadog Summit Tokyo 2019ɹɹʛɹ KARTEΛࢧ͑Δ
ࢹͷϙΠϯτ ͢͜ͱ → 1.Datadogʹશͯू 2.Custom metricsͷੵۃ׆༻ 3.LayerΛ͚ͨDashboard
35 ɹɹʛɹɹ© 2019 PLAID Inc. 2019.11.13ɹɹʛɹɹDatadog Summit Tokyo 2019ɹɹʛɹ →
1.Datadogʹશͯू 2.Custom metricsͷੵۃ׆༻ 3.LayerΛ͚ͨDashboard 1.Ϛϧνςφϯτ 2.ϨΠςϯγʔ/ߴෛՙ 3.େنγεςϜ 4.ϚϧνΫϥυߏ
ɹɹʛɹɹ© 2019 PLAID Inc. 36 2019.11.13ɹɹʛɹɹDatadog Summit Tokyo 2019ɹɹʛɹ Layer
1 ɾKARTE͕ੜ͖͍ͯΔ͔Ͳ͏͔ LayerΛ͚ͨDashboard
ɹɹʛɹɹ© 2019 PLAID Inc. 37 2019.11.13ɹɹʛɹɹDatadog Summit Tokyo 2019ɹɹʛɹ Layer
2 ɾॏཁͳίϯϙʔωϯτ ɾॏཁͳΞϓϦέʔγϣϯͷϝτϦοΫ LayerΛ͚ͨDashboard
ɹɹʛɹɹ© 2019 PLAID Inc. 38 2019.11.13ɹɹʛɹɹDatadog Summit Tokyo 2019ɹɹʛɹ Layer
3 ɾ֤αʔό܈ͷࡉ͔͍ϝτϦοΫ LayerΛ͚ͨDashboard
39 2019.11.13ɹɹʛɹɹDatadog Summit Tokyo 2019ɹɹʛɹ ɹɹʛɹɹ© 2019 PLAID Inc. ͢͜ͱ
KARTEͷಛ KARTEΛࢧ͑ΔࢹͷϙΠϯτ ࠓޙΓ͍ͨ͜ͱ ✓ ✓
ɹࠓޙΓ͍ͨ͜ͱ 40 ɹɹʛɹɹ© 2019 PLAID Inc. 2019.11.13ɹɹʛɹɹDatadog Summit Tokyo 2019ɹɹʛɹ
શͯͷΤϯδχΞ͕ োͷҰ࣌ରԠΛͰ͖ΔΑ͏ʹ͢Δ ɾࢹ/োରԠͷϑϩʔͷཧ ɾDatadogΛ͍͜ͳͯ͠Β͏
ɹࠓޙΓ͍ͨ͜ͱ 41 ɹɹʛɹɹ© 2019 PLAID Inc. 2019.11.13ɹɹʛɹɹDatadog Summit Tokyo 2019ɹɹʛɹ
Custom metrics 1ԯԁͷղܾ ɾ͋·ΓʹศརͩͬͨͷͰ͍͢ ͗ͯ͠·ͬͨ… ɾCustom metricsͷཧΛߦ͏
42 2019.11.13ɹɹʛɹɹDatadog Summit Tokyo 2019ɹɹʛɹ ɹɹʛɹɹ© 2019 PLAID Inc. ·ͱΊ
·ͱΊ
43 ɹɹʛɹɹ© 2019 PLAID Inc. 2019.11.13ɹɹʛɹɹDatadog Summit Tokyo 2019ɹɹʛɹ KARTEΛࢧ͑Δ
ࢹͷϙΠϯτ 1.Datadogʹશͯू 2.Custom metricsͷੵۃ׆༻ 3.LayerΛ͚ͨDashboard →
None