Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
AWS で実現した Mackerel 時系列データ1分粒度長期保存の裏側 / Mackerel Meetup #11 Tokyo
Search
astj
PRO
February 05, 2018
Technology
6
28k
AWS で実現した Mackerel 時系列データ1分粒度長期保存の裏側 / Mackerel Meetup #11 Tokyo
2018/02/05
https://mackerelio.connpass.com/event/76678/
astj
PRO
February 05, 2018
Tweet
Share
More Decks by astj
See All by astj
Mackerel の時系列データベースにおける Redis Cluster の利用と Amazon ElastiCache への移行について / AWS Purpose-Built Databases Week
astj
PRO
3
5.3k
Mackerel のコンテナ監視の近況 / Mackerel Meetup #13
astj
PRO
1
2k
Observability: Mackerel による観測と Mackerel の観測 / NoOps Meetup Tokyo #8
astj
PRO
2
1.8k
サービス開発と健全なプロダクトメンテナンスを開発チームで持続的に取り組む / Developers Boost KANSAI
astj
PRO
1
1.2k
Mackerel and Stripe / Qiita x Stripe Meetup
astj
PRO
0
5.2k
Mackerel をオンプレミスから AWS に移してからの1年半を振り返る / Hatena Engineer Seminar #11
astj
PRO
0
960
稼働中の Web サービスの perl のバージョンを上げていく
astj
PRO
0
7.1k
Perl 6 で Web Application Framework をつくる
astj
PRO
0
4.3k
Other Decks in Technology
See All in Technology
Flutter研修【MIXI 24新卒技術研修】
mixi_engineers
PRO
0
160
データベース研修 DB基礎【MIXI 24新卒技術研修】
mixi_engineers
PRO
0
210
推薦システムを本番導入する上で一番優先すべきだったこと~NewsPicks記事推薦機能の改善事例を元に~
morinota
0
120
「単なる OAuth 2.0 を認証に使うと、車が通れるほどのどでかいセキュリティー・ホールができる」のか検証してみた
terara
0
380
年間一億円削減した時系列データベースのアーキテクチャ改善~不確実性の高いプロジェクトへの挑戦~
lycorptech_jp
PRO
3
2.9k
開発と事業を繋ぐ!SREのオブザーバビリティ戦略 ~ Developers Summit 2024 Summer ~
leveragestech
0
630
コンテナ・K8s研修 - 前半 コンテナ基礎・ハンズオン【MIXI 24新卒技術研修】
mixi_engineers
PRO
0
170
ACRiルーム最新情報とAMD GPUサーバーのご紹介
anjn
0
150
AOAI Dev Day - Opening Session
yoshidashingo
2
440
頼られるのが大好きな 皆さんへ - 支援相手との期待の合わせ方、突き放し方 -/For_people_who_like_to_be_relied_on
naitosatoshi
1
290
サービス開発を前に進めるために 新米リードエンジニアが 取り組んだこと / Steps Taken by a Novice Lead Engineer to Advance Service Development
nologyance
0
180
開発生産性をむしろ向上させる セキュリティパートナーの作り方 / Dev Productivity Con 2024
flatt_security
0
360
Featured
See All Featured
Art, The Web, and Tiny UX
lynnandtonic
291
20k
Navigating Team Friction
lara
181
13k
ReactJS: Keep Simple. Everything can be a component!
pedronauck
662
120k
How to Create Impact in a Changing Tech Landscape [PerfNow 2023]
tammyeverts
34
1.9k
GitHub's CSS Performance
jonrohan
1026
450k
Happy Clients
brianwarren
94
6.5k
Robots, Beer and Maslow
schacon
PRO
157
8.1k
Thoughts on Productivity
jonyablonski
64
4.1k
What's new in Ruby 2.0
geeforr
338
31k
Done Done
chrislema
179
15k
How GitHub Uses GitHub to Build GitHub
holman
471
290k
"I'm Feeling Lucky" - Building Great Search Experiences for Today's Users (#IAC19)
danielanewman
224
21k
Transcript
"84Ͱ࣮ݱͨ͠.BDLFSFM ࣌ܥྻσʔλཻظอଘͷཪଆ .BDLFSFM.FFUVQ5PLZP JEBTUKʢגࣜձࣾͯͳʣ
w ͯͳΞϓϦέʔγϣϯΤϯδχΞ w ͯͳϒοΫϚʔΫͯͳϒϩάFUDʜ w डୗαʔϏε w .BDLFSFM d
w αʔόαΠυΤʔδΣϯτத৺ JEBTUK
w NBDLFSFMBHFOUTZTUFNEରԠ w NBDLFSFMQMVHJOBXT ͍͔ͭ͘ w -*/&5XJMJP࿈ܞ w "84Ҡߦ࣌ܥྻσʔλϕʔεͷ৽ w
࣌ܥྻσʔλཻظอଘ w ࢹϧʔϧͷૢ࡞Λ௨νϟϯωϧʹ௨ w ʜ JEBTUK
ຊͷ
w .BDLFSFM࣌ܥྻ%#ͷհ w ཻظอଘͷٕज़എܠ w αʔόϨεϛυϧΣΞͷߏஙࣄྫ w %ZOBNP%#4USFBNT -BNCEB w
ߴεϧʔϓοτ4ॻ͖ࠐΈ
લఏ
࣌ܥྻσʔλ
࣌ܥྻσʔλ hatena.mackerel.host1.cpu.user 2018/02/01T21:15:00Z 44.00 2018/02/01T21:16:00Z 6.00 2018/02/05T21:17:00Z 8.00 … …
࣌ܥྻσʔλϕʔε hatena.mackerel.host1.cpu.user 2018/02/01T21:15:00Z 44.00 2018/02/01T21:16:00Z 6.00 2018/02/05T21:17:00Z 8.00 hatena.mackerel.host1.cpu.user 2018/02/01T21:15:00Z
44.00 2018/02/01T21:16:00Z 6.00 2018/02/05T21:17:00Z 8.00 hatena.mackerel.host1.cpu.user 2018/02/01T21:15:00Z 44.00 2018/02/01T21:16:00Z 6.00 2018/02/05T21:17:00Z 8.00 hatena.mackerel.host1.cpu.user 2018/02/01T21:15:00Z 44.00 2018/02/01T21:16:00Z 6.00 2018/02/05T21:17:00Z 8.00 … …
http://blog.yuuk.io/entry/the-rebuild-of-tsdb-on-cloud
None
NI NE IE #FGPSF E
NE NE IE E "GUFS
None
.BDLFSFMͷ ࣌ܥྻσʔλϕʔε
Ҡߦલ(SBQIJUF IUUQTHSBQIJUFBQQPSH
http://blog.yuuk.io/entry/high-performance-graphite
ҠߦޙEJBNPOE ίʔυωʔϜ
IUUQCMPHZVVLJPFOUSZUIFSFCVJMEPGUTECPODMPVE ࣌ܥྻσʔλϕʔεͱ͍͏֓೦ΛΫϥυͷٕͰ࠶ߏங͢Δ
IUUQJUDIZOZIBUFOBCMPHDPNFOUSZ 4FSWFSMFTTDPOG5PLZPͰʰαʔόϨεΞʔΩςΫνϟʹ ΑΔ࣌ܥྻσʔλϕʔεͷߏஙͱࢹʱͱ͍͏ൃද͖ͯ͠·ͨ͠
http://blog.yuuk.io/entry/the-rebuild-of-tsdb-on-cloud 3FEJT$MVTUFS &$
ίϯηϓτ w εέʔϧ͢ΔΞʔΩςΫνϟ w ϚωʔδυαʔϏεʹΑΔӡ༻লྗԽ w ΞΫηεಛੑʹԠͨ͡֊ܕσʔλετΞ w 3FEJT %ZOBNP%#
4
ετϨʔδ *0ίετ ϨΠςϯγ ༰ྔίετ 3FEJT ˕ ˕ ˚ %ZOBNP%# ˓
˓ ˓ 4 ˚ ˚ ˕ σʔλετϨʔδ
%ZOBNP%# w ϑϧϚωʔδυ/P42- w ҆ఆɾߴύϑΥʔϚϯεɾεέʔϥϏϦςΟ w खࠒͳίετ w σʔλ༰ྔɺεϧʔϓοτ w
ʢͱ͍͑අ༻ΛແࢹͰ͖ͳ͍ʣ
3FEJT $MVTUFS w ΠϯϝϞϦ,74 w *0ίετ͕ %ZOBNP%#ΑΓ खܰ w σʔλྔΛࡹͨ͘Ί3FEJT$MVTUFS
w ࠓͷͱ͜Ζ&$
4 w ,74తʹར༻ w ༰ྔίετඇৗʹ͑ΒΕΔ w ϨΠςϯγɾ*0ίετྼΔ w ॻ͖ࠐΈճΛݮΒ͍ͨ͠
w %ZOBNP%# w ϝΠϯ w 3FEJT w ߋ৽ϦΫΤετΛݮΒͨ͢ΊͷόοϑΝ w 4
w طଘͷอ࣋ظؒͰ ར༻͠ͳ͍ ֊ܕσʔλετΞ
NI NE IE ֊ܕσʔλετΞ %ZOBNP%# 3FEJT E
ཻͷظอଘ
w σʔλྔ͕େ͖͘ΒΉ ഒ w ݹ͍σʔλͷϨΠςϯγཁ݅গ͠؇͍ w ৽͍͠σʔλ΄Ͳසൟʹࢀর͞Εͳ͍ w ݹ͍σʔλͷ্ॻ͖ߋ৽ൃੜ͠ͳ͍
w ݹ͍σʔλΛ4ʹஔ w طଘͷอ࣋ظؒͷύϑΥʔϚϯεΛҡ࣋ w ॻ͖ࠐΈճΛݮΒ͢ ֊ܕσʔλετΞ
NE NE IE ֊ܕσʔλετΞ %ZOBNP%# 3FEJT 4
E
None
55-&YQJSF %ZOBNP%# %%#4USFBN -BNCEB 4 1VU0CKFDU
%ZOBNP%#55-
%ZOBNP%#ͷΞΠςϜ͝ͱʹ༗ޮظݶ 55- Λࢦఆͯ͠ɺࣗಈతʹআͤ͞Δ
%ZOBNP%# hatena.mackerel.host1.cpu.user 2018/02/03 00:00~03:59 ExpireAt: 2018/02/04 05:00 2018/02/02 16:00~19:59 ExpireAt:
2018/02/03 21:00 hatena.mackerel.host1.cpu.user 2018/02/03 04:00~07:59 ExpireAt: 2018/02/04 09:00 hatena.mackerel.host1.cpu.user hatena.mackerel.host1.cpu.user 2018/02/02 20:00~23:59 ExpireAt: 2018/02/04 01:00
%ZOBNP%# hatena.mackerel.host1.cpu.user 2018/02/03 00:00~03:59 ExpireAt: 2018/02/04 05:00 hatena.mackerel.host1.cpu.user 2018/02/03 04:00~07:59
ExpireAt: 2018/02/04 09:00 hatena.mackerel.host1.cpu.user hatena.mackerel.host1.cpu.user 2018/02/02 20:00~23:59 ExpireAt: 2018/02/04 01:00
%ZOBNP%# hatena.mackerel.host1.cpu.user 2018/02/03 00:00~03:59 ExpireAt: 2018/02/04 05:00 hatena.mackerel.host1.cpu.user 2018/02/03 04:00~07:59
ExpireAt: 2018/02/04 09:00 hatena.mackerel.host1.cpu.user
hatena.mackerel.host1.cpu.user 2018/02/03 04:00~07:59 ExpireAt: 2018/02/04 09:00 hatena.mackerel.host1.cpu.user %ZOBNP%#
%ZOBNP%# hatena.mackerel.host1.cpu.user
%ZOBNP%#
ೳಈతͳૢ࡞ͳ͘%ZOBNP%#্ͷ ϨίʔυΛআͰ͖Δ
%ZOBNP%#4USFBNT
w %ZOBNP%#্ͷΞΠςϜߋ৽͕ Πϕϯτͱͯ͠ྲྀΕΔετϦʔϜ w ςʔϒϧͷσʔλมߋΛτϦΨʹ -BNCEBؔΛىಈͰ͖Δ w ϖΠϩʔυʹมߋલޙͷΞΠςϜ༰Λ ؚΊΔ͜ͱ͕Ͱ͖Δ
%ZOBNP%# 4USFBN -BNCEB ৽نΞΠςϜՃ طଘΞΠςϜߋ৽ ΞΠςϜআ
EJBNPOEFYQPSUFS %ZOBNP%#55- %ZOBNP%#4USFBNT
w 55-ʹ౸ୡͨ͠ΞΠςϜͷআ࣌ʹ ΞΠςϜͷ༰Λͯ͠-BNCEBؔΛ ىಈ w -BNCEBؔͰ4ʹॻ͖ग़͠
͜ͷΞΠςϜ͕55-ʹ౸ୡͯ͠ আ͞Ε·ͨ͠ hatena.mackerel.host1.cpu.user 2018/02/01T21:15:00Z 44.00 2018/02/01T21:16:00Z 6.00 2018/02/05T21:17:00Z 8.00 …
… hatena.mackerel.host1.cpu.user 2018/02/01T21:15:00Z 44.00 2018/02/01T21:16:00Z 6.00 2018/02/05T21:17:00Z 8.00 … …
hatena.mackerel.host1.cpu.user 2018/02/01T21:15:00Z 44.00 2018/02/01T21:16:00Z 6.00 2018/02/05T21:17:00Z 8.00 … …
55-&YQJSF %ZOBNP%# %%#4USFBN -BNCEB 4 1VU0CKFDU
55-&YQJSF %ZOBNP%# %%#4USFBN -BNCEB 4 1VU0CKFDU w ϝτϦοΫ͝ͱO࣌ؒ͝ͱʹΞΠςϜ w %ZOBNP%#ͷΞΠςϜʹ55-Λઃఆ
w 55-ܦաʹΑΓΞΠςϜআ w ΞΠςϜআΠϕϯτ͕4USFBNʹ௨ w 4USFBN͔Β-BNCEBؔىಈ w -BNCEB͕ؔΞΠςϜΛ4ʹॻࠐ
55-&YQJSF %ZOBNP%# %%#4USFBN -BNCEB 4 1VU0CKFDU w 55-༩Ҏ֎ͷ ೳಈతͳૢ࡞͕ෆཁ w
-BNCEBͷࣦഊ࣌ϦτϥΠࣗಈ w 4USFBNҎ֎ʹঢ়ଶΛ࣋ͨͳ͍ w %ZOBNP%#ΞΠςϜ4Φϒ δΣΫτ
4 EJBNPOE"11 NBDLFSFM"11 (FU0CKFDU )551 )551
γϯϓϧ
"84ͷΞοϓσʔτʹΑ࣮ͬͯݱ
ͷ%ZOBNP%#Ξοϓσʔτ w 55-d w "VUP4DBMF %"9d w 71$&OEQPJOUd w (MPCBM5BCMFT
ΦϯσϚϯυόοΫΞοϓd
4ͷߴසॻ͖ࠐΈ
w 4ͷߴසॻ͖ࠐΈΛߦ͏߹ରԠ͕ඞཁ w ෦γϟʔσΟϯάରԠͷͨΊͷQSFpY༩ w ٸܹͳ૿Ճͷ߹ "84αϙʔτܦ༝Ͱͷ ࣄલରԠґཔ
Amazon S3 όέοτͷϫʔΫϩʔυ͕ຖඵ 100 ճͷ PUT/LIST/DELETE ϦΫΤετ·ͨຖඵ 300 ճͷ GET
ϦΫΤετΛසൟʹ͑Δ߹ ɺ࠷ߴͷύϑΥʔϚϯεͱεέʔϥϏϦςΟ Λ֬อ͢ΔͨΊɺ͜ͷτϐοΫͷΨΠυϥΠϯ ʹै͍ͬͯͩ͘͞ɻ ϦΫΤετ͓ΑͼϦΫΤετύϑΥʔϚϯεʹؔ͢Δཹҙࣄ߲ https://docs.aws.amazon.com/ja_jp/AmazonS3/latest/dev/request- rate-perf-considerations.html
EJBNPOEͰ4ͷॻ͖ࠐΈίϯελϯτʹൃੜ SQT༏ʹ͑Δ
55-&YQJSF %ZOBNP%# %%#4USFBN -BNCEB 4 1VU0CKFDU w ϝτϦοΫ͝ͱO࣌ؒ͝ͱʹΞΠςϜ w %ZOBNP%#ͷΞΠςϜʹ55-Λઃఆ
w 55-ܦաʹΑΓΞΠςϜআ w ΞΠςϜআΠϕϯτ͕4USFBNʹ௨ w 4USFBN͔Β-BNCEBؔىಈ w -BNCEB͕ؔΞΠςϜΛ4ʹॻࠐ ࠶ܝ
55-&YQJSF %ZOBNP%# %%#4USFBN -BNCEB 4 1VU0CKFDU w 55-༩Ҏ֎ͷ ೳಈతͳૢ࡞͕ෆཁ w
-BNCEBͷࣦഊ࣌ϦτϥΠࣗಈ w %ZOBNP%#ͷΞΠςϜ4Φϒ δΣΫτ ࠶ܝ
ϦΫΤετ͓ΑͼϦΫΤετύϑΥʔϚϯεʹؔ͢Δཹҙࣄ߲ https://docs.aws.amazon.com/ja_jp/AmazonS3/latest/dev/request- rate-perf-considerations.html
w ΦϒδΣΫτ໊ઌ಄ʹ.%ϋογϡͷ༩ w ॻ͖ࠐΈ։࢝લʹαϙʔτʹରԠΛґཔ w Φʔμʔͷ͕ͪ࣌ؒൃੜ͢Δ
աͳϖʔεͰͷߋ৽)551YYΤϥʔ͕ ൃੜ͢ΔʢΒ͍͠ʣ ϞχλϦϯάʂ
"844ͷϞχλϦϯά
w 4ͷඪ४$MPVE8BUDI.FUSJDTݶఆత w όέοτͷ࣍ͷετϨʔδϝτϦΫε w ϦΫΤετͷੳͰ͖ͳ͍ w ΦϓγϣϯͰϦΫΤετͷϞχλϦϯάՄೳ w 4όέοτͷઃఆͰ༗ޮʹͰ͖Δ
https://docs.aws.amazon.com/ja_jp/AmazonS3/latest/dev/ cloudwatch-monitoring.html
None
$MPVE8BUDI.BDLFSFM
mackerel-plugin-aws-s3-requests [plugin.metrics.diamond-s3] command = "/path/to/mackerel-plugin-aws-s3-requests --bucket- name=XXX --region=ap-northeast-1 --filter-id YYY"
ެࣜϓϥάΠϯͱͯ͠ϦϦʔε༧ఆ ͖ͬͱࠓिதʹʜ
None
w (FU 1VUͦΕͧΕͷϦΫΤετ w ظతͳόʔετɺظతͳτϨϯυ w YYΤϥʔͷ ࢹϧʔϧ
͓ΘΓʹ
w ࣌ܥྻσʔλཻظอଘͷ෦࣮ w ߴස4ॻ͖ࠐΈ w ཹҙࣄ߲ϞχλϦϯά
༗Γ͏͍͟͝·ͨ͠ʂ