Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
AWS で実現した Mackerel 時系列データ1分粒度長期保存の裏側 / Mackerel...
Search
astj
PRO
February 05, 2018
Technology
6
30k
AWS で実現した Mackerel 時系列データ1分粒度長期保存の裏側 / Mackerel Meetup #11 Tokyo
2018/02/05
https://mackerelio.connpass.com/event/76678/
astj
PRO
February 05, 2018
Tweet
Share
More Decks by astj
See All by astj
ISUCON14公式反省会LT: 社内ISUCONの話
astj
PRO
0
330
Mackerel の時系列データベースにおける Redis Cluster の利用と Amazon ElastiCache への移行について / AWS Purpose-Built Databases Week
astj
PRO
3
5.8k
Mackerel のコンテナ監視の近況 / Mackerel Meetup #13
astj
PRO
1
2.3k
Observability: Mackerel による観測と Mackerel の観測 / NoOps Meetup Tokyo #8
astj
PRO
2
2.1k
サービス開発と健全なプロダクトメンテナンスを開発チームで持続的に取り組む / Developers Boost KANSAI
astj
PRO
1
1.4k
Mackerel and Stripe / Qiita x Stripe Meetup
astj
PRO
0
5.5k
Mackerel をオンプレミスから AWS に移してからの1年半を振り返る / Hatena Engineer Seminar #11
astj
PRO
0
1.1k
稼働中の Web サービスの perl のバージョンを上げていく
astj
PRO
0
7.4k
Perl 6 で Web Application Framework をつくる
astj
PRO
0
4.6k
Other Decks in Technology
See All in Technology
オーティファイ会社紹介資料 / Autify Company Deck
autifyhq
10
130k
FOSS4G 2025 KANSAI QGISで点群データをいろいろしてみた
kou_kita
0
410
CDK Toolkit Libraryにおけるテストの考え方
smt7174
0
120
American airlines ®️ USA Contact Numbers: Complete 2025 Support Guide
airhelpsupport
0
390
SEQUENCE object comparison - db tech showcase 2025 LT2
nori_shinoda
0
180
fukabori.fm 出張版: 売上高617億円と高稼働率を陰で支えた社内ツール開発のあれこれ話 / 20250704 Yoshimasa Iwase & Tomoo Morikawa
shift_evolve
PRO
2
8.1k
[SRE NEXT] ARR150億円_エンジニア140名_27チーム_17プロダクトから始めるSLO.pdf
satos
2
650
20250705 Headlamp: 專注可擴展性的 Kubernetes 用戶界面
pichuang
0
290
NewSQLや分散データベースを支えるRaftの仕組み - 仕組みを理解して知る得意不得意
hacomono
PRO
3
190
SRE不在の開発チームが障害対応と 向き合った100日間 / 100 days dealing with issues without SREs
shin1988
1
410
いつの間にか入れ替わってる!?新しいAWS Security Hubとは?
cmusudakeisuke
0
140
開発生産性を組織全体の「生産性」へ! 部門間連携の壁を越える実践的ステップ
sudo5in5k
3
7.5k
Featured
See All Featured
Evolution of real-time – Irina Nazarova, EuRuKo, 2024
irinanazarova
8
830
Raft: Consensus for Rubyists
vanstee
140
7k
[RailsConf 2023] Rails as a piece of cake
palkan
55
5.7k
Optimizing for Happiness
mojombo
379
70k
Site-Speed That Sticks
csswizardry
10
690
Improving Core Web Vitals using Speculation Rules API
sergeychernyshev
18
980
Documentation Writing (for coders)
carmenintech
72
4.9k
Facilitating Awesome Meetings
lara
54
6.4k
Sharpening the Axe: The Primacy of Toolmaking
bcantrill
44
2.4k
A better future with KSS
kneath
238
17k
Art, The Web, and Tiny UX
lynnandtonic
299
21k
Statistics for Hackers
jakevdp
799
220k
Transcript
"84Ͱ࣮ݱͨ͠.BDLFSFM ࣌ܥྻσʔλཻظอଘͷཪଆ .BDLFSFM.FFUVQ5PLZP JEBTUKʢגࣜձࣾͯͳʣ
w ͯͳΞϓϦέʔγϣϯΤϯδχΞ w ͯͳϒοΫϚʔΫͯͳϒϩάFUDʜ w डୗαʔϏε w .BDLFSFM d
w αʔόαΠυΤʔδΣϯτத৺ JEBTUK
w NBDLFSFMBHFOUTZTUFNEରԠ w NBDLFSFMQMVHJOBXT ͍͔ͭ͘ w -*/&5XJMJP࿈ܞ w "84Ҡߦ࣌ܥྻσʔλϕʔεͷ৽ w
࣌ܥྻσʔλཻظอଘ w ࢹϧʔϧͷૢ࡞Λ௨νϟϯωϧʹ௨ w ʜ JEBTUK
ຊͷ
w .BDLFSFM࣌ܥྻ%#ͷհ w ཻظอଘͷٕज़എܠ w αʔόϨεϛυϧΣΞͷߏஙࣄྫ w %ZOBNP%#4USFBNT -BNCEB w
ߴεϧʔϓοτ4ॻ͖ࠐΈ
લఏ
࣌ܥྻσʔλ
࣌ܥྻσʔλ hatena.mackerel.host1.cpu.user 2018/02/01T21:15:00Z 44.00 2018/02/01T21:16:00Z 6.00 2018/02/05T21:17:00Z 8.00 … …
࣌ܥྻσʔλϕʔε hatena.mackerel.host1.cpu.user 2018/02/01T21:15:00Z 44.00 2018/02/01T21:16:00Z 6.00 2018/02/05T21:17:00Z 8.00 hatena.mackerel.host1.cpu.user 2018/02/01T21:15:00Z
44.00 2018/02/01T21:16:00Z 6.00 2018/02/05T21:17:00Z 8.00 hatena.mackerel.host1.cpu.user 2018/02/01T21:15:00Z 44.00 2018/02/01T21:16:00Z 6.00 2018/02/05T21:17:00Z 8.00 hatena.mackerel.host1.cpu.user 2018/02/01T21:15:00Z 44.00 2018/02/01T21:16:00Z 6.00 2018/02/05T21:17:00Z 8.00 … …
http://blog.yuuk.io/entry/the-rebuild-of-tsdb-on-cloud
None
NI NE IE #FGPSF E
NE NE IE E "GUFS
None
.BDLFSFMͷ ࣌ܥྻσʔλϕʔε
Ҡߦલ(SBQIJUF IUUQTHSBQIJUFBQQPSH
http://blog.yuuk.io/entry/high-performance-graphite
ҠߦޙEJBNPOE ίʔυωʔϜ
IUUQCMPHZVVLJPFOUSZUIFSFCVJMEPGUTECPODMPVE ࣌ܥྻσʔλϕʔεͱ͍͏֓೦ΛΫϥυͷٕͰ࠶ߏங͢Δ
IUUQJUDIZOZIBUFOBCMPHDPNFOUSZ 4FSWFSMFTTDPOG5PLZPͰʰαʔόϨεΞʔΩςΫνϟʹ ΑΔ࣌ܥྻσʔλϕʔεͷߏஙͱࢹʱͱ͍͏ൃද͖ͯ͠·ͨ͠
http://blog.yuuk.io/entry/the-rebuild-of-tsdb-on-cloud 3FEJT$MVTUFS &$
ίϯηϓτ w εέʔϧ͢ΔΞʔΩςΫνϟ w ϚωʔδυαʔϏεʹΑΔӡ༻লྗԽ w ΞΫηεಛੑʹԠͨ͡֊ܕσʔλετΞ w 3FEJT %ZOBNP%#
4
ετϨʔδ *0ίετ ϨΠςϯγ ༰ྔίετ 3FEJT ˕ ˕ ˚ %ZOBNP%# ˓
˓ ˓ 4 ˚ ˚ ˕ σʔλετϨʔδ
%ZOBNP%# w ϑϧϚωʔδυ/P42- w ҆ఆɾߴύϑΥʔϚϯεɾεέʔϥϏϦςΟ w खࠒͳίετ w σʔλ༰ྔɺεϧʔϓοτ w
ʢͱ͍͑අ༻ΛແࢹͰ͖ͳ͍ʣ
3FEJT $MVTUFS w ΠϯϝϞϦ,74 w *0ίετ͕ %ZOBNP%#ΑΓ खܰ w σʔλྔΛࡹͨ͘Ί3FEJT$MVTUFS
w ࠓͷͱ͜Ζ&$
4 w ,74తʹར༻ w ༰ྔίετඇৗʹ͑ΒΕΔ w ϨΠςϯγɾ*0ίετྼΔ w ॻ͖ࠐΈճΛݮΒ͍ͨ͠
w %ZOBNP%# w ϝΠϯ w 3FEJT w ߋ৽ϦΫΤετΛݮΒͨ͢ΊͷόοϑΝ w 4
w طଘͷอ࣋ظؒͰ ར༻͠ͳ͍ ֊ܕσʔλετΞ
NI NE IE ֊ܕσʔλετΞ %ZOBNP%# 3FEJT E
ཻͷظอଘ
w σʔλྔ͕େ͖͘ΒΉ ഒ w ݹ͍σʔλͷϨΠςϯγཁ݅গ͠؇͍ w ৽͍͠σʔλ΄Ͳසൟʹࢀর͞Εͳ͍ w ݹ͍σʔλͷ্ॻ͖ߋ৽ൃੜ͠ͳ͍
w ݹ͍σʔλΛ4ʹஔ w طଘͷอ࣋ظؒͷύϑΥʔϚϯεΛҡ࣋ w ॻ͖ࠐΈճΛݮΒ͢ ֊ܕσʔλετΞ
NE NE IE ֊ܕσʔλετΞ %ZOBNP%# 3FEJT 4
E
None
55-&YQJSF %ZOBNP%# %%#4USFBN -BNCEB 4 1VU0CKFDU
%ZOBNP%#55-
%ZOBNP%#ͷΞΠςϜ͝ͱʹ༗ޮظݶ 55- Λࢦఆͯ͠ɺࣗಈతʹআͤ͞Δ
%ZOBNP%# hatena.mackerel.host1.cpu.user 2018/02/03 00:00~03:59 ExpireAt: 2018/02/04 05:00 2018/02/02 16:00~19:59 ExpireAt:
2018/02/03 21:00 hatena.mackerel.host1.cpu.user 2018/02/03 04:00~07:59 ExpireAt: 2018/02/04 09:00 hatena.mackerel.host1.cpu.user hatena.mackerel.host1.cpu.user 2018/02/02 20:00~23:59 ExpireAt: 2018/02/04 01:00
%ZOBNP%# hatena.mackerel.host1.cpu.user 2018/02/03 00:00~03:59 ExpireAt: 2018/02/04 05:00 hatena.mackerel.host1.cpu.user 2018/02/03 04:00~07:59
ExpireAt: 2018/02/04 09:00 hatena.mackerel.host1.cpu.user hatena.mackerel.host1.cpu.user 2018/02/02 20:00~23:59 ExpireAt: 2018/02/04 01:00
%ZOBNP%# hatena.mackerel.host1.cpu.user 2018/02/03 00:00~03:59 ExpireAt: 2018/02/04 05:00 hatena.mackerel.host1.cpu.user 2018/02/03 04:00~07:59
ExpireAt: 2018/02/04 09:00 hatena.mackerel.host1.cpu.user
hatena.mackerel.host1.cpu.user 2018/02/03 04:00~07:59 ExpireAt: 2018/02/04 09:00 hatena.mackerel.host1.cpu.user %ZOBNP%#
%ZOBNP%# hatena.mackerel.host1.cpu.user
%ZOBNP%#
ೳಈతͳૢ࡞ͳ͘%ZOBNP%#্ͷ ϨίʔυΛআͰ͖Δ
%ZOBNP%#4USFBNT
w %ZOBNP%#্ͷΞΠςϜߋ৽͕ Πϕϯτͱͯ͠ྲྀΕΔετϦʔϜ w ςʔϒϧͷσʔλมߋΛτϦΨʹ -BNCEBؔΛىಈͰ͖Δ w ϖΠϩʔυʹมߋલޙͷΞΠςϜ༰Λ ؚΊΔ͜ͱ͕Ͱ͖Δ
%ZOBNP%# 4USFBN -BNCEB ৽نΞΠςϜՃ طଘΞΠςϜߋ৽ ΞΠςϜআ
EJBNPOEFYQPSUFS %ZOBNP%#55- %ZOBNP%#4USFBNT
w 55-ʹ౸ୡͨ͠ΞΠςϜͷআ࣌ʹ ΞΠςϜͷ༰Λͯ͠-BNCEBؔΛ ىಈ w -BNCEBؔͰ4ʹॻ͖ग़͠
͜ͷΞΠςϜ͕55-ʹ౸ୡͯ͠ আ͞Ε·ͨ͠ hatena.mackerel.host1.cpu.user 2018/02/01T21:15:00Z 44.00 2018/02/01T21:16:00Z 6.00 2018/02/05T21:17:00Z 8.00 …
… hatena.mackerel.host1.cpu.user 2018/02/01T21:15:00Z 44.00 2018/02/01T21:16:00Z 6.00 2018/02/05T21:17:00Z 8.00 … …
hatena.mackerel.host1.cpu.user 2018/02/01T21:15:00Z 44.00 2018/02/01T21:16:00Z 6.00 2018/02/05T21:17:00Z 8.00 … …
55-&YQJSF %ZOBNP%# %%#4USFBN -BNCEB 4 1VU0CKFDU
55-&YQJSF %ZOBNP%# %%#4USFBN -BNCEB 4 1VU0CKFDU w ϝτϦοΫ͝ͱO࣌ؒ͝ͱʹΞΠςϜ w %ZOBNP%#ͷΞΠςϜʹ55-Λઃఆ
w 55-ܦաʹΑΓΞΠςϜআ w ΞΠςϜআΠϕϯτ͕4USFBNʹ௨ w 4USFBN͔Β-BNCEBؔىಈ w -BNCEB͕ؔΞΠςϜΛ4ʹॻࠐ
55-&YQJSF %ZOBNP%# %%#4USFBN -BNCEB 4 1VU0CKFDU w 55-༩Ҏ֎ͷ ೳಈతͳૢ࡞͕ෆཁ w
-BNCEBͷࣦഊ࣌ϦτϥΠࣗಈ w 4USFBNҎ֎ʹঢ়ଶΛ࣋ͨͳ͍ w %ZOBNP%#ΞΠςϜ4Φϒ δΣΫτ
4 EJBNPOE"11 NBDLFSFM"11 (FU0CKFDU )551 )551
γϯϓϧ
"84ͷΞοϓσʔτʹΑ࣮ͬͯݱ
ͷ%ZOBNP%#Ξοϓσʔτ w 55-d w "VUP4DBMF %"9d w 71$&OEQPJOUd w (MPCBM5BCMFT
ΦϯσϚϯυόοΫΞοϓd
4ͷߴසॻ͖ࠐΈ
w 4ͷߴසॻ͖ࠐΈΛߦ͏߹ରԠ͕ඞཁ w ෦γϟʔσΟϯάରԠͷͨΊͷQSFpY༩ w ٸܹͳ૿Ճͷ߹ "84αϙʔτܦ༝Ͱͷ ࣄલରԠґཔ
Amazon S3 όέοτͷϫʔΫϩʔυ͕ຖඵ 100 ճͷ PUT/LIST/DELETE ϦΫΤετ·ͨຖඵ 300 ճͷ GET
ϦΫΤετΛසൟʹ͑Δ߹ ɺ࠷ߴͷύϑΥʔϚϯεͱεέʔϥϏϦςΟ Λ֬อ͢ΔͨΊɺ͜ͷτϐοΫͷΨΠυϥΠϯ ʹै͍ͬͯͩ͘͞ɻ ϦΫΤετ͓ΑͼϦΫΤετύϑΥʔϚϯεʹؔ͢Δཹҙࣄ߲ https://docs.aws.amazon.com/ja_jp/AmazonS3/latest/dev/request- rate-perf-considerations.html
EJBNPOEͰ4ͷॻ͖ࠐΈίϯελϯτʹൃੜ SQT༏ʹ͑Δ
55-&YQJSF %ZOBNP%# %%#4USFBN -BNCEB 4 1VU0CKFDU w ϝτϦοΫ͝ͱO࣌ؒ͝ͱʹΞΠςϜ w %ZOBNP%#ͷΞΠςϜʹ55-Λઃఆ
w 55-ܦաʹΑΓΞΠςϜআ w ΞΠςϜআΠϕϯτ͕4USFBNʹ௨ w 4USFBN͔Β-BNCEBؔىಈ w -BNCEB͕ؔΞΠςϜΛ4ʹॻࠐ ࠶ܝ
55-&YQJSF %ZOBNP%# %%#4USFBN -BNCEB 4 1VU0CKFDU w 55-༩Ҏ֎ͷ ೳಈతͳૢ࡞͕ෆཁ w
-BNCEBͷࣦഊ࣌ϦτϥΠࣗಈ w %ZOBNP%#ͷΞΠςϜ4Φϒ δΣΫτ ࠶ܝ
ϦΫΤετ͓ΑͼϦΫΤετύϑΥʔϚϯεʹؔ͢Δཹҙࣄ߲ https://docs.aws.amazon.com/ja_jp/AmazonS3/latest/dev/request- rate-perf-considerations.html
w ΦϒδΣΫτ໊ઌ಄ʹ.%ϋογϡͷ༩ w ॻ͖ࠐΈ։࢝લʹαϙʔτʹରԠΛґཔ w Φʔμʔͷ͕ͪ࣌ؒൃੜ͢Δ
աͳϖʔεͰͷߋ৽)551YYΤϥʔ͕ ൃੜ͢ΔʢΒ͍͠ʣ ϞχλϦϯάʂ
"844ͷϞχλϦϯά
w 4ͷඪ४$MPVE8BUDI.FUSJDTݶఆత w όέοτͷ࣍ͷετϨʔδϝτϦΫε w ϦΫΤετͷੳͰ͖ͳ͍ w ΦϓγϣϯͰϦΫΤετͷϞχλϦϯάՄೳ w 4όέοτͷઃఆͰ༗ޮʹͰ͖Δ
https://docs.aws.amazon.com/ja_jp/AmazonS3/latest/dev/ cloudwatch-monitoring.html
None
$MPVE8BUDI.BDLFSFM
mackerel-plugin-aws-s3-requests [plugin.metrics.diamond-s3] command = "/path/to/mackerel-plugin-aws-s3-requests --bucket- name=XXX --region=ap-northeast-1 --filter-id YYY"
ެࣜϓϥάΠϯͱͯ͠ϦϦʔε༧ఆ ͖ͬͱࠓिதʹʜ
None
w (FU 1VUͦΕͧΕͷϦΫΤετ w ظతͳόʔετɺظతͳτϨϯυ w YYΤϥʔͷ ࢹϧʔϧ
͓ΘΓʹ
w ࣌ܥྻσʔλཻظอଘͷ෦࣮ w ߴස4ॻ͖ࠐΈ w ཹҙࣄ߲ϞχλϦϯά
༗Γ͏͍͟͝·ͨ͠ʂ