Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
やらかした話 - ログ集約編
Search
uraura
January 21, 2021
Technology
0
690
やらかした話 - ログ集約編
AWS Startup Tech Meetup Online#1
uraura
January 21, 2021
Tweet
Share
Other Decks in Technology
See All in Technology
MapLibreとAmazon Location Service
dayjournal
1
150
Azure Container Apps + Bicep 〜 こんな感じで運用しています
kaz29
2
450
どうするコスト最適化のトレードオフ
tetsuyaooooo
1
500
プロデザ! BY リクルート vol.18_リクルートのリサーチ実践組織「リサーチブーストコミュニティ」
recruitengineers
PRO
3
270
最近たまに見かけるTiDBってなんだ? - Findy
pingcap0315
2
760
反実仮想機械学習とは何か
usaito
PRO
11
3.8k
継続的な改善 x ⾮連続的な進化
sansantech
PRO
3
140
Oracle Cloud Infrastructure:2024年4月度サービス・アップデート
oracle4engineer
PRO
1
190
開発パフォーマンスを最大化するための開発体制
ham0215
2
200
[新卒向け研修資料] テスト文字列に「うんこ」と入れるな(2024年版)
infiniteloop_inc
2
9.3k
複雑な構成要素を持つUIとの向き合い方 〜新・支出グラフでの実例〜 / B43 TECH TALK
nakamuuu
0
140
ServiceNow Knowledge 24の歩き方 EYストラテジー・アンド・コンサルティング
manarobot
0
190
Featured
See All Featured
Embracing the Ebb and Flow
colly
80
4.1k
The Brand Is Dead. Long Live the Brand.
mthomps
49
28k
Understanding Cognitive Biases in Performance Measurement
bluesmoon
7
1k
Reflections from 52 weeks, 52 projects
jeffersonlam
345
19k
Fashionably flexible responsive web design (full day workshop)
malarkey
398
65k
No one is an island. Learnings from fostering a developers community.
thoeni
16
2.1k
The Mythical Team-Month
searls
216
42k
Git: the NoSQL Database
bkeepers
PRO
422
63k
Put a Button on it: Removing Barriers to Going Fast.
kastner
58
3k
Building a Scalable Design System with Sketch
lauravandoore
456
32k
Code Reviewing Like a Champion
maltzj
514
39k
Optimizing for Happiness
mojombo
370
69k
Transcript
Β͔ͨ͠ - ϩάूฤ Tomohiro Urakawa @ AWS Startup Tech Meetup
Online#1
@corn3rl0ve • ΫϥελʔגࣜձࣾͰαʔόʔαΠυΤϯδχΞΛ͍ͬͯ·͢ • golangͰΞϓϦΛॻ͍ͨΓɼTerraformͰAWSʑͷϦιʔεཧͯͨ͠Γɼ ࠷ۙϞχλϦϯά·ΘΓʹڵຯ͕͋Γ·͢
cluster ͱ • PC/VR/εϚϗ͔ΒࢀՃͰ͖Δόʔνϟϧ SNS • ษڧձϥΠϒͳͲͷΠϕϯτ͕Ͱ͖ͨ Γɼࣗ࡞ήʔϜΛ࡞ͬͯެ։Ͱ͖ͨΓ • https://resources.awscloud.com/aws-
summit-online-japan-2020-on-demand- industry-4-10523/cus-64-aws-summit- online-2020-cluster-inc ͰͥΜͿ͠ΌΓ· ͨ͠
clusterͷαʔόʔαΠυ֓ཁ ग़ల: https://resources.awscloud.com/aws-summit-online-japan-2020-on-demand-industry-3-12034/cus-64-aws-summit-online-2020-cluster-inc
clusterͷαʔόʔαΠυ֓ཁ • ECS • HTTP API Server • ଓதͷϢʔβʔΛཧ͢ΔͨΊͷdaemonతͳαʔϏε •
ఆ࣌ͷόονδϣϒ (scheduled task) • EC2 • ۭؒͷମݧΛఏڙ͢ΔͨΊͷMQTTαʔόʔ • ۭؒͰඞཁͳใΛੜʗૹड৴͢ΔͨΊͷಠࣗαʔϏε • Lambda
clusterͷαʔόʔαΠυ֓ཁ • ECS • HTTP API Server • ଓதͷϢʔβʔΛཧ͢ΔͨΊͷdaemonతͳαʔϏε •
ఆ࣌ͷόονδϣϒ (scheduled task) • EC2 • roomͷମݧΛఏڙ͢ΔͨΊͷMQTTαʔόʔ • roomͰඞཁͳใΛੜʗૹड৴͢ΔͨΊͷಠࣗαʔϏε • Lambda CloudWatch Logs
clusterͷαʔόʔαΠυ֓ཁ • ECS • HTTP API Server • ଓதͷϢʔβʔΛཧ͢ΔͨΊͷdaemonతͳαʔϏε •
ఆ࣌ͷόονδϣϒ (scheduled task) • EC2 • roomͷମݧΛఏڙ͢ΔͨΊͷMQTTαʔόʔ • roomͰඞཁͳใΛੜʗૹड৴͢ΔͨΊͷಠࣗαʔϏε • Lambda CloudWatch Logs ΄΅ಉ͡ߏͰEFWTUHQSE ڥ "84ΞΧϯτ ଘࡏ
αʔόʔͷϩάΛؾܰʹݟΕΔͱௐࠪ ͕ḿͬͯศར͚ͩͲɽɽɽ
CloudWatch Logs͍͠ • ·ͣAWSͷϢʔβʔΞΧϯτ͕ඞཁ • ڥ͝ͱʹΞΧϯτผͳͷͰswitch-role͕ඞཁ • log-group, log-stream ͱɽɽɽ?
• f i lter͕͍͠ (׳Ε͕ඞཁ • Insights͕͍͠ (׳Ε͕ඞཁ
ϩάΛݟ͍͚ͨͩͳͷʹ ߟ͑Δ͜ͱ͕ଟ͗͢Δ
Elasticsearch ServiceʹೖΕͨ • Kibana͕ศར • dev͔Β͡Ίͯstg/prdల։ CloudWatch Logs Elasticsearch Service
Subscription f i lter
҆͗ͨ͢ • ར༻͢Δͷ99%prd༻ɼ͕ى͜Δͷ99%dev༻ • prd༻Ҏ֎ීஈ͋·Γར༻͠ͳ͍͕ɼ͕ى͖ͨΒରԠ͕ඞཁ • ઃఆมߋ͕͋Εશ෦ʹద༻͠ͳ͍ͱ͍͚ͳ͍ • ܽଛͨ͠ͱ͖ͷิ͕͍͠ •
ϩά͕ݟ͔ͭΒͳ͍? ͱࢥͬͨΒผڥͷKibanaΛݟ͍ͯͨ
Elasticsearch ServiceΛ1ͭʹ͠Α͏
1ͭͷElasticsearch Serviceʹूͨ͠ • ్தʹS3ΛڬΉ͜ͱͰηʔϒϙΠϯτ͕Ͱ͖ͨ • Lambdaࣦഊ࣌ʹϦτϥΠͯ͘͠ΕΔͷͰ΄΅ख์͠ʹͳͬͨ dev Elasticsearch Service Kinesis
Streams Kinesis Firehose S3 Output Lambda Event Noti f i cation Bulk API stg prd Subscription f i lter
ΊͰͨ͠
ͱ͍͔ͳ͔ͬͨ
͠Βͨͬͨ͋͘Δɽɽɽ • Domain status͕ Upgrade processing ͷ··ԿͰ͖ͳ͍ঢ়ଶʹ • • clusterͷstatusਅͬ
(※ը૾ࡱΓΕ) • Lambdaશ෦ࣦഊͯ͠ϦτϥΠ͍ͯͨ͠ͷͰɼݺग़ճʗॲཧ࣌ؒʗ ϩάग़ྗྔ͕͍͢͜͝ͱʹ • →݁ہ͜ͷDomainআ͢Δ͜ͱʹͳͬͯ͠·͍·ͨ͠
ഊҼ • Elasticsearch ServiceͷෛՙΛແବʹ૿େ͍ͤͯͨ͞ • ରͷlog-groupʹsubscription f i lterΛઃఆ͢Δ͚ͩɼͱ͍͏ঢ়ଶͩͬͨͷ Ͱɼ͋ΒΏΔlog-groupʹઃఆΛ͍Εͯ͠·ͬͨ
• ͋·ΓॏཁͰͳ͍ (ௐࠪʹར༻͠ʹ͍͘) ϩάେྔʹElasticsearch Serviceʹอଘ͞ΕΔ͜ͱʹͳͬͯ͠·ͬͨ • ϦτϥΠͷཞ༻ • ޭ͢ΔݟࠐΈ͕ͳ͍ͷʹϦτϥΠ͠ଓ͚Δঢ়ଶͩͬͨ
·ͱΊ • αʔϏεͷಛੑʗѻ͏σʔλʹ͍͋ͬͯΔ͔ • CloudWatch LogsΑΓElasticsearch Service͕ྑ͍ɼͱ͍͏Ͱͳ ͍ • উखʹϦτϥΠͯ͘͠ΕͯศརʙɼͰ͍͚ͳ͍
• త͕ζϨͯ͠·ͬͯͳ͍͔ • ʮௐࠪΛΓ͍ͨ͘͢͠ʯˠʮू͍ͨ͠ʯʹมΘͬͯ͠·ͬͯ ͍ͨ
͓ΘΓ
https://www.wantedly.com/projects/562252