Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
AWSのマネージドサービスを活用したログ可視化
Search
Shinji Fujimoto
February 11, 2017
Technology
1
3.3k
AWSのマネージドサービスを活用したログ可視化
2017/02/11 合同勉強会 in 福岡
Shinji Fujimoto
February 11, 2017
Tweet
Share
More Decks by Shinji Fujimoto
See All by Shinji Fujimoto
AWS のサービスを活用して CI/CD #akibaaws
shinjifujimoto
0
18k
Amazon Elasticsearch Service の使いドコロ in 福岡
shinjifujimoto
1
740
Amazon Elasticsearch Service の使いドコロ
shinjifujimoto
3
54k
Amazon Elasticsearch Serviceを利用したAWSのログ活用 /amazones-aws-integration
shinjifujimoto
3
2.7k
はじめてのIoT
shinjifujimoto
0
5.1k
Beats開発の始め方 #cmdevio
shinjifujimoto
2
2.4k
Other Decks in Technology
See All in Technology
Building a cloud native business on open source
lizrice
0
170
現場データから見える、開発生産性の変化コード生成AI導入・運用のリアル〜 / Changes in Development Productivity and Operational Challenges Following the Introduction of Code Generation AI
nttcom
1
440
Implementing and Evaluating a High-Level Language with WasmGC and the Wasm Component Model: Scala’s Case
tanishiking
0
170
CNCFの視点で捉えるPlatform Engineering - 最新動向と展望 / Platform Engineering from the CNCF Perspective
hhiroshell
0
130
Claude Code Subagents 再入門 ~cc-sddの実装で学んだこと~
gotalab555
10
17k
Microsoft 365 の認証と承認を理解する / Understanding Microsoft 365 Authentication and Authorization
karamem0
0
110
ヘンリー会社紹介資料(エンジニア向け) / company deck for engineer
henryofficial
0
320
[OCI Skill Mapping] AWSユーザーのためのOCI – IaaS編(Compute/Storage/Networking) (2025年10月8日開催)
oracle4engineer
PRO
1
180
データ戦略部門 紹介資料
sansan33
PRO
1
3.8k
RDS の負荷が高い場合に AWS で取りうる具体策 N 連発/a-series-of-specific-countermeasures-available-on-aws-when-rds-is-under-high-load
emiki
7
4.5k
AWS UG Grantでグローバル20名に選出されてre:Inventに行く話と、マルチクラウドセキュリティの教科書を執筆した話 / The Story of Being Selected for the AWS UG Grant to Attending re:Invent, and Writing a Multi-Cloud Security Textbook
yuj1osm
1
120
AIフル活用で挑む!空間アプリ開発のリアル
taat
0
140
Featured
See All Featured
Building a Modern Day E-commerce SEO Strategy
aleyda
44
7.8k
10 Git Anti Patterns You Should be Aware of
lemiorhan
PRO
657
61k
The Art of Delivering Value - GDevCon NA Keynote
reverentgeek
16
1.7k
Practical Orchestrator
shlominoach
190
11k
GraphQLとの向き合い方2022年版
quramy
49
14k
Optimizing for Happiness
mojombo
379
70k
Responsive Adventures: Dirty Tricks From The Dark Corners of Front-End
smashingmag
253
22k
BBQ
matthewcrist
89
9.8k
A better future with KSS
kneath
239
18k
Refactoring Trust on Your Teams (GOTO; Chicago 2020)
rmw
35
3.2k
Fight the Zombie Pattern Library - RWD Summit 2016
marcelosomers
234
17k
Site-Speed That Sticks
csswizardry
13
920
Transcript
AWS ͷ αʔϏεΛ׆༻ͨ͠ ϩάՄࢹԽ ߹ಉษڧձ in Ԭ 2017/02/11 ౻ຊ ਅ࢘
ࣗݾհ • ౻ຊ ਅ࢘ • Work AWS Solutions Architect @Classmethod,
Inc. • Interest Elastic Stack
ձࣾհ
ձࣾհ "84ϞόΠϧɺϏοάσʔλͳͲ ࣾһ૯ग़Ͱ͖ͳΑ͏ʹϒϩάॻ͍͍ͯ·͢
AWSࣄۀ෦հ
AWSࣄۀ෦հ ओʹ"84ʹؔ͢Δϒϩάॻ͍͍ͯ·͢
ຊ͍͑ͨ͜ͱ
ϩάͷՄࢹԽ͕ ؆୯ʹͳΓ·ͨ͠
ΞδΣϯμ • ϩάՄࢹԽͬͯʁ • AWSͷαʔϏεΛ׆༻ͨ͠ϩάՄࢹԽ • Demo
ϩάՄࢹԽͬͯʁ
Έͳ͞Μ ϩάΛ׆༻͍ͯ͠·͔͢ʁ
ϩάͬͯԿ͚ͩͬʁ • ޠݯ • ߤւͷધͷΛଌΔͨΊʹ ؙଠ(log)ʹҰఆִؒͷ͋ΔೄΛׅΓ͚ ͨͷ ʢग़యɿWikipediaʣ
ԿͰϩά͕ඞཁʁ • ࢹɺΞϥʔςΟϯά • োௐࠪ • ύϑΥʔϚϯεղੳ • σʔλੳ
ԿͰϩάͷՄࢹԽ͕ඞཁʁ • มԽͷؾ͖ͮΛಘΔ • োͷ༧ஹɺधཁͷ༧ଌ • આ໌ͷઆಘྗ • จࣈࣈΑΓҰྎવ
ϩάՄࢹԽͷ՝
ϩάՄࢹԽͷ՝ • δϣϒεέδϡʔϦϯά • ETL • ՄࢹԽπʔϧͷબఆ
ϩάՄࢹԽͷ՝ • δϣϒεέδϡʔϥ • ϦΞϧλΠϜͳͷ͔ɺόονͳͷ͔ • ޭɺࣦഊͷఆ • ϦτϥΠॲཧ
ϩάՄࢹԽͷ՝ • Extract • Ͳ͔͜ΒϩάΛऔಘ͢Δ͔ • Transform • ՄࢹԽ͍ͨ͠ใɺܗࣜͷม •
Load • ՄࢹԽπʔϧͷσʔλೖ
ϩάՄࢹԽͷ՝ • ՄࢹԽπʔϧͷબఆ • Kibana • Grafana • re:Dash •
Superset • Google Data Studio • etc.. etc..
AWS ʹ͓͚ΔϩάՄࢹԽ
Amazon Elasticsearch Service
Amazon Elasticsearch Service • Elasticsearch • ݕࡧʗੳΤϯδϯ • Kibana •
ՄࢹԽɺμογϡϘʔυԽ • ४ϦΞϧλΠϜʹՄࢹԽ
Amazon Elasticsearch Service • Elasticsearch + Kibana ͷϚωʔδυαʔϏε • ΫϦοΫͰڥߏங
• ӡ༻ AWS ʹ͓ͤ • ࣍εφοϓγϣοτͷऔಘ • ো࣌ͷ෮چରԠ
Amazon Elasticsearch Service Lambda S3 EC2 Elasticsearch Service
Amazon Elasticsearch Service • 2017/01/31 Ξοϓσʔτ • Elasticsearch / Kibana
5 ܥʹରԠ • εΫϦϓτͷར༻Մ • Ingest Node • Kibana ͷ DevTools Timelion Λར༻Մ
Amazon Athena & Amazon QuickSight
͜ͷαʔϏε͍ͬͯ·͔͢ʁ Amazon Athena Amazon QuickSight
Amazon Athena • AWS re:Invent 2016 ൃදˍϦϦʔε • S3 ͷඇߏԽσʔλʹ
SQL ͰߏԽσʔλ ΛऔΓग़͢
Amazon Athena • αʔόʔϨε • ӡ༻ෆཁɻ͍͍ͨ࣌ʹ͑Δ • ߴͳύϑΥʔϚϯε • 1000ສ݅ͷςΩετͷूܭ͕ඵ
• ίετ • εΩϟϯରαΠζ՝ۚɹ5$ / TB
Amazon QuickSight • AWS re:Invent 2015 ൃදɻ2016/11ϦϦʔε • AWS ۘ
BI πʔϧ
Amazon QuickSight • αʔόʔϨε • SPICE ʹΑΔߴͳՄࢹԽ • ଟ͘ͷ AWS
αʔϏεʹରԠ • ίετͳ BI πʔϧ
Amazon Athena & QuickSight
͍ॲ ʢ˞ϩάՄࢹԽͷ؍ʣ
Ϣʔεέʔε • Elasticsearch Service • χΞϦΞϧλΠϜͳଟ࣍ݩੳ • Athena & QuickSight
• ΞυϗοΫͳੳ
ར༻ίετ • Elasticsearch Service • ElasticsearchΫϥελͷىಈ࣌ؒ • ྫ͑ɺr3.large 50GBσΟεΫ x
3 ౦ژϦʔδϣϯͷ߹ɺ$659 / ݄
ར༻ίετ • Athena & QuickSight • Athena ΫΤϦͨ͠ϑΝΠϧαΠζ • QuickSight
ϢʔβʔˍՃσΟεΫ • ྫ͑ɺ5Ϣʔβʔ 10TB εΩϟϯ $95 / ݄
ͦͷଞίετ • Elasticsearch Service • Elasticsearch ϩάΛೖΕΔͨΊͷΈ ͕ඞཁ • Kibana
ॳݟͰ৮Δʹਏ͍ • Athena & QuickSight • Athena SQL ͕৮ΕΕ͑ͨ • QuickSight ॳݟͰ৮Εͨ
Demo
·ͱΊ • ϩάͬͯຯ͚ͩͲେࣄ • ϩάՄࢹԽͷબࢶ͕Ճ͞Εͨ • ༻్ʹ͋ͬͨՄࢹԽΛબ • ՄࢹԽָ͍ͬͯ͠