Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
AWSのマネージドサービスを活用したログ可視化
Search
Shinji Fujimoto
February 11, 2017
Technology
1
3.3k
AWSのマネージドサービスを活用したログ可視化
2017/02/11 合同勉強会 in 福岡
Shinji Fujimoto
February 11, 2017
Tweet
Share
More Decks by Shinji Fujimoto
See All by Shinji Fujimoto
AWS のサービスを活用して CI/CD #akibaaws
shinjifujimoto
0
18k
Amazon Elasticsearch Service の使いドコロ in 福岡
shinjifujimoto
1
740
Amazon Elasticsearch Service の使いドコロ
shinjifujimoto
3
54k
Amazon Elasticsearch Serviceを利用したAWSのログ活用 /amazones-aws-integration
shinjifujimoto
3
2.7k
はじめてのIoT
shinjifujimoto
0
5.1k
Beats開発の始め方 #cmdevio
shinjifujimoto
2
2.4k
Other Decks in Technology
See All in Technology
Findy Team+のSOC2取得までの道のり
rvirus0817
0
310
AWSにおけるTrend Vision Oneの効果について
shimak
0
120
PLaMoの事後学習を支える技術 / PFN LLMセミナー
pfn
PRO
9
3.7k
Sidekiq その前に:Webアプリケーションにおける非同期ジョブ設計原則
morihirok
17
7.2k
about #74462 go/token#FileSet
tomtwinkle
1
280
Flaky Testへの現実解をGoのプロポーザルから考える | Go Conference 2025
upamune
1
400
SOC2取得の全体像
shonansurvivors
1
350
SoccerNet GSRの紹介と技術応用:選手視点映像を提供するサッカー作戦盤ツール
mixi_engineers
PRO
1
160
Green Tea Garbage Collector の今
zchee
PRO
2
380
stupid jj tricks
indirect
0
7.8k
BirdCLEF+2025 Noir 5位解法紹介
myso
0
190
SwiftUIのGeometryReaderとScrollViewを基礎から応用まで学び直す:設計と活用事例
fumiyasac0921
0
130
Featured
See All Featured
The Success of Rails: Ensuring Growth for the Next 100 Years
eileencodes
46
7.6k
Scaling GitHub
holman
463
140k
Writing Fast Ruby
sferik
629
62k
JavaScript: Past, Present, and Future - NDC Porto 2020
reverentgeek
52
5.6k
A better future with KSS
kneath
239
17k
Documentation Writing (for coders)
carmenintech
75
5k
Stop Working from a Prison Cell
hatefulcrawdad
271
21k
A designer walks into a library…
pauljervisheath
209
24k
The Pragmatic Product Professional
lauravandoore
36
6.9k
The Art of Programming - Codeland 2020
erikaheidi
56
14k
Visualization
eitanlees
148
16k
Unsuck your backbone
ammeep
671
58k
Transcript
AWS ͷ αʔϏεΛ׆༻ͨ͠ ϩάՄࢹԽ ߹ಉษڧձ in Ԭ 2017/02/11 ౻ຊ ਅ࢘
ࣗݾհ • ౻ຊ ਅ࢘ • Work AWS Solutions Architect @Classmethod,
Inc. • Interest Elastic Stack
ձࣾհ
ձࣾհ "84ϞόΠϧɺϏοάσʔλͳͲ ࣾһ૯ग़Ͱ͖ͳΑ͏ʹϒϩάॻ͍͍ͯ·͢
AWSࣄۀ෦հ
AWSࣄۀ෦հ ओʹ"84ʹؔ͢Δϒϩάॻ͍͍ͯ·͢
ຊ͍͑ͨ͜ͱ
ϩάͷՄࢹԽ͕ ؆୯ʹͳΓ·ͨ͠
ΞδΣϯμ • ϩάՄࢹԽͬͯʁ • AWSͷαʔϏεΛ׆༻ͨ͠ϩάՄࢹԽ • Demo
ϩάՄࢹԽͬͯʁ
Έͳ͞Μ ϩάΛ׆༻͍ͯ͠·͔͢ʁ
ϩάͬͯԿ͚ͩͬʁ • ޠݯ • ߤւͷધͷΛଌΔͨΊʹ ؙଠ(log)ʹҰఆִؒͷ͋ΔೄΛׅΓ͚ ͨͷ ʢग़యɿWikipediaʣ
ԿͰϩά͕ඞཁʁ • ࢹɺΞϥʔςΟϯά • োௐࠪ • ύϑΥʔϚϯεղੳ • σʔλੳ
ԿͰϩάͷՄࢹԽ͕ඞཁʁ • มԽͷؾ͖ͮΛಘΔ • োͷ༧ஹɺधཁͷ༧ଌ • આ໌ͷઆಘྗ • จࣈࣈΑΓҰྎવ
ϩάՄࢹԽͷ՝
ϩάՄࢹԽͷ՝ • δϣϒεέδϡʔϦϯά • ETL • ՄࢹԽπʔϧͷબఆ
ϩάՄࢹԽͷ՝ • δϣϒεέδϡʔϥ • ϦΞϧλΠϜͳͷ͔ɺόονͳͷ͔ • ޭɺࣦഊͷఆ • ϦτϥΠॲཧ
ϩάՄࢹԽͷ՝ • Extract • Ͳ͔͜ΒϩάΛऔಘ͢Δ͔ • Transform • ՄࢹԽ͍ͨ͠ใɺܗࣜͷม •
Load • ՄࢹԽπʔϧͷσʔλೖ
ϩάՄࢹԽͷ՝ • ՄࢹԽπʔϧͷબఆ • Kibana • Grafana • re:Dash •
Superset • Google Data Studio • etc.. etc..
AWS ʹ͓͚ΔϩάՄࢹԽ
Amazon Elasticsearch Service
Amazon Elasticsearch Service • Elasticsearch • ݕࡧʗੳΤϯδϯ • Kibana •
ՄࢹԽɺμογϡϘʔυԽ • ४ϦΞϧλΠϜʹՄࢹԽ
Amazon Elasticsearch Service • Elasticsearch + Kibana ͷϚωʔδυαʔϏε • ΫϦοΫͰڥߏங
• ӡ༻ AWS ʹ͓ͤ • ࣍εφοϓγϣοτͷऔಘ • ো࣌ͷ෮چରԠ
Amazon Elasticsearch Service Lambda S3 EC2 Elasticsearch Service
Amazon Elasticsearch Service • 2017/01/31 Ξοϓσʔτ • Elasticsearch / Kibana
5 ܥʹରԠ • εΫϦϓτͷར༻Մ • Ingest Node • Kibana ͷ DevTools Timelion Λར༻Մ
Amazon Athena & Amazon QuickSight
͜ͷαʔϏε͍ͬͯ·͔͢ʁ Amazon Athena Amazon QuickSight
Amazon Athena • AWS re:Invent 2016 ൃදˍϦϦʔε • S3 ͷඇߏԽσʔλʹ
SQL ͰߏԽσʔλ ΛऔΓग़͢
Amazon Athena • αʔόʔϨε • ӡ༻ෆཁɻ͍͍ͨ࣌ʹ͑Δ • ߴͳύϑΥʔϚϯε • 1000ສ݅ͷςΩετͷूܭ͕ඵ
• ίετ • εΩϟϯରαΠζ՝ۚɹ5$ / TB
Amazon QuickSight • AWS re:Invent 2015 ൃදɻ2016/11ϦϦʔε • AWS ۘ
BI πʔϧ
Amazon QuickSight • αʔόʔϨε • SPICE ʹΑΔߴͳՄࢹԽ • ଟ͘ͷ AWS
αʔϏεʹରԠ • ίετͳ BI πʔϧ
Amazon Athena & QuickSight
͍ॲ ʢ˞ϩάՄࢹԽͷ؍ʣ
Ϣʔεέʔε • Elasticsearch Service • χΞϦΞϧλΠϜͳଟ࣍ݩੳ • Athena & QuickSight
• ΞυϗοΫͳੳ
ར༻ίετ • Elasticsearch Service • ElasticsearchΫϥελͷىಈ࣌ؒ • ྫ͑ɺr3.large 50GBσΟεΫ x
3 ౦ژϦʔδϣϯͷ߹ɺ$659 / ݄
ར༻ίετ • Athena & QuickSight • Athena ΫΤϦͨ͠ϑΝΠϧαΠζ • QuickSight
ϢʔβʔˍՃσΟεΫ • ྫ͑ɺ5Ϣʔβʔ 10TB εΩϟϯ $95 / ݄
ͦͷଞίετ • Elasticsearch Service • Elasticsearch ϩάΛೖΕΔͨΊͷΈ ͕ඞཁ • Kibana
ॳݟͰ৮Δʹਏ͍ • Athena & QuickSight • Athena SQL ͕৮ΕΕ͑ͨ • QuickSight ॳݟͰ৮Εͨ
Demo
·ͱΊ • ϩάͬͯຯ͚ͩͲେࣄ • ϩάՄࢹԽͷબࢶ͕Ճ͞Εͨ • ༻్ʹ͋ͬͨՄࢹԽΛબ • ՄࢹԽָ͍ͬͯ͠