Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
AWSで始めるビッグデータ分析 Developers.IO 2017 WORLD in 名古屋
Search
Yutaka Yawata
September 28, 2017
Technology
1
160
AWSで始めるビッグデータ分析 Developers.IO 2017 WORLD in 名古屋
Developers.IO 2017 WORLD in 名古屋 登壇資料
Yutaka Yawata
September 28, 2017
Tweet
Share
More Decks by Yutaka Yawata
See All by Yutaka Yawata
AWSで始めるビッグデータ分析 Developers.IO 2017
yuyawata
1
3.1k
AWSで作るオムニチャネル戦略実現のためのマーケティング・プラットフォーム
yuyawata
1
2k
Other Decks in Technology
See All in Technology
Кто отправит outbox? Валентин Удальцов, автор канала Пых
lamodatech
0
340
AIエージェント最前線! Amazon Bedrock、Amazon Q、そしてMCPを使いこなそう
minorun365
PRO
14
5.1k
米国国防総省のDevSecOpsライフサイクルをAWSのセキュリティサービスとOSSで実現
syoshie
2
1.1k
変化する開発、進化する体系時代に適応するソフトウェアエンジニアの知識と考え方(JaSST'25 Kansai)
mizunori
1
210
Navigation3でViewModelにデータを渡す方法
mikanichinose
0
220
~宇宙最速~2025年AWS Summit レポート
satodesu
1
1.8k
Oracle Cloud Infrastructure:2025年6月度サービス・アップデート
oracle4engineer
PRO
2
240
2年でここまで成長!AWSで育てたAI Slack botの軌跡
iwamot
PRO
4
700
SalesforceArchitectGroupOsaka#20_CNX'25_Report
atomica7sei
0
170
Prox Industries株式会社 会社紹介資料
proxindustries
0
290
Wasm元年
askua
0
140
エンジニア向け技術スタック情報
kauche
1
250
Featured
See All Featured
The Illustrated Children's Guide to Kubernetes
chrisshort
48
50k
Practical Orchestrator
shlominoach
188
11k
Building a Modern Day E-commerce SEO Strategy
aleyda
42
7.3k
The World Runs on Bad Software
bkeepers
PRO
69
11k
Keith and Marios Guide to Fast Websites
keithpitt
411
22k
Product Roadmaps are Hard
iamctodd
PRO
54
11k
A Modern Web Designer's Workflow
chriscoyier
694
190k
It's Worth the Effort
3n
185
28k
Cheating the UX When There Is Nothing More to Optimize - PixelPioneers
stephaniewalter
281
13k
Making Projects Easy
brettharned
116
6.3k
Optimizing for Happiness
mojombo
379
70k
個人開発の失敗を避けるイケてる考え方 / tips for indie hackers
panda_program
107
19k
Transcript
"84Ͱ࢝ΊΔϏοάσʔλੳ %FWFMPQFST*0803-%JO໊ݹ ݄ Ϋϥεϝιουגࣜձࣾീ഼๛ DNEFWJP
ࣗݾհ wീ഼๛ʢ:VUBLB:BXBUBʣ wϓϦηʔϧε୲ wݩ&31ύοέʔδͷج൫ίϯαϧ ࣗݾհ
͜ͷηογϣϯʹ͍ͭͯ ύΠϓϥΠϯͰߟ͑Δσʔλੳج൫ɹ σʔλੳϦϑΝϨϯεΞʔΩςΫνϟ
ηογϣϯͷରऀ w"84ͰσʔλੳΛ࢝Ί͍ͨ w"84Ͱσʔλੳج൫Λߏங͢ΔͨΊͷ ϕετϓϥΫςΟεΛΓ͍ͨ w"84ͷར༻ܦݧ͕͋Δ 71$ &$ 3%4 4 FUD
͜ͷηογϣϯͰ͓͠͠ͳ͍͜ͱ wͳͥ"84͔ʁ w֤"84αʔϏεͷৄࡉͳػೳεϖοΫ wػցֶशɺ"*
͜ͷηογϣϯʹ͍ͭͯ ύΠϓϥΠϯͰߟ͑Δσʔλੳج൫ɹ σʔλੳϦϑΝϨϯεΞʔΩςΫνϟ
༷ʑͳϏοάσʔλ ɨ 104σʔλ ϞόΠϧΞϓϦ ߦಈϩά ΫϦοΫ ετϦʔϜ 4/4
ηϯλʔσʔλ ɓ
σʔλੳͰར༻Ͱ͖Δ"84αʔϏε Amazon DynamoDB Amazon RDS Amazon Redshift Amazon S3
AWS Snowball AWS Lambda AWS Batch Amazon EC2 AWS DMS Amazon ES Amazon EMR Amazon Kinesis Amazon Athena Amazon CloudSearch Amazon QuickSight AWS Data Pipeline Amazon Machine Learning AWS IoT ଟछଟ༷ͳαʔϏε܈ σʔλҠߦ σʔλϕʔε ετϨʔδ ੳ ݕࡧ *P5 ίϯϐϡʔςΟϯά ػցֶश AWS Glue
ͲͷαʔϏεΛ͏͖͔ʁ Ͳ͏ͬͯ"84ʹσʔλΛΞοϓϩʔυ͢Δʁ σʔλͷՃͲ͜ͰΔʁ&$ʁ 3FETIJGUͬͨํ͕Α͍ʁ3%4Ͱେৎʁ
σʔλੳΞʔΩςΫνϟͷݪଇ wσʔλੳΛύΠϓϥΠϯͰߟ͑Δ w ऩूɺલॲཧɺอଘɺੳɺՄࢹԽ w"84ͷ֤αʔϏε͕Կʹ࠷దԽ͞Ε͍ͯΔ͔ʁ·ͨ ࠷దԽ͞Εͨ݁ՌͷτϨʔυΦϑԿ͔ʁΛཧղ͢Δ w εϧʔϓοτɺϨΠςϯγʔɺεέʔϥϏϦςΟɺ ߹ੑɺѻ͑ΔσʔλλΠϓʗσʔλߏɺ
ΧελϚΠζੑɺίετɺ্ݶ w֤αʔϏεͷಛΛཧղ্ͨ͠ͰɺύΠϓϥΠϯ্ͷ ͦΕͧΕͷδϣϒʹ࠷దͳαʔϏεΛબ͢Δ
ϏοάσʔλύΠϓϥΠϯ લॲཧ ऩू อଘ ੳ ՄࢹԽ |
ϏοάσʔλύΠϓϥΠϯอଘ લॲཧ ऩू อଘ ੳ ՄࢹԽ |
σʔλ4ʹू Amazon S3 Amazon EC2 AWS Aurora Amazon Redshift AWS
Lambda Amazon ES Amazon EMR Amazon CloudSearch Amazon QuickSight Amazon Athena AWS IoT Amazon Kinesis Streams Amazon Kinesis Firehose AWS DMS AWS Batch AWS DynamoDB
ϏοάσʔλύΠϓϥΠϯऩू લॲཧ ऩू อଘ ੳ ՄࢹԽ |
ऩू ϑΝΠϧ ετϦʔϜ w ച্σʔλ w ސ٬Ϛελ w Ϛελ w
ߦಈϩά w ΫϦοΫετϦʔϜ w *P5ηϯαʔσʔλ ɏ σʔλλΠϓʹண AWS IoT Amazon S3 Amazon Kinesis Streams Amazon Kinesis Firehose … … …
ϏοάσʔλύΠϓϥΠϯલॲཧ લॲཧ ऩू อଘ ੳ ՄࢹԽ |
લॲཧ ϦΞϧλΠϜ όον w ࣌ؒຖɺ࣍ɺि࣍ɺ w εϧʔϓοτॏࢹ w ஞ࣍ॲཧ w
ϨΠςϯγʔॏࢹ Amazon EMR Amazon EC2 AWS Lambda Amazon Kinesis Analytics ɏ ϫʔΫϩʔυʹண ƽ
ϏοάσʔλύΠϓϥΠϯੳ લॲཧ ऩू อଘ ੳ ՄࢹԽ |
ੳ σʔλߏ ΞΫηεύλʔϯ σʔλετΞ ੳ ݻఆ͞ΕͨεΩʔϚ ෳࡶͳϦϨʔγϣφϧΫΤϦɺ ଟͷϢʔβʔ͔Βͷಉ࣌ΞΫηε 3%4 େنσʔλͷूܭɾநग़
3FETIJGU 4 3FETIJGU4QFDUSVN εΩʔϚϨε γϯϓϧͳΫΤϦɺ͍ϨΠςϯγʔ %ZOBNP%# ੜσʔλʹର͢ΔΞυϗοΫੳ 4 &.3 1SFTUP *NQBMB 4QBSL "UIFOB "UIFOB શจݕࡧ &MBTUJD4FBSDI4FSWJDF $MPVE4FBSDI ɏ σʔλߏɺΞΫηεύλʔϯʹண
ϏοάσʔλύΠϓϥΠϯՄࢹԽ લॲཧ ऩू อଘ ੳ ՄࢹԽ |
ՄࢹԽ ఏڙํ๏ ՄࢹԽπʔϧ ηϧϑ#* "NB[PO2VJDL4JHIU ఆܕϨϙʔτ ΧελϜ8FCΞϓϦέʔγϣϯ w %ZOBNP%# -BNCEB
"1*(BUFXBZ 4 w 3%4 &$ ϩάੳͷμογϡϘʔυ "NB[PO&MBTUJD4FBSDI4FSWJDF ,JCBOB ɏ σʔλͷఏڙํ๏ʹண͢Δ w "84αʔϏεͷબࢶগͳ͍αʔυύʔςΟͷ πʔϧݕ౼ ǟ ǟ
͜ͷηογϣϯʹ͍ͭͯ ύΠϓϥΠϯͰߟ͑Δσʔλੳج൫ɹ σʔλੳϦϑΝϨϯεΞʔΩςΫνϟ
ϦϑΝϨϯεΞʔΩςΫνϟ wǡ ച্ੳ wΦϯϓϨϛεͷ%#ʹ͋Δച্σʔλ wҎԼͷੳڥΛఏڙ wੜσʔλʹର͢ΔΞυϗοΫੳ wηϧϑ#* wఆܕϨϙʔτ
ച্ੳσʔλऩू ੜσʔλ σʔλ4ʹू σʔλΞοϓϩʔυ Internet ੳ୲ऀ (ਓʙेਓ)
ച্ੳੜσʔλͷΞυϗοΫੳ Amazon Athena ੜσʔλ σʔλΞοϓϩʔυ ΞυϗοΫੳ "UIFOBͰ ΞυϗοΫੳ ǟ
Internet ੳ୲ऀ (ਓʙेਓ)
ച্ੳσʔλมɺܗ Amazon Athena ੜσʔλ ՃࡁΈσʔλ Amazon EMR σʔλΞοϓϩʔυ ΞυϗοΫੳ
σʔλมɺܗ &.3Ͱσʔλมɺܗ ؆қͳॲཧͰ͋Ε&$Ͱࡁ·ͤΔ͜ͱ ǟ Internet ੳ୲ऀ (ਓʙेਓ)
ച্ੳ3FETIJGUऔࠐΈ Amazon Athena ੜσʔλ ՃࡁΈσʔλ Amazon Redshift όοναʔόʔ Amazon
EMR σʔλϩʔυ ίϚϯυ࣮ߦ σʔλΞοϓϩʔυ ΞυϗοΫੳ σʔλมɺܗ όοναʔόʔͰ $01:ίϚϯυΛ࣮ߦ ǟ Internet ੳ୲ऀ (ਓʙेਓ)
ച্ੳ3FETIJGUऔࠐΈ Amazon Athena ੜσʔλ ՃࡁΈσʔλ Amazon Redshift όοναʔόʔ Amazon
EMR σʔλϩʔυ ίϚϯυ࣮ߦ σʔλΞοϓϩʔυ ΞυϗοΫੳ σʔλมɺܗ σΟϝϯγϣϯ ςʔϒϧ 3FETIJGU 4QFDUSVN ϑΝΫτςʔϒϧ4ʹɻ 4QFDUSVNͰΞΫηε ǟ Internet ੳ୲ऀ (ਓʙेਓ)
ച্ੳ#*πʔϧͰੳɺՄࢹԽ Amazon Athena ੜσʔλ ՃࡁΈσʔλ Amazon QuickSight Amazon Redshift
όοναʔόʔ Amazon EMR σʔλϩʔυ ίϚϯυ࣮ߦ σʔλΞοϓϩʔυ ΞυϗοΫੳ σʔλมɺܗ 4QFDUSVN 2VJDL4JHIUͰՄࢹԽɻ ͦͷଞ#*πʔϧ༻Մೳ ՄࢹԽ ǟ Internet ੳ୲ऀ (ਓʙेਓ)
ച্ੳϨϙʔτఏڙ Amazon Athena ੜσʔλ ՃࡁΈσʔλ Amazon Redshift όοναʔόʔ Amazon
EMR 4QFDUSVN Amazon RDS for PostgreSQL dblink NBUFSJBMJ[FEWJFX Ϩϙʔτදࣔ αʔόʔ ੳ୲ऀ (ਓʙेਓ) σʔλϩʔυ ίϚϯυ࣮ߦ σʔλΞοϓϩʔυ ΞυϗοΫੳ σʔλมɺܗ ϨϙʔτӾཡऀ (100ਓʙ) Amazon QuickSight ǟ ՄࢹԽ Internet Internet σʔλϚʔτΛ3%4ʹอଘ͠ɺ ϨϙʔτӾཡऀʹ։์
ച্ੳશମ૾ Amazon Athena ੜσʔλ ՃࡁΈσʔλ Amazon Redshift όοναʔόʔ Amazon
EMR 4QFDUSVN Amazon RDS for PostgreSQL dblink NBUFSJBMJ[FEWJFX Ϩϙʔτදࣔ αʔόʔ ੳ୲ऀ (ਓʙेਓ) σʔλϩʔυ ίϚϯυ࣮ߦ σʔλΞοϓϩʔυ ΞυϗοΫੳ σʔλมɺܗ ϨϙʔτӾཡऀ (100ਓʙ) Amazon QuickSight ǟ ՄࢹԽ Internet Internet
ϦϑΝϨϯεΞʔΩςΫνϟ w | 8FCථγεςϜ w8FC͔ΒථσʔλΛड͚͚ wථσʔλΛɺ wϦΞϧλΠϜʹूܭ wޙ#*πʔϧͰੳ
8FCථγεςϜථσʔλड ୲ऀ Amazon Kinesis Streams Internet ථσʔλड Internet "1*(BUFXBZΛϓϩΩγʹͯ͠
,JOFTJT4USFBNʹථσʔλΛొ Amazon API Gateway Amazon CloudFront ථϖʔδ4 $MPVE'SPOUͰ৴ Amazon S3 ථը໘
8FCථγεςϜ%#ͷొ ୲ऀ Amazon Kinesis Streams Amazon DynamoDB AWS Lambda
ථσʔλड ථσʔλొ Internet ථσʔλ -BNCEBͰ%ZOBNP%#ʹ ථσʔλΛొ Internet Amazon CloudFront ථσʔλड Amazon S3 Amazon API Gateway ථը໘
8FCථγεςϜϦΞϧλΠϜूܭ ୲ऀ Amazon Kinesis Streams Amazon Kinesis Analytics Amazon
Kinesis Streams AWS Lambda Amazon DynamoDB AWS Lambda ථσʔλड ථσʔλूܭ αϚϦσʔλొ ථσʔλొ Internet αϚϦσʔλ ථσʔλ ूܭͨ͠αϚϦσʔλΛ %ZOBNP%#ʹొ Internet Amazon CloudFront Amazon API Gateway Amazon S3 ථը໘ ,JOFTJT"OBMZUJDTͰ ථσʔλΛϦΞϧλΠϜूܭ
8FCථγεςϜΞʔΧΠϒ ୲ऀ Amazon Kinesis Streams Amazon DynamoDB AWS Lambda
ථσʔλड ථσʔλొ Amazon Kinesis Firehose S3 Internet αϚϦσʔλ ථσʔλ Internet Amazon CloudFront Amazon API Gateway Amazon S3 ථը໘ શථσʔλ,JOFTJT'JSFIPTFܦ༝Ͱ 4ʹΞʔΧΠϒ Amazon Kinesis Analytics Amazon Kinesis Streams ථσʔλूܭ AWS Lambda αϚϦσʔλొ
8FCථγεςϜσʔλࢀর ୲ऀ ౦ژϦʔδϣϯ Amazon Kinesis Streams Amazon DynamoDB AWS
Lambda ථσʔλड ථσʔλొ Amazon Kinesis Firehose S3 Internet αϚϦσʔλ ථσʔλ AWS Lambda %ZOBNP%# 4USFBN ථσʔλΛϦΞϧλΠϜʹࢀর Internet Amazon API Gateway Amazon CloudFront ूܭ݁Ռ4ʹΞοϓϩʔυ͠ $MPVE'SPOU͔Β৴ Amazon S3 ථը໘ Amazon Kinesis Analytics Amazon Kinesis Streams ථσʔλूܭ AWS Lambda αϚϦσʔλొ Amazon S3 ථཧը໘
8FCථγεςϜશମ૾ ୲ऀ ౦ژϦʔδϣϯ Amazon Kinesis Streams Amazon DynamoDB AWS
Lambda ථσʔλड ථσʔλొ Amazon Kinesis Firehose S3 Internet αϚϦσʔλ ථσʔλ AWS Lambda %ZOBNP%# 4USFBN Internet Amazon API Gateway Amazon CloudFront Amazon S3 ථཧը໘ Amazon S3 ථը໘ Amazon Kinesis Analytics Amazon Kinesis Streams ථσʔλूܭ AWS Lambda αϚϦσʔλొ
·ͱΊ wσʔλੳΛύΠϓϥΠϯͰߟ͑Δ w"84ͷ֤αʔϏε͕Կʹ࠷దԽ͞Ε͍ͯΔ͔ʁ·ͨ ࠷దԽ͞Εͨ݁ՌͷτϨʔυΦϑԿ͔ʁΛཧղ͢Δ w֤αʔϏεͷಛΛཧղ্ͨ͠ͰɺύΠϓϥΠϯ্ͷ ͦΕͧΕͷδϣϒʹ࠷దͳαʔϏεΛબ͢Δ
׀幠耮ָ֮הֲ׀ְׂת׃կ
None