Amazon Kinesis + socket.io + D3.jsを使った webブラウザで行うリアルタイム可視化の仕組み/inovationegg-8
by
堀家隆宏
Link
Embed
Share
Beginning
This slide
Copy link URL
Copy link URL
Copy iframe embed code
Copy iframe embed code
Copy javascript embed code
Copy javascript embed code
Share
Tweet
Share
Tweet
Slide 1
Slide 1 text
Amazon Kinesis + socket.io + D3.jsΛͬͨ webϒϥβͰߦ͏ϦΞϧλΠϜՄࢹԽͷΈ @Amimoto_Ami amimoto-ami.com
Slide 2
Slide 2 text
ࣗݾհ • ໊લɿງՈོ • ॴଐɿAMIMOTO TEAM • Githubɿhttps://github.com/horike37 • Qiitaɿhttp://qiita.com/horike37 • ಘҙͳ͜ͱɿAWS, WordPress
Slide 3
Slide 3 text
https://ja.amimoto-ami.com/ AMIMOTO
Slide 4
Slide 4 text
Qiita http://qiita.com/horike37
Slide 5
Slide 5 text
https://github.com/horike37/ Github
Slide 6
Slide 6 text
ࠓ͢͜ͱ
Slide 7
Slide 7 text
AWSͰՄࢹԽΞʔΩςΫνϟΛ࡞Δٕज़
Slide 8
Slide 8 text
ࠓ͞ͳ͍͜ͱ
Slide 9
Slide 9 text
ՄࢹԽͷతɺ֓೦ɺֶ
Slide 10
Slide 10 text
ΞδΣϯμ
Slide 11
Slide 11 text
• TweetՄࢹԽπʔϧʮSocial Stream MonitorʯͷσϞ • Social Stream MonitorΞʔΩςΫνϟͷ • ϦΞϧλΠϜՄࢹԽ+σʔλͷӬଓԽ
Slide 12
Slide 12 text
Social Straem Monitor
Slide 13
Slide 13 text
Social Straem Monitorͱ TwitterͷετϦʔϛϯάσʔλΛ ϦΞϧλΠϜʹՄࢹԽ͢Δπʔϧ
Slide 14
Slide 14 text
https://github.com/megumiteam/social-stream-monitor Github
Slide 15
Slide 15 text
·ͣɺσϞ
Slide 16
Slide 16 text
ΞʔΩςΫνϟ
Slide 17
Slide 17 text
Amazon Kinesis ϦΞϧλΠϜ ετϦʔϛϯάॲཧ
Slide 18
Slide 18 text
ΞʔΩςΫνϟ ετϦʔϛϯά σʔλΛॲཧ
Slide 19
Slide 19 text
Amazon Kinesisͱ • ϑϧϚωʔδυͳϦΞϧλΠϜσʔλॲ ཧαʔϏε • େྔͷετϦʔϛϯάσʔλΛγϟʔ υʹอଘɻόοΫΤϯυͷΞϓϦέʔ γϣϯͰऔΓग़ͯ͠ॲཧ
Slide 20
Slide 20 text
Amazon Kinesis͕ແ͍ͱ͖ ετϦʔϛϯάσʔλ σʔλॲཧܥ
Slide 21
Slide 21 text
Amazon Kinesis͕͋Δ࣌ ετϦʔϛϯάσʔλ σʔλॲཧܥ ॲཧܥ͕ѻ͍͍͢Α͏ʹσʔλΛอଘ
Slide 22
Slide 22 text
Social Stream Metor্ͷׂ • TwitterͷετϦʔϛϯάσʔλΛҰ࣌ తʹอଘͯ͠ΞϓϦέʔγϣϯ͔Βσʔ λͱͯ͠ॲཧ͘͢͢͠Δ
Slide 23
Slide 23 text
Lambda ίʔυͷ࣮ߦ
Slide 24
Slide 24 text
ΞʔΩςΫνϟ Kinesis͔Βड͚ औͬͨσʔλΛ SNSʹ͢
Slide 25
Slide 25 text
Lambdaͱ • AWS ΠϯϑϥετϥΫνϟΛ༻ͯ͠ ίʔυͷ࣮ߦΛߦ͢ΔίϯϐϡʔςΟ ϯάαʔϏε • ΠϕϯτυϦϒϯͳॲཧ͕Մೳ
Slide 26
Slide 26 text
SNS http endpointσʔλΛ͢ σʔλϨίʔυΛSNSʹpublish
Slide 27
Slide 27 text
Social Stream Metor্ͷׂ • webϒϥβͰදࣔ͢ΔͨΊʹhttpϓϩ τίϧͰσʔλΛྲྀ͢ඞཁ͕͋Δ • ͦͷͨΊͷSNS http push௨σʔλ Λ͢
Slide 28
Slide 28 text
ΞʔΩςΫνϟ HTTPΤϯυϙΠϯ τσʔλΛpush
Slide 29
Slide 29 text
SNS HTTP notification HTTPΤϯυϙΠϯτ PUSH௨
Slide 30
Slide 30 text
SNS HTTP notificationͱ • Amazon SNS Λ༻ͯ͠ɺ1 ͭҎ্ͷ HTTPΤϯυϙΠϯτʹ௨ϝοηʔδ Λૹ৴ • HTTP POST ϦΫΤετΛૹ৴͠ɺ௨ ͷ༰Λड৴ొͨ͠ΤϯυϙΠϯ τʹ৴
Slide 31
Slide 31 text
SNS HTTP notificationͱ Topicʹड৴ొ͞ΕͨΤϯυϙΠϯτʹσʔλΛPOST
Slide 32
Slide 32 text
Social Stream Metor্ͷׂ • webϒϥβͰදࣔ͢ΔͨΊʹhttpϓϩτίϧͰ σʔλΛྲྀ͢ඞཁ͕͋Δ • websocketͰϒϥβʹσʔλΛૹ৴͢ΔͨΊͷ socket.ioαʔόʹσʔλΛpush͢Δ
Slide 33
Slide 33 text
socket.io ϦΞϧλΠϜͳ WebΞϓϦ
Slide 34
Slide 34 text
ΞʔΩςΫνϟ WebSocketͰϒϥ βʹσʔλૹ৴
Slide 35
Slide 35 text
socket.io • ͯ͢ͷϒϥβɾϞόΠϧσόΠεͰ ϦΞϧλΠϜ௨৴ΛՄೳͱ͢Δ͜ͱΛ తʹ࡞ΒΕͨJavaScriptϥΠϒϥϦ • WebSocketʹΑΔϒϥβͱαʔόͷ ํ௨৴
Slide 36
Slide 36 text
Social Stream Metor্ͷׂ • SNS͔ΒૹΒΕ͖ͯͨTweetΛWebSocketΛ͍ ϒϥβϦΞϧλΠϜʹૹ৴
Slide 37
Slide 37 text
SNS͔ΒૹΒΕ͖ͯͨσʔλΛड৴ https://github.com/megumiteam/social-stream-monitor/blob/master/app.js
Slide 38
Slide 38 text
ϒϥβσʔλΛૹ৴ https://github.com/megumiteam/social-stream-monitor/blob/master/app.js
Slide 39
Slide 39 text
D3.js σʔλɾυϦϒϯɾ υΩϡϝϯτ
Slide 40
Slide 40 text
ΞʔΩςΫνϟ D3.jsʹΑΔඳը
Slide 41
Slide 41 text
D3.jsͱ • javascriptͷσʔλϏδϡΞϥΠθʔ γϣϯΤϯδϯ • σʔλΛجʹDOMΛૢ࡞ • άϥϑඳըϥΠϒϥϦͰͳ͍ɻD3Λ ϕʔεʹͨ͠άϥϑඳըϥΠϒϥϦଟ ଘࡏ
Slide 42
Slide 42 text
ϦΞϧλΠϜνϟʔτ
Slide 43
Slide 43 text
Real-time Charting Library http://epochjs.github.io/epoch/
Slide 44
Slide 44 text
εϐʔυϝʔλʔ
Slide 45
Slide 45 text
http://iop.io/iopctrl Speedometer
Slide 46
Slide 46 text
͜͜·ͰͷྲྀΕΛཧ͢Δͱ • KinesisʹΑΓTweetσʔλΛऩू • LambdaʹΑΓKinesis͔ΒσʔλΛऔಘ ͯ͠SNS Topicʹσʔλૹ৴ • SNS HTTP notification pushʹΑΓ socket.ioαʔόʹσʔλૹ৴ • socket.ioαʔό͔Βϒϥβ͔Βૹ৴ ͞ΕͨσʔλΛD3.jsͰՄࢹԽ
Slide 47
Slide 47 text
σʔλͷӬଓԽ
Slide 48
Slide 48 text
՝ • ϦΞϧλΠϜͳՄࢹԽʹ߹ΘͤͯɺσʔλΛੵͯ͠ੳΛߦ͍͍ͨ
Slide 49
Slide 49 text
DynamoDB Stream σʔλϕʔεͷ σʔλΩϟϓνϟ
Slide 50
Slide 50 text
DynamoDB Streamͱ • DynamoDB ςʔϒϧʹอଘ͞Ε߲ͨͷมߋΛɺม ߋͷൃੜ࣌ʹΩϟϓνϟͰ͖Δ Ϣʔεέʔε • ৽͍͠ը૾ΛΞοϓϩʔυ͢Δͱ͙͢ʹɺάϧʔϓ ͷͯ͢ͷ༑ਓͷϞόΠϧσόΠεʹ௨Λࣗಈૹ৴ • Ϣʔβొ͕͋ΕɺΑ͏ͦ͜ϝʔϧΛૹ৴͢Δ
Slide 51
Slide 51 text
ΞʔΩςΫνϟ
Slide 52
Slide 52 text
ΞʔΩςΫνϟ σʔλͷӬଓԽ+ ετϦʔϛϯάσʔλ৴
Slide 53
Slide 53 text
ΞʔΩςΫνϟ τϦΨʔΛ༗ޮԽͯ͠ DynamoDB Streamͷ σʔλॲཧ
Slide 54
Slide 54 text
σʔλͷӬଓԽ
Slide 55
Slide 55 text
LambdaϑΝϯΫγϣϯͷτϦΨʔ༗ޮԽ
Slide 56
Slide 56 text
τϦΨʔϑΝϯΫγϣϯ SNS Topicʹσʔλૹ৴ DynamoDBϨίʔυऔಘ
Slide 57
Slide 57 text
ӬଓԽ͞ΕͨσʔλΛޙ͔ΒऔΓग़͢ DymanoDB Lambda API Gateway
Slide 58
Slide 58 text
͜͜·ͰͷྲྀΕΛཧ͢Δͱ • DynamoDBΛΈࠐΉ͜ͱͰΓσʔλ ͷӬଓԽ͕Մೳ • ӬଓԽͤͨ͞σʔλɺϨϙʔτσʔ λͱͯ͠ޙ͔ΒऔΓग़͢͜ͱ͕ग़དྷΔ
Slide 59
Slide 59 text
CloudFormation ͰߏΛݻΊΔ
Slide 60
Slide 60 text
՝ • ͜͜·ͰෳࡶͳߏΛຖճ࡞Δͷݱ࣮తͰͳ͍
Slide 61
Slide 61 text
CloudFormation ΞʔΩςΫνϟΛ ΞʔΧΠϒ
Slide 62
Slide 62 text
CloudFormation https://github.com/megumiteam/social-stream-monitor/blob/master/ social_stream_processing.template
Slide 63
Slide 63 text
CloudFormationϝϦοτ • ಉ͡ߏΛ͍ͭ͘Ͱ্ཱͪ͛ΒΕΔ • ࣅͨΑ͏ͳՄࢹԽߏΛͭ͘ΔͷͰ͋Ε ϑΥʔΫͯ͠վม͢ΕޮΑ͘ΞʔΩςΫ νϟ͕ΊΔ
Slide 64
Slide 64 text
·ͱΊ • ࠓճհͨ͠ߏΛϕʔεʹ͢ΕTwitterҎ֎ͷՄࢹԽՄೳ • AWS IoTʹMQTT over WebSocketͱ͍͏Έ͕͋Δͷ ͰηϯαʔσʔλͷϒϥβՄࢹԽͬͱΓ͍͢ • ࡞ͬͨߏCloudFormationͰݻΊͯ࠶ར༻Մೳʹ͢Δͷ ͕Φεεϝ • ࠓճհͨ͠ߏͰϦΞϧλΠϜՄࢹԽ+σʔλͷӬଓԽ͕Մೳ
Slide 65
Slide 65 text
·ͱΊ http://qiita.com/horike37 ࠓհͨ͠ΞʔΩςΫνϟͷৄࡉQiitaͷํʹιʔείʔυؚΊͯॻ͍͍ͯ·͢ ڵຯ͋Δํੋඇͷ͍ͧͯΈ͍ͯͩ͘͞
Slide 66
Slide 66 text
@Amimoto_Ami amimoto-ami.com THANK YOU! Amazon Kinesis + socket.io + D3.jsΛͬͨ webϒϥβͰߦ͏ϦΞϧλΠϜՄࢹԽͷΈ
Slide 67
Slide 67 text
No content