Amazon Kinesis + socket.io + D3.jsを使った webブラウザで行うリアルタイム可視化の仕組み/inovationegg-8

Transcript

1. Amazon Kinesis + socket.io + D3.jsΛ࢖ͬͨ webϒϥ΢βͰߦ͏ϦΞϧλΠϜՄࢹԽͷ࢓૊Έ @Amimoto_Ami amimoto-ami.com
2. ࣗݾ঺հ • ໊લɿງՈོ޺ • ॴଐɿAMIMOTO TEAM • Githubɿhttps://github.com/horike37 • Qiitaɿhttp://qiita.com/horike37 • ಘҙͳ͜ͱɿAWS, WordPress
3. https://ja.amimoto-ami.com/ AMIMOTO
4. Qiita http://qiita.com/horike37
5. https://github.com/horike37/ Github
6. ࠓ೔࿩͢͜ͱ
7. AWSͰՄࢹԽΞʔΩςΫνϟΛ࡞Δٕज़
8. ࠓ೔࿩͞ͳ͍͜ͱ
9. ՄࢹԽͷ໨తɺ֓೦ɺ఩ֶ
10. ΞδΣϯμ
11. • TweetՄࢹԽπʔϧʮSocial Stream MonitorʯͷσϞ • Social Stream MonitorΞʔΩςΫνϟͷ࿩ • ϦΞϧλΠϜՄࢹԽ+σʔλͷӬଓԽ
12. Social Straem Monitor
13. Social Straem Monitorͱ͸ TwitterͷετϦʔϛϯάσʔλΛ ϦΞϧλΠϜʹՄࢹԽ͢Δπʔϧ
14. https://github.com/megumiteam/social-stream-monitor Github
15. ·ͣ͸ɺσϞ
16. ΞʔΩςΫνϟ
17. Amazon 
 Kinesis ϦΞϧλΠϜ ετϦʔϛϯάॲཧ
18. ΞʔΩςΫνϟ ετϦʔϛϯά σʔλΛॲཧ
19. Amazon Kinesisͱ͸ • ϑϧϚωʔδυͳϦΞϧλΠϜσʔλॲ ཧαʔϏε • େྔͷετϦʔϛϯάσʔλΛγϟʔ υʹอଘɻόοΫΤϯυͷΞϓϦέʔ γϣϯͰऔΓग़ͯ͠ॲཧ
20. Amazon Kinesis͕ແ͍ͱ͖ ετϦʔϛϯάσʔλ σʔλॲཧܥ
21. Amazon Kinesis͕͋Δ࣌ ετϦʔϛϯάσʔλ σʔλॲཧܥ ॲཧܥ͕ѻ͍΍͍͢Α͏ʹσʔλΛอଘ
22. Social Stream Metor্ͷ໾ׂ • TwitterͷετϦʔϛϯάσʔλΛҰ࣌ తʹอଘͯ͠ΞϓϦέʔγϣϯ͔Βσʔ λͱͯ͠ॲཧ͠΍͘͢͢Δ
23. Lambda ίʔυͷ࣮ߦ
24. ΞʔΩςΫνϟ Kinesis͔Βड͚ औͬͨσʔλΛ SNSʹ౉͢
25. Lambdaͱ͸ • AWS ΠϯϑϥετϥΫνϟΛ࢖༻ͯ͠ ίʔυͷ࣮ߦΛ୅ߦ͢ΔίϯϐϡʔςΟ ϯάαʔϏε • ΠϕϯτυϦϒϯͳॲཧ͕Մೳ
26. SNS http endpoint΁σʔλΛ౉͢ σʔλϨίʔυΛSNSʹpublish
27. Social Stream Metor্ͷ໾ׂ • webϒϥ΢βͰදࣔ͢ΔͨΊʹhttpϓϩ τίϧͰσʔλΛྲྀ͢ඞཁ͕͋Δ • ͦͷͨΊͷSNS http push௨஌΁σʔλ Λ౉͢
28. ΞʔΩςΫνϟ HTTPΤϯυϙΠϯ τ΁σʔλΛpush
29. SNS HTTP notification HTTPΤϯυϙΠϯτ
 PUSH௨஌
30. SNS HTTP notificationͱ͸ • Amazon SNS Λ࢖༻ͯ͠ɺ1 ͭҎ্ͷ HTTPΤϯυϙΠϯτʹ௨஌ϝοηʔδ Λૹ৴ • HTTP POST ϦΫΤετΛૹ৴͠ɺ௨ ஌ͷ಺༰Λड৴ొ࿥ͨ͠ΤϯυϙΠϯ τʹ഑৴
31. SNS HTTP notificationͱ͸ Topicʹड৴ొ࿥͞ΕͨΤϯυϙΠϯτʹσʔλΛPOST
32. Social Stream Metor্ͷ໾ׂ • webϒϥ΢βͰදࣔ͢ΔͨΊʹhttpϓϩτίϧͰ σʔλΛྲྀ͢ඞཁ͕͋Δ • websocketͰϒϥ΢βʹσʔλΛૹ৴͢ΔͨΊͷ socket.ioαʔόʹσʔλΛpush͢Δ
33. socket.io ϦΞϧλΠϜͳ WebΞϓϦ
34. ΞʔΩςΫνϟ WebSocketͰϒϥ ΢βʹσʔλૹ৴
35. socket.io • ͢΂ͯͷϒϥ΢βɾϞόΠϧσόΠεͰ ϦΞϧλΠϜ௨৴ΛՄೳͱ͢Δ͜ͱΛ໨ తʹ࡞ΒΕͨJavaScriptϥΠϒϥϦ • WebSocketʹΑΔϒϥ΢βͱαʔόͷ ૒ํ޲௨৴
36. Social Stream Metor্ͷ໾ׂ • SNS͔ΒૹΒΕ͖ͯͨTweetΛWebSocketΛ࢖͍ ϒϥ΢β΁ϦΞϧλΠϜʹૹ৴
37. SNS͔ΒૹΒΕ͖ͯͨσʔλΛड৴ https://github.com/megumiteam/social-stream-monitor/blob/master/app.js
38. ϒϥ΢β΁σʔλΛૹ৴ https://github.com/megumiteam/social-stream-monitor/blob/master/app.js
39. D3.js σʔλɾυϦϒϯɾ υΩϡϝϯτ
40. ΞʔΩςΫνϟ D3.jsʹΑΔඳը
41. D3.jsͱ͸ • javascriptͷσʔλϏδϡΞϥΠθʔ γϣϯΤϯδϯ • σʔλΛجʹDOMΛૢ࡞ • άϥϑඳըϥΠϒϥϦͰ͸ͳ͍ɻD3Λ ϕʔεʹͨ͠άϥϑඳըϥΠϒϥϦ͸ଟ ਺ଘࡏ
42. ϦΞϧλΠϜνϟʔτ
43. Real-time Charting Library http://epochjs.github.io/epoch/
44. εϐʔυϝʔλʔ
45. http://iop.io/iopctrl Speedometer
46. ͜͜·ͰͷྲྀΕΛ੔ཧ͢Δͱ • KinesisʹΑΓTweetσʔλΛऩू • LambdaʹΑΓKinesis͔ΒσʔλΛऔಘ ͯ͠SNS Topicʹσʔλૹ৴ • SNS HTTP notification pushʹΑΓ socket.ioαʔόʹσʔλૹ৴ • socket.ioαʔό͔Βϒϥ΢β͔Βૹ৴ ͞ΕͨσʔλΛD3.jsͰՄࢹԽ
47. σʔλͷӬଓԽ
48. ՝୊ • ϦΞϧλΠϜͳՄࢹԽʹ߹ΘͤͯɺσʔλΛ஝ੵͯ͠෼ੳΛߦ͍͍ͨ
49. DynamoDB 
 Stream σʔλϕʔεͷ σʔλΩϟϓνϟ
50. DynamoDB Streamͱ͸ • DynamoDB ςʔϒϧʹอଘ͞Ε߲ͨ໨ͷมߋΛɺม ߋͷൃੜ࣌ʹΩϟϓνϟͰ͖Δ Ϣʔεέʔε • ৽͍͠ը૾ΛΞοϓϩʔυ͢Δͱ͙͢ʹɺάϧʔϓ಺ ͷ͢΂ͯͷ༑ਓͷϞόΠϧσόΠεʹ௨஌Λࣗಈૹ৴ • Ϣʔβొ࿥͕͋Ε͹ɺΑ͏ͦ͜ϝʔϧΛૹ৴͢Δ
51. ΞʔΩςΫνϟ
52. ΞʔΩςΫνϟ σʔλͷӬଓԽ+ ετϦʔϛϯάσʔλ഑৴
53. ΞʔΩςΫνϟ τϦΨʔΛ༗ޮԽͯ͠ DynamoDB Streamͷ σʔλॲཧ
54. σʔλͷӬଓԽ
55. LambdaϑΝϯΫγϣϯͷτϦΨʔ༗ޮԽ
56. τϦΨʔϑΝϯΫγϣϯ SNS Topicʹσʔλૹ৴ DynamoDBϨίʔυऔಘ
57. ӬଓԽ͞ΕͨσʔλΛޙ͔ΒऔΓग़͢ DymanoDB Lambda API Gateway
58. ͜͜·ͰͷྲྀΕΛ੔ཧ͢Δͱ • DynamoDBΛ૊ΈࠐΉ͜ͱͰΓσʔλ ͷӬଓԽ͕Մೳ • ӬଓԽͤͨ͞σʔλ͸ɺϨϙʔτσʔ λͱͯ͠ޙ͔ΒऔΓग़͢͜ͱ͕ग़དྷΔ
59. CloudFormation Ͱߏ੒ΛݻΊΔ
60. ՝୊ • ͜͜·Ͱෳࡶͳߏ੒Λຖճ࡞Δͷ͸ݱ࣮తͰ͸ͳ͍
61. CloudFormation ΞʔΩςΫνϟΛ ΞʔΧΠϒ
62. CloudFormation https://github.com/megumiteam/social-stream-monitor/blob/master/ social_stream_processing.template
63. CloudFormationϝϦοτ • ಉ͡ߏ੒Λ͍ͭ͘Ͱ΋্ཱͪ͛ΒΕΔ • ࣅͨΑ͏ͳՄࢹԽߏ੒Λͭ͘ΔͷͰ͋Ε͹ ϑΥʔΫͯ͠վม͢Ε͹ޮ཰Α͘ΞʔΩςΫ νϟ͕૊ΊΔ
64. ·ͱΊ • ࠓճ঺հͨ͠ߏ੒Λϕʔεʹ͢Ε͹TwitterҎ֎ͷՄࢹԽ΋Մೳ • AWS IoTʹ͸MQTT over WebSocketͱ͍͏࢓૊Έ͕͋Δͷ Ͱηϯαʔσʔλͷϒϥ΢βՄࢹԽ͸΋ͬͱ΍Γ΍͍͢ • ࡞ͬͨߏ੒͸CloudFormationͰݻΊͯ࠶ར༻Մೳʹ͢Δͷ ͕Φεεϝ • ࠓճ঺հͨ͠ߏ੒ͰϦΞϧλΠϜՄࢹԽ+σʔλͷӬଓԽ͕Մೳ
65. ·ͱΊ http://qiita.com/horike37 ࠓ೔঺հͨ͠ΞʔΩςΫνϟͷৄࡉ͸QiitaͷํʹιʔείʔυؚΊͯॻ͍͍ͯ·͢ ڵຯ͋Δํ͸ੋඇͷ͍ͧͯΈ͍ͯͩ͘͞
66. @Amimoto_Ami amimoto-ami.com THANK YOU! Amazon Kinesis + socket.io + D3.jsΛ࢖ͬͨ webϒϥ΢βͰߦ͏ϦΞϧλΠϜՄࢹԽͷ࢓૊Έ