Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Amazon Kinesis + socket.io + D3.jsを使った webブラウザで行うリアルタイム可視化の仕組み/inovationegg-8

Amazon Kinesis + socket.io + D3.jsを使った webブラウザで行うリアルタイム可視化の仕組み/inovationegg-8

53c9bf20050e5457f054eac61c388a15?s=128

堀家隆宏

July 02, 2016
Tweet

Transcript

  1. Amazon Kinesis + socket.io + D3.jsΛ࢖ͬͨ webϒϥ΢βͰߦ͏ϦΞϧλΠϜՄࢹԽͷ࢓૊Έ @Amimoto_Ami amimoto-ami.com

  2. ࣗݾ঺հ • ໊લɿງՈོ޺ • ॴଐɿAMIMOTO TEAM • Githubɿhttps://github.com/horike37 • Qiitaɿhttp://qiita.com/horike37

    • ಘҙͳ͜ͱɿAWS, WordPress
  3. https://ja.amimoto-ami.com/ AMIMOTO

  4. Qiita http://qiita.com/horike37

  5. https://github.com/horike37/ Github

  6. ࠓ೔࿩͢͜ͱ

  7. AWSͰՄࢹԽΞʔΩςΫνϟΛ࡞Δٕज़

  8. ࠓ೔࿩͞ͳ͍͜ͱ

  9. ՄࢹԽͷ໨తɺ֓೦ɺ఩ֶ

  10. ΞδΣϯμ

  11. • TweetՄࢹԽπʔϧʮSocial Stream MonitorʯͷσϞ • Social Stream MonitorΞʔΩςΫνϟͷ࿩ • ϦΞϧλΠϜՄࢹԽ+σʔλͷӬଓԽ

  12. Social Straem Monitor

  13. Social Straem Monitorͱ͸ TwitterͷετϦʔϛϯάσʔλΛ ϦΞϧλΠϜʹՄࢹԽ͢Δπʔϧ

  14. https://github.com/megumiteam/social-stream-monitor Github

  15. ·ͣ͸ɺσϞ

  16. ΞʔΩςΫνϟ

  17. Amazon 
 Kinesis ϦΞϧλΠϜ ετϦʔϛϯάॲཧ

  18. ΞʔΩςΫνϟ ετϦʔϛϯά σʔλΛॲཧ

  19. Amazon Kinesisͱ͸ • ϑϧϚωʔδυͳϦΞϧλΠϜσʔλॲ ཧαʔϏε • େྔͷετϦʔϛϯάσʔλΛγϟʔ υʹอଘɻόοΫΤϯυͷΞϓϦέʔ γϣϯͰऔΓग़ͯ͠ॲཧ

  20. Amazon Kinesis͕ແ͍ͱ͖ ετϦʔϛϯάσʔλ σʔλॲཧܥ

  21. Amazon Kinesis͕͋Δ࣌ ετϦʔϛϯάσʔλ σʔλॲཧܥ ॲཧܥ͕ѻ͍΍͍͢Α͏ʹσʔλΛอଘ

  22. Social Stream Metor্ͷ໾ׂ • TwitterͷετϦʔϛϯάσʔλΛҰ࣌ తʹอଘͯ͠ΞϓϦέʔγϣϯ͔Βσʔ λͱͯ͠ॲཧ͠΍͘͢͢Δ

  23. Lambda ίʔυͷ࣮ߦ

  24. ΞʔΩςΫνϟ Kinesis͔Βड͚ औͬͨσʔλΛ SNSʹ౉͢

  25. Lambdaͱ͸ • AWS ΠϯϑϥετϥΫνϟΛ࢖༻ͯ͠ ίʔυͷ࣮ߦΛ୅ߦ͢ΔίϯϐϡʔςΟ ϯάαʔϏε • ΠϕϯτυϦϒϯͳॲཧ͕Մೳ

  26. SNS http endpoint΁σʔλΛ౉͢ σʔλϨίʔυΛSNSʹpublish

  27. Social Stream Metor্ͷ໾ׂ • webϒϥ΢βͰදࣔ͢ΔͨΊʹhttpϓϩ τίϧͰσʔλΛྲྀ͢ඞཁ͕͋Δ • ͦͷͨΊͷSNS http push௨஌΁σʔλ

    Λ౉͢
  28. ΞʔΩςΫνϟ HTTPΤϯυϙΠϯ τ΁σʔλΛpush

  29. SNS HTTP notification HTTPΤϯυϙΠϯτ
 PUSH௨஌

  30. SNS HTTP notificationͱ͸ • Amazon SNS Λ࢖༻ͯ͠ɺ1 ͭҎ্ͷ HTTPΤϯυϙΠϯτʹ௨஌ϝοηʔδ Λૹ৴

    • HTTP POST ϦΫΤετΛૹ৴͠ɺ௨ ஌ͷ಺༰Λड৴ొ࿥ͨ͠ΤϯυϙΠϯ τʹ഑৴
  31. SNS HTTP notificationͱ͸ Topicʹड৴ొ࿥͞ΕͨΤϯυϙΠϯτʹσʔλΛPOST

  32. Social Stream Metor্ͷ໾ׂ • webϒϥ΢βͰදࣔ͢ΔͨΊʹhttpϓϩτίϧͰ σʔλΛྲྀ͢ඞཁ͕͋Δ • websocketͰϒϥ΢βʹσʔλΛૹ৴͢ΔͨΊͷ socket.ioαʔόʹσʔλΛpush͢Δ

  33. socket.io ϦΞϧλΠϜͳ WebΞϓϦ

  34. ΞʔΩςΫνϟ WebSocketͰϒϥ ΢βʹσʔλૹ৴

  35. socket.io • ͢΂ͯͷϒϥ΢βɾϞόΠϧσόΠεͰ ϦΞϧλΠϜ௨৴ΛՄೳͱ͢Δ͜ͱΛ໨ తʹ࡞ΒΕͨJavaScriptϥΠϒϥϦ • WebSocketʹΑΔϒϥ΢βͱαʔόͷ ૒ํ޲௨৴

  36. Social Stream Metor্ͷ໾ׂ • SNS͔ΒૹΒΕ͖ͯͨTweetΛWebSocketΛ࢖͍ ϒϥ΢β΁ϦΞϧλΠϜʹૹ৴

  37. SNS͔ΒૹΒΕ͖ͯͨσʔλΛड৴ https://github.com/megumiteam/social-stream-monitor/blob/master/app.js

  38. ϒϥ΢β΁σʔλΛૹ৴ https://github.com/megumiteam/social-stream-monitor/blob/master/app.js

  39. D3.js σʔλɾυϦϒϯɾ υΩϡϝϯτ

  40. ΞʔΩςΫνϟ D3.jsʹΑΔඳը

  41. D3.jsͱ͸ • javascriptͷσʔλϏδϡΞϥΠθʔ γϣϯΤϯδϯ • σʔλΛجʹDOMΛૢ࡞ • άϥϑඳըϥΠϒϥϦͰ͸ͳ͍ɻD3Λ ϕʔεʹͨ͠άϥϑඳըϥΠϒϥϦ͸ଟ ਺ଘࡏ

  42. ϦΞϧλΠϜνϟʔτ

  43. Real-time Charting Library http://epochjs.github.io/epoch/

  44. εϐʔυϝʔλʔ

  45. http://iop.io/iopctrl Speedometer

  46. ͜͜·ͰͷྲྀΕΛ੔ཧ͢Δͱ • KinesisʹΑΓTweetσʔλΛऩू • LambdaʹΑΓKinesis͔ΒσʔλΛऔಘ ͯ͠SNS Topicʹσʔλૹ৴ • SNS HTTP

    notification pushʹΑΓ socket.ioαʔόʹσʔλૹ৴ • socket.ioαʔό͔Βϒϥ΢β͔Βૹ৴ ͞ΕͨσʔλΛD3.jsͰՄࢹԽ
  47. σʔλͷӬଓԽ

  48. ՝୊ • ϦΞϧλΠϜͳՄࢹԽʹ߹ΘͤͯɺσʔλΛ஝ੵͯ͠෼ੳΛߦ͍͍ͨ

  49. DynamoDB 
 Stream σʔλϕʔεͷ σʔλΩϟϓνϟ

  50. DynamoDB Streamͱ͸ • DynamoDB ςʔϒϧʹอଘ͞Ε߲ͨ໨ͷมߋΛɺม ߋͷൃੜ࣌ʹΩϟϓνϟͰ͖Δ Ϣʔεέʔε • ৽͍͠ը૾ΛΞοϓϩʔυ͢Δͱ͙͢ʹɺάϧʔϓ಺ ͷ͢΂ͯͷ༑ਓͷϞόΠϧσόΠεʹ௨஌Λࣗಈૹ৴

    • Ϣʔβొ࿥͕͋Ε͹ɺΑ͏ͦ͜ϝʔϧΛૹ৴͢Δ
  51. ΞʔΩςΫνϟ

  52. ΞʔΩςΫνϟ σʔλͷӬଓԽ+ ετϦʔϛϯάσʔλ഑৴

  53. ΞʔΩςΫνϟ τϦΨʔΛ༗ޮԽͯ͠ DynamoDB Streamͷ σʔλॲཧ

  54. σʔλͷӬଓԽ

  55. LambdaϑΝϯΫγϣϯͷτϦΨʔ༗ޮԽ

  56. τϦΨʔϑΝϯΫγϣϯ SNS Topicʹσʔλૹ৴ DynamoDBϨίʔυऔಘ

  57. ӬଓԽ͞ΕͨσʔλΛޙ͔ΒऔΓग़͢ DymanoDB Lambda API Gateway

  58. ͜͜·ͰͷྲྀΕΛ੔ཧ͢Δͱ • DynamoDBΛ૊ΈࠐΉ͜ͱͰΓσʔλ ͷӬଓԽ͕Մೳ • ӬଓԽͤͨ͞σʔλ͸ɺϨϙʔτσʔ λͱͯ͠ޙ͔ΒऔΓग़͢͜ͱ͕ग़དྷΔ

  59. CloudFormation Ͱߏ੒ΛݻΊΔ

  60. ՝୊ • ͜͜·Ͱෳࡶͳߏ੒Λຖճ࡞Δͷ͸ݱ࣮తͰ͸ͳ͍

  61. CloudFormation ΞʔΩςΫνϟΛ ΞʔΧΠϒ

  62. CloudFormation https://github.com/megumiteam/social-stream-monitor/blob/master/ social_stream_processing.template

  63. CloudFormationϝϦοτ • ಉ͡ߏ੒Λ͍ͭ͘Ͱ΋্ཱͪ͛ΒΕΔ • ࣅͨΑ͏ͳՄࢹԽߏ੒Λͭ͘ΔͷͰ͋Ε͹ ϑΥʔΫͯ͠վม͢Ε͹ޮ཰Α͘ΞʔΩςΫ νϟ͕૊ΊΔ

  64. ·ͱΊ • ࠓճ঺հͨ͠ߏ੒Λϕʔεʹ͢Ε͹TwitterҎ֎ͷՄࢹԽ΋Մೳ • AWS IoTʹ͸MQTT over WebSocketͱ͍͏࢓૊Έ͕͋Δͷ Ͱηϯαʔσʔλͷϒϥ΢βՄࢹԽ͸΋ͬͱ΍Γ΍͍͢ •

    ࡞ͬͨߏ੒͸CloudFormationͰݻΊͯ࠶ར༻Մೳʹ͢Δͷ ͕Φεεϝ • ࠓճ঺հͨ͠ߏ੒ͰϦΞϧλΠϜՄࢹԽ+σʔλͷӬଓԽ͕Մೳ
  65. ·ͱΊ http://qiita.com/horike37 ࠓ೔঺հͨ͠ΞʔΩςΫνϟͷৄࡉ͸QiitaͷํʹιʔείʔυؚΊͯॻ͍͍ͯ·͢ ڵຯ͋Δํ͸ੋඇͷ͍ͧͯΈ͍ͯͩ͘͞

  66. @Amimoto_Ami amimoto-ami.com THANK YOU! Amazon Kinesis + socket.io + D3.jsΛ࢖ͬͨ

    webϒϥ΢βͰߦ͏ϦΞϧλΠϜՄࢹԽͷ࢓૊Έ
  67. None