Spark Streamingで全世界の 混雑状況を20分ごとに集計 / All over the world with Spark Streaming Congested situation every 20 minutes

0431014d47f66420bb69ffcf6eea13f6?s=47 yu
March 14, 2019

Spark Streamingで全世界の 混雑状況を20分ごとに集計 / All over the world with Spark Streaming Congested situation every 20 minutes

0431014d47f66420bb69ffcf6eea13f6?s=128

yu

March 14, 2019
Tweet

Transcript

  1. ೥݄೔ ιϑτόϯΫגࣜձࣾᖛా༎ 4QBSL4USFBNJOHͰશੈքͷ ࠞࡶঢ়گΛ෼͝ͱʹूܭ )BEPPQ4QBSL$POGFSFODF+BQBO

  2. ᖛాɹ༎ ιϑτόϯΫגࣜձࣾ ݉ גࣜձࣾ"HPPQ ޷͖ͳ"84αʔϏε "UIFOB(MVF όοΫΤϯυશൠ

  3. "HPPQʹ͍ͭͯ اۀ໊ גࣜձࣾ"HPPQ ઃཱ࣌ظ ೥݄ ୅දऔక໾ ࣲࢁ࿨ٱ גओ ιϑτόϯΫגࣜձࣾ 

    ࢿຊۚ ԯ ສԁ ैۀһ਺ ໊ ॅॴ ˟౦ژ౎ौ୩۠ਆٶલ ࣄۀ಺༰ ɹҐஔ৘ใϏοάσʔλࣄۀ ɹɹɾҐஔ৘ใϏοάσʔλऩूղੳ ɹɹɾҐஔ৘ใؔ࿈σʔλൢച ɹɹɾҐஔ৘ใαʔϏεΞϓϦఏڙ ˞೥݄೔ݱࡏ Ґஔ৘ใºϏοάσʔλઐ໳اۀ
  4. ਓྲྀάϩʔόϧϏοάσʔλ ਓʑͷߦಈΛৄࡉʹ೺Ѳ

  5. Strictly Confidential άϩʔόϧϏοάσʔλ   εϚʔτϑΥϯΞϓϦέʔγϣϯ͔Β ੈքͷ(14Ґஔ৘ใσʔλΛऔಘ /VNCFSPG-PHT %BUB4J[F ਺ԯ݅

    EBZ 5#ڃ EBZ
  6. None
  7. ࠞࡶϚοϓ

  8. ΞʔΩςΫνϟ &-# "VUP4DBMJOH ,JOFTJT "VUP4DBMJOH "VUP4DBMJOH ,JOFTJT 4 &MBTUJD4FBSDI &.3

    4 ࠞࡶϚοϓ ϩάऩूج൫ "1* ࠞࡶϚοϓ &5- ɾɾɾ
  9. "1* ࠞࡶϚοϓ ΞʔΩςΫνϟ &-# "VUP4DBMJOH ,JOFTJT "VUP4DBMJOH "VUP4DBMJOH ,JOFTJT 4

    &MBTUJD4FBSDI &.3 4 ࠞࡶϚοϓ ϩάऩूج൫ &5- ɾɾɾ
  10. ϩάऩूج൫ ,JOFTJTʢେن໛ετϦʔϛϯάσʔλ༻ΩϡʔΠϯάαʔϏεʣΛத৺ͱͨ͠ߏ੒ લஈͷ"VUP4DBMJOH͸,JOFTJT1SPEVDFS-JCSBSZΛ࢖༻͠ɺͦͷ··,JOFTJTʹ౤ೖ ޙஈͷ"VUP4DBMJOH͸̎ͭͷॲཧΛ࣮ࢪ ,JOFTJT$POTVNFS-JCSBSZΛ࢖༻͠ɺ,JOFTJT͔Β+40/Λऔಘɾύʔεͯ͠ޙ ଓͷ෼ੳͰ࢖༻͢Δ৘ใΛ෇Ճ qVFOUEΛ༻͍ͯɺҰఆִؒͰ4ʹ([JQܗࣜͰग़ྗ ,JOFTJTʹೖΕ͓͚ͯ͹࣌ؒ ࠷௕೔ؒ σʔλϩετͷ৺഑͕ແ͍ͨΊ

    <1PJOU>,JOFTJTಋೖͰ҆ఆੑ͕͔ͳΓ޲্ˠσʔλऩूܥʹ͸͔ͳΓΦεεϝ &-# "VUP4DBMJOH ,JOFTJT "VUP4DBMJOH 4 1SPEVDFS -JCSBSZ $POTVNFS -JCSBSZ &5- ɾɾɾ
  11. ΞʔΩςΫνϟ &-# "VUP4DBMJOH ,JOFTJT "VUP4DBMJOH "VUP4DBMJOH ,JOFTJT 4 &MBTUJD4FBSDI &.3

    4 ࠞࡶϚοϓ "1* ࠞࡶϚοϓ ϩάऩूج൫ &5- ɾɾɾ
  12. ࠞࡶϚοϓલॲཧ ,JOFTJT1SPEVDFS$POTVNFS-JCSBSZ྆ํར༻ લॲཧͰ͸ҎԼͭͷॲཧΛ࣮ࢪ ඞཁͳ߲໨ͷΈʹߜΔޙଓॲཧͰඞཁͳσʔλΛΞλον͢Δ ࠞࡶϚοϓઐ༻ͷ,JOFTJT΁ॻ͖ࠐΈ ,JOFTJT͸γϟʔυ਺ઃఆʹΑΓಡΈࠐΈɾॻ͖ࠐΈΩϟύγςΟ͕ܾఆ͞Εɺ ॻ͖ࠐΈಡΈࠐΈͱͳ͍ͬͯΔ ➡ϩάऩूج൫ɺࠞࡶϚοϓʢຊ൪ɾ։ൃʣͷ؀ڥߏங͢Δͱɺίετޮ཰ѱԽ "VUP4DBMJOH ,JOFTJT

    &MBTUJD4FBSDI &.3 4 ,JOFTJT ʢϩάऩूج൫ʣ 1SPEVDFS $POTVNFS -JCSBSZ "1* ࠞࡶϚοϓ
  13. ࠞࡶϚοϓσʔλೖग़ྗ &.3্Ͱ4QBSLΛՔಇʢݴޠɿ4DBMBʣͤ͞ॲཧΛ࣮ࢪ ,JOFTJTͷσʔλऔಘʹ͸"84੡4%,ɿ"NB[PO,JOFTJT$MJFOUΛ࢖༻ TUSFBNJOHॲཧͰ໰୊ͱͳΓ΍͍͢νΣοΫϙΠϯτॲཧΛ4%,಺෦Ͱࣗಈ؅ཧ %ZOBNP%#ʹνΣοΫϙΠϯτΛॻ͖ࠐΈɺෳ਺αʔόʔ͔Βॻ͖ࠐΈɾࢀর &MBTUJD4FBSDI΁ͷσʔλ౤ೖ͸4QBSLͷجຊػೳΛ༻͍ͯɺ3%%Λ%BUB'SBNFʹม ׵ͨ͠ޙɺ4BWF5P&4Ͱอଘ "VUP4DBMJOH ,JOFTJT &.3

    4 ,JOFTJT ʢϩάऩूج൫ʣ 1SPEVDFS $POTVNFS -JCSBSZ &MBTUJD4FBSDI "1* ࠞࡶϚοϓ ೔࣍ॲཧ༻&.3
  14. ࠞࡶϚοϓूܭॲཧ 4QBSL4USFBNJOHʹͯ෼ִؒͰूܭ͠ɺ݁ՌΛ&MBTUJD4FBSDIʹอଘ ΢Οϯυ΢αΠζΛ෼ͱ͍͏͔ͳΓେ͖ͳ஋Ͱӡ༻ ूܭ୯Ґ͸λΠϧʢϝογϡʣΛ࠾༻͓ͯ͠ΓɺζʔϜϨϕϧɾλΠϧຖʹूܭ ฐࣾϩά͸೔ຊɾ64ɾ&6Λத৺ʹϩάྔ͕ଟ͘ɺࠃຖʹϩάྔʹେ͖͘ภΓ͕͋Δ ೔࣍ॲཧͰࠃຖͷฏۉϩά਺Λूܭ͠ɺϩάྔͷҧ͍Λฏ४Խ ϩάྔ͕গͳ͍ࠃͰ΋ࠞࡶঢ়گΛ೺Ѳ͢Δ͜ͱ͕Ͱ͖Δ ࠷΋޻෉ͨ͠఺͸ɺ଺ࡏΛՃຯͨࠞ͠ࡶ౓ूܭ "VUP4DBMJOH ,JOFTJT

    &.3 4 ,JOFTJT ʢϩάऩूج൫ʣ 1SPEVDFS $POTVNFS -JCSBSZ &MBTUJD4FBSDI "1* ࠞࡶϚοϓ ೔࣍ॲཧ༻&.3
  15. ۱ా઒ՖՐେձ มߋޙ վળޙ վળલ ۱ా઒ՖՐେձ ՖՐਅͬ࠷த

  16. ۱ా઒ՖՐେձ վળલ ϝΠϯձ৔ ઙ૲Ӻ ϝΠϯձ৔͕ ࠞΜͰ͍ͳ͍

  17. ͳͥձ৔͕ࠞ·ͳ͍ͷ͔ʁ

  18. ͳͥձ৔͕ࠞ·ͳ͍ͷ͔ʁ ूܭϩδοΫมߋ

  19. มߋલ ࠞࡶϚοϓूܭॲཧ ,JOFTJT &.3 &MBTUJD4FBSDI ,JOFTJT &.3 4 &MBTUJD4FBSDI &.3

    ,JOFTJT͔Β෼ؒͷϩάΛऔಘ ूܭͨ݁͠ՌΛ &MBTUJD4FBSDIʹ֨ೲ มߋޙ &.3ॲཧΛҎԼʹ෼ׂ ,JOFTJT͔Β෼ؒͷϩάΛऔಘ͠ ɹɹIJWFςʔϒϧʹͦͷ··ग़ྗ IJWFςʔϒϧ͔Β࣌ؒ෼ͷσʔλΛऔ ಘ͠ɺ࠷ޙʹग़ྗ͞ΕͨϩάͷΈΛूܭ
  20. վળޙ ۱ా઒ՖՐେձ վળޙ վળલ ୈҰձ৔ ઙ૲Ӻ ୈҰձ৔͕ ࠞΜͩʂ ࣮ଶʹ͍ۙ

  21. ࠓޙͷల๬ ଺ࡏΛՃຯͨࠞ͠ࡶ౓ूܭΛ4QBSL4USFBNJOHͷ 4MJEJOH8JOEPXΛ༻͍࣮ͯࢪ͢Δ &.3Ϋϥελʔͷ֤छઃఆͷνϡʔχϯάΛ࣮ࢪ͠ɺί ετޮ཰޲্ɺूܭظؒͷ୹ॖʢ෼෼ʣΛ࣮ݱ

  22. THANK YOU! ANY QUESTIONS?