Data aggregation design with Goroutine, Channel, "sync" package, and Simple diagram

Data aggregation design with Goroutine, Channel, "sync" package, and Simple diagram

GoConference ’19 Summer in Fukuoka

DEMOはこちら https://gist.github.com/k1LoW/732360772475203fd6b7d8eb62309c83

69b93af68320a590f607c296e8edff73?s=128

Ken’ichiro Oyama

July 13, 2019
Tweet

Transcript

  1. খࢁ݈Ұ࿠(.01FQBCP *OD (P$POGFSFODF`4VNNFSJO'VLVPLB %BUBBHHSFHBUJPOEFTJHO XJUI(PSPVUJOF $IBOOFM  TZODQBDLBHF  BOE4JNQMFEJBHSBN

  2. ϓϦϯγύϧΤϯδχΞ খࢁ݈Ұ࿠!L-P8 ϗεςΟϯάࣄۀ෦ϗεςΟϯάάϧʔϓϚωʔδυΫϥ΢υνʔϜ IUUQTHJUIVCDPNL-P8  

  3. w UCMTJTB$*'SJFOEMZUPPMGPS EPDVNFOUBEBUBCBTF XSJUUFOJO(P w σʔλϕʔεʹ઀ଓͯ͠υΩϡϝϯτΛ ੜ੒ͯ͘͠ΕΔίϚϯυϥΠϯπʔϧ w 1PTUHSF42-.Z42-42-JUF42-4FSWFS #JH2VFSZ"NB[PO3FETIJGU

    w IUUQTHJUIVCDPNL-P8UCMT
  4. None
  5. None
  6. %BUBBHHSFHBUJPO

  7. %BUBBHHSFHBUJPO wʮσʔλΛऩूͯ͠ॲཧ͢Δʯ wେن໛ͳσʔλू໿ج൫ͷ͓࿩Ͱ͸͋Γ·ͤΜɻ͝ΊΜͳ͍͞ wຊηογϣϯͰ͸ɺ୯७ʹʮೖྗ͕͋ͬͯɺͦΕΛ͍͍ײ͡ʹॲཧ͢ ΔɻٻΊΔग़ྗΛಘΔʯͱ͍͏΋ͷΛࢦ͠·͢

  8. Ϟνϕʔγϣϯ wαʔόʔΦϖϨʔγϣϯΛΑΓޮ཰Խ͍ͨ͠ wʮٕܳͷίʔυʹΑΔϘτϜΞοϓʯ wʮϩάʯಛʹੜϩάΛޮՌతʹऔΓѻ͑ΔΑ͏ʹͳΕ͹ɺΦϖϨʔ γϣϯ΋ޮ཰తʹͳΔͷͰ͸ʁ wϩάͷऔΓѻ͍ʮೖྗ͕͋ͬͯɺͦΕΛ͍͍ײ͡ʹॲཧͯ͠ɺٻΊ Δग़ྗΛಘΔʯ

  9. ྫ'MVFOUE w ೖྗΛड͚ͯʢJOQVUʣ w ॲཧΛͯ͠ʢpMUFSCV⒎FSSPVUJOH  w ग़ྗ͢ΔʢPVUQVUʣ l8IBUJT'MVFOUE zΑΓIUUQTXXXqVFOUEPSHBSDIJUFDUVSF

  10. ೖྗॲཧग़ྗ

  11. ೖྗॲཧग़ྗ wσʔλΛऩू͢Δͱ͍͏͜ͱ͸ɺͲ͔͜ʹσʔλ͕͋Δ wͦͷσʔλΛೖྗͱͯ͠ॲཧ͍ͯ͘͜͠ͱʹͳΔ wٻΊΔग़ྗ͸σʔλͷ֨ೲ΍ɺ୯७ʹඪ४ग़ྗ΁ͷදࣔͳͲɻ w֨ೲ΋ɺೖྗσʔλΛͦͷ··֨ೲ͢Δ͜ͱ΋͋Ε͹ɺϑΟϧλ΍Ճ޻ͷॲ ཧ͕͸͍Δ͜ͱ΋͋Δ wೖྗΛޮ཰తʹॲཧ͠ग़ྗ͢Δ͜ͱΛߟ͑Δͱɺඞવతʹೖྗ͔Βग़ྗ·Ͱͷ ؒ͸ετϦʔϜσʔλͷॲཧʹͳ͍ͬͯ͘ σʔλऩूॲཧ͸ύΠϓϥΠϯॲཧ

  12. ύΠϓϥΠϯ wೖྗͱॲཧͱग़ྗΛεςʔδͱͯ͠෼͚ͯߟ͑Δ w ,BUIFSJOF$PY#VEBZஶɺࢁޱೳ᫫༁ʮ(PݴޠʹΑΔฒߦॲཧʯύΠϓϥΠϯ ೖྗ ॲཧ ग़ྗ ೖྗ ॲཧ ग़ྗ

    ॲཧ εςʔδ εςʔδ ྫ
  13. ύΠϓϥΠϯͱฒߦॲཧ wεςʔδͦΕͧΕͰฒߦʹॲཧΛ૸ΒͤΔ w(PSPVUJOF wσʔλ͸ͦΕͧΕͷεςʔδͰฒߦʹॲཧ͞Εͭͭɺ࣍ͷεςʔδ΁ྲྀΕ͍ͯ ͘

  14. εςʔδ4UBHF wೖྗΛॲཧͯ͠ग़ྗ͢Δͷ͕·ͱ·Γʹͳ͍ͬͯΔ ग़ྗ JP3FBEFSPSDIBO ೖྗ JP3FBEFSPSDIBO ೖྗ͔ΒͷσʔλΛ଴ͪड͚ͯ ೖྗ͕དྷͨΒ౎౓ॲཧΛͯ͠ ग़ྗʹॲཧͨ͠σʔλΛॻ͖ࠐΉ

  15. εςʔδͱεςʔδΛͭͳ͛Δ wલஈͷग़ྗͷܕͱޙஈͷೖྗͷܕ͕߹͍ͬͯΕ͹ܨ͛ΒΕΔ ೖྗ JP3FBEFS ग़ྗ JP3FBEFS

  16. 4UBHFJP3FBEFSWFS w ೖྗJP3FBEFS w ग़ྗJP3FBEFS w )BOEMF಺ͰJP1JQF Ͱ࿈݁ w DBODFM

    ͕ൃՐͨ͠Βऴྃ w ೖྗ͔ΒJP&0'͕དྷͯ΋ऴྃ ˞஫ҙޙड़͠·͢
  17. 4UBHFDIBOWFS w ೖྗDIBO*O w ग़ྗDIBO0VU w DUY%POF ͕ൃՐͨ͠Βऴྃ w DIBO*O͕DMPTFͯ͠΋ऴྃ

    ˞஫ҙޙड़͠·͢
  18. ͪΐͬͱ଴ͬͨʂ

  19. ⚠ %POPUTUPSF$POUFYUTJOTJEFBTUSVDUUZQF JOTUFBE QBTTB$POUFYUFYQMJDJUMZUPFBDI GVODUJPOUIBUOFFETJU IUUQTHPMBOHPSHQLHDPOUFYU ߏ଄ମʹ$POUFYU͸࣋ͨͤͳ͍Α͏ʹ͍ͯͩ͘͠͞ ୅ΘΓʹ$POUFYUΛඞཁͱ͍ͯ͠Δؔ਺ʹ౉͍ͯͩ͘͠͞

  20. ͝ΊΜͳ͍͞

  21. 4UBHFJP3FBEFSWFS w ೖྗJP3FBEFS w ग़ྗJP3FBEFS w )BOEMF಺ͰJP1JQF Ͱ࿈݁ w DBODFM

    ͕ൃՐͨ͠Βऴྃ w ೖྗ͔ΒJP&0'͕དྷͯ΋ऴྃ
  22. 4UBHFDIBOWFS w ೖྗDIBO*O w ग़ྗDIBO0VU w DUY%POF ͕ൃՐͨ͠Βऴྃ w DIBO*O͕DMPTFͯ͠΋ऴྃ

  23. ύΠϓϥΠϯͷਖ਼ৗऴྃ wग़ྗΛࢭΊΒΕΔͷ͸ɺͦͷग़ྗΛੜ੒͍ͯ͠Δεςʔδ wύΠϓϥΠϯͷ࠷ॳͷεςʔδͷग़ྗ͕ऴྃ͢Ε͹ɺ࣍ʑʹ࿈࠯ͯ͠ εςʔδ͕ऴྃ͢Δ wJP&0'ɺDMPTF DIBO

  24. ύΠϓϥΠϯͷΩϟϯηϧ wύΠϓϥΠϯͷ్தͷεςʔδΛ͍͖ͳΓΩϟϯηϧͰ͖ͯ͠·͏ͷ ͸ྑ͘ͳ͍ લஈͷεςʔδ͸ग़ྗΛ΍Ίͯ͘Εͳ͍ ޙஈͷεςʔδͷ(PSPVUJOF΋ࢭ·ͬͯ͘Εͳ͍ wύΠϓϥΠϯͷશͯΛΩϟϯηϧͰ͖ΔΑ͏ʹ͢Δ DPOUFYU$POUFYUɺDPOUFYU$BODFM'VOD શͯͷεςʔδʹDPOUFYU$POUFYUΛ౉͓ͯ͘͠

  25. $POUFYUUSFF w $POUFYUͱ͔ͦ͜Β೿ੜͨ͠$POUFYUͰ਌ࢠؔ܎ͷπϦʔ͕ܗ੒͞ΕΔΠϝʔδ w $POUFYU8JUI$BODFM w ֤$POUFYU೿ੜ$POUFYUͷ$BODFM'VODͷ఻ൖൣғΛ͏·͘ར༻ͯ͠ύΠϓϥ Πϯʹ૊ΈࠐΜͰ͍͘

  26. έʔεελσΟ

  27. ඪ४ग़ྗʹ৭Λ͚͍ͭͨ

  28. DPMS w೚ҙͷ਺ͷจࣈྻΛࢦఆ࣮ͯ͠ߦ͢Δͱɺඪ४ೖྗͰड͚औͬͨετ ϦʔϜʹͦͷจࣈྻ͕͋ͬͨΒ೚ҙͷ৭Λ͚ͭͯ͘ΕΔ wIUUQTHJUIVCDPNL-P8DPMS wUBJM'͍ͯ͠Δϩάͷจࣈྻʹ৭Λ͚ͭͯϩάͷ܏޲ͳͲΛࢹ֮తʹ Θ͔Γ΍͔ͨͬͨ͘͢͠ wطଘπʔϧ͸ଘࡏ͔͕ͨͬͨ͠ɺ୯७Ͱࣗ෼ͷ࢖͍উख͕ྑ͍΋ͷΛ ࡞ͬͨ

  29. %&.0

  30. DPMS wೖྗʜඪ४ೖྗ wॲཧʜࢦఆͨ͠จࣈྻʹ৭Λ͚ͭΔ wग़ྗʜඪ४ग़ྗ

  31. DPMS

  32. UBJM'ͷετϦʔϜΛࢭΊͣʹ Կ౓΋HSFQΛࢼ͍ͨ͠

  33. pMU wετϦʔϜͷύΠϓઌʹpMUΛࢦఆ͓ͯ͘͜͠ͱͰɺετϦʔϜΛࢭΊ ͣʹύΠϓઌͷίϚϯυΛԿ౓Ͱ΋มߋͰ͖Δ wIUUQTHJUIVCDPNL-P8pMU wHSFQ΍BXL΍TFE͕Լखͦ͘ͳͷͰɺUSJBMFSSPSͰࢼ͔ͨͬͨ͠ wTUFSOͳͲɺ1PEʹϩά͕ཷ·͍ͬͯΔͱɺ࠷ॳͷग़ྗ·Ͱʹ࣌ؒ ͕͔͔ΔͷͰɺ$USM$ͰࢭΊΔͷΛ᪳᪯͢Δ͕࣌͋ͬͨ

  34. %&.0

  35. pMU w ೖྗʜඪ४ೖྗ w ॲཧʜ w ͦͷ··ೖྗΛग़ྗʹͭͳ͛Δ w αϒϓϩηεʹೖྗΛ౉͠ɺίϚϯυΛ࣮ߦͨ݁͠ՌΛग़ྗʹͭͳ͛Δ w

    ग़ྗʜ w ඪ४ग़ྗ w EFWOVMM
  36. pMU

  37. pMU wೖྗͷྲྀྔΛ͏·੍͘ޚͰ͖Ε͹ɺεςʔδΛ੾Γସ͑ΔΑ͏ͳύΠ ϓϥΠϯΛߏஙՄೳ wpMUͷ৔߹͸ϓϩϯϓτ͕ग़͍ͯΔؒ͸ग़ྗΛEFWOVMMʹࣺ͍ͯͯ Δ w1JQF͢Δ4VCQSPDFTT΋෇͚֎͠Ͱ͖ΔΑ͏ʹ͍ͯ͠Δ

  38. ෺ཧαʔόͷϩά΋ LT1PEͷϩά΋ ೚ҙͷλʔήοτΛࢦఆͯ͠ ԣஅతʹ࣌ܥྻͰऔಘͨ͠Γ UBJM'ͨ͠Γ͍ͨ͠

  39. )BSWFTU w͋Β͔͡ΊઃఆΛ༻ҙ͓ͯ͘͜͠ͱͰɺෳ਺αʔό΍ίϯςφʹ෼ࢄ ͨ͠ϩά͔Β೚ҙͷϩάΛࢦఆͯ͠ͷऔಘ΍UBJM͕Ͱ͖Δϙʔλϒϧͳ ϩάऩूπʔϧ wIUUQTHJUIVCDPNL-P8IBSWFTU wΤʔδΣϯτϨεɻ w44)઀ଓͰ͖Δαʔό͚ͩͰͳ͘,VCFSOFUFTͷ1PEͷϩά΋αϙʔ τɻͦΕͧΕΛ۠ผ͢Δ͜ͱͳ͘ಁաతʹऔಘՄೳɻ

  40. %&.0

  41. )BSWFTU

  42. )BSWFTU

  43. )BSWFTU w ೖྗʜ w αʔό಺ͷϩάϑΝΠϧʢ44)ͰϩάΠϯʣ w ,VCFSOFUFT"1*ܦ༝Ͱͷ1PEͷϩά w ॲཧʜ w

    ϩά͔Β࣌ࠁΛύʔεͯ͠औಘ͢Δ w ग़ྗʜ w ࡞੒ͨ͠42-JUFʹϩά৘ใΛ*/4&35͢Δ w ඪ४ग़ྗ
  44. )BSWFTU

  45. TZOD8BJU(SPVQ wTZOD8BJU(SPVQͰϑΝϯΞ΢ τɾϑΝϯΠϯΛ࣮૷ w$IBOOFMͰฒྻ౓Λௐ੔ ࢀߟIUUQTNBUUOLBPSJZBOFUTPGUXBSFMBOHHPIUN

  46. UBJM'ͷ݁ՌΛ /ඵ͝ͱʹΧ΢ϯτ͍ͨ͠

  47. FWSZ wඪ४ೖྗͷ಺༰ΛɺҰఆ࣌ؒ͝ͱʢ΋͘͠͸Ұఆߦ͝ͱʣʹ·ͱΊͯ ࢦఆͷίϚϯυʹ౉͠ɺͦͷ࣮ߦ݁ՌΛγʔέϯγϟϧʹग़ྗ͢Δί Ϛϯυ wIUUQTHJUIVCDPNL-P8FWSZ

  48. %&.0

  49. FWSZ

  50. FWSZ

  51. ฒྻ࣮ߦͭͭ͠ ݩͷॱͰγʔέϯγϟϧʹग़ྗ͢Δ

  52. TZOD.VUFY

  53. ϩοΫ෇͖࣮ߦ݁ՌσʔλΧϓηϧ wίϚϯυͷ࣮ߦ݁ՌΛอ࣋͢Δ $BQTVMFΛ༻ҙ w/FX$BQTVMF ͰD-PDL ͯ͠౉͢ wD0VUQVU Λ࣮ߦͯ͠΋ɺD6OMPDL ͕͞ΕΔ·Ͱग़ྗ͞Εͳ͍ wग़ྗΛ଴ͭ

  54. 4FD4QMJUUFS wFYFDVUFS&YFDVUFS D ETU ͷதͰɺ ίϚϯυͷ࣮ߦ݁Ռ͕ͰͨΒ D6OMPDL ͕ൃՐ UJDLFSͷॱͰγʔέϯγϟϧʹग़ྗ

  55. ·ͱΊ

  56. ·ͱΊ wσʔλऩूॲཧ͸ύΠϓϥΠϯ wࣗ෼ͷதͰύλʔϯΛ࡞ΕΕ͹ॻ͚Δɻಈ͘ w(Pͷ͓͔͛Ͱɺϓϩάϥϛϯάͷ෯͕޿͕ͬͨɻͰ͖Δ͜ͱ͕޿ ͕ͬͨɻ΍Γ͍ͨ͜ͱ͕޿͕ͬͨ

  57. (PݴޠʹΑΔฒߦॲཧ w ຊηογϣϯͰ͓࿩͢Δ͜ͱͷ΄ͱ ΜͲ͕ຊॻ͔Βಘͨ஌ݟ͕ݩʹͳͬ ͍ͯΔ w ಘͨ஌ݟΛݩʹ࣮૷ͨ͠Β͏·͘ ͍ͬͨ w ຊ౰ʹΦεεϝʂʂ

  58. 5IBOLZPV ࠷৽ͷ࠾༻৘ใΛνΣοΫˠ !QC@SFDSVJU