Gaurun / A general push notification server in Go

Gaurun / A general push notification server in Go

Gaurun 〜A general push notification server in Go〜

5d74d743eabd2bf7d4d2f68b9d3c727d?s=128

Tatsuhiko Kubo

June 21, 2015
Tweet

Transcript

  1. A general push notification server in Go Tatsuhiko Kubo@cubicdaiya Go

    Conference summer@2015/06/21 Gaurun
  2. @cubicdaiya / Tatsuhiko Kubo w 4PGUXBSF&OHJOFFS!.FSDBSJ *OD w *OGSBTUSVDUVSF&OHJOFFSJOH w

    (P w OHJOYCVJME TMBDLCPBSE (BVSVO w OHJOY w OHY@TNBMM@MJHIU OHY@EZOBNJD@VQTUSFBN
  3. Other Activities IUUQNP[BJDGNQPTUOHJOY

  4. None
  5. None
  6. mercari.go w DVCJDEBJZBOHJOYCVJME w DVCJDEBJZBDBDIFDUM w DVCJDEBJZBTMBDLCPBSE w NFSDBSJ(BVSVO (PJOQSPEVDUJPO!.FSDBSJ

  7. mercari.go w DVCJDEBJZBOHJOYCVJME w DVCJDEBJZBDBDIFDUM w DVCJDEBJZBTMBDLCPBSE w NFSDBSJ(BVSVO (PJOQSPEVDUJPO!.FSDBSJ

  8. Gaurun https://github.com/mercari/gaurun

  9. Gaurun w "HFOFSBMQVTIOPUJpDBUJPOTFSWFS GPSTNBSUQIPOFBQQMJDBUJPOT w 8SJUUFOJO(P w QSPYJOHQVTISFRVFTUTUP"1/TBOE($. w 4JNQMF)551"1*T

  10. Overview ($. "1/T (BVSVO )5514 "1/T QSPUPDPM )551 1045QVTI #PEZJT+40/

    $MJFOU
  11. Why Go? w 3FRVJSFNFOUT w (PPEQFSGPSNBODF w )JHIDPODVSSFODZ w OFUIUUQJTHPPEDIPJDF

  12. Why Go? w 1VTIJTTMPX w %VFUPOFUXPSLMBUFODZ w 4PNFPQFSBUJPO!.FSDBSJ w 4FOENJMMJPOQVTIBUPODFʂ

    w )JHIDPODVSSFODZJTSFRVJSFEʂ
  13. Why Go? w "TZODSPOPVTQVTIJTSFRVJSFE w QVTIJTUSJHHFSFEJO"1*TFSWFS w DPNNFOU MJLF CVZ

    TFMM w "1*TFSWFSNVTUSFTQPOTFRVJDLMZ w "TZODISPOJ[BUJPOJTDPNQMJDBUFE w "1*TFSWFS!.FSDBSJJT"QBDIF 1)1
  14. Push System @ Mercari

  15. a year ago… ($. "1/T "1*4FSWFS "1*4FSWFS "1*4FSWFS "MMQVTIOPUJpDBUJPOTBSFTZODISPOPVT QVTI

  16. half a year ago… ($. "1/T 8PSLFS "MMQVTIOPUJpDBUJPOTBSFBTZODISPOPVT 2VFVF 8PSLFS

    8PSLFS QVTI EFRVFVF "1*4FSWFS "1*4FSWFS #BUDI4FSWFS FORVFVF "1*4FSWFS
  17. But very complicated… w +PC2VFVF w 2. w .VMUJ1SPDFTT8PSLFSXJUI1)1 w

    1)11BSBMMFMT1SFGPSL w 5IFBSDIJUFDUVSFJTOPUTDBMBCMF
  18. Now… ($. "1/T (BVSVO (BVSVO (BVSVO /(*/9 "1*4FSWFS "1*4FSWFS "1*4FSWFS

    #BUDI4FSWFS QVTI XJUI LFFQBMJWF )551 1045QVTI
  19. Gaurun Internals

  20. How Gaurun pushesʁ ($. "1/T (BVSVO )5514 "1/T QSPUPDPM )551

    1045QVTI #PEZJT+40/ $MJFOU
  21. Bootstrap Gaurun UIFDPEFCFMPXJTTJNQMJpFEGPSFYQMBOBUJPO TUBSUXPSLFSTGPSQVTI HBVSVO4UBSU1VTI8PSLFST XPSLFS/VN RVFVF/VN  )551"1*T IUUQ)BOEMF'VOD

    lQVTIz HBVSVO1VTI/PUJpDBUJPO)BOEMFS  IUUQ)BOEMF'VOD lTUBUHPz TUBUT(P)BOEMFS  IUUQ)BOEMF'VOD lTUBUBQQz HBVSVO4UBUT(BVSVO)BOEMFS  ʜ TUBSU)551TFSWFS IUUQ-JTUFO"OE4FSWF lz OJM
  22. start workers for push UIFDPEFCFMPXJTTJNQMJpFEGPSFYQMBOBUJPO GVOD4UBSU1VTI8PSLFST XPSLFS/VN RVFVF/VNJOU \ 2VFVFNBLF

    DIBO/PUJpDBUJPO RVFVF/VN  GPSJJXPSLFS/VNJ \ HPQVTI/PUJpDBUJPO8PSLFS  ^ ^
  23. POST /push UIFDPEFCFMPXJTTJNQMJpFEGPSFYQMBOBUJPO GVOD1VTI/PUJpDBUJPO)BOEMFS XIUUQ3FTQPOTF8SJUFS S IUUQ3FRVFTU \ BDDFQUTPOMZ1045NFUIPE NFUIPE$IFDL

     SFBESFRVFTUCPEZ SCPEZ @JPVUJM3FBE"MM S#PEZ  QBSTFSFRVFTUCPEZ WBSSFR(BVSVO3FRVFTU(BVSVO @KTPO6ONBSTIBM SCPEZ SFR(BVSVO   FORVFVFOPUJpDBUJPOBTZODISPOPVTMZ HPFORVFVF/PUJpDBUJPOT SFR(BVSVO/PUJpDBUJPOT  SFUVSOUIFSFTQPOTF TFOE3FQPOTF X lPLz IUUQ4UBUVT0,  ^
  24. enqueue Notifications UIFDPEFCFMPXJTTJNQMJpFEGPSFYQMBOBUJPO GVODFORVFVF/PUJpDBUJPOT OT<>3FRVFTU/PUJpDBUJPO \ GPS@ OPUJpDBUJPOSBOHFOT\ 2VFVFOPUJpDBUJPO ^

    ^
  25. pushNotificationWorker UIFDPEFCFMPXJTTJNQMJpFEGPSFYQMBOBUJPO GVODQVTI/PUJpDBUJPO8PSLFS \ GPS\ OPUJpDBUJPO2VFVF TXJUDIOPUJpDBUJPO1MBUGPSN\ DBTF1MBUGPSN*PT QVTI/PUJpDBUJPO*PT 

    DBTF1MBUGPSN"OESPJE QVTI/PUJpDBUJPO*"OESPJE  ^ ^ ^
  26. Why workers with goroutine and queue with channel are static?

    w 6OMJNJUFEHPSPVUJOFT w 1FSGPSNBODFTMPXTEPXO w #FDBVTFQVTIJTTMPX w DIBOOFMJTSFRVJSFEGPSLFFQJOHXPSLFSTXBJUJOH w (P`TDIBOOFMTJ[FJTOPUVOMJNJUFE
  27. Gaurun Features

  28. HTTP APIs w 1045QVTI w (&5TUBUHP w (&5TUBUBQQ

  29. POST /push "DDFQUTCVMLFEQVTISFRVFTUXJUI+40/ \ lOPUJpDBUJPOz< \ lUPLFOTz<lYYYz>  lQMBUGPSNz 

    lNFTTBHFzl)FMMP J04z  lCBEHFz  lTPVOEzlEFGBVMUz  lFYQJSZz ^  \ lUPLFOTz<lZZZz>  lQMBUGPSNz  lNFTTBHFzl)FMMP "OESPJEz  lDPMMBQTF@LFZzlVQEBUFz  lEFMBZ@XIJMF@JEMFzUSVF  lUJNF@UP@MJWFz ^ > ^
  30. GET /stat/go DVSMTIUUQTUBUHP \ lUJNFz  lHP@WFSTJPOzlHPz  lHP@PTzlMJOVYz 

    lHP@BSDIzlBNEz  lDQV@OVNz  lHPSPVUJOF@OVNz  lHPNBYQSPDTz  lDHP@DBMM@OVNz  lNFNPSZ@BMMPDz  lNFNPSZ@UPUBM@BMMPDz  lNFNPSZ@TZTz  lNFNPSZ@MPPLVQTz  lNFNPSZ@NBMMPDTz  lNFNPSZ@GSFFTz  ɾ ɾ ɾ 3FUVSOTBTUBUJDUJDTGPS(PSVOUJNF
  31. GET /stat/app DVSMTIUUQTUBUBQQ \ lRVFVF@NBYz  lRVFVF@VTBHFz  lJPTz\ lQVTI@TVDDFTTz

     lQVTI@FSSPSz ^  lBOESPJEz\ lQVTI@TVDDFTTz  lQVTI@FSSPSz ^ ^  3FUVSOTBTUBUJDUJDTGPS(BVSVO
  32. More details… IUUQTHJUIVCDPNNFSDBSJHBVSVOCMPCNBTUFS41&$NE

  33. Gaurun Configuration HBVSVODHBVSVOUPNM

  34. Gaurun Configuration (TOML) <DPSF> QPSUlz XPSLFST RVFVFT <BOESPJE> BQJLFZlʜz <JPT>

    QFN@DFSU@QBUIlQBUIDFSUQFNz FN@LFZ@QBUIlQFNLFZQFNz TBOECPYGBMTF UJNFPVU@FSSPS
  35. Optimization w DPSFXPSLFST w OVNCFSPGXPSLFSTGPSQVTI w DPSFRVFVFT w TJ[FPGDIBOOFMGPSQVTIXPSLFST w

    JPTUJNFPVU@FSSPS w UJNFPVUGPSXBJUJOHFSSPSGSPN"1/T w EFGBVMUDPOpHVSBUJPOJTDPTFSWBUJWF
  36. More details… IUUQTHJUIVCDPNNFSDBSJHBVSVOCMPCNBTUFS$0/'*(63"5*0/NE

  37. Monitoring Gaurun

  38. Monitoring Gaurun w (&5TUBUHP w 3FUVSOTBTUBUJDUJDTGPS(PSVOUJNF w (&5TUBUBQQ w 3FUVSOTBTUBUJDUJDTGPS(BVSVO

  39. Monitoring Gaurun with Zabbix VTFSQBSBNFUFS@HPMBOH@TUBUTDPOG 6TFS1BSBNFUFSHPMBOHTUBUT< > DVSMTIUUQTUBUHPcKR

  40. Monitoring server written in Go w .FNPSZVTBHFBOEBMMPDBUJPOT w ($DPOEJUJPO w

    OVNCFSPGHPSPVUJOFT w DIBOOFMVTBHF w OVNCFSPGDPOOFDUJPOT SFRVFTUTTFD FUDʜ
  41. The case of Gaurun w (&5TUBUHP w .FNPSZVTBHFBOEBMMPDBUJPOT w ($DPOEJUJPO

    w OVNCFSPGHPSPVUJOFT w (&5TUBUBQQ w DIBOOFMVTBHF w OHY@IUUQ@TUVC@TUBUVT@NPEVMF w OVNCFSPGDPOOFDUJPOT SFRVFTUTTFD FUDʜ
  42. Potential problems w HPSPVUJOFMFBL w -JOFBSJODSFBTFPGOVNCFSPGHPSPVUJOFT w *U`TBNPSFTFSJPVTCVHUIBONFNPSZMFBL w DIBOOFMCMPDLJOH

    w DIBOOFMTJ[FJTMJNJUFE
  43. Library For Monitoring Go-runtime w GVLBUBHPMBOHTUBUTBQJIBOEMFS w 7FSZVTFGVM w (&5TUBUHPJTQPXFSFECZUIJT

  44. Summary w (PJTTVJUBCMFGPSTJNQMF)551"1*TFSWFS w (BVSVOXPSLTpOF w .POJUPSJOHTFSWFSXSJUUFOJO(P w $BSFQPUFOUJBMQSPCMFNTCMPX w

    HPSPVUJOFMFBL w DIBOOFMCMPDLJOH
  45. ਓࡐืूதʂ