Gaurun / A general push notification server in Go

Gaurun 〜A general push notification server in Go〜

Transcript

1. A general push notification server in Go
   Tatsuhiko Kubo@cubicdaiya
   Go Conference summer@2015/06/21
   Gaurun
   

   View Slide

2. @cubicdaiya / Tatsuhiko Kubo
   w 4PGUXBSF&OHJOFFS!.FSDBSJ *OD
   w *OGSBTUSVDUVSF&OHJOFFSJOH
   w (P
   w OHJOYCVJME TMBDLCPBSE (BVSVO
   w OHJOY
   w OHY@TNBMM@MJHIU OHY@EZOBNJD@VQTUSFBN
   

   View Slide

3. Other Activities
   IUUQNP[BJDGNQPTUOHJOY
   

   View Slide

4. View Slide

5. View Slide

6. mercari.go
   w DVCJDEBJZBOHJOYCVJME
   w DVCJDEBJZBDBDIFDUM
   w DVCJDEBJZBTMBDLCPBSE
   w NFSDBSJ(BVSVO
   (PJOQSPEVDUJPO!.FSDBSJ
   

   View Slide

7. mercari.go
   w DVCJDEBJZBOHJOYCVJME
   w DVCJDEBJZBDBDIFDUM
   w DVCJDEBJZBTMBDLCPBSE
   w NFSDBSJ(BVSVO
   (PJOQSPEVDUJPO!.FSDBSJ
   

   View Slide

8. Gaurun
   https://github.com/mercari/gaurun
   

   View Slide

9. Gaurun
   w "HFOFSBMQVTIOPUJpDBUJPOTFSWFS
   GPSTNBSUQIPOFBQQMJDBUJPOT
   w 8SJUUFOJO(P
   w QSPYJOHQVTISFRVFTUTUP"1/TBOE($.
   w 4JNQMF)551"1*T
   

   View Slide

10. Overview
    ($.
    "1/T
    (BVSVO
    )5514
    "1/T
    QSPUPDPM
    )551
    1045QVTI
    #PEZJT+40/
    $MJFOU
    

    View Slide

11. Why Go?
    w 3FRVJSFNFOUT
    w (PPEQFSGPSNBODF
    w )JHIDPODVSSFODZ
    w OFUIUUQJTHPPEDIPJDF
    

    View Slide

12. Why Go?
    w 1VTIJTTMPX
    w %VFUPOFUXPSLMBUFODZ
    w 4PNFPQFSBUJPO!.FSDBSJ
    w 4FOENJMMJPOQVTIBUPODFʂ
    w )JHIDPODVSSFODZJTSFRVJSFEʂ
    

    View Slide

13. Why Go?
    w "TZODSPOPVTQVTIJTSFRVJSFE
    w QVTIJTUSJHHFSFEJO"1*TFSWFS
    w DPNNFOU MJLF CVZ TFMM
    w "1*TFSWFSNVTUSFTQPOTFRVJDLMZ
    w "TZODISPOJ[BUJPOJTDPNQMJDBUFE
    w "1*TFSWFS!.FSDBSJJT"QBDIF1)1
    

    View Slide

14. Push System @ Mercari
    

    View Slide

15. a year ago…
    ($.
    "1/T
    "1*4FSWFS
    "1*4FSWFS
    "1*4FSWFS
    "MMQVTIOPUJpDBUJPOTBSFTZODISPOPVT
    QVTI
    

    View Slide

16. half a year ago…
    ($.
    "1/T
    8PSLFS
    "MMQVTIOPUJpDBUJPOTBSFBTZODISPOPVT
    2VFVF
    8PSLFS
    8PSLFS
    QVTI
    EFRVFVF
    "1*4FSWFS
    "1*4FSWFS
    #BUDI4FSWFS
    FORVFVF
    "1*4FSWFS
    

    View Slide

17. But very complicated…
    w +PC2VFVF
    w 2.
    w .VMUJ1SPDFTT8PSLFSXJUI1)1
    w 1)11BSBMMFMT1SFGPSL
    w 5IFBSDIJUFDUVSFJTOPUTDBMBCMF
    

    View Slide

18. Now…
    ($.
    "1/T
    (BVSVO
    (BVSVO
    (BVSVO
    /(*/9
    "1*4FSWFS
    "1*4FSWFS
    "1*4FSWFS
    #BUDI4FSWFS
    QVTI
    XJUI
    LFFQBMJWF
    )551
    1045QVTI
    

    View Slide

19. Gaurun Internals
    

    View Slide

20. How Gaurun pushesʁ
    ($.
    "1/T
    (BVSVO
    )5514
    "1/T
    QSPUPDPM
    )551
    1045QVTI
    #PEZJT+40/
    $MJFOU
    

    View Slide

21. Bootstrap Gaurun
    UIFDPEFCFMPXJTTJNQMJpFEGPSFYQMBOBUJPO
    TUBSUXPSLFSTGPSQVTI
    HBVSVO4UBSU1VTI8PSLFST XPSLFS/VN RVFVF/VN
    
    )551"1*T
    IUUQ)BOEMF'VOD lQVTIz HBVSVO1VTI/PUJpDBUJPO)BOEMFS
    
    IUUQ)BOEMF'VOD lTUBUHPz TUBUT(P)BOEMFS
    
    IUUQ)BOEMF'VOD lTUBUBQQz HBVSVO4UBUT(BVSVO)BOEMFS
    
    ʜ
    TUBSU)551TFSWFS
    IUUQ-JTUFO"OE4FSWF lz OJM
    
    

    View Slide

22. start workers for push
    UIFDPEFCFMPXJTTJNQMJpFEGPSFYQMBOBUJPO
    GVOD4UBSU1VTI8PSLFST XPSLFS/VN RVFVF/VNJOU
    \
    2VFVFNBLF DIBO/PUJpDBUJPO RVFVF/VN
    
    GPSJJXPSLFS/VNJ\
    HPQVTI/PUJpDBUJPO8PSLFS
    
    ^
    ^
    

    View Slide

23. POST /push
    UIFDPEFCFMPXJTTJNQMJpFEGPSFYQMBOBUJPO
    GVOD1VTI/PUJpDBUJPO)BOEMFS XIUUQ3FTQPOTF8SJUFS SIUUQ3FRVFTU
    \
    BDDFQUTPOMZ1045NFUIPE
    NFUIPE$IFDL
    
    SFBESFRVFTUCPEZ
    SCPEZ @JPVUJM3FBE"MM S#PEZ
    
    QBSTFSFRVFTUCPEZ
    WBSSFR(BVSVO3FRVFTU(BVSVO
    @KTPO6ONBSTIBM SCPEZ SFR(BVSVO
    
    
    FORVFVFOPUJpDBUJPOBTZODISPOPVTMZ
    HPFORVFVF/PUJpDBUJPOT SFR(BVSVO/PUJpDBUJPOT
    
    SFUVSOUIFSFTQPOTF
    TFOE3FQPOTF X lPLz IUUQ4UBUVT0,
    
    ^
    

    View Slide

24. enqueue Notifications
    UIFDPEFCFMPXJTTJNQMJpFEGPSFYQMBOBUJPO
    GVODFORVFVF/PUJpDBUJPOT OT<>3FRVFTU/PUJpDBUJPO
    \
    GPS@ OPUJpDBUJPOSBOHFOT\
    2VFVFOPUJpDBUJPO
    ^
    ^
    

    View Slide

25. pushNotificationWorker
    UIFDPEFCFMPXJTTJNQMJpFEGPSFYQMBOBUJPO
    GVODQVTI/PUJpDBUJPO8PSLFS
    \
    GPS\
    OPUJpDBUJPO2VFVF
    TXJUDIOPUJpDBUJPO1MBUGPSN\
    DBTF1MBUGPSN*PT
    QVTI/PUJpDBUJPO*PT
    
    DBTF1MBUGPSN"OESPJE
    QVTI/PUJpDBUJPO*"OESPJE
    
    ^
    ^
    ^
    

    View Slide

26. Why workers with goroutine and
    queue with channel are static?
    w 6OMJNJUFEHPSPVUJOFT
    w 1FSGPSNBODFTMPXTEPXO
    w #FDBVTFQVTIJTTMPX
    w DIBOOFMJTSFRVJSFEGPSLFFQJOHXPSLFSTXBJUJOH
    w (P`TDIBOOFMTJ[FJTOPUVOMJNJUFE
    

    View Slide

27. Gaurun Features
    

    View Slide

28. HTTP APIs
    w 1045QVTI
    w (&5TUBUHP
    w (&5TUBUBQQ
    

    View Slide

29. POST /push
    "DDFQUTCVMLFEQVTISFRVFTUXJUI+40/
    \
    lOPUJpDBUJPOz
    \
    lUPLFOTz
    lQMBUGPSNz
    lNFTTBHFzl)FMMP J04z
    lCBEHFz
    lTPVOEzlEFGBVMUz
    lFYQJSZz
    ^
    \
    lUPLFOTz
    lQMBUGPSNz
    lNFTTBHFzl)FMMP "OESPJEz
    lDPMMBQTF@LFZzlVQEBUFz
    lEFMBZ@XIJMF@JEMFzUSVF
    lUJNF@UP@MJWFz
    ^
    >
    ^
    

    View Slide

30. GET /stat/go
    DVSMTIUUQTUBUHP
    \
    lUJNFz
    lHP@WFSTJPOzlHPz
    lHP@PTzlMJOVYz
    lHP@BSDIzlBNEz
    lDQV@OVNz
    lHPSPVUJOF@OVNz
    lHPNBYQSPDTz
    lDHP@DBMM@OVNz
    lNFNPSZ@BMMPDz
    lNFNPSZ@UPUBM@BMMPDz
    lNFNPSZ@TZTz
    lNFNPSZ@MPPLVQTz
    lNFNPSZ@NBMMPDTz
    lNFNPSZ@GSFFTz
    ɾ
    ɾ
    ɾ
    3FUVSOTBTUBUJDUJDTGPS(PSVOUJNF
    

    View Slide

31. GET /stat/app
    DVSMTIUUQTUBUBQQ
    \
    lRVFVF@NBYz
    lRVFVF@VTBHFz
    lJPTz\
    lQVTI@TVDDFTTz
    lQVTI@FSSPSz
    ^
    lBOESPJEz\
    lQVTI@TVDDFTTz
    lQVTI@FSSPSz
    ^
    ^
    
    3FUVSOTBTUBUJDUJDTGPS(BVSVO
    

    View Slide

32. More details…
    IUUQTHJUIVCDPNNFSDBSJHBVSVOCMPCNBTUFS41&$NE
    

    View Slide

33. Gaurun Configuration
    HBVSVODHBVSVOUPNM
    

    View Slide

34. Gaurun Configuration (TOML)
    
    QPSUlz
    XPSLFST
    RVFVFT
    
    BQJLFZlʜz
    
    QFN@DFSU@QBUIlQBUIDFSUQFNz
    FN@LFZ@QBUIlQFNLFZQFNz
    TBOECPYGBMTF
    UJNFPVU@FSSPS
    

    View Slide

35. Optimization
    w DPSFXPSLFST
    w OVNCFSPGXPSLFSTGPSQVTI
    w DPSFRVFVFT
    w TJ[FPGDIBOOFMGPSQVTIXPSLFST
    w JPTUJNFPVU@FSSPS
    w UJNFPVUGPSXBJUJOHFSSPSGSPN"1/T
    w EFGBVMUDPOpHVSBUJPOJTDPTFSWBUJWF
    

    View Slide

36. More details…
    IUUQTHJUIVCDPNNFSDBSJHBVSVOCMPCNBTUFS$0/'*(63"5*0/NE
    

    View Slide

37. Monitoring Gaurun
    

    View Slide

38. Monitoring Gaurun
    w (&5TUBUHP
    w 3FUVSOTBTUBUJDUJDTGPS(PSVOUJNF
    w (&5TUBUBQQ
    w 3FUVSOTBTUBUJDUJDTGPS(BVSVO
    

    View Slide

39. Monitoring Gaurun with Zabbix
    VTFSQBSBNFUFS@HPMBOH@TUBUTDPOG
    6TFS1BSBNFUFSHPMBOHTUBUT> DVSMTIUUQTUBUHPcKR
    

    View Slide

40. Monitoring server written in Go
    w .FNPSZVTBHFBOEBMMPDBUJPOT
    w ($DPOEJUJPO
    w OVNCFSPGHPSPVUJOFT
    w DIBOOFMVTBHF
    w OVNCFSPGDPOOFDUJPOT SFRVFTUTTFD FUDʜ
    

    View Slide

41. The case of Gaurun
    w (&5TUBUHP
    w .FNPSZVTBHFBOEBMMPDBUJPOT
    w ($DPOEJUJPO
    w OVNCFSPGHPSPVUJOFT
    w (&5TUBUBQQ
    w DIBOOFMVTBHF
    w OHY@IUUQ@TUVC@TUBUVT@NPEVMF
    w OVNCFSPGDPOOFDUJPOT SFRVFTUTTFD FUDʜ
    

    View Slide

42. Potential problems
    w HPSPVUJOFMFBL
    w -JOFBSJODSFBTFPGOVNCFSPGHPSPVUJOFT
    w *U`TBNPSFTFSJPVTCVHUIBONFNPSZMFBL
    w DIBOOFMCMPDLJOH
    w DIBOOFMTJ[FJTMJNJUFE
    

    View Slide

43. Library For Monitoring Go-runtime
    w GVLBUBHPMBOHTUBUTBQJIBOEMFS
    w 7FSZVTFGVM
    w (&5TUBUHPJTQPXFSFECZUIJT
    

    View Slide

44. Summary
    w (PJTTVJUBCMFGPSTJNQMF)551"1*TFSWFS
    w (BVSVOXPSLTpOF
    w .POJUPSJOHTFSWFSXSJUUFOJO(P
    w $BSFQPUFOUJBMQSPCMFNTCMPX
    w HPSPVUJOFMFBL
    w DIBOOFMCMPDLJOH
    

    View Slide

45. ਓࡐืूதʂ
    

    View Slide