Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Gaurun / A general push notification server in Go

Gaurun / A general push notification server in Go

Gaurun 〜A general push notification server in Go〜

Tatsuhiko Kubo

June 21, 2015
Tweet

More Decks by Tatsuhiko Kubo

Other Decks in Technology

Transcript

  1. A general push notification server in Go
    Tatsuhiko Kubo@cubicdaiya
    Go Conference summer@2015/06/21
    Gaurun

    View full-size slide

  2. @cubicdaiya / Tatsuhiko Kubo
    w 4PGUXBSF&OHJOFFS!.FSDBSJ *OD
    w *OGSBTUSVDUVSF&OHJOFFSJOH
    w (P
    w OHJOYCVJME TMBDLCPBSE (BVSVO
    w OHJOY
    w OHY@TNBMM@MJHIU OHY@EZOBNJD@VQTUSFBN

    View full-size slide

  3. Other Activities
    IUUQNP[BJDGNQPTUOHJOY

    View full-size slide

  4. mercari.go
    w DVCJDEBJZBOHJOYCVJME
    w DVCJDEBJZBDBDIFDUM
    w DVCJDEBJZBTMBDLCPBSE
    w NFSDBSJ(BVSVO
    (PJOQSPEVDUJPO!.FSDBSJ

    View full-size slide

  5. mercari.go
    w DVCJDEBJZBOHJOYCVJME
    w DVCJDEBJZBDBDIFDUM
    w DVCJDEBJZBTMBDLCPBSE
    w NFSDBSJ(BVSVO
    (PJOQSPEVDUJPO!.FSDBSJ

    View full-size slide

  6. Gaurun
    https://github.com/mercari/gaurun

    View full-size slide

  7. Gaurun
    w "HFOFSBMQVTIOPUJpDBUJPOTFSWFS
    GPSTNBSUQIPOFBQQMJDBUJPOT
    w 8SJUUFOJO(P
    w QSPYJOHQVTISFRVFTUTUP"1/TBOE($.
    w 4JNQMF)551"1*T

    View full-size slide

  8. Overview
    ($.
    "1/T
    (BVSVO
    )5514
    "1/T
    QSPUPDPM
    )551
    1045QVTI
    #PEZJT+40/
    $MJFOU

    View full-size slide

  9. Why Go?
    w 3FRVJSFNFOUT
    w (PPEQFSGPSNBODF
    w )JHIDPODVSSFODZ
    w OFUIUUQJTHPPEDIPJDF

    View full-size slide

  10. Why Go?
    w 1VTIJTTMPX
    w %VFUPOFUXPSLMBUFODZ
    w 4PNFPQFSBUJPO!.FSDBSJ
    w 4FOENJMMJPOQVTIBUPODFʂ
    w )JHIDPODVSSFODZJTSFRVJSFEʂ

    View full-size slide

  11. Why Go?
    w "TZODSPOPVTQVTIJTSFRVJSFE
    w QVTIJTUSJHHFSFEJO"1*TFSWFS
    w DPNNFOU MJLF CVZ TFMM
    w "1*TFSWFSNVTUSFTQPOTFRVJDLMZ
    w "TZODISPOJ[BUJPOJTDPNQMJDBUFE
    w "1*TFSWFS!.FSDBSJJT"QBDIF1)1

    View full-size slide

  12. Push System @ Mercari

    View full-size slide

  13. a year ago…
    ($.
    "1/T
    "1*4FSWFS
    "1*4FSWFS
    "1*4FSWFS
    "MMQVTIOPUJpDBUJPOTBSFTZODISPOPVT
    QVTI

    View full-size slide

  14. half a year ago…
    ($.
    "1/T
    8PSLFS
    "MMQVTIOPUJpDBUJPOTBSFBTZODISPOPVT
    2VFVF
    8PSLFS
    8PSLFS
    QVTI
    EFRVFVF
    "1*4FSWFS
    "1*4FSWFS
    #BUDI4FSWFS
    FORVFVF
    "1*4FSWFS

    View full-size slide

  15. But very complicated…
    w +PC2VFVF
    w 2.
    w .VMUJ1SPDFTT8PSLFSXJUI1)1
    w 1)11BSBMMFMT1SFGPSL
    w 5IFBSDIJUFDUVSFJTOPUTDBMBCMF

    View full-size slide

  16. Now…
    ($.
    "1/T
    (BVSVO
    (BVSVO
    (BVSVO
    /(*/9
    "1*4FSWFS
    "1*4FSWFS
    "1*4FSWFS
    #BUDI4FSWFS
    QVTI
    XJUI
    LFFQBMJWF
    )551
    1045QVTI

    View full-size slide

  17. Gaurun Internals

    View full-size slide

  18. How Gaurun pushesʁ
    ($.
    "1/T
    (BVSVO
    )5514
    "1/T
    QSPUPDPM
    )551
    1045QVTI
    #PEZJT+40/
    $MJFOU

    View full-size slide

  19. Bootstrap Gaurun
    UIFDPEFCFMPXJTTJNQMJpFEGPSFYQMBOBUJPO
    TUBSUXPSLFSTGPSQVTI
    HBVSVO4UBSU1VTI8PSLFST XPSLFS/VN RVFVF/VN

    )551"1*T
    IUUQ)BOEMF'VOD lQVTIz HBVSVO1VTI/PUJpDBUJPO)BOEMFS

    IUUQ)BOEMF'VOD lTUBUHPz TUBUT(P)BOEMFS

    IUUQ)BOEMF'VOD lTUBUBQQz HBVSVO4UBUT(BVSVO)BOEMFS

    ʜ
    TUBSU)551TFSWFS
    IUUQ-JTUFO"OE4FSWF lz OJM

    View full-size slide

  20. start workers for push
    UIFDPEFCFMPXJTTJNQMJpFEGPSFYQMBOBUJPO
    GVOD4UBSU1VTI8PSLFST XPSLFS/VN RVFVF/VNJOU
    \
    2VFVFNBLF DIBO/PUJpDBUJPO RVFVF/VN

    GPSJJXPSLFS/VNJ\
    HPQVTI/PUJpDBUJPO8PSLFS

    ^
    ^

    View full-size slide

  21. POST /push
    UIFDPEFCFMPXJTTJNQMJpFEGPSFYQMBOBUJPO
    GVOD1VTI/PUJpDBUJPO)BOEMFS XIUUQ3FTQPOTF8SJUFS SIUUQ3FRVFTU
    \
    BDDFQUTPOMZ1045NFUIPE
    NFUIPE$IFDL

    SFBESFRVFTUCPEZ
    SCPEZ @JPVUJM3FBE"MM S#PEZ

    QBSTFSFRVFTUCPEZ
    WBSSFR(BVSVO3FRVFTU(BVSVO
    @KTPO6ONBSTIBM SCPEZ SFR(BVSVO


    FORVFVFOPUJpDBUJPOBTZODISPOPVTMZ
    HPFORVFVF/PUJpDBUJPOT SFR(BVSVO/PUJpDBUJPOT

    SFUVSOUIFSFTQPOTF
    TFOE3FQPOTF X lPLz IUUQ4UBUVT0,

    ^

    View full-size slide

  22. enqueue Notifications
    UIFDPEFCFMPXJTTJNQMJpFEGPSFYQMBOBUJPO
    GVODFORVFVF/PUJpDBUJPOT OT<>3FRVFTU/PUJpDBUJPO
    \
    GPS@ OPUJpDBUJPOSBOHFOT\
    2VFVFOPUJpDBUJPO
    ^
    ^

    View full-size slide

  23. pushNotificationWorker
    UIFDPEFCFMPXJTTJNQMJpFEGPSFYQMBOBUJPO
    GVODQVTI/PUJpDBUJPO8PSLFS
    \
    GPS\
    OPUJpDBUJPO2VFVF
    TXJUDIOPUJpDBUJPO1MBUGPSN\
    DBTF1MBUGPSN*PT
    QVTI/PUJpDBUJPO*PT

    DBTF1MBUGPSN"OESPJE
    QVTI/PUJpDBUJPO*"OESPJE

    ^
    ^
    ^

    View full-size slide

  24. Why workers with goroutine and
    queue with channel are static?
    w 6OMJNJUFEHPSPVUJOFT
    w 1FSGPSNBODFTMPXTEPXO
    w #FDBVTFQVTIJTTMPX
    w DIBOOFMJTSFRVJSFEGPSLFFQJOHXPSLFSTXBJUJOH
    w (P`TDIBOOFMTJ[FJTOPUVOMJNJUFE

    View full-size slide

  25. Gaurun Features

    View full-size slide

  26. HTTP APIs
    w 1045QVTI
    w (&5TUBUHP
    w (&5TUBUBQQ

    View full-size slide

  27. POST /push
    "DDFQUTCVMLFEQVTISFRVFTUXJUI+40/
    \
    lOPUJpDBUJPOz<
    \
    lUPLFOTz
    lQMBUGPSNz
    lNFTTBHFzl)FMMP J04z
    lCBEHFz
    lTPVOEzlEFGBVMUz
    lFYQJSZz
    ^
    \
    lUPLFOTz
    lQMBUGPSNz
    lNFTTBHFzl)FMMP "OESPJEz
    lDPMMBQTF@LFZzlVQEBUFz
    lEFMBZ@XIJMF@JEMFzUSVF
    lUJNF@UP@MJWFz
    ^
    >
    ^

    View full-size slide

  28. GET /stat/go
    DVSMTIUUQTUBUHP
    \
    lUJNFz
    lHP@WFSTJPOzlHPz
    lHP@PTzlMJOVYz
    lHP@BSDIzlBNEz
    lDQV@OVNz
    lHPSPVUJOF@OVNz
    lHPNBYQSPDTz
    lDHP@DBMM@OVNz
    lNFNPSZ@BMMPDz
    lNFNPSZ@UPUBM@BMMPDz
    lNFNPSZ@TZTz
    lNFNPSZ@MPPLVQTz
    lNFNPSZ@NBMMPDTz
    lNFNPSZ@GSFFTz
    ɾ
    ɾ
    ɾ
    3FUVSOTBTUBUJDUJDTGPS(PSVOUJNF

    View full-size slide

  29. GET /stat/app
    DVSMTIUUQTUBUBQQ
    \
    lRVFVF@NBYz
    lRVFVF@VTBHFz
    lJPTz\
    lQVTI@TVDDFTTz
    lQVTI@FSSPSz
    ^
    lBOESPJEz\
    lQVTI@TVDDFTTz
    lQVTI@FSSPSz
    ^
    ^

    3FUVSOTBTUBUJDUJDTGPS(BVSVO

    View full-size slide

  30. More details…
    IUUQTHJUIVCDPNNFSDBSJHBVSVOCMPCNBTUFS41&$NE

    View full-size slide

  31. Gaurun Configuration
    HBVSVODHBVSVOUPNM

    View full-size slide

  32. Gaurun Configuration (TOML)

    QPSUlz
    XPSLFST
    RVFVFT

    BQJLFZlʜz

    QFN@DFSU@QBUIlQBUIDFSUQFNz
    FN@LFZ@QBUIlQFNLFZQFNz
    TBOECPYGBMTF
    UJNFPVU@FSSPS

    View full-size slide

  33. Optimization
    w DPSFXPSLFST
    w OVNCFSPGXPSLFSTGPSQVTI
    w DPSFRVFVFT
    w TJ[FPGDIBOOFMGPSQVTIXPSLFST
    w JPTUJNFPVU@FSSPS
    w UJNFPVUGPSXBJUJOHFSSPSGSPN"1/T
    w EFGBVMUDPOpHVSBUJPOJTDPTFSWBUJWF

    View full-size slide

  34. More details…
    IUUQTHJUIVCDPNNFSDBSJHBVSVOCMPCNBTUFS$0/'*(63"5*0/NE

    View full-size slide

  35. Monitoring Gaurun

    View full-size slide

  36. Monitoring Gaurun
    w (&5TUBUHP
    w 3FUVSOTBTUBUJDUJDTGPS(PSVOUJNF
    w (&5TUBUBQQ
    w 3FUVSOTBTUBUJDUJDTGPS(BVSVO

    View full-size slide

  37. Monitoring Gaurun with Zabbix
    VTFSQBSBNFUFS@HPMBOH@TUBUTDPOG
    6TFS1BSBNFUFSHPMBOHTUBUT<> DVSMTIUUQTUBUHPcKR

    View full-size slide

  38. Monitoring server written in Go
    w .FNPSZVTBHFBOEBMMPDBUJPOT
    w ($DPOEJUJPO
    w OVNCFSPGHPSPVUJOFT
    w DIBOOFMVTBHF
    w OVNCFSPGDPOOFDUJPOT SFRVFTUTTFD FUDʜ

    View full-size slide

  39. The case of Gaurun
    w (&5TUBUHP
    w .FNPSZVTBHFBOEBMMPDBUJPOT
    w ($DPOEJUJPO
    w OVNCFSPGHPSPVUJOFT
    w (&5TUBUBQQ
    w DIBOOFMVTBHF
    w OHY@IUUQ@TUVC@TUBUVT@NPEVMF
    w OVNCFSPGDPOOFDUJPOT SFRVFTUTTFD FUDʜ

    View full-size slide

  40. Potential problems
    w HPSPVUJOFMFBL
    w -JOFBSJODSFBTFPGOVNCFSPGHPSPVUJOFT
    w *U`TBNPSFTFSJPVTCVHUIBONFNPSZMFBL
    w DIBOOFMCMPDLJOH
    w DIBOOFMTJ[FJTMJNJUFE

    View full-size slide

  41. Library For Monitoring Go-runtime
    w GVLBUBHPMBOHTUBUTBQJIBOEMFS
    w 7FSZVTFGVM
    w (&5TUBUHPJTQPXFSFECZUIJT

    View full-size slide

  42. Summary
    w (PJTTVJUBCMFGPSTJNQMF)551"1*TFSWFS
    w (BVSVOXPSLTpOF
    w .POJUPSJOHTFSWFSXSJUUFOJO(P
    w $BSFQPUFOUJBMQSPCMFNTCMPX
    w HPSPVUJOFMFBL
    w DIBOOFMCMPDLJOH

    View full-size slide

  43. ਓࡐืूதʂ

    View full-size slide