Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Gaurun / A general push notification server in Go

Gaurun / A general push notification server in Go

Gaurun 〜A general push notification server in Go〜

Tatsuhiko Kubo

June 21, 2015
Tweet

More Decks by Tatsuhiko Kubo

Other Decks in Technology

Transcript

  1. A general push notification server in Go
    Tatsuhiko [email protected]
    Go Conference [email protected]/06/21
    Gaurun

    View Slide

  2. @cubicdaiya / Tatsuhiko Kubo
    w 4PGUXBSF&OHJOFFS!.FSDBSJ *OD
    w *OGSBTUSVDUVSF&OHJOFFSJOH
    w (P
    w OHJOYCVJME TMBDLCPBSE (BVSVO
    w OHJOY
    w [email protected]@MJHIU [email protected]@VQTUSFBN

    View Slide

  3. Other Activities
    IUUQNP[BJDGNQPTUOHJOY

    View Slide

  4. View Slide

  5. View Slide

  6. mercari.go
    w DVCJDEBJZBOHJOYCVJME
    w DVCJDEBJZBDBDIFDUM
    w DVCJDEBJZBTMBDLCPBSE
    w NFSDBSJ(BVSVO
    (PJOQSPEVDUJPO!.FSDBSJ

    View Slide

  7. mercari.go
    w DVCJDEBJZBOHJOYCVJME
    w DVCJDEBJZBDBDIFDUM
    w DVCJDEBJZBTMBDLCPBSE
    w NFSDBSJ(BVSVO
    (PJOQSPEVDUJPO!.FSDBSJ

    View Slide

  8. Gaurun
    https://github.com/mercari/gaurun

    View Slide

  9. Gaurun
    w "HFOFSBMQVTIOPUJpDBUJPOTFSWFS
    GPSTNBSUQIPOFBQQMJDBUJPOT
    w 8SJUUFOJO(P
    w QSPYJOHQVTISFRVFTUTUP"1/TBOE($.
    w 4JNQMF)551"1*T

    View Slide

  10. Overview
    ($.
    "1/T
    (BVSVO
    )5514
    "1/T
    QSPUPDPM
    )551
    1045QVTI
    #PEZJT+40/
    $MJFOU

    View Slide

  11. Why Go?
    w 3FRVJSFNFOUT
    w (PPEQFSGPSNBODF
    w )JHIDPODVSSFODZ
    w OFUIUUQJTHPPEDIPJDF

    View Slide

  12. Why Go?
    w 1VTIJTTMPX
    w %VFUPOFUXPSLMBUFODZ
    w 4PNFPQFSBUJPO!.FSDBSJ
    w 4FOENJMMJPOQVTIBUPODFʂ
    w )JHIDPODVSSFODZJTSFRVJSFEʂ

    View Slide

  13. Why Go?
    w "TZODSPOPVTQVTIJTSFRVJSFE
    w QVTIJTUSJHHFSFEJO"1*TFSWFS
    w DPNNFOU MJLF CVZ TFMM
    w "1*TFSWFSNVTUSFTQPOTFRVJDLMZ
    w "TZODISPOJ[BUJPOJTDPNQMJDBUFE
    w "1*TFSWFS!.FSDBSJJT"QBDIF1)1

    View Slide

  14. Push System @ Mercari

    View Slide

  15. a year ago…
    ($.
    "1/T
    "1*4FSWFS
    "1*4FSWFS
    "1*4FSWFS
    "MMQVTIOPUJpDBUJPOTBSFTZODISPOPVT
    QVTI

    View Slide

  16. half a year ago…
    ($.
    "1/T
    8PSLFS
    "MMQVTIOPUJpDBUJPOTBSFBTZODISPOPVT
    2VFVF
    8PSLFS
    8PSLFS
    QVTI
    EFRVFVF
    "1*4FSWFS
    "1*4FSWFS
    #BUDI4FSWFS
    FORVFVF
    "1*4FSWFS

    View Slide

  17. But very complicated…
    w +PC2VFVF
    w 2.
    w .VMUJ1SPDFTT8PSLFSXJUI1)1
    w 1)11BSBMMFMT1SFGPSL
    w 5IFBSDIJUFDUVSFJTOPUTDBMBCMF

    View Slide

  18. Now…
    ($.
    "1/T
    (BVSVO
    (BVSVO
    (BVSVO
    /(*/9
    "1*4FSWFS
    "1*4FSWFS
    "1*4FSWFS
    #BUDI4FSWFS
    QVTI
    XJUI
    LFFQBMJWF
    )551
    1045QVTI

    View Slide

  19. Gaurun Internals

    View Slide

  20. How Gaurun pushesʁ
    ($.
    "1/T
    (BVSVO
    )5514
    "1/T
    QSPUPDPM
    )551
    1045QVTI
    #PEZJT+40/
    $MJFOU

    View Slide

  21. Bootstrap Gaurun
    UIFDPEFCFMPXJTTJNQMJpFEGPSFYQMBOBUJPO
    TUBSUXPSLFSTGPSQVTI
    HBVSVO4UBSU1VTI8PSLFST XPSLFS/VN RVFVF/VN

    )551"1*T
    IUUQ)BOEMF'VOD lQVTIz HBVSVO1VTI/PUJpDBUJPO)BOEMFS

    IUUQ)BOEMF'VOD lTUBUHPz TUBUT(P)BOEMFS

    IUUQ)BOEMF'VOD lTUBUBQQz HBVSVO4UBUT(BVSVO)BOEMFS

    ʜ
    TUBSU)551TFSWFS
    IUUQ-JTUFO"OE4FSWF lz OJM

    View Slide

  22. start workers for push
    UIFDPEFCFMPXJTTJNQMJpFEGPSFYQMBOBUJPO
    GVOD4UBSU1VTI8PSLFST XPSLFS/VN RVFVF/VNJOU
    \
    2VFVFNBLF DIBO/PUJpDBUJPO RVFVF/VN

    GPSJJXPSLFS/VNJ\
    HPQVTI/PUJpDBUJPO8PSLFS

    ^
    ^

    View Slide

  23. POST /push
    UIFDPEFCFMPXJTTJNQMJpFEGPSFYQMBOBUJPO
    GVOD1VTI/PUJpDBUJPO)BOEMFS XIUUQ3FTQPOTF8SJUFS SIUUQ3FRVFTU
    \
    BDDFQUTPOMZ1045NFUIPE
    NFUIPE$IFDL

    SFBESFRVFTUCPEZ
    SCPEZ @JPVUJM3FBE"MM S#PEZ

    QBSTFSFRVFTUCPEZ
    WBSSFR(BVSVO3FRVFTU(BVSVO
    @KTPO6ONBSTIBM SCPEZ SFR(BVSVO


    FORVFVFOPUJpDBUJPOBTZODISPOPVTMZ
    HPFORVFVF/PUJpDBUJPOT SFR(BVSVO/PUJpDBUJPOT

    SFUVSOUIFSFTQPOTF
    TFOE3FQPOTF X lPLz IUUQ4UBUVT0,

    ^

    View Slide

  24. enqueue Notifications
    UIFDPEFCFMPXJTTJNQMJpFEGPSFYQMBOBUJPO
    GVODFORVFVF/PUJpDBUJPOT OT<>3FRVFTU/PUJpDBUJPO
    \
    [email protected] OPUJpDBUJPOSBOHFOT\
    2VFVFOPUJpDBUJPO
    ^
    ^

    View Slide

  25. pushNotificationWorker
    UIFDPEFCFMPXJTTJNQMJpFEGPSFYQMBOBUJPO
    GVODQVTI/PUJpDBUJPO8PSLFS
    \
    GPS\
    OPUJpDBUJPO2VFVF
    TXJUDIOPUJpDBUJPO1MBUGPSN\
    DBTF1MBUGPSN*PT
    QVTI/PUJpDBUJPO*PT

    DBTF1MBUGPSN"OESPJE
    QVTI/PUJpDBUJPO*"OESPJE

    ^
    ^
    ^

    View Slide

  26. Why workers with goroutine and
    queue with channel are static?
    w 6OMJNJUFEHPSPVUJOFT
    w 1FSGPSNBODFTMPXTEPXO
    w #FDBVTFQVTIJTTMPX
    w DIBOOFMJTSFRVJSFEGPSLFFQJOHXPSLFSTXBJUJOH
    w (P`TDIBOOFMTJ[FJTOPUVOMJNJUFE

    View Slide

  27. Gaurun Features

    View Slide

  28. HTTP APIs
    w 1045QVTI
    w (&5TUBUHP
    w (&5TUBUBQQ

    View Slide

  29. POST /push
    "DDFQUTCVMLFEQVTISFRVFTUXJUI+40/
    \
    lOPUJpDBUJPOz
    \
    lUPLFOTz
    lQMBUGPSNz
    lNFTTBHFzl)FMMP J04z
    lCBEHFz
    lTPVOEzlEFGBVMUz
    lFYQJSZz
    ^
    \
    lUPLFOTz
    lQMBUGPSNz
    lNFTTBHFzl)FMMP "OESPJEz
    [email protected]
    [email protected]@JEMFzUSVF
    [email protected]@MJWFz
    ^
    >
    ^

    View Slide

  30. View Slide

  31. GET /stat/app
    DVSMTIUUQTUBUBQQ
    \
    [email protected]Yz
    [email protected]
    lJPTz\
    [email protected]
    [email protected]
    ^
    lBOESPJEz\
    [email protected]
    [email protected]
    ^
    ^

    3FUVSOTBTUBUJDUJDTGPS(BVSVO

    View Slide

  32. More details…
    IUUQTHJUIVCDPNNFSDBSJHBVSVOCMPCNBTUFS41&$NE

    View Slide

  33. Gaurun Configuration
    HBVSVODHBVSVOUPNM

    View Slide

  34. Gaurun Configuration (TOML)

    QPSUlz
    XPSLFST
    RVFVFT

    BQJLFZlʜz

    [email protected]@QBUIlQBUIDFSUQFNz
    [email protected]@QBUIlQFNLFZQFNz
    TBOECPYGBMTF
    [email protected]

    View Slide

  35. Optimization
    w DPSFXPSLFST
    w OVNCFSPGXPSLFSTGPSQVTI
    w DPSFRVFVFT
    w TJ[FPGDIBOOFMGPSQVTIXPSLFST
    w [email protected]
    w UJNFPVUGPSXBJUJOHFSSPSGSPN"1/T
    w EFGBVMUDPOpHVSBUJPOJTDPTFSWBUJWF

    View Slide

  36. More details…
    IUUQTHJUIVCDPNNFSDBSJHBVSVOCMPCNBTUFS$0/'*(63"5*0/NE

    View Slide

  37. Monitoring Gaurun

    View Slide

  38. Monitoring Gaurun
    w (&5TUBUHP
    w 3FUVSOTBTUBUJDUJDTGPS(PSVOUJNF
    w (&5TUBUBQQ
    w 3FUVSOTBTUBUJDUJDTGPS(BVSVO

    View Slide

  39. Monitoring Gaurun with Zabbix
    VTFSQBSBNF[email protected]@TUBUTDPOG
    6TFS1BSBNFUFSHPMBOHTUBUT> DVSMTIUUQTUBUHPcKR

    View Slide

  40. Monitoring server written in Go
    w .FNPSZVTBHFBOEBMMPDBUJPOT
    w ($DPOEJUJPO
    w OVNCFSPGHPSPVUJOFT
    w DIBOOFMVTBHF
    w OVNCFSPGDPOOFDUJPOT SFRVFTUTTFD FUDʜ

    View Slide

  41. The case of Gaurun
    w (&5TUBUHP
    w .FNPSZVTBHFBOEBMMPDBUJPOT
    w ($DPOEJUJPO
    w OVNCFSPGHPSPVUJOFT
    w (&5TUBUBQQ
    w DIBOOFMVTBHF
    w [email protected]@[email protected]@NPEVMF
    w OVNCFSPGDPOOFDUJPOT SFRVFTUTTFD FUDʜ

    View Slide

  42. Potential problems
    w HPSPVUJOFMFBL
    w -JOFBSJODSFBTFPGOVNCFSPGHPSPVUJOFT
    w *U`TBNPSFTFSJPVTCVHUIBONFNPSZMFBL
    w DIBOOFMCMPDLJOH
    w DIBOOFMTJ[FJTMJNJUFE

    View Slide

  43. Library For Monitoring Go-runtime
    w GVLBUBHPMBOHTUBUTBQJIBOEMFS
    w 7FSZVTFGVM
    w (&5TUBUHPJTQPXFSFECZUIJT

    View Slide

  44. Summary
    w (PJTTVJUBCMFGPSTJNQMF)551"1*TFSWFS
    w (BVSVOXPSLTpOF
    w .POJUPSJOHTFSWFSXSJUUFOJO(P
    w $BSFQPUFOUJBMQSPCMFNTCMPX
    w HPSPVUJOFMFBL
    w DIBOOFMCMPDLJOH

    View Slide

  45. ਓࡐืूதʂ

    View Slide