Gaurun / A general push notification server in Go

by Tatsuhiko Kubo

Slide 1

Slide 1 text

A general push notiﬁcation server in Go Tatsuhiko Kubo@cubicdaiya Go Conference summer@2015/06/21 Gaurun

Slide 2

Slide 2 text

@cubicdaiya / Tatsuhiko Kubo w 4PGUXBSF&OHJOFFS!.FSDBSJ *OD w *OGSBTUSVDUVSF&OHJOFFSJOH w (P w OHJOYCVJME TMBDLCPBSE (BVSVO w OHJOY w OHY@TNBMM@MJHIU OHY@EZOBNJD@VQTUSFBN

Slide 3

Slide 3 text

Other Activities IUUQNP[BJDGNQPTUOHJOY

Slide 4

Slide 4 text

No content

Slide 5

Slide 5 text

No content

Slide 6

Slide 6 text

mercari.go w DVCJDEBJZBOHJOYCVJME w DVCJDEBJZBDBDIFDUM w DVCJDEBJZBTMBDLCPBSE w NFSDBSJ(BVSVO (PJOQSPEVDUJPO!.FSDBSJ

Slide 7

Slide 7 text

mercari.go w DVCJDEBJZBOHJOYCVJME w DVCJDEBJZBDBDIFDUM w DVCJDEBJZBTMBDLCPBSE w NFSDBSJ(BVSVO (PJOQSPEVDUJPO!.FSDBSJ

Slide 8

Slide 8 text

Gaurun https://github.com/mercari/gaurun

Slide 9

Slide 9 text

Gaurun w "HFOFSBMQVTIOPUJpDBUJPOTFSWFS GPSTNBSUQIPOFBQQMJDBUJPOT w 8SJUUFOJO(P w QSPYJOHQVTISFRVFTUTUP"1/TBOE($. w 4JNQMF)551"1*T

Slide 10

Slide 10 text

Overview ($. "1/T (BVSVO )5514 "1/T QSPUPDPM )551 1045QVTI #PEZJT+40/ $MJFOU

Slide 11

Slide 11 text

Why Go? w 3FRVJSFNFOUT w (PPEQFSGPSNBODF w )JHIDPODVSSFODZ w OFUIUUQJTHPPEDIPJDF

Slide 12

Slide 12 text

Why Go? w 1VTIJTTMPX w %VFUPOFUXPSLMBUFODZ w 4PNFPQFSBUJPO!.FSDBSJ w 4FOENJMMJPOQVTIBUPODFʂ w )JHIDPODVSSFODZJTSFRVJSFEʂ

Slide 13

Slide 13 text

Why Go? w "TZODSPOPVTQVTIJTSFRVJSFE w QVTIJTUSJHHFSFEJO"1*TFSWFS w DPNNFOU MJLF CVZ TFMM w "1*TFSWFSNVTUSFTQPOTFRVJDLMZ w "TZODISPOJ[BUJPOJTDPNQMJDBUFE w "1*TFSWFS!.FSDBSJJT"QBDIF1)1

Slide 14

Slide 14 text

Push System @ Mercari

Slide 15

Slide 15 text

a year ago… ($. "1/T "1*4FSWFS "1*4FSWFS "1*4FSWFS "MMQVTIOPUJpDBUJPOTBSFTZODISPOPVT QVTI

Slide 16

Slide 16 text

half a year ago… ($. "1/T 8PSLFS "MMQVTIOPUJpDBUJPOTBSFBTZODISPOPVT 2VFVF 8PSLFS 8PSLFS QVTI EFRVFVF "1*4FSWFS "1*4FSWFS #BUDI4FSWFS FORVFVF "1*4FSWFS

Slide 17

Slide 17 text

But very complicated… w +PC2VFVF w 2. w .VMUJ1SPDFTT8PSLFSXJUI1)1 w 1)11BSBMMFMT1SFGPSL w 5IFBSDIJUFDUVSFJTOPUTDBMBCMF

Slide 18

Slide 18 text

Now… ($. "1/T (BVSVO (BVSVO (BVSVO /(*/9 "1*4FSWFS "1*4FSWFS "1*4FSWFS #BUDI4FSWFS QVTI XJUI LFFQBMJWF )551 1045QVTI

Slide 19

Slide 19 text

Gaurun Internals

Slide 20

Slide 20 text

How Gaurun pushesʁ ($. "1/T (BVSVO )5514 "1/T QSPUPDPM )551 1045QVTI #PEZJT+40/ $MJFOU

Slide 21

Slide 21 text

Bootstrap Gaurun UIFDPEFCFMPXJTTJNQMJpFEGPSFYQMBOBUJPO TUBSUXPSLFSTGPSQVTI HBVSVO4UBSU1VTI8PSLFST XPSLFS/VN RVFVF/VN )551"1*T IUUQ)BOEMF'VOD lQVTIz HBVSVO1VTI/PUJpDBUJPO)BOEMFS IUUQ)BOEMF'VOD lTUBUHPz TUBUT(P)BOEMFS IUUQ)BOEMF'VOD lTUBUBQQz HBVSVO4UBUT(BVSVO)BOEMFS ʜ TUBSU)551TFSWFS IUUQ-JTUFO"OE4FSWF lz OJM

Slide 22

Slide 22 text

start workers for push UIFDPEFCFMPXJTTJNQMJpFEGPSFYQMBOBUJPO GVOD4UBSU1VTI8PSLFST XPSLFS/VN RVFVF/VNJOU \ 2VFVFNBLF DIBO/PUJpDBUJPO RVFVF/VN GPSJJXPSLFS/VNJ\ HPQVTI/PUJpDBUJPO8PSLFS ^ ^

Slide 23

Slide 23 text

POST /push UIFDPEFCFMPXJTTJNQMJpFEGPSFYQMBOBUJPO GVOD1VTI/PUJpDBUJPO)BOEMFS XIUUQ3FTQPOTF8SJUFS SIUUQ3FRVFTU \ BDDFQUTPOMZ1045NFUIPE NFUIPE$IFDL SFBESFRVFTUCPEZ SCPEZ @JPVUJM3FBE"MM S#PEZ QBSTFSFRVFTUCPEZ WBSSFR(BVSVO3FRVFTU(BVSVO @KTPO6ONBSTIBM SCPEZ SFR(BVSVO FORVFVFOPUJpDBUJPOBTZODISPOPVTMZ HPFORVFVF/PUJpDBUJPOT SFR(BVSVO/PUJpDBUJPOT SFUVSOUIFSFTQPOTF TFOE3FQPOTF X lPLz IUUQ4UBUVT0, ^

Slide 24

Slide 24 text

enqueue Notiﬁcations UIFDPEFCFMPXJTTJNQMJpFEGPSFYQMBOBUJPO GVODFORVFVF/PUJpDBUJPOT OT<>3FRVFTU/PUJpDBUJPO \ GPS@ OPUJpDBUJPOSBOHFOT\ 2VFVFOPUJpDBUJPO ^ ^

Slide 25

Slide 25 text

pushNotiﬁcationWorker UIFDPEFCFMPXJTTJNQMJpFEGPSFYQMBOBUJPO GVODQVTI/PUJpDBUJPO8PSLFS \ GPS\ OPUJpDBUJPO2VFVF TXJUDIOPUJpDBUJPO1MBUGPSN\ DBTF1MBUGPSN*PT QVTI/PUJpDBUJPO*PT DBTF1MBUGPSN"OESPJE QVTI/PUJpDBUJPO*"OESPJE ^ ^ ^

Slide 26

Slide 26 text

Why workers with goroutine and queue with channel are static? w 6OMJNJUFEHPSPVUJOFT w 1FSGPSNBODFTMPXTEPXO w #FDBVTFQVTIJTTMPX w DIBOOFMJTSFRVJSFEGPSLFFQJOHXPSLFSTXBJUJOH w (P`TDIBOOFMTJ[FJTOPUVOMJNJUFE

Slide 27

Slide 27 text

Gaurun Features

Slide 28

Slide 28 text

HTTP APIs w 1045QVTI w (&5TUBUHP w (&5TUBUBQQ

Slide 29

Slide 29 text

POST /push "DDFQUTCVMLFEQVTISFRVFTUXJUI+40/ \ lOPUJpDBUJPOz< \ lUPLFOTz lQMBUGPSNz lNFTTBHFzl)FMMP J04z lCBEHFz lTPVOEzlEFGBVMUz lFYQJSZz ^ \ lUPLFOTz lQMBUGPSNz lNFTTBHFzl)FMMP "OESPJEz lDPMMBQTF@LFZzlVQEBUFz lEFMBZ@XIJMF@JEMFzUSVF lUJNF@UP@MJWFz ^ > ^

Slide 30

Slide 30 text

GET /stat/go DVSMTIUUQTUBUHP \ lUJNFz lHP@WFSTJPOzlHPz lHP@PTzlMJOVYz lHP@BSDIzlBNEz lDQV@OVNz lHPSPVUJOF@OVNz lHPNBYQSPDTz lDHP@DBMM@OVNz lNFNPSZ@BMMPDz lNFNPSZ@UPUBM@BMMPDz lNFNPSZ@TZTz lNFNPSZ@MPPLVQTz lNFNPSZ@NBMMPDTz lNFNPSZ@GSFFTz ɾ ɾ ɾ 3FUVSOTBTUBUJDUJDTGPS(PSVOUJNF

Slide 31

Slide 31 text

GET /stat/app DVSMTIUUQTUBUBQQ \ lRVFVF@NBYz lRVFVF@VTBHFz lJPTz\ lQVTI@TVDDFTTz lQVTI@FSSPSz ^ lBOESPJEz\ lQVTI@TVDDFTTz lQVTI@FSSPSz ^ ^ 3FUVSOTBTUBUJDUJDTGPS(BVSVO

Slide 32

Slide 32 text

More details… IUUQTHJUIVCDPNNFSDBSJHBVSVOCMPCNBTUFS41&$NE

Slide 33

Slide 33 text

Gaurun Conﬁguration HBVSVODHBVSVOUPNM

Slide 34

Slide 34 text

Gaurun Conﬁguration (TOML) QPSUlz XPSLFST RVFVFT BQJLFZlʜz QFN@DFSU@QBUIlQBUIDFSUQFNz FN@LFZ@QBUIlQFNLFZQFNz TBOECPYGBMTF UJNFPVU@FSSPS

Slide 35

Slide 35 text

Optimization w DPSFXPSLFST w OVNCFSPGXPSLFSTGPSQVTI w DPSFRVFVFT w TJ[FPGDIBOOFMGPSQVTIXPSLFST w JPTUJNFPVU@FSSPS w UJNFPVUGPSXBJUJOHFSSPSGSPN"1/T w EFGBVMUDPOpHVSBUJPOJTDPTFSWBUJWF

Slide 36

Slide 36 text

More details… IUUQTHJUIVCDPNNFSDBSJHBVSVOCMPCNBTUFS$0/'*(63"5*0/NE

Slide 37

Slide 37 text

Monitoring Gaurun

Slide 38

Slide 38 text

Monitoring Gaurun w (&5TUBUHP w 3FUVSOTBTUBUJDUJDTGPS(PSVOUJNF w (&5TUBUBQQ w 3FUVSOTBTUBUJDUJDTGPS(BVSVO

Slide 39

Slide 39 text

Monitoring Gaurun with Zabbix VTFSQBSBNFUFS@HPMBOH@TUBUTDPOG 6TFS1BSBNFUFSHPMBOHTUBUT<> DVSMTIUUQTUBUHPcKR

Slide 40

Slide 40 text

Monitoring server written in Go w .FNPSZVTBHFBOEBMMPDBUJPOT w ($DPOEJUJPO w OVNCFSPGHPSPVUJOFT w DIBOOFMVTBHF w OVNCFSPGDPOOFDUJPOT SFRVFTUTTFD FUDʜ

Slide 41

Slide 41 text

The case of Gaurun w (&5TUBUHP w .FNPSZVTBHFBOEBMMPDBUJPOT w ($DPOEJUJPO w OVNCFSPGHPSPVUJOFT w (&5TUBUBQQ w DIBOOFMVTBHF w OHY@IUUQ@TUVC@TUBUVT@NPEVMF w OVNCFSPGDPOOFDUJPOT SFRVFTUTTFD FUDʜ

Slide 42

Slide 42 text

Potential problems w HPSPVUJOFMFBL w -JOFBSJODSFBTFPGOVNCFSPGHPSPVUJOFT w *U`TBNPSFTFSJPVTCVHUIBONFNPSZMFBL w DIBOOFMCMPDLJOH w DIBOOFMTJ[FJTMJNJUFE

Slide 43

Slide 43 text

Library For Monitoring Go-runtime w GVLBUBHPMBOHTUBUTBQJIBOEMFS w 7FSZVTFGVM w (&5TUBUHPJTQPXFSFECZUIJT

Slide 44

Slide 44 text

Summary w (PJTTVJUBCMFGPSTJNQMF)551"1*TFSWFS w (BVSVOXPSLTpOF w .POJUPSJOHTFSWFSXSJUUFOJO(P w $BSFQPUFOUJBMQSPCMFNTCMPX w HPSPVUJOFMFBL w DIBOOFMCMPDLJOH

Slide 45

Slide 45 text

ਓࡐืूதʂ