Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Speaker Deck
PRO
Sign in
Sign up
for free
Gaurun / A general push notification server in Go
Tatsuhiko Kubo
June 21, 2015
Technology
11
42k
Gaurun / A general push notification server in Go
Gaurun 〜A general push notification server in Go〜
Tatsuhiko Kubo
June 21, 2015
Tweet
Share
More Decks by Tatsuhiko Kubo
See All by Tatsuhiko Kubo
cubicdaiya
2
280
cubicdaiya
0
290
cubicdaiya
4
15k
cubicdaiya
4
2.6k
cubicdaiya
0
170
cubicdaiya
0
1.9k
cubicdaiya
5
2.4k
cubicdaiya
11
10k
cubicdaiya
8
5.3k
Other Decks in Technology
See All in Technology
ryusa
2
190
kraj
0
5.3k
noir_neo
0
130
kentaro
1
430
ocise
0
170
kakka
0
3.7k
muras
0
110
cmwatanabeseigo
0
360
takem001
0
910
satoryu
0
2.4k
_kensh
1
170
texmeijin
1
210
Featured
See All Featured
bryan
30
3.3k
sstephenson
144
12k
jnunemaker
PRO
40
4.6k
hatefulcrawdad
257
17k
jacobian
255
20k
dotmariusz
94
5.1k
marcelosomers
220
15k
afnizarnur
176
14k
keathley
18
650
edds
56
9.3k
yeseniaperezcruz
302
31k
holman
288
130k
Transcript
A general push notification server in Go Tatsuhiko Kubo@cubicdaiya Go
Conference summer@2015/06/21 Gaurun
@cubicdaiya / Tatsuhiko Kubo w 4PGUXBSF&OHJOFFS!.FSDBSJ *OD w *OGSBTUSVDUVSF&OHJOFFSJOH w
(P w OHJOYCVJME TMBDLCPBSE (BVSVO w OHJOY w OHY@TNBMM@MJHIU OHY@EZOBNJD@VQTUSFBN
Other Activities IUUQNP[BJDGNQPTUOHJOY
None
None
mercari.go w DVCJDEBJZBOHJOYCVJME w DVCJDEBJZBDBDIFDUM w DVCJDEBJZBTMBDLCPBSE w NFSDBSJ(BVSVO (PJOQSPEVDUJPO!.FSDBSJ
mercari.go w DVCJDEBJZBOHJOYCVJME w DVCJDEBJZBDBDIFDUM w DVCJDEBJZBTMBDLCPBSE w NFSDBSJ(BVSVO (PJOQSPEVDUJPO!.FSDBSJ
Gaurun https://github.com/mercari/gaurun
Gaurun w "HFOFSBMQVTIOPUJpDBUJPOTFSWFS GPSTNBSUQIPOFBQQMJDBUJPOT w 8SJUUFOJO(P w QSPYJOHQVTISFRVFTUTUP"1/TBOE($. w 4JNQMF)551"1*T
Overview ($. "1/T (BVSVO )5514 "1/T QSPUPDPM )551 1045QVTI #PEZJT+40/
$MJFOU
Why Go? w 3FRVJSFNFOUT w (PPEQFSGPSNBODF w )JHIDPODVSSFODZ w OFUIUUQJTHPPEDIPJDF
Why Go? w 1VTIJTTMPX w %VFUPOFUXPSLMBUFODZ w 4PNFPQFSBUJPO!.FSDBSJ w 4FOENJMMJPOQVTIBUPODFʂ
w )JHIDPODVSSFODZJTSFRVJSFEʂ
Why Go? w "TZODSPOPVTQVTIJTSFRVJSFE w QVTIJTUSJHHFSFEJO"1*TFSWFS w DPNNFOU MJLF CVZ
TFMM w "1*TFSWFSNVTUSFTQPOTFRVJDLMZ w "TZODISPOJ[BUJPOJTDPNQMJDBUFE w "1*TFSWFS!.FSDBSJJT"QBDIF 1)1
Push System @ Mercari
a year ago… ($. "1/T "1*4FSWFS "1*4FSWFS "1*4FSWFS "MMQVTIOPUJpDBUJPOTBSFTZODISPOPVT QVTI
half a year ago… ($. "1/T 8PSLFS "MMQVTIOPUJpDBUJPOTBSFBTZODISPOPVT 2VFVF 8PSLFS
8PSLFS QVTI EFRVFVF "1*4FSWFS "1*4FSWFS #BUDI4FSWFS FORVFVF "1*4FSWFS
But very complicated… w +PC2VFVF w 2. w .VMUJ1SPDFTT8PSLFSXJUI1)1 w
1)11BSBMMFMT1SFGPSL w 5IFBSDIJUFDUVSFJTOPUTDBMBCMF
Now… ($. "1/T (BVSVO (BVSVO (BVSVO /(*/9 "1*4FSWFS "1*4FSWFS "1*4FSWFS
#BUDI4FSWFS QVTI XJUI LFFQBMJWF )551 1045QVTI
Gaurun Internals
How Gaurun pushesʁ ($. "1/T (BVSVO )5514 "1/T QSPUPDPM )551
1045QVTI #PEZJT+40/ $MJFOU
Bootstrap Gaurun UIFDPEFCFMPXJTTJNQMJpFEGPSFYQMBOBUJPO TUBSUXPSLFSTGPSQVTI HBVSVO4UBSU1VTI8PSLFST XPSLFS/VN RVFVF/VN )551"1*T IUUQ)BOEMF'VOD
lQVTIz HBVSVO1VTI/PUJpDBUJPO)BOEMFS IUUQ)BOEMF'VOD lTUBUHPz TUBUT(P)BOEMFS IUUQ)BOEMF'VOD lTUBUBQQz HBVSVO4UBUT(BVSVO)BOEMFS ʜ TUBSU)551TFSWFS IUUQ-JTUFO"OE4FSWF lz OJM
start workers for push UIFDPEFCFMPXJTTJNQMJpFEGPSFYQMBOBUJPO GVOD4UBSU1VTI8PSLFST XPSLFS/VN RVFVF/VNJOU \ 2VFVFNBLF
DIBO/PUJpDBUJPO RVFVF/VN GPSJJXPSLFS/VNJ \ HPQVTI/PUJpDBUJPO8PSLFS ^ ^
POST /push UIFDPEFCFMPXJTTJNQMJpFEGPSFYQMBOBUJPO GVOD1VTI/PUJpDBUJPO)BOEMFS XIUUQ3FTQPOTF8SJUFS S IUUQ3FRVFTU \ BDDFQUTPOMZ1045NFUIPE NFUIPE$IFDL
SFBESFRVFTUCPEZ SCPEZ @JPVUJM3FBE"MM S#PEZ QBSTFSFRVFTUCPEZ WBSSFR(BVSVO3FRVFTU(BVSVO @KTPO6ONBSTIBM SCPEZ SFR(BVSVO FORVFVFOPUJpDBUJPOBTZODISPOPVTMZ HPFORVFVF/PUJpDBUJPOT SFR(BVSVO/PUJpDBUJPOT SFUVSOUIFSFTQPOTF TFOE3FQPOTF X lPLz IUUQ4UBUVT0, ^
enqueue Notifications UIFDPEFCFMPXJTTJNQMJpFEGPSFYQMBOBUJPO GVODFORVFVF/PUJpDBUJPOT OT<>3FRVFTU/PUJpDBUJPO \ GPS@ OPUJpDBUJPOSBOHFOT\ 2VFVFOPUJpDBUJPO ^
^
pushNotificationWorker UIFDPEFCFMPXJTTJNQMJpFEGPSFYQMBOBUJPO GVODQVTI/PUJpDBUJPO8PSLFS \ GPS\ OPUJpDBUJPO2VFVF TXJUDIOPUJpDBUJPO1MBUGPSN\ DBTF1MBUGPSN*PT QVTI/PUJpDBUJPO*PT
DBTF1MBUGPSN"OESPJE QVTI/PUJpDBUJPO*"OESPJE ^ ^ ^
Why workers with goroutine and queue with channel are static?
w 6OMJNJUFEHPSPVUJOFT w 1FSGPSNBODFTMPXTEPXO w #FDBVTFQVTIJTTMPX w DIBOOFMJTSFRVJSFEGPSLFFQJOHXPSLFSTXBJUJOH w (P`TDIBOOFMTJ[FJTOPUVOMJNJUFE
Gaurun Features
HTTP APIs w 1045QVTI w (&5TUBUHP w (&5TUBUBQQ
POST /push "DDFQUTCVMLFEQVTISFRVFTUXJUI+40/ \ lOPUJpDBUJPOz< \ lUPLFOTz<lYYYz> lQMBUGPSNz
lNFTTBHFzl)FMMP J04z lCBEHFz lTPVOEzlEFGBVMUz lFYQJSZz ^ \ lUPLFOTz<lZZZz> lQMBUGPSNz lNFTTBHFzl)FMMP "OESPJEz lDPMMBQTF@LFZzlVQEBUFz lEFMBZ@XIJMF@JEMFzUSVF lUJNF@UP@MJWFz ^ > ^
GET /stat/go DVSMTIUUQTUBUHP \ lUJNFz lHP@WFSTJPOzlHPz lHP@PTzlMJOVYz
lHP@BSDIzlBNEz lDQV@OVNz lHPSPVUJOF@OVNz lHPNBYQSPDTz lDHP@DBMM@OVNz lNFNPSZ@BMMPDz lNFNPSZ@UPUBM@BMMPDz lNFNPSZ@TZTz lNFNPSZ@MPPLVQTz lNFNPSZ@NBMMPDTz lNFNPSZ@GSFFTz ɾ ɾ ɾ 3FUVSOTBTUBUJDUJDTGPS(PSVOUJNF
GET /stat/app DVSMTIUUQTUBUBQQ \ lRVFVF@NBYz lRVFVF@VTBHFz lJPTz\ lQVTI@TVDDFTTz
lQVTI@FSSPSz ^ lBOESPJEz\ lQVTI@TVDDFTTz lQVTI@FSSPSz ^ ^ 3FUVSOTBTUBUJDUJDTGPS(BVSVO
More details… IUUQTHJUIVCDPNNFSDBSJHBVSVOCMPCNBTUFS41&$NE
Gaurun Configuration HBVSVODHBVSVOUPNM
Gaurun Configuration (TOML) <DPSF> QPSUlz XPSLFST RVFVFT <BOESPJE> BQJLFZlʜz <JPT>
QFN@DFSU@QBUIlQBUIDFSUQFNz FN@LFZ@QBUIlQFNLFZQFNz TBOECPYGBMTF UJNFPVU@FSSPS
Optimization w DPSFXPSLFST w OVNCFSPGXPSLFSTGPSQVTI w DPSFRVFVFT w TJ[FPGDIBOOFMGPSQVTIXPSLFST w
JPTUJNFPVU@FSSPS w UJNFPVUGPSXBJUJOHFSSPSGSPN"1/T w EFGBVMUDPOpHVSBUJPOJTDPTFSWBUJWF
More details… IUUQTHJUIVCDPNNFSDBSJHBVSVOCMPCNBTUFS$0/'*(63"5*0/NE
Monitoring Gaurun
Monitoring Gaurun w (&5TUBUHP w 3FUVSOTBTUBUJDUJDTGPS(PSVOUJNF w (&5TUBUBQQ w 3FUVSOTBTUBUJDUJDTGPS(BVSVO
Monitoring Gaurun with Zabbix VTFSQBSBNFUFS@HPMBOH@TUBUTDPOG 6TFS1BSBNFUFSHPMBOHTUBUT< > DVSMTIUUQTUBUHPcKR
Monitoring server written in Go w .FNPSZVTBHFBOEBMMPDBUJPOT w ($DPOEJUJPO w
OVNCFSPGHPSPVUJOFT w DIBOOFMVTBHF w OVNCFSPGDPOOFDUJPOT SFRVFTUTTFD FUDʜ
The case of Gaurun w (&5TUBUHP w .FNPSZVTBHFBOEBMMPDBUJPOT w ($DPOEJUJPO
w OVNCFSPGHPSPVUJOFT w (&5TUBUBQQ w DIBOOFMVTBHF w OHY@IUUQ@TUVC@TUBUVT@NPEVMF w OVNCFSPGDPOOFDUJPOT SFRVFTUTTFD FUDʜ
Potential problems w HPSPVUJOFMFBL w -JOFBSJODSFBTFPGOVNCFSPGHPSPVUJOFT w *U`TBNPSFTFSJPVTCVHUIBONFNPSZMFBL w DIBOOFMCMPDLJOH
w DIBOOFMTJ[FJTMJNJUFE
Library For Monitoring Go-runtime w GVLBUBHPMBOHTUBUTBQJIBOEMFS w 7FSZVTFGVM w (&5TUBUHPJTQPXFSFECZUIJT
Summary w (PJTTVJUBCMFGPSTJNQMF)551"1*TFSWFS w (BVSVOXPSLTpOF w .POJUPSJOHTFSWFSXSJUUFOJO(P w $BSFQPUFOUJBMQSPCMFNTCMPX w
HPSPVUJOFMFBL w DIBOOFMCMPDLJOH
ਓࡐืूதʂ