Development and Deployment with Docker at Dwango @ixixi 

%8"/(0
$P -UE :VSJ
0EBHJSJ

!JYJYJ
3FDPNNFOEBUJPO
TZTUFN
BSDIJUFDU 

Development with Docker 

/JDPOJDP
"QQMJDBUJPO
4FSWFS )551
SFRVFTU 8FC
4FSWFS $BTTBOESB 3FDPNNFOE
 "1*
4FSWFS Recommendation API 3FDPNNFOE
4FSWFS 0UIFS
4FSWJDFT MPHT $BDIF
4FSWFS MPHT w%PDLFSҎલ͔ΒՔಇ͍ͯ͠ΔϨίϝϯυ"1*
wχίχίಈըͷʮؔ࿈ಈըʯͳͲΛฦ͢ɻ 

Ϩίϝϯυͷ֬ೝͱςετ wϨίϝϯυͷγεςϜʹखΛՃ͑ͨ৔߹ʹɺϨίϝϯυ݁Ռ͕Ͳ͏
 มԽ͢Δ͔ͷ֬ೝ͸ɺίʔυϨϏϡʔɾ6OJU5FTUͰ͸อূͮ͠Β͍ɻ
! wQVMM
SFRΛNBTUFSʹऔΓࠐΉຊ൪σϓϩΠͳͷͰɺ֬ೝແ͠Ͱͷ౤ ೖ͸ͨ͘͠ͳ͍ɻ
! wͲ͏ʹ͔ͯ͠ɺखܰʹʮ࣮ࡍͷ؀ڥΛ֬ೝʯग़དྷΔΑ͏ʹ͍ͨ͠ɻ 

΍Γ͍ͨ͜ͱ wQVMM
SFRग़ͨ͠ΒɺϨίϝϯυʹؔΘΔ%PDLFSͷ
 ίϯςφҰἧ͍ lܥz ͕উखʹཱͭɻ
wࣗ෼Ͱ؀ڥΛ࡞ͬͯࢼ͞ͳͯ͘΋ɺ
 ʮ͜͜Ͱطʹಈ͍͍ͯΔΑʯͱ͍͏ঢ়ଶ͕࠷ߴɻ
! wϨίϝϯυ"1*Ͱɺ࣮ࡍʹԿ͕Ϩίϝϯυ͞ΕΔ͔
 දࣔग़དྷΔ3FDPNNFOE
7JFXFS ಺෦πʔϧ Ͱݟ͍ͨɻ طʹಈ͍͍ͯΔ 

/JDPOJDP
"QQMJDBUJPO
4FSWFS )551
SFRVFTU 8FC
4FSWFS $BTTBOESB 3FDPNNFOE

"1*
4FSWFS Containerize! 3FDPNNFOE
4FSWFS 0UIFS
4FSWJDFT MPHT $BDIF
4FSWFS MPHT 3FDPNNFOE
7JFXFS )551
SFRVFTU 

pull request builderͰҰ্ཱࣜͪ͛&௨஌ wKFOLJOTͷQVMM
SFRVFTU
CVJMEFSΛར༻ɻ
wKPC
DIBJOͰςετ͔Βίϯςφ࡞੒ɺίϝϯτॻ͖ࠐΈ·Ͱ࣮ߦɻ QVMM
SFRVFTU
CVJMEFS QVMM
SFRVFTU ௨ৗͷςετ QVMM
SFRVFTUΛ൓өͨ͠
EPDLFSίϯςφ࡞੒ EPDLFSίϯςφͰ
ܥΛҰࣜىಈ ઀ଓઌϙʔτ৘ใΛ
QVMM
SFRίϝϯτʹॻࠐ ίϝϯτʹॻ͖ࠐΈ 

ίϯςφੜ੒ wQVMMSFRVFTUʹରԠͨ͠ίϯςφ໊ͷQSFpYΛ࡞Δɻ
 wྫ QVMMSFR൪߸͕Ͱ࣌ࠁ͕ͳΒ
 ɹɹQVMMSFR@SFDPNNFOE@BQJ@@U
 wಉҰQVMMSFR಺Ͱ΋ɺߋ৽͞ΕΔ౓ʹίϯςφੜ੒͍ͨ͠ͷͰɺ೔͕࣌ඞཁɻ
 wQVMMSFRऴྃ࣌ ޙड़ ʹ͜ͷQSFpYΛར༻ɻ 

ίϯςφͰʮܥʯΛཱͯΔ wHJU
QVMM͔ͯ͠ΒpH
CVJME͢ΔSBLF
UBTLΛ࣮ߦ
wQVMM
SFRʹରԠͨ͠CSBODIΛ࢖ͬͯCVJME͕૸Δɻ
wΞϓϦߋ৽෦෼Ҏલ͸Ωϟογϡ͕ޮ͘ͷͰCVJME͸ߴ଎
! wੜ੒ͨ͠QSFpYΛ෇Ճͨ͠ίϯςφ໊ͰpH
VQ͢ΔSBLF
UBTL࣮ߦ
! wDBTTBOESBSFEJTͷαʔό͸ɺμϛʔσʔλ͕
 ೖͬͨίϯςφ্ཱ͕͕ͪΔɻ
! wQVMM
SFRͷίϝϯτͰɺىಈͤ͞Δ֤ίϯςφͷ
 CSBODIΛ໌ࣔࢦఆ͢Δ͜ͱ΋Մೳɻ
 ผͷSFQPTJUPSZͷQVMM
SFR΋Ұॹʹద༻͢Δඞཁ͕͋Δ৔߹౳ $BTTBOESB 3FDPNNFE

"1*
4FSWFS 3FDPNNFOE 4FSWFS $BDIF
4FSWFS 3FDPNNFOE
7JFXFS MJOL MJOL MJOL MJOL 

PullReqίϝϯτ΁ͷॻ͖ࠐΈ w֤ίϯςφׂ͕Γ౰ͯΒΕͨϙʔτΛऔಘɻ
! w઀ଓ৘ใΛ()FͷQVMM
SFRίϝϯτʹॻ͘ɻ 

Garbage Collection (daily) wQVMM
SFRVFTU͕DMPTF͞ΕͨΒίϯςφΛࢭΊΔ࡟আ͢Δɻ
 wKFOLJOTͰQVMM
SFRVFTUͷDMPTF
IPPL͢Δ͜ͱ͕೉͔ͬͨͨ͠Ίɺ೔࣍Ͱ($ɻ DSPO 
 wEPDLFSͷίϯςφ໊ͷQSFpYΛQBSTFͯ͠ɺSFQPTJUPSZͱQVMM
SFRVFTU൪߸Λऔಘɻ
 w౰֘QVMM
SFRVFTU͕DMPTF͞Ε͍ͯΔ͔֬ೝ͠ɺด͍ͯͨ͡Βίϯςφఀࢭ࡟আɻ
! wίϯςφ໊ͷQSFpYͰ͸ͳ͘FUDE΍SFEJTʹରԠΛอ࣋ͯ͠΋ྑ͔͕ͬͨɺߏ੒ͷγϯϓ ϧ͞Λॏࢹɻ 

developmentฤ Ή͢ͼ wطଘͷϓϩμΫτΛ%PDLFSԽ͢Δͷ͸ਏ͍৔߹͕͋ͬͨɻ
w؀ڥߏஙΛDIFGͰߦ͏લఏͷϓϩμΫτΛޙ͔Β%PDLFSԽ͢Δͷ͸େมʹײͨ͡ɻ
! wطଘ෦෼Λ%PDLFSԽ͢ΔΑΓɺ৽نͰ࡞Δ΋ͷΛ%PDLFSԽલఏͰߏங͢Δํ͕ྑ͍ɻ w1VMM
3FRVFTUͰʮͦͷ৔Ͱ࣮ࡍʹ৮ΕΔ؀ڥ͕طʹಈ͍͍ͯΔʯͷ͸࠷ߴɻ 

Deployment with Docker 

Personalize API /JDPOJDP
"QQMJDBUJPO
4FSWFS )551
SFRVFTU 8FC
4FSWFS $BDIF %BUBCBTF 3FWFSTF
1SPYZ 1FSTPOBMJ[F
"1* • ύʔιφϥΠζʹར༻͢ΔAPI (golang࣮૷) • DockerલఏͰ৽نʹߏஙɻ 

Containerize! • ύʔιφϥΠζAPI • ϩάసૹ&ू໿(fluentd) • CEPΤϯδϯ(Norikra) • ϩάอଘ (ES&InfluxDB) • ՄࢹԽ (Kibana&Grafana) • ϦόʔεϓϩΩγ 1FSTPOBMJ[F
"1* /JDPOJDP
"QQMJDBUJPO
4FSWFS )551
SFRVFTU $POUBJOFSJ[F 8FC
4FSWFS $BDIF %BUBCBTF MPHTUPSF .FUSJDT
7JTVBMJ[FS $&1 -PH
$PMMFDUPS 3FWFSTF
1SPYZ 

Dataflow DP
4FSWFS )551
SFRVFTU $BDIF %BUBCBTF MPHTUPSF .FUSJDT
7JTVBMJ[FS $&1 3FWFSTF
1SPYZ -PBE
#BMBODFS -PH
"HHSFHBUPS MPH
pMF %PDLFS
)PTU %PDLFS
)PTU %PDLFS
)PTU %PDLFS
)PTU "1* -PH
$PMMFDUPS w ֤IPTU಺ͷϦόʔεϓϩΩγ OHJOY ܦ༝Ͱ"1*ʹϦΫΤετɻ
w ϩάϑΝΠϧ͸ϗετଆϘϦϡʔϜΛϚ΢ϯτͯ͠ॻ͖ग़͠ɻ
w qVFOUEίϯςφଆͰ΋ϩάσΟϨΫτϦΛϚ΢ϯτͯ͠ɺಡΈࠐΈɻ
w ϩάू໿ίϯςφͰϩάΛूΊͯɺ$&1 /PSJLSB Ͱूܭͭͭ͠ʹอଘˠՄࢹԽ 

Docker Build Architecture MPDBM ()F +FOLJOT 3FHJTUSZ
$PNNPO QVTI
TPVSDFT )PPL w %PDLFS
*NBHFϏϧυ
w ίϯςφͰܥΛ্ཱͪ͛ͯ݁߹ςετ
w υΩϡϝϯτͷੜ੒
w %PDLFS
3FHJTUSZʹΠϝʔδొ࿥ "1* qVFOUE /HJOY SFHJTUFS
"1*
DPOUBJOFS
*NBHF VCVOUV CVJMEͷى఺ͱͳΔJNBHF͸ɺ1SJWBUF
SFHJTUSZʹஔ͍͓͖ͯɺ%PDLFS
IVC
SFHJTUSZͷ౎౓ར༻͸͠ͳ͍ɻ 

Deploy Manager WebUI • WebUIσϓϩΠπʔϧ • DockerԽҎલ͔Β࢖͍ͬͯͨ಺੡πʔϧ • Capistrano task ࣮ߦ • Ϣʔβೝূ • σϓϩΠϩάͷϦΞϧλΠϜදࣔ&อଘ
 • DockerσϓϩΠ΋capistrano taskԽͯ͠ར༻
 σϓϩΠઌαʔόΛબ୒ͯ͠ɺϙνοͱ
 1click͚ͩͰແఀࢭσϓϩΠɻ • σϓϩΠͷrollbackλεΫ΋༻ҙɻ 

Docker Deployment Architecture %FQMPZ
.BOBHFS
8FC6* 3FHJTUSZ
ڞ௨ 3FHJTUSZ
 QSPEVDUJPOઃఆ "1* qVFOUE /HJOY "1*
TFSWFS
DPOpH 0QFSBUF QVMM qVFOUE 3FWFSTF
1SPYZ "1* QVMM "1*
TFSWFS
DPOpH %PDLFS
)PTU 0UIFS
4FSWJDFT %FQMPZ
$BQJTUSBOP
UBTL )551
SFRVFTU NPVOU • Capistrano taskͰɺίϯςφͷ
 ΠϝʔδΛ֤ϗετ͕pullͯ͠σϓϩΠɻ ! • ࣮ߦόΠφϦͷίϯςφΠϝʔδ͸։ൃऀ શһ͕pullՄೳɻ ! • production؀ڥͰ࢖͏configίϯςφ͸ɺ registry͝ͱ෼཭(ೝূ͕ҟͳΔ)ɻ 

Deployment ~ step.1 1.APIίϯςφΠϝʔδͱɺઃఆΠϝʔδ ΛɺDockerHostଆͰpull͢Δ %PDLFS
)PTU 3FHJTUSZ
ڞ௨ 3FHJTUSZ
ઃఆ "1* qVFOUE /HJOY "1*W EPDLFS
JNBHFT "1*
TFSWFS
DPOpH "1*
TFSWFS
DPOpH "1*
TFSWFS
DPOpH NPVOU NPVOU 3FWFSTF
1SPYZ "1*
W qVFOUEMBUFTU 

Deployment ~ step.1 1.APIίϯςφΠϝʔδͱɺઃఆΠϝʔδ ΛɺDockerHostଆͰpull͢Δ %PDLFS
)PTU 3FHJTUSZ
ڞ௨ 3FHJTUSZ
ઃఆ "1* qVFOUE /HJOY EPDLFS
QVMM
BQJMBUFTU EPDLFS
QVMM
BQJ@DPOpHMBUFTU "1*W EPDLFS
JNBHFT "1*W
MBUFTU "1*
TFSWFS
DPOpH "1*
TFSWFS
DPOpH "1*
TFSWFS
DPOpH NPVOU NPVOU 3FWFSTF
1SPYZ "1*
W qVFOUEMBUFTU 

Deployment ~ step.2 1.৽͍͠imageͷίϯςφΛىಈ 2.API͕ਖ਼͘͠ϦΫΤετΛฦͤΔ·Ͱ଴ͭ
 ɹ(࣮ࡍʹϨεϙϯεΛ֬ೝ) %PDLFS
)PTU "1*
W qVFOUEMBUFTU DPOpH 3FWFSTF
1SPYZ 

Deployment ~ step.2 1.৽͍͠imageͷίϯςφΛىಈ 2.API͕ਖ਼͘͠ϦΫΤετΛฦͤΔ·Ͱ଴ͭ
 ɹ(࣮ࡍʹϨεϙϯεΛ֬ೝ) %PDLFS
)PTU "1*
W "1*
W qVFOUEMBUFTU qVFOUEMBUFTU DPOpH DPOpH 3FWFSTF
1SPYZ 

Deployment ~ step.2 1.৽͍͠imageͷίϯςφΛىಈ 2.API͕ਖ਼͘͠ϦΫΤετΛฦͤΔ·Ͱ଴ͭ
 ɹ(࣮ࡍʹϨεϙϯεΛ֬ೝ) %PDLFS
)PTU "1*
W "1*
W qVFOUEMBUFTU qVFOUEMBUFTU DPOpH DPOpH IFBMUIDIFDL EVNNZ
SFRVFTU 3FWFSTF
1SPYZ 

Deployment ~ step.3 1.৽͍͠IPΛnginxͷconfʹॻ͖ࠐΉɻ 2.NginxΛgraceful restartͯ͠ɺແఀࢭͰίϯςφ ͷ޲͚ઌΛมߋ͢Δɻ
 Reverse Proxy(nginx)ͷDockerίϯςφ಺Ͱ 1.$ kill -USR2 `cat /var/run/nginx.pid` 2.$ kill -WINCH `cat /var/run/nginx.pid.oldbin` 3.$ kill -QUIT `cat /var/run/nginx.pid.oldbin`
 
 
 
 ※࣮ࡍʹ͸ɺgracefull restart෦෼͸
 γΣϧεΫϦϓτԽͯ͠ɺίϯςφ಺ʹಉࠝɻ %PDLFS
)PTU "1*
W "1*
W qVFOUEMBUFTU qVFOUEMBUFTU 3FWFSTF
1SPYZ DPOpH DPOpH OHJOYDPOG
  OHJOY 
QPSU


Deployment ~ step.3 1.৽͍͠IPΛnginxͷconfʹॻ͖ࠐΉɻ 2.NginxΛgraceful restartͯ͠ɺແఀࢭͰίϯςφ ͷ޲͚ઌΛมߋ͢Δɻ
 Reverse Proxy(nginx)ͷDockerίϯςφ಺Ͱ 1.$ kill -USR2 `cat /var/run/nginx.pid` 2.$ kill -WINCH `cat /var/run/nginx.pid.oldbin` 3.$ kill -QUIT `cat /var/run/nginx.pid.oldbin`
 
 
 
 ※࣮ࡍʹ͸ɺgracefull restart෦෼͸
 γΣϧεΫϦϓτԽͯ͠ɺίϯςφ಺ʹಉࠝɻ %PDLFS
)PTU "1*
W "1*
W qVFOUEMBUFTU qVFOUEMBUFTU 3FWFSTF
1SPYZ DPOpH DPOpH OHJOYDPOG


MPDBUJPO

\




QSPYZ@QBTT
IUUQ


^   OHJOY 
QPSU


Deployment ~ step.3 1.৽͍͠IPΛnginxͷconfʹॻ͖ࠐΉɻ 2.NginxΛgraceful restartͯ͠ɺແఀࢭͰίϯςφ ͷ޲͚ઌΛมߋ͢Δɻ
 Reverse Proxy(nginx)ͷDockerίϯςφ಺Ͱ 1.$ kill -USR2 `cat /var/run/nginx.pid` 2.$ kill -WINCH `cat /var/run/nginx.pid.oldbin` 3.$ kill -QUIT `cat /var/run/nginx.pid.oldbin`
 
 
 
 ※࣮ࡍʹ͸ɺgracefull restart෦෼͸
 γΣϧεΫϦϓτԽͯ͠ɺίϯςφ಺ʹಉࠝɻ %PDLFS
)PTU "1*
W "1*
W qVFOUEMBUFTU qVFOUEMBUFTU 3FWFSTF
1SPYZ DPOpH DPOpH OHJOYDPOG


MPDBUJPO

\




QSPYZ@QBTT
IUUQ


^  

MPDBUJPO

\




QSPYZ@QBTT
IUUQ


^ OHJOY 
QPSU


Deployment ~ step.3 1.৽͍͠IPΛnginxͷconfʹॻ͖ࠐΉɻ 2.NginxΛgraceful restartͯ͠ɺແఀࢭͰίϯςφ ͷ޲͚ઌΛมߋ͢Δɻ
 Reverse Proxy(nginx)ͷDockerίϯςφ಺Ͱ 1.$ kill -USR2 `cat /var/run/nginx.pid` 2.$ kill -WINCH `cat /var/run/nginx.pid.oldbin` 3.$ kill -QUIT `cat /var/run/nginx.pid.oldbin`
 
 
 
 ※࣮ࡍʹ͸ɺgracefull restart෦෼͸
 γΣϧεΫϦϓτԽͯ͠ɺίϯςφ಺ʹಉࠝɻ %PDLFS
)PTU "1*
W "1*
W qVFOUEMBUFTU qVFOUEMBUFTU 3FWFSTF
1SPYZ DPOpH DPOpH OHJOYDPOG
  OHJOY 
QPSU


Deployment ~ step.3 1.৽͍͠IPΛnginxͷconfʹॻ͖ࠐΉɻ 2.NginxΛgraceful restartͯ͠ɺແఀࢭͰίϯςφ ͷ޲͚ઌΛมߋ͢Δɻ
 Reverse Proxy(nginx)ͷDockerίϯςφ಺Ͱ 1.$ kill -USR2 `cat /var/run/nginx.pid` 2.$ kill -WINCH `cat /var/run/nginx.pid.oldbin` 3.$ kill -QUIT `cat /var/run/nginx.pid.oldbin`
 
 
 
 ※࣮ࡍʹ͸ɺgracefull restart෦෼͸
 γΣϧεΫϦϓτԽͯ͠ɺίϯςφ಺ʹಉࠝɻ %PDLFS
)PTU "1*
W "1*
W qVFOUEMBUFTU qVFOUEMBUFTU 3FWFSTF
1SPYZ DPOpH DPOpH OHJOYDPOG
  OHJOY OHJOY 
QPSU


Deployment ~ step.3 1.৽͍͠IPΛnginxͷconfʹॻ͖ࠐΉɻ 2.NginxΛgraceful restartͯ͠ɺແఀࢭͰίϯςφ ͷ޲͚ઌΛมߋ͢Δɻ
 Reverse Proxy(nginx)ͷDockerίϯςφ಺Ͱ 1.$ kill -USR2 `cat /var/run/nginx.pid` 2.$ kill -WINCH `cat /var/run/nginx.pid.oldbin` 3.$ kill -QUIT `cat /var/run/nginx.pid.oldbin`
 
 
 
 ※࣮ࡍʹ͸ɺgracefull restart෦෼͸
 γΣϧεΫϦϓτԽͯ͠ɺίϯςφ಺ʹಉࠝɻ %PDLFS
)PTU "1*
W "1*
W qVFOUEMBUFTU qVFOUEMBUFTU 3FWFSTF
1SPYZ DPOpH DPOpH OHJOYDPOG
  OHJOY 
QPSU


Deployment ~ step.4 1.ݹ͍ίϯςφͷఀࢭ&࡟আ 2.ݹ͍ίϯςφΠϝʔδͷ࡟আ
 
 ˞ߴ଎ʹrollbackग़དྷΔΑ͏ɺ࣮ࡍʹ͸
 ɹ5ੈ୅લͷΠϝʔδΛ࡟আ %PDLFS
)PTU "1*
W "1*
W qVFOUEMBUFTU qVFOUEMBUFTU 3FWFSTF
1SPYZ DPOpH DPOpH OHJOY "1*W EPDLFS
JNBHFT "1*W
MBUFTU "1*
TFSWFS
DPOpH 

Deployment ~ step.4 1.ݹ͍ίϯςφͷఀࢭ&࡟আ 2.ݹ͍ίϯςφΠϝʔδͷ࡟আ
 
 ˞ߴ଎ʹrollbackग़དྷΔΑ͏ɺ࣮ࡍʹ͸
 ɹ5ੈ୅લͷΠϝʔδΛ࡟আ %PDLFS
)PTU "1*
W qVFOUEMBUFTU 3FWFSTF
1SPYZ DPOpH OHJOY "1*W EPDLFS
JNBHFT "1*W
MBUFTU "1*
TFSWFS
DPOpH 

Deployment ~ step.4 1.ݹ͍ίϯςφͷఀࢭ&࡟আ 2.ݹ͍ίϯςφΠϝʔδͷ࡟আ
 
 ˞ߴ଎ʹrollbackग़དྷΔΑ͏ɺ࣮ࡍʹ͸
 ɹ5ੈ୅લͷΠϝʔδΛ࡟আ %PDLFS
)PTU "1*
W qVFOUEMBUFTU 3FWFSTF
1SPYZ DPOpH OHJOY EPDLFS
JNBHFT "1*W
MBUFTU "1*
TFSWFS
DPOpH 

deploymentฤ Ή͢ͼ wμ΢ϯλΠϜແ͠ͷσϓϩΠ͸Մೳ͕ͩɺݱঢ়ɺࣗલͰগ͠࡞Γ͜Ήඞཁ͕͋Δɻ
wࣗલͷίϯςφ੾ସલϔϧενΣοΫɻ
! wϗετଆͷ؀ڥ͕ԚΕͳ͍ͷ͸ૉ੖Β͍͠ɻ
! w௕ظӡ༻͢ΔલఏͳΒΰϛΛย෇͚ΔػߏΛ࣋ͱ͏ɻ
 wLTΛ࢖͏ʹ͸ɺNBTUFSͷ৑௕Խ΍ߴՄ༻ੑͳFUDEΫϥελͷߏஙɾӡ༻౳ɺ
 ن໛ͷׂʹ߹Θͳ͔ͬͨͷͰʮϦιʔεͷந৅Խʯ·Ͱ͸౿Έࠐ·ͳͳ͔ͬͨɻ
 wαʔϏεԣஅͰར༻Մೳͳେن໛σϓϩΠઌαʔόࢿݯͱͯ͠ந৅Խ͢Δ৔߹ʹ͸ɺ
 LT౳͕ඞཁʹͳΔͱࢥ͏ɻن໛ͱ໨తʹԠͯ͡ΞʔΩςΫνϟઃܭ͠Α͏ɻ 

υϫϯΰ޿ࠂ 
 ͜ͷهࣄ͸υϫϯΰۈ຿࣌ؒதʹॻ͔Εͨɻ ! υϫϯΰ͸ຊ෺ͷ
 (C++|Elrang|ݕࡧ|Ϩίϝϯυ|σʔλ෼ੳ|σʔλ࿈ܞ|FPGA|Πϯϑϥ|WEBΞϓϦ)
 ΤϯδχΞΛืू͍ͯ͠·͢ɻ ࠾༻৘ใ | גࣜձࣾυϫϯΰ