Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Speaker Deck
PRO
Sign in
Sign up
for free
DAU 100 万人突破! 急成長を支える Shadowverse のインフラ技術
Cygames
June 02, 2017
Technology
8
12k
DAU 100 万人突破! 急成長を支える Shadowverse のインフラ技術
2017/06/01 Game Tech Session ~AWS Summit Tokyo 2017~
Cygames
June 02, 2017
Tweet
Share
More Decks by Cygames
See All by Cygames
cygames
1
340
cygames
0
780
cygames
2
1.3k
cygames
2
1k
cygames
7
4.7k
cygames
1
1k
cygames
2
19k
cygames
1
2.2k
cygames
1
990
Other Decks in Technology
See All in Technology
nihonbuson
2
1.5k
yuhta28
1
200
shirayanagiryuji
0
1.6k
shirayanagiryuji
1
400
sylph01
0
170
yukitodate
2
320
masakick
0
110
clustervr
0
160
yuuturn
1
130
noir_neo
0
120
y0hgi
1
370
smzksts
0
210
Featured
See All Featured
zakiwarfel
88
3.3k
qrush
285
18k
lara
16
2.6k
ufuk
56
5.4k
lara
590
61k
samanthasiow
56
6.3k
phodgson
87
3.9k
malarkey
192
8.5k
chriscoyier
499
130k
smashingmag
283
47k
carmenhchung
26
1.3k
3n
163
22k
Transcript
ΞδΣϯμ 4IBEPXWFSTFΛࢧ͑ ΔΠ ϯϑϥٕज़ εέʔϥϒϧͳϩάऩूج൫ େنڥγεςϜͱͭͷνϟ Ϩϯδ 5#ͷϩάऩूج൫
4IBEPXWFSTFΛࢧ͑ Δ Π ϯϑϥٕज़
େنڥ
"QQMJDBUJPO4FSWFST Node.js Nginx 6TFS$MJFOU "QQMJDBUJPO4FSWFST Apache / PHP %BUBCBTF4FSWFST
MySQL HTTPS WebSocket Elastic Load Balancing Elastic Load Balancing $BTIF4FSWFST Memcached Redis
ͭͷνϟ Ϩϯδ શσʔλͷอଘͱσʔλϩʔυ ϦϓϨΠػೳͷϦ Ϧʔε /HJOYʹΑ Δಠ࣮ࣗͷ-ϩʔυόϥϯα ରઓγεςϜͷεέʔϧΞ τ
ํϦ ΞϧλΠ Ϝ௨৴ ࣋ଓతଓͷཱ֬ 8FC4PDLFU ରઓ૬खʹ Ϧ ΞϧλΠ Ϝʹ
σʔλө
Ϧ ΞϧλΠ Ϝ௨৴ͷαʔόߏ ΫϥΠ Ξϯ τ3PPN*%Ͱ άϧʔϓԽ͞ΕΔ αʔόϝ οηʔδΛ ΫϥΠ
Ξϯ τʹڞ༗͢Δ ΈΛ࣋ͭ ࢄϦ ΞϧλΠ Ϝ௨৴ͷ ߏͷݕ౼ Client A (RoomID 1) Client B (RoomID 2) Client C (RoomID 1) RoomID 1 RoomID 2 4FSWFS
1VC4VCϝ οηʔδϯάϞσϧ 1VC4VCϞσϧબఆ ࣮͕༰қ ୯Ұো ʢ410'ʣ 1VC4VC͕Ϙ τϧωο Ϋʹ
εέʔϧΞ τͷ՝ ແఀࢭͷมߋ͕ߦ͑ͳ͍ Client A (RoomID 1) Client B (RoomID 2) Client C (RoomID 1) RoomID 1 /PEF RoomID 2 /PEF RoomID 1 RoomID 3 /PEF RoomID 2 3FEJT1VC4VC RoomID 1 RoomID 3 Client D (RoomID 3) Layer4 LB(Elastic Load Balancing)
ϧʔϜ*%ରԠ-ϩʔυόϥϯαͷ։ൃ 410'ͷഉআ ϊ ʔυΛಈతʹมߋ Ϛ ονϯάϩδο Ϋ Λ࣮ ಉҰϧʔϜ*%Λ ಉҰαʔόʹಈతϧʔς
Ο ϯά Client A (RoomID 1) Client B (RoomID 2) Client C (RoomID 1) Lua Nginx 1 Layer4 LB(Elastic Load Balancing) Lua Application Load Balancer Cluster Lua Nginx N Client D (RoomID 3) Nginx 2 … Room 1 Node 1 Room 2 Room 3 Node 3 Node 2 Dynamic Routing
/HJOY-ϩʔυόϥϯαͱ-VBεΫ Ϧϓ τ /HJOY 8FC4PDLFU௨৴ ʢํ௨৴ʣ ͷϩʔυόϥϯα -VBϞδϡʔϧʹΑ Γϩδο
ΫػೳΛ࣮ -VBεΫ Ϧϓ τ ϔο μʹ༩͞ΕͨϧʔϜ*%Λݩʹಈతࢄ 8FC"1*ʹΑ Γແఀࢭʹઃఆมߋ
ࢄϩδο Ϋͷ࣮ ࢄϩδο Ϋ ϧʔϜ*%ͷϋο γϡͱ/PEFαʔόͷ༨ࢉ ʢNPEʣ Ͱ ଓઌϊ
ʔυ͕ܾఆ͞ΕΔ $POTJTUFOU)BTIJOH ʢϋο γϡܭࢉʣ ϊ ʔυͷՃ ɾ আΛ࠷খݶͷมߋͰϚ ο ϐϯά
ສηο γϣϯ ಉ࣌ରઓສୡ
ͭͷνϟ Ϩϯδ શσʔλͷอଘͱσʔλϩʔυ ϦϓϨΠػೳͷϦ Ϧʔε /HJOYʹΑ Δಠ࣮ࣗͷ-ϩʔυόϥϯα ରઓγεςϜͷεέʔϧΞ τ
ϦϓϨΠػೳ ࠷৽݅ͷότϧΛ ϦϓϨΠ ͢Δ શରઓσʔλͷอଘ շదͳԠ
(BNF4ZTUFN 3FQMBZ4ZTUFN ϦϓϨΠσʔλͷอଘͱ࠶ੜ Amazon S3 Redis SQS Web Servers
(PHP) Worker Servers (PHP)
ϦϓϨΠػೳͷσʔλ
ϦϓϨΠػೳͷϦ Ϧʔε ࣮ͷ๛ͳ"84αʔϏεΛ࠾༻ 424ʹΑ ΔඇಉظԽͰػೳ "NB[PO4ͷ҆ఆͨ͠σʔλऔಘ ຊ൪ڥʹ͓͚Δظؒͷݕূ Ϣʔβʹݟͤͳ͍ඇެ։Ϧ Ϧʔε
· ͱΊ ୯Ұোͷͳ͍֦ுՄೳͳαʔόߏ ϐʔΫ࣌ͷ௨৴ྔ ɾ ΞΫηε͕ଟ͘ɺ ༧ଌࠔ ରઓγεςϜΛಠٕࣗज़Ͱ࣮ େ༰ྔσʔλͷอଘɺ
σʔλશੑͷ֬อ "NB[PO3%4 "NB[PO4Λར༻
εέʔϥϒϧͳ ϩάऩूج൫
ιʔγϟϧήʔϜͱ ϩά $4ରԠ ,1*ࢦඪͷऔಘ γεςϜෆ۩߹ͷௐࠪ
σʔλͷछྨͱྔ ςΩε τϩά ΞΫηεϩάɺ Τϥʔϩάɺ ό τϧϩάɺ 42-ϩάʜ %BUBCBTF*OTFSUϩά
՝ۚϩάɺ ΞΠ ςϜϩάɺ ߦಈϩά
˞4IBEPXWFSTFҎ֎ؚΉ
ςΩε τϩάऩूͷ՝ ϩάऔΓ ͜΅͠ ҰߦͷαΠ ζ੍ݶ όο ϑ ΝΩϡʔҲΕ தܧαʔόͷෛՙ
ϩάσʔλ૿େʹΑ Δ εέʔϧΞ τͷ՝ Web Server 1 Error log KPI log Log Proxy 3 4UPSBHF"SSBZ Web Server 2 Web Server N SQL log Log Proxy 2 Access log Log Proxy 1
.Z42-Π ϯαʔ τ ϩάͷ՝ ϨεϙϯεԆ .Z42-*OTFSUॲཧͷԆ ϩάσʔλ૿େ ϨϓϦέʔγϣϯԆ Web
Server 1 Action log Web Server 2 Web Server N Item log MySQL Server Payment log
ಠࣗϩάసૹΤʔδΣϯ τͷ։ൃ "NB[PO4ʹϩάΛू ̎ͭͷϩάసૹΤʔδΣϯ τ Λ։ൃ 4UBMLFSςΩε τϩάͷసૹ 1PSUFS%#*OTFSUϩάͷඇಉظసૹ
ߏԽσʔλͷసૹͱ424ϝ οηʔδ࡞
4UBMLFSͱ1PSUFS औΓ ͜΅͠ࢭͱ࠶ଓ సૹͰ͖ͳ͔ͬͨϩάશͯϩʔΧϧʹอଘ &YQPOFOUJBM#BDLP⒎Ϧ τϥΠִؒͷௐ +JUUFS ʢΒ͖ͭʣ Λಋೖ͠ɺ
࠶ଓཁٻΛࢄ
4UBMLFSͱ1PSUFS "NB[PO4ͷύϑ ΥʔϚϯε্ ઌ಄จࣈʹϋο γϡϓϨϑ Ο ο Ϋ εՃ͠ɺ
JOEFYΛෳύʔς Ο γϣϯʹࢄ ྫ ɿ 4CVDLFU/MpMF໊
"TZODISPOPVT-PH 4ZTUFN (BNF 4ZTUFN CVTJOFTT 4ZTUFN 1PSUFSͱඇಉظ%#ϩάγεςϜ Worker Servers
(Amazon EC2) Log Datebase (Amazon Aurora) σʔλղੳ CSରԠ Porter Web Servers Amazon S3 SQS TSV
"844ϩάू Kibana Google BigQuery Mackerel Amazon Aurora Redshift Porter/Stalker
Amazon S3
ϩά౷ܭͱ άϥϑඳը )551εςʔλεͷΞΫηε
ώʔ τϚοϓ ΞΫηεϩάͱ࠲ඪใΛϚο ϐϯά
.BDLFSFM "QBDIFϨεϙϯελΠ ϜඵҎ্ͷ
"NB[PO4ͱ֎෦πʔϧͷ࿈ܞ εέʔϥϒϧʹ Π ϯελϯεΛ૿ݮ ϩάͷྔϐʔΫ࣌ͱ ࠷খͷ࣌Ͱഒͷࠩ αʔόϨεͰ ӡ༻Λܰݮ Google
BigQuery Porter/Stalker Amazon S3
1.Put Event Trigger 2.Create Message 3.C ount M essages
&$4Λར༻ͨ͠εέʔϥϒϧͳϩά࿈ܞ Google Cloud Storage Google BigQuery Auto Scaling 4.Run Task 6.Save Objects SQL Query Cron Job Amazon S3 Amazon Lamdba Amazon Lamdba SQS Amazon EC2 Container Service(ECS) Ϋϥελͷ$16༻ʹԠͯ͡ Π ϯελϯεΛ૿ݮ 5.Get Files ϩάྔʹԠͯ͡ίϯςφىಈΛ૿ݮ
ͭͷεέʔϦϯάϙϦ γͷ࿈ಈ -BNCEB'VODUJPO ϩάྔʹԠͯ͡ɺ ίϯςφىಈΛ্͛Δ &$"VUP4DBMJOH ىಈதίϯςφ͕ ্ঢʹΑ Γ$16༻্ঢ
&$εέʔϦϯάϙϦ γʹΑ Γ $POUBJOFS*OTUBODF૿Ճ &$4Ϋϥελʔͷ$16༻ $POUBJOFS*OTUBODFىಈ
ίε τݮͱࣗಈԽͷ࣮ݱ αʔόϨεʺίε τݮ -BNCEB&$4ʹΑ Γ ίε τ Λʹݮ
ίϯςφٕज़ͷ࠾༻ -BNCEBͱ&$4ͷద༻ൣғ όονॲཧɺ Π ϕϯ τ τ ϦΨʔ-BNCEB ৗ࣮࣌ߦ͢Δॲཧ&$4
"NB[PO"UIFOBͷར༻ 1.Put Event Trigger 2.Create Message 3.C ount M
essages Auto Scaling 4.Run Task Cron Job Amazon S3 Amazon Lamdba Amazon Lamdba SQS Amazon EC2 Container Service(ECS) 5.Get Files SQL Query Amazon S3 Amazon Athena 6.Save Objects Google Cloud Storage Google BigQuery SQL Query
"NB[PO"UIFOBͷར༻ ύεมʹΑ Δύʔς Ο γϣϯԽ 4UBMLFSͷ4ύεܗࣜ "NB[PO"UIFOBͷ4ύεܗࣜ S3://athena-examples/l2xa-<file>.20150101 s3://athena-examples/ex/2015/01/01/<file>
· ͱΊ ϩάूʹΑ Γσʔλ׆༻͕ଅਐ ඇಉظԽΛߦ͍ϨεϙϯεΛվળ αʔόϨε ͱࣗಈԽͰίε τ ͱݮ
$ZHBNFTΠ ϯϑϥ͕ େʹ͍ͯ͠Δ ͜ ͱ
ͨ Γલͷ͜ ͱ Λ ͨ ΓલʹΔ
Π ϯϑϥͷϨΠ ϠʔͰ ՝Λղܾ͢Δ
ίΞٕज़ ࣗͨͪͰ࣮͢Δ
5IBOLZPV