DAU 100 万人突破！ 急成長を支える Shadowverse のインフラ技術

Transcript

1. 

2.  

3.  ΞδΣϯμ 4IBEPXWFSTFΛࢧ͑ ΔΠ ϯϑϥٕज़
   εέʔϥϒϧͳϩάऩूج൫ େن໛؀ڥγεςϜͱͭͷνϟ Ϩϯδ ೔5#ͷϩάऩूج൫

4.  4IBEPXWFSTFΛࢧ͑ Δ
   Π ϯϑϥٕज़

5. 

6.  େن໛؀ڥ

7.  "QQMJDBUJPO
   4FSWFST Node.js Nginx 6TFS$MJFOU "QQMJDBUJPO
   4FSWFST Apache / PHP %BUBCBTF
   4FSWFST

   MySQL HTTPS WebSocket Elastic Load Balancing Elastic Load Balancing $BTIF
   4FSWFST Memcached Redis

8.  ͭͷνϟ Ϩϯδ શσʔλͷอଘͱσʔλϩʔυ ϦϓϨΠػೳͷ೥຤Ϧ Ϧʔε /HJOYʹΑ Δಠ࣮ࣗ૷ͷ-
   ϩʔυόϥϯα ରઓγεςϜͷεέʔϧΞ΢ τ

9. ૒ํ޲Ϧ ΞϧλΠ Ϝ௨৴ ࣋ଓత઀ଓͷཱ֬ 8FC4PDLFU
   ରઓ૬खʹ Ϧ ΞϧλΠ Ϝʹ

   σʔλ൓ө 

10. Ϧ ΞϧλΠ Ϝ௨৴ͷαʔόߏ੒ ΫϥΠ Ξϯ τ͸3PPN*%Ͱ άϧʔϓԽ͞ΕΔ
    αʔό͸ϝ οηʔδΛ
 ΫϥΠ

    Ξϯ τʹڞ༗͢Δ
 ࢓૊ΈΛ࣋ͭ
    ෼ࢄϦ ΞϧλΠ Ϝ௨৴ͷ
 ߏ੒ͷݕ౼  Client A (RoomID 1) Client B (RoomID 2) Client C (RoomID 1) RoomID 1 RoomID 2 4FSWFS

11. 1VC4VCϝ οηʔδϯάϞσϧ 1VC4VCϞσϧબఆ
    ࣮૷͕༰қ
    ୯Ұো֐఺ ʢ410'ʣ
    1VC4VC͕Ϙ τϧωο Ϋʹ
    

    εέʔϧΞ΢ τͷ՝୊
    ແఀࢭͷมߋ͕ߦ͑ͳ͍  Client A (RoomID 1) Client B (RoomID 2) Client C (RoomID 1) RoomID 1 /PEF
     RoomID 2 /PEF
     RoomID 1 RoomID 3 /PEF
     RoomID 2 3FEJT
    
    1VC4VC RoomID 1 RoomID 3 Client D (RoomID 3) Layer4 LB(Elastic Load Balancing)

12. ϧʔϜ*%ରԠ
    -ϩʔυόϥϯαͷ։ൃ 410'ͷഉআ
    ϊ ʔυ୆਺Λಈతʹมߋ
    Ϛ ονϯάϩδο Ϋ Λ࣮૷
    ಉҰϧʔϜ*%Λ
 ಉҰαʔόʹಈతϧʔς

    Ο ϯά  Client A (RoomID 1) Client B (RoomID 2) Client C (RoomID 1) Lua Nginx 1 Layer4 LB(Elastic Load Balancing) Lua Application Load Balancer Cluster Lua Nginx N Client D (RoomID 3) Nginx 2 … Room 1 Node 1 Room 2 Room 3 Node 3 Node 2 Dynamic Routing

13.  /HJOY
    -ϩʔυόϥϯαͱ-VBεΫ Ϧϓ τ /HJOY
    8FC4PDLFU௨৴ ʢ૒ํ޲௨৴ʣ ͷϩʔυόϥϯα
    -VBϞδϡʔϧʹΑ Γϩδο

    ΫػೳΛ࣮૷
    -VBεΫ Ϧϓ τ
    ϔο μʹ෇༩͞ΕͨϧʔϜ*%Λݩʹಈత෼ࢄ
    8FC"1*ʹΑ Γແఀࢭʹઃఆมߋ

14.  ෼ࢄϩδο Ϋͷ࣮૷ ෼ࢄϩδο Ϋ
    ϧʔϜ*%ͷϋο γϡ஋ͱ/PEFαʔό਺ͷ৒༨ࢉ ʢNPEʣ Ͱ ઀ଓઌϊ

    ʔυ͕ܾఆ͞ΕΔ
    $POTJTUFOU
    )BTIJOH ʢϋο γϡܭࢉʣ
    ϊ ʔυͷ௥Ճ ɾ ࡟আΛ࠷খݶͷมߋͰϚ ο ϐϯά

15.  ສηο γϣϯ
    ಉ࣌ରઓ਺
    ສୡ੒

16.  ͭͷνϟ Ϩϯδ શσʔλͷอଘͱσʔλϩʔυ ϦϓϨΠػೳͷ೥຤Ϧ Ϧʔε /HJOYʹΑ Δಠ࣮ࣗ૷ͷ-
    ϩʔυόϥϯα ରઓγεςϜͷεέʔϧΞ΢ τ

17. ϦϓϨΠػೳ ࠷৽݅ͷότϧΛ
 ϦϓϨΠ ͢Δ
    શରઓσʔλͷอଘ
    շదͳԠ౴଎౓ 

18.  (BNF
    4ZTUFN 3FQMBZ
    4ZTUFN ϦϓϨΠσʔλͷอଘͱ࠶ੜ Amazon S3 Redis SQS Web Servers


    (PHP) Worker Servers
 (PHP)

19.  ϦϓϨΠػೳͷσʔλ

20.  ϦϓϨΠػೳͷ೥຤Ϧ Ϧʔε ࣮੷ͷ๛෋ͳ"84αʔϏεΛ࠾༻
    424ʹΑ ΔඇಉظԽͰػೳ෼཭
    "NB[PO
    4ͷ҆ఆͨ͠σʔλऔಘ
    ຊ൪؀ڥʹ͓͚Δ௕ظؒͷݕূ
    Ϣʔβʹ͸ݟͤͳ͍ඇެ։Ϧ Ϧʔε

21.  · ͱΊ ୯Ұো֐఺ͷͳ͍֦ுՄೳͳαʔόߏ੒
    ϐʔΫ࣌ͷ௨৴ྔ ɾ ΞΫηε͕ଟ͘ɺ ༧ଌࠔ೉
    ରઓγεςϜΛಠٕࣗज़Ͱ࣮૷
    େ༰ྔσʔλͷอଘɺ

    σʔλ׬શੑͷ֬อ
    "NB[PO
    3%4
    "NB[PO
    4Λར༻

22.  εέʔϥϒϧͳ
 ϩάऩूج൫

23.  ιʔγϟϧήʔϜͱ ϩά $4ରԠ
    ,1*ࢦඪͷऔಘ
    γεςϜෆ۩߹ͷௐࠪ

24.  σʔλͷछྨͱྔ ςΩε τϩά
    ΞΫηεϩάɺ Τϥʔϩάɺ ό τϧϩάɺ 42-ϩάʜ
    %BUBCBTF
    *OTFSU
    ϩά
    

    ՝ۚϩάɺ ΞΠ ςϜϩάɺ ߦಈϩά

25.  ˞4IBEPXWFSTFҎ֎΋ؚΉ

26. ςΩε τϩάऩूͷ՝୊ ϩάऔΓ ͜΅͠
    ҰߦͷαΠ ζ੍ݶ
    όο ϑ ΝΩϡʔҲΕ
    தܧαʔόͷෛՙ
    

    ϩάσʔλ૿େʹΑ Δ
 εέʔϧΞ΢ τͷ՝୊  Web Server 1 Error log KPI log Log Proxy 3 4UPSBHF
    "SSBZ Web Server 2 Web Server N SQL log Log Proxy 2 Access log Log Proxy 1

27. .Z42-Π ϯαʔ τ ϩάͷ՝୊ Ϩεϙϯε஗Ԇ
    .Z42-
    *OTFSUॲཧͷ஗Ԇ
    ϩάσʔλ૿େ
    ϨϓϦέʔγϣϯ஗Ԇ  Web

    Server 1 Action log Web Server 2 Web Server N Item log MySQL Server Payment log

28.  ಠࣗϩάసૹΤʔδΣϯ τͷ։ൃ "NB[PO
    4ʹϩάΛू໿
    ̎ͭͷϩάసૹΤʔδΣϯ τ Λ։ൃ
    4UBMLFS
    ςΩε τϩάͷసૹ
    1PSUFS
    
    %#
    *OTFSUϩάͷඇಉظసૹ
    

    ߏ଄Խσʔλͷసૹͱ424ϝ οηʔδ࡞੒

29.  4UBMLFS
    ͱ
    1PSUFS औΓ ͜΅͠๷ࢭͱ࠶઀ଓ
    సૹͰ͖ͳ͔ͬͨϩά͸શͯϩʔΧϧʹอଘ
    &YQPOFOUJBM
    #BDLP⒎
    Ϧ τϥΠִؒͷௐ੔
    +JUUFS ʢ͹Β͖ͭʣ Λಋೖ͠ɺ

    ࠶઀ଓཁٻΛ෼ࢄ

30.  4UBMLFS
    ͱ
    1PSUFS "NB[PO
    4
    ͷύϑ ΥʔϚϯε޲্
    ઌ಄จࣈʹϋο γϡϓϨϑ Ο ο Ϋ ε௥Ճ͠ɺ

    
 JOEFYΛෳ਺ύʔς Ο γϣϯʹ෼ࢄ ྫ ɿ 4CVDLFU/MpMF໊

31.  "TZODISPOPVT
    -PH
    4ZTUFN (BNF
    4ZTUFN CVTJOFTT
    4ZTUFN 1PSUFSͱඇಉظ%#ϩάγεςϜ Worker Servers

    (Amazon EC2) Log Datebase (Amazon Aurora) σʔλղੳ CSରԠ Porter Web Servers Amazon S3 SQS TSV

32.  "84
    4΁ϩάू໿ Kibana Google BigQuery Mackerel Amazon Aurora Redshift Porter/Stalker

    Amazon S3

33. 

34.  ϩά౷ܭͱ άϥϑඳը )551εςʔλε
    
    ͷΞΫηε਺

35.  ώʔ τϚοϓ ΞΫηεϩάͱ࠲ඪ৘ใΛϚο ϐϯά

36.  .BDLFSFM "QBDIFϨεϙϯελΠ Ϝ
    ඵҎ্ͷ෼෍

37. "NB[PO
    4
    ͱ֎෦πʔϧͷ࿈ܞ εέʔϥϒϧʹ
 Π ϯελϯεΛ૿ݮ
    ϩάͷྔ͸ϐʔΫ࣌ͱ
 ࠷খͷ࣌Ͱ໿ഒͷࠩ
    αʔόϨεͰ
 ӡ༻޻਺Λܰݮ  Google

    BigQuery Porter/Stalker Amazon S3

38.  1.Put Event Trigger 2.Create Message 3.C ount M essages

    &$4Λར༻ͨ͠εέʔϥϒϧͳϩά࿈ܞ Google Cloud Storage Google BigQuery Auto Scaling 4.Run Task 6.Save Objects SQL Query Cron Job Amazon S3 Amazon Lamdba Amazon Lamdba SQS Amazon EC2 Container Service(ECS) Ϋϥελͷ$16࢖༻཰ʹԠͯ͡
    Π ϯελϯε਺Λ૿ݮ 5.Get Files ϩάྔʹԠͯ͡ίϯςφىಈ਺Λ૿ݮ

39. ͭͷεέʔϦϯάϙϦ γͷ࿈ಈ -BNCEB
    'VODUJPO
    ϩάྔʹԠͯ͡ɺ 
 ίϯςφىಈ਺Λ্͛Δ
    &$
    "VUP4DBMJOH
    ىಈதίϯςφ਺͕
 ্ঢʹΑ Γ$16࢖༻཰্ঢ
    

    &$εέʔϦϯάϙϦ γʹΑ Γ
 $POUBJOFS
    *OTUBODF
    ૿Ճ  &$4Ϋϥελʔͷ$16࢖༻཰ $POUBJOFS
    *OTUBODFىಈ਺

40.  ίε τ࡟ݮͱࣗಈԽͷ࣮ݱ αʔόϨε
    ʺ
    ίε τ࡟ݮ
    -BNCEB
    
    &$4
    ʹΑ Γ ίε τ Λ
    
    ʹ࡟ݮ
    

    ίϯςφٕज़ͷ࠾༻
    -BNCEBͱ&$4
    ͷద༻ൣғ
    όονॲཧɺ Π ϕϯ τ τ ϦΨʔ͸-BNCEB
    
    ৗ࣮࣌ߦ͢Δॲཧ͸&$4

41.  "NB[PO
    "UIFOBͷར༻ 1.Put Event Trigger 2.Create Message 3.C ount M

    essages Auto Scaling 4.Run Task Cron Job Amazon S3 Amazon Lamdba Amazon Lamdba SQS Amazon EC2 Container Service(ECS) 5.Get Files SQL Query Amazon S3 Amazon Athena 6.Save Objects Google Cloud Storage Google BigQuery SQL Query

42. "NB[PO
    "UIFOBͷར༻ ύεม׵ʹΑ Δύʔς Ο γϣϯԽ
    4UBMLFSͷ4ύεܗࣜ
    "NB[PO
    "UIFOBͷ4ύεܗࣜ
     S3://athena-examples/l2xa-<file>.20150101 s3://athena-examples/ex/2015/01/01/<file>

43.  · ͱΊ ϩάू໿ʹΑ Γσʔλ׆༻͕ଅਐ
    ඇಉظԽΛߦ͍ϨεϙϯεΛվળ
    αʔόϨε ͱࣗಈԽͰίε τ ͱ޻਺࡟ݮ

44.  $ZHBNFTΠ ϯϑϥ͕
    େ੾ʹ͍ͯ͠Δ ͜ ͱ

45.  ౰ͨ Γલͷ͜ ͱ Λ
 ౰ͨ Γલʹ΍Δ

46.  Π ϯϑϥͷϨΠ ϠʔͰ
 ՝୊Λղܾ͢Δ

47.  ίΞٕज़͸
 ࣗ෼ͨͪͰ࣮૷͢Δ

48.  5IBOL
    ZPV