Upgrade to Pro — share decks privately, control downloads, hide ads and more …

DeNA AIシステム部におけるクラウドを活用した機械学習基盤の構築

Naotoshi Seo
February 17, 2017

DeNA AIシステム部におけるクラウドを活用した機械学習基盤の構築

Naotoshi Seo

February 17, 2017
Tweet

More Decks by Naotoshi Seo

Other Decks in Technology

Transcript

  1. Copyright (C) DeNA Co.,Ltd. All Rights Reserved. DeNA AIγεςϜ෦ʹ͓ ͚ΔΫϥ΢υΛ׆༻ͨ͠

    ػցֶशج൫ͷߏங #denatechcon 2017 1 Feb 10, 2017 Naotoshi Seo AI System Division
 DeNA Co., Ltd.
  2. ࣗݾ঺հ 2 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. ɾ੉ඌ௚ར!TPOPUT

    ɾ"*γεςϜ෦ چ෼ੳج൫෦  ɾ"OBMZUJDT5PPMTνʔϜ $MPVE*OGSBνʔϜ ϦʔυΤϯδχΞ ݉೚  ɾ'MVFOUE3VCZίϛολ
  3. ର৅ऀ 3 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. •

    ػցֶशΘ͔Βͳ͍͚Ͳػցֶशج൫Λ࡞Β ͳ͚Ε͹͍͚ͳ͘ͳͬͨΠϯϑϥ୲౰ऀ • ΠϯϑϥΘ͔Βͳ͍͚Ͳػցֶशج൫Λ࡞Β ͳ͚Ε͹͍͚ͳ͘ͳͬͨػցֶशΤϯδχΞ
  4. ໨࣍ 4 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. •

    ػցֶशج൫ʹٻΊΒΕΔ ͨ ΋ͷ • ५୔ͳ(16 • ִ཭͞Εͨ؀ڥ • ૉૣ͍ߏங • ӡ༻ָ͕ • ࣗ༝౓Λߴ͘ • ϛε͕͓͖ʹ͍͘ • (16αʔόͷΦϯσϚϯυΦʔτεέʔϧ • "1*αʔό αʔϏεఏڙ؀ڥ  • 'VUVSF8PSL • ͓·͚"84ͱ($1ͷൺֱ
  5. 8 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. • σΟʔϓϥʔχϯά

    • (16Λ࢖ͬͨ΄͏͕଎͍ • ྫ $16ͩͱ೔ˠ(16ͩͱ೔ ͳͥ(16͕ඞཁ͔ GPUͷجૅ஌ࣝ
  6. (16ͷಛ௃ 9 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. •

    ฒྻॲཧ͕ಘҙ • $16ͩͱίΞͳͲͷΦʔμʔ • (16ͩͱ d ίΞ • ෼ذॲཧ͸ۤख • ෼ذ༧ଌɾΞ΢τΦϒΦʔμʔͷ ػೳΛ؆ૉԽ͢Δ͜ͱͰίΞ਺Λ ૿΍͍ͯ͠Δ • ߦྻԋࢉʹ޲͍͍ͯΔ GPUͷجૅ஌ࣝ
  7. (16੡඼ 10 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. •

    /7*%*"5FTMB • /7*%*"ͷ)1$޲͚(16γϦʔζ • /7*%*"(F'PSDF(3*% • Ϋϥ΢υήʔϛϯά޲͚ • ".%'JSF1SP • ".%ͷ)1$޲͚γϦʔζ GPUͷجૅ஌ࣝ
  8. /7*%*"5FTMB 11 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. •

    ੈ୅ • 'FSNJ,FQMFS • .BYXFMM1BTDBM • ྫ LˠLFQMFS • "1* • $6%" /7*%*"ಠࣗ  • 0QFO$- • %JSFDU$PNQVUF GPUͷجૅ஌ࣝ http://pc.watch.impress.co.jp/img/pcw/docs/755/994/html/Visio- NvidiaGPUComparisonGFLOPSvsMemorybandwidth.png.html
  9. $6%"ͷΞʔΩςΫνϟ 12 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. •

    ϗετ $16 ͔ΒσόΠ ε (16 ʹσʔλΛసૹ • (16Ͱॲཧ • ݁ՌΛσόΠε (16 ͔ Βϗετʹసૹ $16 GPUͷجૅ஌ࣝ http://www.nvidia.co.jp/docs/IO/51174/ NVIDIA_CUDA_Programming_Guide_1.1_JPN.pdf
  10. Copyright (C) DeNA Co.,Ltd. All Rights Reserved. DeNAͷΦϯϓϨGPU؀ڥ (2017/02ݱࡏ) 14

    CPUɿXeon E5-2630 v3 2.4GHz *2 GPUɿNVIDIA Tesla M40 *2 Memɿ64GB NICɿ10Gb ୆ ίΞ਺ MHz GPUϝϞϦ ൃද೥ M40 3072 1114 12GB 2015/11 ५୔ͳGPU
  11. "*Ҋ݅ʹ͓͚Δ(16ར༻ 16 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. •

    ͋Ε͹͋Δ͚ͩ࢖͍͍ͨ • ύϥϝʔλΛগ͠ม͑ͯฒྻͰֶशͤ͞ ͯɺྑ͍ํΛબͿ • ֶश͢Δ࣌ʹ͚ͩ࢖͏ • ऴΘͬͨΒഇغͯ͠΋ߏΘͳ͍ ५୔ͳGPU
  12. HΠϯελϯε 19 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. AWSͷGPU

    GPU ίΞ਺ MHz ൃද೥ NVIDIA GRID K520 1536 * 2 800 2013/5 Πϯελϯε໊ GPU
 ݸ vCPU ݸ ϝϞϦ (GiB) ฒྻί Ξ GPUϝ ϞϦ (GB) ωο τϫ ʔΫ Ձ֨/࣌ (us- east-1) g2.2xlarge 1 8 15 1,536 4 $0.65 g2.8xlarge 4 32 60 6,144 32 $2.6
  13. QΠϯελϯε Copyright (C) DeNA Co.,Ltd. All Rights Reserved. ౦ژϦʔδϣϯͰ(·ͩ)࢖͑ͳ͍ AWSͷGPU

    GPU ίΞ਺ MHz ൃද೥ NVIDIA Tesla K80 2496 * 2 875 2014/11 Πϯελϯε໊ GPU ݸ vCPU ݸ ϝϞ Ϧ (GiB) ฒྻί Ξ GPUϝ ϞϦ (GB) ωοτ ϫʔΫ Ձ֨/࣌ (us- east-1) p2.xlarge 1 4 61 2,496 12 ߴ $0.9 p2.8xlarge 8 32 488 19,968 96 10G $7.2 p2.16xlarge 16 64 732 39,936 192 20G $14.4
  14. %F/"Ͱͷ(16ར༻ 21 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. •

    VTͰQΠϯελϯε L Λ࢖͏ • ֶशͤ͞Δ͚࣌ͩىಈ͠ɺ͋ͱ͸ࢭΊΔ • ϋΠύʔύϥϝʔλνϡʔχϯάͳͲͰͨ ͘͞Μฒ΂࣮ͯߦ͢ΔͷʹΫϥ΢υ؀ڥ͸ ༗ར‷ ŋŷŋ و ̑̑ ५୔ͳGPU
  15. ($1ͷ(16 23 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. •

    ΞϧϑΝ • ਃ੥͢Ε͹L QΠ ϯελϯεͱಉ͡ ͕࢖ ͑Δ • ͦΕҎ֎͸·ͩμϝͩͬ ͨ ɾТɾʆ  • L͕௨ৗΠϯελϯ εʹΞλον͞Εͨܗ GCPͷGPU
  16. ($1ͷ(16 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. ௨ৗΠϯελϯε୅ۚʹΞλον෼͚ͩϓϥε GCP͸Πϯελϯεͷ՝͕ۚ෼୯Ґ(ͨͩ͠ɺ

    ࠷௿Ͱ΋10෼͸औΒΕΔ) GCPͷGPU GPU ݸ ฒྻί Ξ GPUϝ ϞϦ (GB) FP32
 (TFLOP S) Ձ֨/࣌ (us) Ϧʔδϣ ϯ Tesla K80x1 1 2,496 12 4.3 $0.7 ౦ژҎ֎ Tesla P100x1 1 3,584 16 10.6 ෆ໌ N/A AMD FirePro S9300 x2 1 8 13.9 ෆ໌ N/A
  17. ػցֶशج൫ʹٻΊΒΕΔ ͨ ΋ͷ 25 Copyright (C) DeNA Co.,Ltd. All Rights

    Reserved. ɾ५୔ͳ(16 ɾ ִ཭͞Εͨ؀ڥ ɾૉૣ͍ߏங ɾӡ༻ָ͕ ɾࣗ༝౓Λߴ͘ ɾϛε͕͓͖ʹ͍͘
  18. ִ཭͞Εͨ؀ڥ 26 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. •

    %F/"ͷ"*Ҋ݅  • ൿಗҊ݅ • ڠۀҊ݅ • ඇؔ༩ࣾһ͕σʔλʹΞΫηεͰ͖ͳ͍Α͏ʹִ ཭͞ΕͨηΩϡΞͳ؀ڥΛ࡞Δඞཁ͕͋Δ • ͨͩ͠ɺڠۀձࣾͷؔ༩ࣾһ͸ΞΫηεͰ͖Δ Ϋϥ΢υΞΧ΢ϯτΛ෼͚ͯ ผݸͷ؀ڥͱִͯ͠཭ ִ཭͞Εͨ؀ڥ
  19. ωοτϫʔΫߏ੒ 27 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. Subnet

    service components step DeNA PC Gateway ghe, etc Security Group ssh Security Group ִ཭͞Εͨ؀ڥ • ΦϯϓϨ(BUFXBZ ͔ΒͷΈ౿Έ୆αʔ όʹTTIՄೳ • (BUFXBZͰάϧʔ ϓݖݶ؅ཧ • ίʔυ͸HJUIVCF ʹઐ༻PSHΛ࡞ͬ ͯQSJWBUFӡ༻
  20. ؂ࠪϩά 28 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. •

    ίϚϯυ࣮ߦϩά • %F/"ͷ(BUFXBZʹ͋ΔΩʔϩΨʔ • Ϋϥ΢υͷૢ࡞ϩά • "84$MPVE5SBJM • ($14UBDL%SJWFS ִ཭͞Εͨ؀ڥ
  21. ػցֶशج൫ʹٻΊΒΕΔ ͨ ΋ͷ 29 Copyright (C) DeNA Co.,Ltd. All Rights

    Reserved. ɾ५୔ͳ(16 ɾִ཭͞Εͨ؀ڥ ɾ ૉૣ͍ߏங ɾӡ༻ָ͕ ɾࣗ༝౓Λߴ͘ ɾϛε͕͓͖ʹ͍͘
  22. ૉૣ͍ߏங 30 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. •

    "*Ҋ͕݅ͲΜͲΜ্ཱ͕ͪͬͯ͘Δ • ̍ϲ݄ʹ̑ϓϩδΣΫτ ݄  • ઐ೚͡Όͳ͍ͷͰɺखؒͳ͘ૉૣ͘ ૉૣ͍ߏங
  23. ࣗಈԽίʔυԽ 31 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. •

    5FSSBGPSNͰΦʔέετϨʔγϣϯͷ ࣗಈԽˍίʔυԽ • "84 ($1྆ํʹରԠ • *UBNBFͰϓϩϏδϣχϯά • ੿࡞ͷLPOEBUFϥούʔ ૉૣ͍ߏங
  24. ػցֶश؀ڥͷ ϓϩϏδϣχϯά 32 Copyright (C) DeNA Co.,Ltd. All Rights Reserved.

    • VCVOUV$IBJOFSΦεεϝ  • OWJEJBESJWFS • DVEBOWJEJBͷDVEBBQUSFQP͔ΒೖΕΔͱOWJEJBESJWFS΋ೖָͬͯ • DVEOOOWJEJBͷNBDIJOFMFBSOJOHBQUSFQP͔ΒೖΕΔͱָ  • QZUIPO • DIBJOFS ૉૣ͍ߏங
  25. ػցֶशج൫ʹٻΊΒΕΔ ͨ ΋ͷ 33 Copyright (C) DeNA Co.,Ltd. All Rights

    Reserved. ɾ५୔ͳ(16 ɾִ཭͞Εͨ؀ڥ ɾૉૣ͍ߏங  ɾ ӡ༻ָ͕ ɾࣗ༝౓Λߴ͘ ɾϛε͕͓͖ʹ͍͘
  26. ӡ༻ָ͕ 34 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. •

    "*Ҋ͕݅ͲΜͲΜ্ཱ͕ͪͬͯ͘Δ • ͲΜͲΜ؀ڥ͕૿͑Δ • ઐ೚͡Όͳ͍ͷͰɺָʹ͠ͳ͍ͱ݉ۀͷ ΞϓϦ։ൃۀ͕Ͱ͖ͳ͍ ӡ༻ָ͕
  27. Ϋϥ΢υͷػೳΛ࢖͏ 36 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. •

    Ͱ͖Δ͚ͩΫϥ΢υͷػೳΛ࢖͏ • ؂ࢹπʔϧ • "84$MPVE8BUDI • ($14UBDL%SJWFS • .Z42- "843%4 ($1$MPVE42-  • 3FEJT "84&MBTUJDBDIF ͳͲͳͲ ӡ༻ָ͕
  28. ར༻தͷ"84αʔϏε 37 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. •

    &$ • 71$ • &#4 • 3PVUF • 4 • "VSPSBPS3%4 .Z42-  • &MBTUJ$BDIF 3FEJT  • &MBTUJD-PBE#BMBODJOH • $MPVE'SPOU • &MBTUJD'JMF4ZTUFN /'4 • 4/4 • 4&4 4.51  • &.3 )BEPPQ 4QBSL  • $MPVE8BUDI • $MPVE5SBJM ӡ༻ָ͕
  29. ར༻தͷ($1αʔϏε 38 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. •

    (PPHMF$PNQVUF&OHJOF ($&  • (PPHMF"QQ&OHJOF • (PPHMF-PBE#BMBODJOH • (PPHMF$MPVE$%/ • (PPHMF$MPVE%/4 • (PPHMF$MPVE42- • 4UBDLESJWFS.POJUPSJOH • (PPHMF$MPVE4UPSBHF • (PPHMF#JH2VFSZ • (PPHMF$MPVE%BUBQSPD ӡ༻ָ͕
  30. ػցֶशج൫ʹٻΊΒΕΔ ͨ ΋ͷ 39 Copyright (C) DeNA Co.,Ltd. All Rights

    Reserved. ɾ५୔ͳ(16 ɾִ཭͞Εͨ؀ڥ ɾૉૣ͍ߏங ɾӡ༻ָ͕ ɾ ࣗ༝౓Λߴ͘ ɾϛε͕͓͖ʹ͍͘
  31. ࣗ༝౓Λߴ͘ 40 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. •

    γε ΤϯδχΞ ʹ΋̎ύλʔϯ  Πϯϑϥ͍͡Γ͍ͨ  Πϯϑϥ͓೚ͤ͠·͢ • ͍͡Γ͍ͨͳΒੋඇ΍ͬͯ͘Εʂ ࣗ༝౓Λߴ͘
  32. ͍ͬͯ͡΋ྑ͍ 42 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. •

    Πϯελϯεͷ্ཱͪ͛͸ࣗ༝ • Ϋϥ΢υαʔϏεͷઃఆ • 5FSSBGPSNʹ13 • ϓϩϏδϣχϯά • ج൫ͱ͸ผͷπʔϧͰίʔυԽ΋0, • खಈηοτΞοϓͯ͠ΠϝʔδʹݻΊ ͯ΋0, ࣗ༝౓Λߴ͘
  33. ػցֶशج൫ʹٻΊΒΕΔ ͨ ΋ͷ 43 Copyright (C) DeNA Co.,Ltd. All Rights

    Reserved. ɾ५୔ͳ(16 ɾִ཭͞Εͨ؀ڥ ɾૉૣ͍ߏங ɾӡ༻ָ͕ ɾࣗ༝౓Λߴ͘ ɾ ϛε͕͓͖ʹ͍͘
  34. ϛε͕͓͖ʹ͍͘ 44 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. •

    ࣗ༝౓Λߴ͘͢Δͱࠓ౓͸ϛε͕͓͖Δ • ͨͱ͑͹ɺΠϯελϯε࡞੒ • ద੾ͳηΩϡϦςΟάϧʔϓΛ෇͚Δ • λάΛ෇͚Δ • 3PVUFʹϗετ໊ొ࿥ • ؂ࢹ౤ೖ ࣗ࡞πʔϧͰ୲อ ϛε͕͓͖ʹ͍͘
  35. FDDSFBUFJOTUBODFT HDFDSFBUFJOTUBODFT 45 Copyright (C) DeNA Co.,Ltd. All Rights Reserved.

    • ΠϯελϯεΛ࡞੒ • 3PMFTλάΛ෇͚Δ • ద੾ͳΠϝʔδΛબ୒ ࣍΃ʔδ  • ద੾ͳηΩϡϦςΟάϧʔϓΛ෇͚Δ • 3PVUFʹϗετ໊ొ࿥ • ($1ͷ৔߹͸ඪ४Ͱ%/4ొ࿥͞ΕΔͷͰෆཁ • ؂ࢹ౤ೖ $ ec2-create-instances -n host --role chainer:dev ϛε͕͓͖ʹ͍͘
  36. FDDSFBUFBNJ HDFDSFBUFTOBQTIPU 46 Copyright (C) DeNA Co.,Ltd. All Rights Reserved.

    • ࢦఆͨ͠ϗετΛΠϝʔδԽ • Πϝʔδʹ΋ϗετͱಉ͡ϩʔϧΛ෇͚Δ • ࣍ճFDDSFBUFJOTUBODFT͢Δࡍʹ͸ɺ ಉ͡ϩʔϧͷ".*ͷ͏ͪɺ࠷৽ͷ΋ͷ͔ΒΠ ϯελϯεΛཱͪ͋͛Δ $ ec2-create-ami -n host ϛε͕͓͖ʹ͍͘
  37. FDMJTUBNJ  HDFMJTUTOBQTIPUT 47 Copyright (C) DeNA Co.,Ltd. All Rights

    Reserved. • ࢦఆͨ͠ϩʔϧͷ͍ͭͨ".*ͷ͏ͪɺ࠷৽ͷ ΋ͷΛදࣔ͢Δ • FDDSFBUFJOTUBODFT͸಺෦తʹ͜Ε Λ ϥΠϒϥϦͱͯ͠ ར༻ͯ͠ɺద੾ͳ".* Λબ୒͢Δ $ ec2-list-ami --role chainer:dev ami-XXXXXXXX 2017-02-04T03:53:10+09:00 chainer:dev ϛε͕͓͖ʹ͍͘
  38. FDIPTU HDFIPTU 48 Copyright (C) DeNA Co.,Ltd. All Rights Reserved.

    • &$Πϯελϯεʹ 3PMFTλάΛ෇͚ͯɺϩʔ ϧͰ؅ཧɺݕࡧ • ($&ͷ৔߹͸ NFUBEBUBΛར༻  • *UBNBFͷϩʔϧࢦఆ࣌ ʹ΋ར༻ $ ec2-host --role chainer:dev asaa5002 ϛε͕͓͖ʹ͍͘
  39. ໨࣍ 49 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. ɾػցֶशج൫ʹٻΊΒΕΔ

    ͨ ΋ͷ ɾ(16αʔόͷΦϯσϚϯυΦʔτεέʔϧ ɾ"1*αʔό αʔϏεఏڙ؀ڥ  ɾ'VUVSF8PSL ɾ͓·͚"84ͱ($1ͷൺֱ
  40. (16αʔόͷ ΦϯσϚϯυΦʔτεέʔϧ 50 Copyright (C) DeNA Co.,Ltd. All Rights Reserved.

    • ฒྻͰֶशΛ૸Β͍ͤͨ • (16ΠϯελϯεΛඞཁͳ෼͚ͩࣗಈ Ͱ্ཱͪ͛ͯཉ͍͠ • ֶश͕ऴΘͬͨΒࣗಈͰམͱͯ͠΄͍͠ • མͱ͠๨Ε͕ͪʼʻ GPUαʔόͷΦϯσϚϯυΦʔτεέʔϧ
  41. ... Chainer Cluster create ssh terminate ... XXX Cluster FDTDBMFSVO

    51 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. • ֶशͤ͞Δͱ͖ʹΠϯελϯεΛ ্ཱͪ͛ͯίϚϯυ࣮ߦ • ͢Ͱʹ্ཱ͕͍ͪͬͯͯ࢖ΘΕͯ ͍ͳ͍Πϯελϯε͕͍Ε͹ੵۃ తʹར༻͢Δ • "84͸̍࣌ؒ՝ۚͳͷͰ͙͢ʹ͸ ࡴ͞ͳ͍ • ىಈ࣌ؒNJONJO Ͱ࢖ΘΕ͍ͯͳ͍Πϯελϯε͕ ͍Ε͹མͱ͢ GPUαʔόͷΦϯσϚϯυΦʔτεέʔϧ $ ec2-scale-run -c config.yml [<‘Some shell code'> | <filename [args ...]>
  42. %PDLFSରԠ 53 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. TDBMFSVOͷதͰOWJEJBEPDLFSSVO

    ྫ $IBJOFS5FOTPS'MPXίϯςφ "84($1Ͱखݩͱಉ͡ίϯςφΛ࢖͑ͯศར 2&$4͸࢖Θͳ͍ͷʁ "ࠓͷͱ͜Ζ࢖ͬͯ·ͤΜ ཧ༝(16Λ࢖͑ΔΑ͏ʹ͢Δઃఆ͕ ৭ʑΊΜͲ͏ͩͬͨ ཧ༝(,&ͱͷΠϯλʔϑΣʔεҧ͍ GPUαʔόͷΦϯσϚϯυΦʔτεέʔϧ
  43. ิ଍OWJEJBEPDLFS 55 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. •

    EPDLFSDMJͷബ͍ϥούʔ • %PDLFSίϯςφ͔Βϗετͷ(16 Λ࢖͏ͱ͖ʹศར • OWJEJBEPDLFSQMVHJO͔Β৘ ใΛಘͯɺEPDLFSίϚϯυʹΦ ϓγϣϯΛ৭ʑ౉ͯ͠΋ྑ͍͕໘౗ $ nvidia-docker run nvidia/cuda nvidia-smi GPUαʔόͷΦϯσϚϯυΦʔτεέʔϧ
  44. /'4Ϛ΢ϯτ 56 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. •

    ΠϯελϯεΛ্ཱͪ͛ ͯ͸མͱ͢ͷͰɺ • ڞ༗ετϨʔδ͕ඞཁ • ֶशݩσʔλ • தؒεφοϓγϣοτ NFS • "NB[PO&'4ΛϚ΢ϯτ • ༰ྔࣗಈ֦ு • ஋ஈ͸͓ߴΊ (#݄  • ౦ژϦʔδϣϯͰ͸࢖͑ͳ͍ GPUαʔόͷΦϯσϚϯυΦʔτεέʔϧ
  45. εϙοτΠϯελϯεͷ׆༻ 57 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. •

    ೖࡳՁ֨ࢢ৔ εϙοτ Ձ֨ͳΒɺࢢ৔Ձ֨ ͰΠϯελϯε্ཱ͕͕ͪΔ • ࢢ৔Ձ͕͕֨͋ͬͨΒɺՁ͕֨มΘΔ • ࢢ৔Ձ֨ೖࡳՁ֨ʹٯసͨ͠ΒɺΠϯελϯ ε͕མͱ͞ΕΔ εϙοτΠϯελϯε <࣮੷> QYMBSHFIˠI 0'' GPUαʔόͷΦϯσϚϯυΦʔτεέʔϧ
  46. "1*αʔό 59 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. •

    ػցֶश̎ͭͷϑΣʔζ • ֶशϑΣʔζ • ༧ଌϑΣʔζ • ֶशϑΣʔζ͸։ൃ؀ڥͰ (16ར༻  • ༧ଌϑΣʔζ͸ϓϩμΫγϣϯ $16ར༻  • )551"1*ͱͯͯ͠͠αʔϏεʹఏڙ APIαʔό
  47. ࣮ྫ 60 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. Elastic

    Load Balancing ... API Server (Blue) ... API Server (Green) • $IBJOFSͷ༧ଌϑΣʔ ζΛ࢖͏ͷͰ1ZUIPO ΞϓϦέʔγϣϯ • %KBOHP • ϩʔΧϧ+VCBUVT • #MVF(SFFO %FQMPZNFOU APIαʔό ap-northeast-1 (cpu)
  48. 'VUVSF8PSL 62 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. •

    "1*αʔόΛ("& (PPMHF"QQ&OHJOF  Ͱӡ༻ָ͍ͯͨ͠͠ • ("&QZUIPO • 4UBOEBSE&OWJSPONFOU͸੍໿͕ݫ͠ ͗͢ΔͷͰ'MFYJCMF&OWJSPONFOU͔  • શࣗಈϋΠύʔύϥϝʔλνϡʔχϯά͞Μ Future Work
  49. 'VUVSF8PSL 63 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. •

    ෺ཧ(16Ϛγϯ΋ཉ͍͠ ɾТɾʆ  • τϥΠΞϯυΤϥʔͳ։ൃ༻్ • खݩͰ։ൃ͍ͨ͠ • େྔʹฒྻͰֶश͍ͤͨ͞৔߹͸Ϋϥ΢υ • ࠷௿Ͱ΋̍ਓ̍୆ߪೖ༧ఆ‷ ŋŷŋ و ̑̑ Future Work
  50. Πϯελϯεྉۚ 66 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. •

    (16Πϯελϯεͷ஋ஈ͸͋·ΓมΘΒͳ͍ • ($&Πϯελϯεͷ՝ۚ͸̍෼͝ͱ • ͨͩ͠ɺ࠷௿Ͱ΋෼͸औΒΕΔ • ($&͸Πϯελϯε͕̍ϲ݄࣮ؒߦ͞ΕΔ ͱɺྉ͕ۚࣗಈͰׂΓҾ͔ΕΔ • &$ͷϦβʔϒυΠϯελϯεͱҧ͍ɺ৽͍͠Π ϯελϯελΠϓ͕ग़ͨ࣌ʹҾͬӽ͠΍͍͢ҹ৅ AWSͱGCPͷൺֱ
  51. ($1ͷϩʔυόϥϯα͕ڧ͍ 67 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. •

    ΢ΥʔϜΞοϓෆཁ • ࣄલεέʔϧΞ΢τਃ੥ෆཁ • ϚϧνϦʔδϣϯରԠ • ̍άϩʔόϧ*1 *1ΤχʔΩϟετ AWSͱGCPͷൺֱ
  52. ($1ͷωοτϫʔΫઃఆ͕ॊೈ 68 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. •

    ΠϯελϯεʹλάΛ෇͚ͯɺλάͰϧʔ ςΟϯάઃఆՄೳ • "84ͷ৔߹͸TVCOFUͰϧʔςΟϯάઃఆ AWSͱGCPͷൺֱ
  53. "84ͷ΄͏͕αʔϏε͸ॆ࣮ 69 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. •

    ($1ʹ͸"84&MBTUJ$BDIF 3FEJT  .FNDBDIFE ૬౰͕ͳ͍ • &'4 /'4 ૬౰͕ͳ͍ • 4.51αʔϏε͕ͳ͍ • SEQBSUZͱఏܞ • ଍Γͳ͍ͱ͜Ζ͕ͪ´ • 4UBDL%SJWFSʹ؂ࢹొ࿥"1*͕·ͩͳ͍ AWSͱGCPͷൺֱ
  54. ·ͱΊ 70 Copyright (C) DeNA Co.,Ltd. All Rights Reserved. •

    ػցֶशج൫ʹٻΊΒΕΔ ͨ ΋ͷ • ५୔ͳ(16 • ִ཭͞Εͨ؀ڥ • ૉૣ͍ߏங • ӡ༻ָ͕ • ࣗ༝౓Λߴ͘ • ϛε͕͓͖ʹ͍͘ • (16αʔόͷΦϯσϚϯυΦʔτεέʔϧ • "1*αʔό αʔϏεఏڙ؀ڥ  • 'VUVSF8PSL ("& શࣗಈύϥϝʔλνϡʔχϯά  ˠͦΕΫϥ΢υͰ
  55. 4QFDJBM5IBOLT ॱෆಉ 71 Copyright (C) DeNA Co.,Ltd. All Rights Reserved.

    • OIBZBTIJ • LB[VLJGVKJLBXB • GVLVEBNB 1'/  • DJWJUBTQP • NB[HJ