Upgrade to Pro — share decks privately, control downloads, hide ads and more …

AWS で加速する機械学習 / Accelerate Machine Learning with AWS

AWS で加速する機械学習 / Accelerate Machine Learning with AWS

Yuichiro Someya

March 14, 2018
Tweet

More Decks by Yuichiro Someya

Other Decks in Programming

Transcript

  1. ৽͍͠௅ઓ  ॳͷػցֶशϓϩμΫτ  ηϛϦΞϧλΠϜͳը૾ೝࣝ  8FCΞϓϦέʔγϣϯͱҧ͏ϫʔΫϩʔυ  ඞཁͩͬͨ΋ͷ 

    ৽͍͠ϫʔΫϩʔυʹରͯ͠εέʔϧ͢ΔΠϯϑϥ  ࣦഊͯ͠΋طଘͷ؀ڥʹӨڹΛ༩͑ͳ͍ಠཱͨ͠؀ڥ  ૉૣ͘ࢼߦࡨޡΛ܁Γฦ͢ͷʹద࣮ͨ͠ݧ؀ڥ
  2. ৽͍͠௅ઓ  ॳͷػցֶशϓϩμΫτ  ηϛϦΞϧλΠϜͳը૾ೝࣝ  8FCΞϓϦέʔγϣϯͱҧ͏ϫʔΫϩʔυ  ඞཁͩͬͨ΋ͷ 

    ৽͍͠ϫʔΫϩʔυʹରͯ͠εέʔϧ͢ΔΠϯϑϥ  ࣦഊͯ͠΋طଘͷ؀ڥʹӨڹΛ༩͑ͳ͍ಠཱͨ͠؀ڥ  ૉૣ͘ࢼߦࡨޡΛ܁Γฦ͢ͷʹద࣮ͨ͠ݧ؀ڥ ৽͍͠αʔϏε࡞ΓΛ lՃ଎z͢Δ؀ڥ
  3. εέʔϥϏϦςΟͷ֬อ  ༧ଌͰ͖ͳ͍τϥϑΟοΫ େྔͷը૾ΞοϓϩʔυͳͲ   ಥવ57ʹ঺հ͞ΕΔ͜ͱ΋͋Δ͔΋͠Εͳ͍ ࣮ࡍ͋ͬͨ  

    ը૾ͷΞοϓϩʔυ΍൑ఆʹ͸͕͔͔࣌ؒΔ d਺ඦNT   ಉظॲཧͯ͠͠·͏ͱϢʔβମݧΛѱ͘͢Δ
  4. ؀ڥͷ෼཭  ͜Ε·Ͱͷ8FCαʔό؀ڥͱҧ͏؀ڥ͕ٻΊΒΕͨ  ҧ͏ݴޠ ҧ͏ϫʔΫϩʔυ ҧ͏ϋʔυ΢ΣΞ (16  

    ίϯςφ؀ڥΛར༻  ϥϯλΠϜ΍(16υϥΠόɺઃఆΛ෼཭  "NB[PO&$4Λར༻ͯ͠؅ཧ  طʹଟ͘ͷαʔϏε͕ίϯςφ؀ڥͰՔಇ
  5. API Server "1*αʔόʹ
 4QSFTJHOFE63-Λཁٻ খ͍͞αϜωΠϧΛ
 4όέοτʹΞοϓϩʔυ TUIVNCOBJMTKQH 4OPUJpDBUJPO͕
 424ʹ౤ೖ͞ΕΔ {

    bucket: thumbnails key: 123.jpg } ϝοηʔδΛ
 424͔ΒऔΓग़͢ ECS Cluster αϜωΠϧͷμ΢ϯϩʔυ ը૾ͷ൑ఆΛߦ͍݁ՌΛ
 "1*αʔόʹ௨஌
  6. API Server "1*αʔόʹ
 4QSFTJHOFE63-Λཁٻ { bucket: thumbnails key: 123.jpg }

    ϝοηʔδΛ
 424͔ΒऔΓग़͢ ECS Cluster αϜωΠϧͷμ΢ϯϩʔυ ը૾ͷ൑ఆΛߦ͍݁ՌΛ
 "1*αʔόʹ௨஌ খ͍͞αϜωΠϧΛ
 4όέοτʹΞοϓϩʔυ 4OPUJpDBUJPO͕
 424ʹ౤ೖ͞ΕΔ ಛผͳίʔυ΍ΠϯϑϥΛ༻ҙͤͣ
 εέʔϥϒϧͳߏ੒ʹ
  7. API Server "1*αʔόʹ
 4QSFTJHOFE63-Λཁٻ খ͍͞αϜωΠϧΛ
 4όέοτʹΞοϓϩʔυ TUIVNCOBJMTKQH 4OPUJpDBUJPO͕
 424ʹ౤ೖ͞ΕΔ {

    bucket: thumbnails key: 123.jpg } αϜωΠϧͷμ΢ϯϩʔυ ը૾ͷ൑ఆΛߦ͍݁ՌΛ
 "1*αʔόʹ௨஌ ϝοηʔδΛ
 424͔ΒऔΓग़͢ ECS Cluster ECS Cluster 424ͷΩϡʔ௕ʹԠͯ͡ &$4λεΫ΋εέʔϧ
  8. "1*αʔόʹ
 4QSFTJHOFE63-Λཁٻ খ͍͞αϜωΠϧΛ
 4όέοτʹΞοϓϩʔυ TUIVNCOBJMTKQH 4OPUJpDBUJPO͕
 424ʹ౤ೖ͞ΕΔ { bucket: thumbnails

    key: 123.jpg } ϝοηʔδΛ
 424͔ΒऔΓग़͢ ECS Cluster αϜωΠϧͷμ΢ϯϩʔυ ը૾ͷ൑ఆΛߦ͍݁ՌΛ
 "1*αʔόʹ௨஌ API Server API Server &$4λεΫ͔Βͷ൑ఆ݁ՌΛ଴͚ͭͩ
  9. IBLPʹΑΔΞϓϦέʔγϣϯఆٛ  IBLPIUUQTHJUIVCDPNFBHMFUNUIBLP  &$4ޓ׵ͷίϯςφσϓϩΠπʔϧ  ZBNMKTPOOFUܗࣜͰͷΞϓϦέʔγϣϯఆٛ  "VUP4DBMFɺ-#ͷࢦఆͳͲ΋Մೳ 

    શ։ൃऀ͕ࣗ෼ͰఆٛΛॻ͍ͯσϓϩΠՄೳ scheduler: type: ecs region: ap-northeast-1 cluster: hako-production-g2 desired_count: 1 app: image: food-photo-classifier cpu: 128 memory: 3072 memory_reservation: 2048 env: AWS_REGION: ap-northeast-1 COOKPADNET_ENV: production ...
  10. ৽͍͠௅ઓ  ॳͷػցֶशϓϩμΫτ  ηϛϦΞϧλΠϜͳը૾ೝࣝ  8FCΞϓϦέʔγϣϯͱҧ͏ϫʔΫϩʔυ  ඞཁͩͬͨ΋ͷ 

    ৽͍͠ϫʔΫϩʔυʹରͯ͠εέʔϧ͢ΔΠϯϑϥ✅  طଘͷ؀ڥʹӨڹΛ༩͑ͳ͍ಠཱͨ͠؀ڥ✅  ࢼߦࡨޡΛ܁Γฦ͢ͷʹద࣮ͨ͠ݧ؀ڥ
  11. ࣮ݧ؀ڥͮ͘Γ  खΛಈ͔ͯ͠ػցֶशͷ஌ݟΛಘΔඞཁ͕͋ͬͨ  "84؀ڥ͸શͯΠϯϑϥ෦͕؅ཧ͍͕ͯͨ͠ʜ  Ϧιʔεͷ༻ҙ͕௥͍͔ͭͳ͍  ͍ΘΏΔຊ൪؀ڥͷӡ༻ʹ߹Θͳ͍ 

    ٻΊΒΕΔͷ͸αʔϏεʹඞཁͳ࣮ݧ݁Ռ  தԝ؅ཧʹݶք͕๚Εͨ ֤ࣄۀ෦ Πϯϑϥ෦ ٕज़ج൫ܥ෦ॺ ݚڀ ։ൃ෦
  12. ར༻࣮੷  "84Λࣗ෼Ͱ৮ͬͯΈΔਓ͕ͱͯ΋૿͑ͨ  ༷ʑͳ"84αʔϏεͷར༻ͱऔΓࠐΈ  "1*(BUFXBZ -BNCEB $PHOJUP %ZOBNP%#

    &MFNFOUBM.FEJB 4FSWJDF (MVF ,JOFTJT 1PMMZ 4BHF.BLFS 93BZ -FY 1JOQPJOU  "QQ4ZODFUDʜ  ৽ػೳɺαʔϏεͷ૑ग़ΛՃ଎  ྉཧ͖Ζ͘ "MFYB4LJMM DPPLQBE57 FUD
  13. എܠΫοΫύουݚڀ։ൃ෦  ೥݄ʹൃ଍  ॳظϝϯόʔ͸໊  ೥݄ݱࡏ͸
 ࠃ಺ʹ໊ ւ֎ʹ໊ 

    ෳ਺ͷਂ૚ֶशϓϩδΣΫτ  (16࣮ݧ؀ڥ͕ඞཁ ϢʔβʔϑΟʔυόοΫ ͷࣗಈλά෇͚ ը૾෼ੳʹΑΔϨγϐͷ෼ྨ Ϩγϐࡐྉ໊ͷࣗಈਖ਼نԽ