Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
mxnet-on-aws-batch
Search
ryo nakamaru
April 01, 2017
Programming
0
790
mxnet-on-aws-batch
JAWS-UG HPC * AI @ 2017.03.031
ryo nakamaru
April 01, 2017
Tweet
Share
More Decks by ryo nakamaru
See All by ryo nakamaru
AWSで楽をするサービスメッシュ入門/appmesh-trial
pottava
1
1.5k
reinforce-2019-recap-lt
pottava
2
4.1k
ScaleShift-jp-2019-summer
pottava
1
210
Firecracker とは何か/what is Firecracker
pottava
12
5.4k
ハイブリッド並列 on Kubernetes/hybrid-parallel-program-on-kubernetes
pottava
1
430
AWS Fargate + Code 兄弟で始める継続的デリバリー / Continuous Delivery with AWS Fargate and Code brothers
pottava
12
3.2k
Singularity と NVIDIA GPU Cloud で作る ハイブリッド機械学習環境の構築 / Building a hybrid environment for Machine Learning with Singularity and NGC
pottava
3
1.3k
明日から始めるちょい足し λ / get-started-with-aws-lambda
pottava
4
2.5k
NGC と Singularity によるハイブリッド機械学習環境 / A hybrid environment for Machine Learning with NGC and Singularity
pottava
0
490
Other Decks in Programming
See All in Programming
Node-RED を(HTTP で)つなげる MCP サーバーを作ってみた
highu
0
110
Discover Metal 4
rei315
2
100
「Cursor/Devin全社導入の理想と現実」のその後
saitoryc
0
610
なんとなくわかった気になるブロックテーマ入門/contents.nagoya 2025 6.28
chiilog
1
240
High-Level Programming Languages in AI Era -Human Thought and Mind-
hayat01sh1da
PRO
0
630
生成AIコーディングとの向き合い方、AIと共創するという考え方 / How to deal with generative AI coding and the concept of co-creating with AI
seike460
PRO
1
340
deno-redisの紹介とJSRパッケージの運用について (toranoana.deno #21)
uki00a
0
160
Java on Azure で LangGraph!
kohei3110
0
170
Team operations that are not burdened by SRE
kazatohiei
1
280
Deep Dive into ~/.claude/projects
hiragram
10
2.1k
AIプログラマーDevinは PHPerの夢を見るか?
shinyasaita
1
170
What Spring Developers Should Know About Jakarta EE
ivargrimstad
0
330
Featured
See All Featured
Faster Mobile Websites
deanohume
307
31k
The Success of Rails: Ensuring Growth for the Next 100 Years
eileencodes
45
7.5k
The Invisible Side of Design
smashingmag
300
51k
Java REST API Framework Comparison - PWX 2021
mraible
31
8.7k
Git: the NoSQL Database
bkeepers
PRO
430
65k
Testing 201, or: Great Expectations
jmmastey
42
7.6k
Fight the Zombie Pattern Library - RWD Summit 2016
marcelosomers
233
17k
Practical Tips for Bootstrapping Information Extraction Pipelines
honnibal
PRO
20
1.3k
What's in a price? How to price your products and services
michaelherold
246
12k
Why Our Code Smells
bkeepers
PRO
337
57k
The Straight Up "How To Draw Better" Workshop
denniskardys
234
140k
Speed Design
sergeychernyshev
32
1k
Transcript
MXNet on AWS Batch JAWS-UG: HPC #9 & AI #5
߹ಉษڧձ @ 2017.03.31
@pottava SUPINF Inc.
• AWS Batch ͷಛ • MXNet on AWS Batch σϞ
• ߏɺϙΠϯτ • ϋϯζΦϯͷ͝հ ͓
AWS Batch ͷಛ
Պֶٕज़ܭࢉɾϋΠύϑΥʔϚϯείϯϐϡʔςΟϯά ༻్ͰਅՁΛൃش͢Δɺେنͳεέʔϧɺδϣϒͷґ ଘఆ͕ٛՄೳͳϚωʔδυฒྻࢄॲཧج൫ɻ AWS Batch
͢Ͱʹ Black Belt ͷࢿྉ͕ެ։͞Ε͍ͯ·͢ɻ AWS Batch http://aws.typepad.com/sajp/2017/02/aws-black-belt-online-seminar-aws-batch.html
ࢲϢʔβࢹͰݱঢ়Λ·ͱΊ·ͨ͠ɻ AWS Batch http://qiita.com/pottava/items/d9886b2e8835c5c0d30f
MXNet on AWS Batch
σϞ
σΟʔϓϥʔχϯάΛར༻ʢLSTM with MXNetʣ AWS നॻɺPDF 27 ϑΝΠϧΛֶशσʔλʹར༻ ग़ͩ͠ͷ୯ޠ͔ΒɺͦΕΒ͍͠ޙଓͷจষΛࣗಈੜ AWS ϗϫΠτϖʔύʔ͘Μ
1. ·ͣҰʢֶशෆेͳʣਪ͕ಈ͘͜ͱΛ֬ೝ 2. ͦͷޙվΊͯɺσʔλͷऔಘɾՃ 3. ֶश 4. ৽൛ਪαʔϏεσϓϩΠ σϞͷྲྀΕ
ʮAmazon EC2 isʯΛ༩͑ͨ࣌ͷɺֶशෆेͳਪྫ ਪͷ༷ࢠ ৽͍͠ݴޠ͕ੜ·Ε·ͨ͠ɻ
ʮAmazon EC2 isʯͰɺͦΕͳΓʹֶशͨ͠ޙͷਪྫ ਪͷ༷ࢠ ݴ͍͍ͨ͜ͱΘ͔Γ·ͤΜ͕ɺ୯ޠจ๏ਵ͠·ͨ͠ɻ
γεςϜߏ
γεςϜߏ AWS Batch S3 2.ֶशδϣϒೖ ΤϯδχΞ SpotFleet DeepLearning AMI v2
1.ֶशσʔλೖ 3.δϣϒεέδϡʔϧ 4.σʔλऔಘ & ֶश 5.݁ՌϞσϧΛอଘ ҰൠϢʔβ AWS Lambda 7.Ϟσϧऔಘ & ਪ APIGateway 6.ਪϦΫΤετ 8.݁ՌԠ ECS EC2 g2.2xlarge EC2 g2.2xlarge
ϙΠϯτ
GPU ར༻ AMI Λ͏ • Unmanaged ڥͰ SpotFleet Ͱ҆͘ʂ •
CloudFormation Ͱڥͷల։Λ༰қʹ
NVIDIA-docker, awslogs • NVIDIA-docker ඞਢͰͳ͍ͷͷೖΕΔͱศར • Unmanaged ڥͰ log
CloudWatch Logs ʹ
privileged ϞʔυͰ job Λఆٛ
git ϦϙδτϦ
git clone on your machine! https://github.com/pottava/mxnet-char-lstm
ϋϯζΦϯ͋Γ·͢
COBOL on AWS Batch http://qiita.com/pottava/items/435c65b1fa72cb643f6e
JAWS-UG AI ࢧ෦
ίϯςϯπ • AWS Ͱ AI αʔϏεΛ࣮ɾӡ༻͢ΔͨΊͷ ɹҰൠతͳٕज़ใɺݟɺࣄྫڞ༗ͷ • ͢Ͱʹ׆༻͍ͯ͠Δํ •
ಋೖΛݕ౼͍ͯ͠Δํ • ԿͦΕ͓͍͍͠ͷʁͳํʢ։࠵͝ͱʹқ͕ଟগҧ͍·͢ʣ
ӡӦϝϯόʔ
ࢀߟจݙ ࢀߟจݙ: • AWS Batch – ؆୯ʹ͑ͯޮతͳόονίϯϐϡʔςΟϯάػೳ – AWS https://aws.amazon.com/jp/batch/
• AWS Black Belt Online SeminarʮAWS Batchʯͷࢿྉ͓ΑͼQAެ։ http://aws.typepad.com/sajp/2017/02/aws-black-belt-online-seminar-aws- batch.html#QCPzBdn.twitter_tweet_count_m • re:Invent 2016: AWS Big Data & Machine Learning Sessionsɻ https://aws.amazon.com/blogs/big-data/reinvent-2016-aws-big-data- machine-learning-sessions/