Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
mxnet-on-aws-batch
Search
ryo nakamaru
April 01, 2017
Programming
0
720
mxnet-on-aws-batch
JAWS-UG HPC * AI @ 2017.03.031
ryo nakamaru
April 01, 2017
Tweet
Share
More Decks by ryo nakamaru
See All by ryo nakamaru
AWSで楽をするサービスメッシュ入門/appmesh-trial
pottava
1
1.2k
reinforce-2019-recap-lt
pottava
2
4k
ScaleShift-jp-2019-summer
pottava
1
180
Firecracker とは何か/what is Firecracker
pottava
13
5.1k
ハイブリッド並列 on Kubernetes/hybrid-parallel-program-on-kubernetes
pottava
1
380
AWS Fargate + Code 兄弟で始める継続的デリバリー / Continuous Delivery with AWS Fargate and Code brothers
pottava
12
2.9k
Singularity と NVIDIA GPU Cloud で作る ハイブリッド機械学習環境の構築 / Building a hybrid environment for Machine Learning with Singularity and NGC
pottava
3
1.1k
明日から始めるちょい足し λ / get-started-with-aws-lambda
pottava
4
2.3k
NGC と Singularity によるハイブリッド機械学習環境 / A hybrid environment for Machine Learning with NGC and Singularity
pottava
0
440
Other Decks in Programming
See All in Programming
Open standards for building event-driven applications in the cloud
meteatamel
0
230
The grand strategy of Ruby Parser
yui_knk
5
280
TypeScriptとGraphQLで実現する 型安全なAPI実装 / TSKaigi 2024
hokaccha
5
2.7k
Criando a Woovi em uma semana
daniloab
0
120
Documentation testsの恩恵 / Documentation testing benefits
ssssota
1
560
戦略的DDDは重いのか? / Is strategic DDD heavy?
pictiny
3
2.1k
CQRS meets modern Java
simas
PRO
2
470
初心者のためのRubyKaigi入門/RubyKaigi Introduction
a_matsuda
10
1.9k
mb_trim関数を作りました
youkidearitai
PRO
1
200
Runtime Objects in Rust
mitsuhiko
0
220
TypeScriptから始める VR生活
tamagokakeg
2
110
GitLab CI/CD で C#/WPFアプリケーションのテストとインストーラーのビルド・デプロイを自動化する
hacarus
0
610
Featured
See All Featured
Let's Do A Bunch of Simple Stuff to Make Websites Faster
chriscoyier
501
140k
Learning to Love Humans: Emotional Interface Design
aarron
267
39k
Design by the Numbers
sachag
274
18k
Large-scale JavaScript Application Architecture
addyosmani
504
110k
Art, The Web, and Tiny UX
lynnandtonic
290
19k
Testing 201, or: Great Expectations
jmmastey
30
6.4k
Scaling GitHub
holman
457
140k
Designing Experiences People Love
moore
136
23k
KATA
mclloyd
16
12k
The Success of Rails: Ensuring Growth for the Next 100 Years
eileencodes
34
6.1k
Mobile First: as difficult as doing things right
swwweet
217
8.6k
For a Future-Friendly Web
brad_frost
172
9k
Transcript
MXNet on AWS Batch JAWS-UG: HPC #9 & AI #5
߹ಉษڧձ @ 2017.03.31
@pottava SUPINF Inc.
• AWS Batch ͷಛ • MXNet on AWS Batch σϞ
• ߏɺϙΠϯτ • ϋϯζΦϯͷ͝հ ͓
AWS Batch ͷಛ
Պֶٕज़ܭࢉɾϋΠύϑΥʔϚϯείϯϐϡʔςΟϯά ༻్ͰਅՁΛൃش͢Δɺେنͳεέʔϧɺδϣϒͷґ ଘఆ͕ٛՄೳͳϚωʔδυฒྻࢄॲཧج൫ɻ AWS Batch
͢Ͱʹ Black Belt ͷࢿྉ͕ެ։͞Ε͍ͯ·͢ɻ AWS Batch http://aws.typepad.com/sajp/2017/02/aws-black-belt-online-seminar-aws-batch.html
ࢲϢʔβࢹͰݱঢ়Λ·ͱΊ·ͨ͠ɻ AWS Batch http://qiita.com/pottava/items/d9886b2e8835c5c0d30f
MXNet on AWS Batch
σϞ
σΟʔϓϥʔχϯάΛར༻ʢLSTM with MXNetʣ AWS നॻɺPDF 27 ϑΝΠϧΛֶशσʔλʹར༻ ग़ͩ͠ͷ୯ޠ͔ΒɺͦΕΒ͍͠ޙଓͷจষΛࣗಈੜ AWS ϗϫΠτϖʔύʔ͘Μ
1. ·ͣҰʢֶशෆेͳʣਪ͕ಈ͘͜ͱΛ֬ೝ 2. ͦͷޙվΊͯɺσʔλͷऔಘɾՃ 3. ֶश 4. ৽൛ਪαʔϏεσϓϩΠ σϞͷྲྀΕ
ʮAmazon EC2 isʯΛ༩͑ͨ࣌ͷɺֶशෆेͳਪྫ ਪͷ༷ࢠ ৽͍͠ݴޠ͕ੜ·Ε·ͨ͠ɻ
ʮAmazon EC2 isʯͰɺͦΕͳΓʹֶशͨ͠ޙͷਪྫ ਪͷ༷ࢠ ݴ͍͍ͨ͜ͱΘ͔Γ·ͤΜ͕ɺ୯ޠจ๏ਵ͠·ͨ͠ɻ
γεςϜߏ
γεςϜߏ AWS Batch S3 2.ֶशδϣϒೖ ΤϯδχΞ SpotFleet DeepLearning AMI v2
1.ֶशσʔλೖ 3.δϣϒεέδϡʔϧ 4.σʔλऔಘ & ֶश 5.݁ՌϞσϧΛอଘ ҰൠϢʔβ AWS Lambda 7.Ϟσϧऔಘ & ਪ APIGateway 6.ਪϦΫΤετ 8.݁ՌԠ ECS EC2 g2.2xlarge EC2 g2.2xlarge
ϙΠϯτ
GPU ར༻ AMI Λ͏ • Unmanaged ڥͰ SpotFleet Ͱ҆͘ʂ •
CloudFormation Ͱڥͷల։Λ༰қʹ
NVIDIA-docker, awslogs • NVIDIA-docker ඞਢͰͳ͍ͷͷೖΕΔͱศར • Unmanaged ڥͰ log
CloudWatch Logs ʹ
privileged ϞʔυͰ job Λఆٛ
git ϦϙδτϦ
git clone on your machine! https://github.com/pottava/mxnet-char-lstm
ϋϯζΦϯ͋Γ·͢
COBOL on AWS Batch http://qiita.com/pottava/items/435c65b1fa72cb643f6e
JAWS-UG AI ࢧ෦
ίϯςϯπ • AWS Ͱ AI αʔϏεΛ࣮ɾӡ༻͢ΔͨΊͷ ɹҰൠతͳٕज़ใɺݟɺࣄྫڞ༗ͷ • ͢Ͱʹ׆༻͍ͯ͠Δํ •
ಋೖΛݕ౼͍ͯ͠Δํ • ԿͦΕ͓͍͍͠ͷʁͳํʢ։࠵͝ͱʹқ͕ଟগҧ͍·͢ʣ
ӡӦϝϯόʔ
ࢀߟจݙ ࢀߟจݙ: • AWS Batch – ؆୯ʹ͑ͯޮతͳόονίϯϐϡʔςΟϯάػೳ – AWS https://aws.amazon.com/jp/batch/
• AWS Black Belt Online SeminarʮAWS Batchʯͷࢿྉ͓ΑͼQAެ։ http://aws.typepad.com/sajp/2017/02/aws-black-belt-online-seminar-aws- batch.html#QCPzBdn.twitter_tweet_count_m • re:Invent 2016: AWS Big Data & Machine Learning Sessionsɻ https://aws.amazon.com/blogs/big-data/reinvent-2016-aws-big-data- machine-learning-sessions/