Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
mxnet-on-aws-batch
Search
ryo nakamaru
April 01, 2017
Programming
0
790
mxnet-on-aws-batch
JAWS-UG HPC * AI @ 2017.03.031
ryo nakamaru
April 01, 2017
Tweet
Share
More Decks by ryo nakamaru
See All by ryo nakamaru
AWSで楽をするサービスメッシュ入門/appmesh-trial
pottava
1
1.5k
reinforce-2019-recap-lt
pottava
2
4.1k
ScaleShift-jp-2019-summer
pottava
1
210
Firecracker とは何か/what is Firecracker
pottava
12
5.4k
ハイブリッド並列 on Kubernetes/hybrid-parallel-program-on-kubernetes
pottava
1
430
AWS Fargate + Code 兄弟で始める継続的デリバリー / Continuous Delivery with AWS Fargate and Code brothers
pottava
12
3.2k
Singularity と NVIDIA GPU Cloud で作る ハイブリッド機械学習環境の構築 / Building a hybrid environment for Machine Learning with Singularity and NGC
pottava
3
1.3k
明日から始めるちょい足し λ / get-started-with-aws-lambda
pottava
4
2.5k
NGC と Singularity によるハイブリッド機械学習環境 / A hybrid environment for Machine Learning with NGC and Singularity
pottava
0
490
Other Decks in Programming
See All in Programming
AIエージェントによるテストフレームワーク Arbigent
takahirom
0
370
セキュリティマネジャー廃止とクラウドネイティブ型サンドボックス活用
kazumura
1
170
Javaのルールをねじ曲げろ!禁断の操作とその代償から学ぶメタプログラミング入門 / A Guide to Metaprogramming: Lessons from Forbidden Techniques and Their Price
nrslib
3
1.9k
ワンバイナリWebサービスのススメ
mackee
10
7.7k
RubyKaigiで得られる10の価値 〜Ruby話を聞くことだけが RubyKaigiじゃない〜
tomohiko9090
0
140
Elixir で IoT 開発、 Nerves なら簡単にできる!?
pojiro
1
120
つよそうにふるまい、つよい成果を出すのなら、つよいのかもしれない
irof
1
280
Create a website using Spatial Web
akkeylab
0
270
AWS CDKの推しポイント 〜CloudFormationと比較してみた〜
akihisaikeda
3
230
統一感のある Go コードを生成 AI の力で手にいれる
otakakot
0
3k
Using AI Tools Around Software Development
inouehi
0
1.2k
既存デザインを変更せずにタップ領域を広げる方法
tahia910
1
190
Featured
See All Featured
Embracing the Ebb and Flow
colly
86
4.7k
Refactoring Trust on Your Teams (GOTO; Chicago 2020)
rmw
34
3k
"I'm Feeling Lucky" - Building Great Search Experiences for Today's Users (#IAC19)
danielanewman
228
22k
YesSQL, Process and Tooling at Scale
rocio
172
14k
How to Ace a Technical Interview
jacobian
276
23k
Navigating Team Friction
lara
186
15k
How to Think Like a Performance Engineer
csswizardry
24
1.7k
GraphQLの誤解/rethinking-graphql
sonatard
71
11k
Principles of Awesome APIs and How to Build Them.
keavy
126
17k
Measuring & Analyzing Core Web Vitals
bluesmoon
7
480
Optimizing for Happiness
mojombo
379
70k
Faster Mobile Websites
deanohume
307
31k
Transcript
MXNet on AWS Batch JAWS-UG: HPC #9 & AI #5
߹ಉษڧձ @ 2017.03.31
@pottava SUPINF Inc.
• AWS Batch ͷಛ • MXNet on AWS Batch σϞ
• ߏɺϙΠϯτ • ϋϯζΦϯͷ͝հ ͓
AWS Batch ͷಛ
Պֶٕज़ܭࢉɾϋΠύϑΥʔϚϯείϯϐϡʔςΟϯά ༻్ͰਅՁΛൃش͢Δɺେنͳεέʔϧɺδϣϒͷґ ଘఆ͕ٛՄೳͳϚωʔδυฒྻࢄॲཧج൫ɻ AWS Batch
͢Ͱʹ Black Belt ͷࢿྉ͕ެ։͞Ε͍ͯ·͢ɻ AWS Batch http://aws.typepad.com/sajp/2017/02/aws-black-belt-online-seminar-aws-batch.html
ࢲϢʔβࢹͰݱঢ়Λ·ͱΊ·ͨ͠ɻ AWS Batch http://qiita.com/pottava/items/d9886b2e8835c5c0d30f
MXNet on AWS Batch
σϞ
σΟʔϓϥʔχϯάΛར༻ʢLSTM with MXNetʣ AWS നॻɺPDF 27 ϑΝΠϧΛֶशσʔλʹར༻ ग़ͩ͠ͷ୯ޠ͔ΒɺͦΕΒ͍͠ޙଓͷจষΛࣗಈੜ AWS ϗϫΠτϖʔύʔ͘Μ
1. ·ͣҰʢֶशෆेͳʣਪ͕ಈ͘͜ͱΛ֬ೝ 2. ͦͷޙվΊͯɺσʔλͷऔಘɾՃ 3. ֶश 4. ৽൛ਪαʔϏεσϓϩΠ σϞͷྲྀΕ
ʮAmazon EC2 isʯΛ༩͑ͨ࣌ͷɺֶशෆेͳਪྫ ਪͷ༷ࢠ ৽͍͠ݴޠ͕ੜ·Ε·ͨ͠ɻ
ʮAmazon EC2 isʯͰɺͦΕͳΓʹֶशͨ͠ޙͷਪྫ ਪͷ༷ࢠ ݴ͍͍ͨ͜ͱΘ͔Γ·ͤΜ͕ɺ୯ޠจ๏ਵ͠·ͨ͠ɻ
γεςϜߏ
γεςϜߏ AWS Batch S3 2.ֶशδϣϒೖ ΤϯδχΞ SpotFleet DeepLearning AMI v2
1.ֶशσʔλೖ 3.δϣϒεέδϡʔϧ 4.σʔλऔಘ & ֶश 5.݁ՌϞσϧΛอଘ ҰൠϢʔβ AWS Lambda 7.Ϟσϧऔಘ & ਪ APIGateway 6.ਪϦΫΤετ 8.݁ՌԠ ECS EC2 g2.2xlarge EC2 g2.2xlarge
ϙΠϯτ
GPU ར༻ AMI Λ͏ • Unmanaged ڥͰ SpotFleet Ͱ҆͘ʂ •
CloudFormation Ͱڥͷల։Λ༰қʹ
NVIDIA-docker, awslogs • NVIDIA-docker ඞਢͰͳ͍ͷͷೖΕΔͱศར • Unmanaged ڥͰ log
CloudWatch Logs ʹ
privileged ϞʔυͰ job Λఆٛ
git ϦϙδτϦ
git clone on your machine! https://github.com/pottava/mxnet-char-lstm
ϋϯζΦϯ͋Γ·͢
COBOL on AWS Batch http://qiita.com/pottava/items/435c65b1fa72cb643f6e
JAWS-UG AI ࢧ෦
ίϯςϯπ • AWS Ͱ AI αʔϏεΛ࣮ɾӡ༻͢ΔͨΊͷ ɹҰൠతͳٕज़ใɺݟɺࣄྫڞ༗ͷ • ͢Ͱʹ׆༻͍ͯ͠Δํ •
ಋೖΛݕ౼͍ͯ͠Δํ • ԿͦΕ͓͍͍͠ͷʁͳํʢ։࠵͝ͱʹқ͕ଟগҧ͍·͢ʣ
ӡӦϝϯόʔ
ࢀߟจݙ ࢀߟจݙ: • AWS Batch – ؆୯ʹ͑ͯޮతͳόονίϯϐϡʔςΟϯάػೳ – AWS https://aws.amazon.com/jp/batch/
• AWS Black Belt Online SeminarʮAWS Batchʯͷࢿྉ͓ΑͼQAެ։ http://aws.typepad.com/sajp/2017/02/aws-black-belt-online-seminar-aws- batch.html#QCPzBdn.twitter_tweet_count_m • re:Invent 2016: AWS Big Data & Machine Learning Sessionsɻ https://aws.amazon.com/blogs/big-data/reinvent-2016-aws-big-data- machine-learning-sessions/