Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
mxnet-on-aws-batch
Search
ryo nakamaru
April 01, 2017
Programming
0
780
mxnet-on-aws-batch
JAWS-UG HPC * AI @ 2017.03.031
ryo nakamaru
April 01, 2017
Tweet
Share
More Decks by ryo nakamaru
See All by ryo nakamaru
AWSで楽をするサービスメッシュ入門/appmesh-trial
pottava
1
1.4k
reinforce-2019-recap-lt
pottava
2
4.1k
ScaleShift-jp-2019-summer
pottava
1
200
Firecracker とは何か/what is Firecracker
pottava
13
5.3k
ハイブリッド並列 on Kubernetes/hybrid-parallel-program-on-kubernetes
pottava
1
410
AWS Fargate + Code 兄弟で始める継続的デリバリー / Continuous Delivery with AWS Fargate and Code brothers
pottava
12
3.1k
Singularity と NVIDIA GPU Cloud で作る ハイブリッド機械学習環境の構築 / Building a hybrid environment for Machine Learning with Singularity and NGC
pottava
3
1.3k
明日から始めるちょい足し λ / get-started-with-aws-lambda
pottava
4
2.4k
NGC と Singularity によるハイブリッド機械学習環境 / A hybrid environment for Machine Learning with NGC and Singularity
pottava
0
470
Other Decks in Programming
See All in Programming
AIプログラミング雑キャッチアップ
yuheinakasaka
20
5.3k
Drawing Heighway’s Dragon- Recursive Function Rewrite- From Imperative Style in Pascal 64 To Functional Style in Scala 3
philipschwarz
PRO
0
160
Expoによるアプリ開発の現在地とReact Server Componentsが切り開く未来
yukukotani
2
240
責務と認知負荷を整える! 抽象レベルを意識した関心の分離
yahiru
9
1.6k
Serverless Rust: Your Low-Risk Entry Point to Rust in Production (and the benefits are huge)
lmammino
1
170
5分で理解する SOLID 原則 #phpcon_nagoya
shogogg
1
410
ナレッジイネイブリングにAIを活用してみる ゆるSRE勉強会 #9
nealle
0
170
LINE messaging APIを使ってGoogleカレンダーと連携した予約ツールを作ってみた
takumakoike
0
130
技術を改善し続ける
gumioji
0
180
Jasprが凄い話
hyshu
0
190
データベースのオペレーターであるCloudNativePGがStatefulSetを使わない理由に迫る
nnaka2992
0
250
「個人開発マネタイズ大全」が教えてくれたこと
bani24884
1
300
Featured
See All Featured
The Success of Rails: Ensuring Growth for the Next 100 Years
eileencodes
44
7.1k
Become a Pro
speakerdeck
PRO
26
5.2k
Dealing with People You Can't Stand - Big Design 2015
cassininazir
366
25k
Refactoring Trust on Your Teams (GOTO; Chicago 2020)
rmw
33
2.8k
Unsuck your backbone
ammeep
669
57k
Rebuilding a faster, lazier Slack
samanthasiow
80
8.9k
Build The Right Thing And Hit Your Dates
maggiecrowley
34
2.5k
Why Our Code Smells
bkeepers
PRO
336
57k
Facilitating Awesome Meetings
lara
53
6.3k
Keith and Marios Guide to Fast Websites
keithpitt
411
22k
Building a Scalable Design System with Sketch
lauravandoore
461
33k
CSS Pre-Processors: Stylus, Less & Sass
bermonpainter
356
29k
Transcript
MXNet on AWS Batch JAWS-UG: HPC #9 & AI #5
߹ಉษڧձ @ 2017.03.31
@pottava SUPINF Inc.
• AWS Batch ͷಛ • MXNet on AWS Batch σϞ
• ߏɺϙΠϯτ • ϋϯζΦϯͷ͝հ ͓
AWS Batch ͷಛ
Պֶٕज़ܭࢉɾϋΠύϑΥʔϚϯείϯϐϡʔςΟϯά ༻్ͰਅՁΛൃش͢Δɺେنͳεέʔϧɺδϣϒͷґ ଘఆ͕ٛՄೳͳϚωʔδυฒྻࢄॲཧج൫ɻ AWS Batch
͢Ͱʹ Black Belt ͷࢿྉ͕ެ։͞Ε͍ͯ·͢ɻ AWS Batch http://aws.typepad.com/sajp/2017/02/aws-black-belt-online-seminar-aws-batch.html
ࢲϢʔβࢹͰݱঢ়Λ·ͱΊ·ͨ͠ɻ AWS Batch http://qiita.com/pottava/items/d9886b2e8835c5c0d30f
MXNet on AWS Batch
σϞ
σΟʔϓϥʔχϯάΛར༻ʢLSTM with MXNetʣ AWS നॻɺPDF 27 ϑΝΠϧΛֶशσʔλʹར༻ ग़ͩ͠ͷ୯ޠ͔ΒɺͦΕΒ͍͠ޙଓͷจষΛࣗಈੜ AWS ϗϫΠτϖʔύʔ͘Μ
1. ·ͣҰʢֶशෆेͳʣਪ͕ಈ͘͜ͱΛ֬ೝ 2. ͦͷޙվΊͯɺσʔλͷऔಘɾՃ 3. ֶश 4. ৽൛ਪαʔϏεσϓϩΠ σϞͷྲྀΕ
ʮAmazon EC2 isʯΛ༩͑ͨ࣌ͷɺֶशෆेͳਪྫ ਪͷ༷ࢠ ৽͍͠ݴޠ͕ੜ·Ε·ͨ͠ɻ
ʮAmazon EC2 isʯͰɺͦΕͳΓʹֶशͨ͠ޙͷਪྫ ਪͷ༷ࢠ ݴ͍͍ͨ͜ͱΘ͔Γ·ͤΜ͕ɺ୯ޠจ๏ਵ͠·ͨ͠ɻ
γεςϜߏ
γεςϜߏ AWS Batch S3 2.ֶशδϣϒೖ ΤϯδχΞ SpotFleet DeepLearning AMI v2
1.ֶशσʔλೖ 3.δϣϒεέδϡʔϧ 4.σʔλऔಘ & ֶश 5.݁ՌϞσϧΛอଘ ҰൠϢʔβ AWS Lambda 7.Ϟσϧऔಘ & ਪ APIGateway 6.ਪϦΫΤετ 8.݁ՌԠ ECS EC2 g2.2xlarge EC2 g2.2xlarge
ϙΠϯτ
GPU ར༻ AMI Λ͏ • Unmanaged ڥͰ SpotFleet Ͱ҆͘ʂ •
CloudFormation Ͱڥͷల։Λ༰қʹ
NVIDIA-docker, awslogs • NVIDIA-docker ඞਢͰͳ͍ͷͷೖΕΔͱศར • Unmanaged ڥͰ log
CloudWatch Logs ʹ
privileged ϞʔυͰ job Λఆٛ
git ϦϙδτϦ
git clone on your machine! https://github.com/pottava/mxnet-char-lstm
ϋϯζΦϯ͋Γ·͢
COBOL on AWS Batch http://qiita.com/pottava/items/435c65b1fa72cb643f6e
JAWS-UG AI ࢧ෦
ίϯςϯπ • AWS Ͱ AI αʔϏεΛ࣮ɾӡ༻͢ΔͨΊͷ ɹҰൠతͳٕज़ใɺݟɺࣄྫڞ༗ͷ • ͢Ͱʹ׆༻͍ͯ͠Δํ •
ಋೖΛݕ౼͍ͯ͠Δํ • ԿͦΕ͓͍͍͠ͷʁͳํʢ։࠵͝ͱʹқ͕ଟগҧ͍·͢ʣ
ӡӦϝϯόʔ
ࢀߟจݙ ࢀߟจݙ: • AWS Batch – ؆୯ʹ͑ͯޮతͳόονίϯϐϡʔςΟϯάػೳ – AWS https://aws.amazon.com/jp/batch/
• AWS Black Belt Online SeminarʮAWS Batchʯͷࢿྉ͓ΑͼQAެ։ http://aws.typepad.com/sajp/2017/02/aws-black-belt-online-seminar-aws- batch.html#QCPzBdn.twitter_tweet_count_m • re:Invent 2016: AWS Big Data & Machine Learning Sessionsɻ https://aws.amazon.com/blogs/big-data/reinvent-2016-aws-big-data- machine-learning-sessions/