Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
mxnet-on-aws-batch
Search
ryo nakamaru
April 01, 2017
Programming
0
800
mxnet-on-aws-batch
JAWS-UG HPC * AI @ 2017.03.031
ryo nakamaru
April 01, 2017
Tweet
Share
More Decks by ryo nakamaru
See All by ryo nakamaru
AWSで楽をするサービスメッシュ入門/appmesh-trial
pottava
1
1.5k
reinforce-2019-recap-lt
pottava
2
4.1k
ScaleShift-jp-2019-summer
pottava
1
210
Firecracker とは何か/what is Firecracker
pottava
12
5.4k
ハイブリッド並列 on Kubernetes/hybrid-parallel-program-on-kubernetes
pottava
1
440
AWS Fargate + Code 兄弟で始める継続的デリバリー / Continuous Delivery with AWS Fargate and Code brothers
pottava
12
3.2k
Singularity と NVIDIA GPU Cloud で作る ハイブリッド機械学習環境の構築 / Building a hybrid environment for Machine Learning with Singularity and NGC
pottava
3
1.4k
明日から始めるちょい足し λ / get-started-with-aws-lambda
pottava
4
2.5k
NGC と Singularity によるハイブリッド機械学習環境 / A hybrid environment for Machine Learning with NGC and Singularity
pottava
0
490
Other Decks in Programming
See All in Programming
testingを眺める
matumoto
1
120
CSC305 Summer Lecture 12
javiergs
PRO
0
130
Processing Gem ベースの、2D レトロゲームエンジンの開発
tokujiros
2
100
Claude Codeで挑むOSSコントリビュート
eycjur
0
180
パスタの技術
yusukebe
1
540
LLMOpsのパフォーマンスを支える技術と現場で実践した改善
po3rin
8
990
実用的なGOCACHEPROG実装をするために / golang.tokyo #40
mazrean
1
110
個人軟體時代
ethanhuang13
0
240
TanStack DB ~状態管理の新しい考え方~
bmthd
2
360
詳解!defer panic recover のしくみ / Understanding defer, panic, and recover
convto
0
190
go test -json そして testing.T.Attr / Kyoto.go #63
utgwkk
1
160
AWS発のAIエディタKiroを使ってみた
iriikeita
1
110
Featured
See All Featured
Designing for humans not robots
tammielis
253
25k
Designing for Performance
lara
610
69k
It's Worth the Effort
3n
187
28k
Fantastic passwords and where to find them - at NoRuKo
philnash
52
3.4k
Improving Core Web Vitals using Speculation Rules API
sergeychernyshev
18
1.1k
XXLCSS - How to scale CSS and keep your sanity
sugarenia
248
1.3M
Faster Mobile Websites
deanohume
309
31k
Large-scale JavaScript Application Architecture
addyosmani
512
110k
Scaling GitHub
holman
463
140k
The Cult of Friendly URLs
andyhume
79
6.6k
Making the Leap to Tech Lead
cromwellryan
134
9.5k
Optimizing for Happiness
mojombo
379
70k
Transcript
MXNet on AWS Batch JAWS-UG: HPC #9 & AI #5
߹ಉษڧձ @ 2017.03.31
@pottava SUPINF Inc.
• AWS Batch ͷಛ • MXNet on AWS Batch σϞ
• ߏɺϙΠϯτ • ϋϯζΦϯͷ͝հ ͓
AWS Batch ͷಛ
Պֶٕज़ܭࢉɾϋΠύϑΥʔϚϯείϯϐϡʔςΟϯά ༻్ͰਅՁΛൃش͢Δɺେنͳεέʔϧɺδϣϒͷґ ଘఆ͕ٛՄೳͳϚωʔδυฒྻࢄॲཧج൫ɻ AWS Batch
͢Ͱʹ Black Belt ͷࢿྉ͕ެ։͞Ε͍ͯ·͢ɻ AWS Batch http://aws.typepad.com/sajp/2017/02/aws-black-belt-online-seminar-aws-batch.html
ࢲϢʔβࢹͰݱঢ়Λ·ͱΊ·ͨ͠ɻ AWS Batch http://qiita.com/pottava/items/d9886b2e8835c5c0d30f
MXNet on AWS Batch
σϞ
σΟʔϓϥʔχϯάΛར༻ʢLSTM with MXNetʣ AWS നॻɺPDF 27 ϑΝΠϧΛֶशσʔλʹར༻ ग़ͩ͠ͷ୯ޠ͔ΒɺͦΕΒ͍͠ޙଓͷจষΛࣗಈੜ AWS ϗϫΠτϖʔύʔ͘Μ
1. ·ͣҰʢֶशෆेͳʣਪ͕ಈ͘͜ͱΛ֬ೝ 2. ͦͷޙվΊͯɺσʔλͷऔಘɾՃ 3. ֶश 4. ৽൛ਪαʔϏεσϓϩΠ σϞͷྲྀΕ
ʮAmazon EC2 isʯΛ༩͑ͨ࣌ͷɺֶशෆेͳਪྫ ਪͷ༷ࢠ ৽͍͠ݴޠ͕ੜ·Ε·ͨ͠ɻ
ʮAmazon EC2 isʯͰɺͦΕͳΓʹֶशͨ͠ޙͷਪྫ ਪͷ༷ࢠ ݴ͍͍ͨ͜ͱΘ͔Γ·ͤΜ͕ɺ୯ޠจ๏ਵ͠·ͨ͠ɻ
γεςϜߏ
γεςϜߏ AWS Batch S3 2.ֶशδϣϒೖ ΤϯδχΞ SpotFleet DeepLearning AMI v2
1.ֶशσʔλೖ 3.δϣϒεέδϡʔϧ 4.σʔλऔಘ & ֶश 5.݁ՌϞσϧΛอଘ ҰൠϢʔβ AWS Lambda 7.Ϟσϧऔಘ & ਪ APIGateway 6.ਪϦΫΤετ 8.݁ՌԠ ECS EC2 g2.2xlarge EC2 g2.2xlarge
ϙΠϯτ
GPU ར༻ AMI Λ͏ • Unmanaged ڥͰ SpotFleet Ͱ҆͘ʂ •
CloudFormation Ͱڥͷల։Λ༰қʹ
NVIDIA-docker, awslogs • NVIDIA-docker ඞਢͰͳ͍ͷͷೖΕΔͱศར • Unmanaged ڥͰ log
CloudWatch Logs ʹ
privileged ϞʔυͰ job Λఆٛ
git ϦϙδτϦ
git clone on your machine! https://github.com/pottava/mxnet-char-lstm
ϋϯζΦϯ͋Γ·͢
COBOL on AWS Batch http://qiita.com/pottava/items/435c65b1fa72cb643f6e
JAWS-UG AI ࢧ෦
ίϯςϯπ • AWS Ͱ AI αʔϏεΛ࣮ɾӡ༻͢ΔͨΊͷ ɹҰൠతͳٕज़ใɺݟɺࣄྫڞ༗ͷ • ͢Ͱʹ׆༻͍ͯ͠Δํ •
ಋೖΛݕ౼͍ͯ͠Δํ • ԿͦΕ͓͍͍͠ͷʁͳํʢ։࠵͝ͱʹқ͕ଟগҧ͍·͢ʣ
ӡӦϝϯόʔ
ࢀߟจݙ ࢀߟจݙ: • AWS Batch – ؆୯ʹ͑ͯޮతͳόονίϯϐϡʔςΟϯάػೳ – AWS https://aws.amazon.com/jp/batch/
• AWS Black Belt Online SeminarʮAWS Batchʯͷࢿྉ͓ΑͼQAެ։ http://aws.typepad.com/sajp/2017/02/aws-black-belt-online-seminar-aws- batch.html#QCPzBdn.twitter_tweet_count_m • re:Invent 2016: AWS Big Data & Machine Learning Sessionsɻ https://aws.amazon.com/blogs/big-data/reinvent-2016-aws-big-data- machine-learning-sessions/